{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Example notebook - Create Cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "This example uses a python library for working with an IAE instance.\n",
    "\n",
    "IBM Analytics Engine Python Library links:\n",
    "\n",
    " - documentation is on [readthedocs](http://ibm-analytics-engine-python.readthedocs.io/en/latest/)\n",
    " - source code repository is on [github](https://github.com/snowch/ibm-analytics-engine-python)\n",
    " - this notebook is on [github](https://github.com/snowch/ibm-analytics-engine-python/blob/master/docs/example_notebooks/CreateCluster.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "! pip install --quiet --upgrade git+https://github.com/snowch/ibm-analytics-engine-python@master"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from ibm_analytics_engine import CloudFoundryAPI, CloudFoundryAPI\n",
    "from ibm_analytics_engine import IAE, IAEServicePlanGuid, IAEClusterSpecificationExamples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use an IBM Cloud API key to work with an IAE Instance.  You can create an API using the bluemix CLI tools, e.g.\n",
    "\n",
    "```bash\n",
    "bluemix iam api-key-create My_IAE_Key -d \"This is my IAE API key\" -f my_api_key.json\n",
    "\n",
    "```\n",
    "\n",
    "Alternatively, follow [these instructions](https://console.bluemix.net/docs/iam/userid_keys.html#userapikey) to create an API key using the IBM Cloud web console and then save it in a secure location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "cf = CloudFoundryAPI(api_key_filename='./my_api_key.json')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "You aren't restricted to just using an API key file.  If you have the API key value, you can do this:\n",
    "\n",
    "```python\n",
    "from getpass import getpass\n",
    "api_key = getpass(\"Enter your api key: \")\n",
    "\n",
    "cf = CloudFoundryAPI(api_key=api_key)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Provide your organizaton name and space name:\n",
    "\n",
    "SPACE_GUID = cf.space_guid(org_name='my_org_name', space_name='my_space_name')\n",
    "print(SPACE_GUID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "If you couldn't find your space guid, try printing out all your orgs and spaces:\n",
    "\n",
    "```python\n",
    "cf.print_orgs_and_spaces()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# We interact with the IBM Analytics Engine through the IAE class.  \n",
    "# Let's create an instance of it:\n",
    "\n",
    "iae = IAE(cf_client=cf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# List the clusters in the space\n",
    "\n",
    "iae.clusters(space_guid=SPACE_GUID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "cluster_guid = iae.create_cluster(service_instance_name = 'MY_SPARK_CLUSTER', \n",
    "                   service_plan_guid = IAEServicePlanGuid.LITE, \n",
    "                   cluster_creation_parameters = {\n",
    "                        \"hardware_config\": \"default\",\n",
    "                        \"num_compute_nodes\": 1,\n",
    "                        \"software_package\": \"ae-1.0-spark\",\n",
    "                    }, \n",
    "                   space_guid = SPACE_GUID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Alternative options for service_plan_guid:\n",
    "    \n",
    " - `IAEServicePlanGuid.STD_HOURLY`\n",
    " - `IAEServicePlanGuid.STD_MONTHLY`\n",
    " \n",
    "There are also some examples of cluster_creation_paramters in `IAEClusterSpecificationExamples` class:\n",
    "\n",
    "```python\n",
    "IAEClusterSpecificationExamples.SINGLE_NODE_BASIC_SPARK = {\n",
    "    'num_compute_nodes': 1,\n",
    "    'hardware_config': 'default', \n",
    "    'software_package': 'ae-1.0-spark'\n",
    "    }\n",
    "```\n",
    "and:\n",
    "```python\n",
    "IAEClusterSpecificationExamples.SINGLE_NODE_BASIC_HADOOP = {\n",
    "    'num_compute_nodes': 1, \n",
    "    'hardware_config': 'default', \n",
    "    'software_package': 'ae-1.0-hadoop-spark'\n",
    "    }\n",
    "```\n",
    "These have been provided so you don't have to remember the parameters for creating a default basic cluster.\n",
    "\n",
    "You would use them like this:\n",
    "\n",
    "```python\n",
    "iae.create_cluster(..., \n",
    "    cluster_creation_parameters = IAEClusterSpecificationExamples.SINGLE_NODE_BASIC_SPARK, \n",
    "    ...)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Poll the cluster until provisioning has finished\n",
    "\n",
    "import time\n",
    "while True:\n",
    "    status = iae.status(cluster_instance_guid=cluster_guid)\n",
    "    print(status)\n",
    "    if status == 'succeeded' or status == 'failed': break\n",
    "    time.sleep(60)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Only run this cell after the previous cell has finished with the status 'succeeded',\n",
    "# otherwise you will receive an error trying to get or create the credentials.\n",
    "\n",
    "import json\n",
    "\n",
    "# get the credentials data for the cluster in vcap json format\n",
    "vcap = iae.get_or_create_credentials(cluster_instance_guid=cluster_guid)\n",
    "\n",
    "# print the credentials out\n",
    "vcap_formatted = json.dumps(vcap, indent=4, separators=(',', ': '))\n",
    "print(vcap_formatted)\n",
    "\n",
    "# save the credentials to a file\n",
    "with open('./vcap.json', 'w') as vcap_file:\n",
    "    vcap_file.write(vcap_formatted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Grab the ambari console url\n",
    "\n",
    "print(vcap['cluster']['service_endpoints']['ambari_console'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Delete the cluster.  Recursive=True will delete service bindings, service keys,\n",
    "# and routes associated with the service instance.\n",
    "\n",
    "iae.delete_cluster(cluster_guid, recursive=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}