add %%px --local to Parallel Magics notebook

docs/examples/parallel/Parallel Magics.ipynb

@@ -1,452 +1,476 b''
 {
  "metadata": {
   "name": "Parallel Magics"
  },
  "nbformat": 3,
  "nbformat_minor": 0,
  "worksheets": [
   {
    "cells": [
     {
      "cell_type": "heading",
      "level": 1,
      "metadata": {},
      "source": [
       "Using Parallel Magics"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "IPython has a few magics for working with your engines.\n",
       "\n",
       "This assumes you have started an IPython cluster, either with the notebook interface,\n",
       "or the `ipcluster/controller/engine` commands."
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "from IPython import parallel\n",
       "rc = parallel.Client()\n",
       "dv = rc[:]\n",
       "rc.ids"
      ],
      "language": "python",
      "metadata": {},
-     "outputs": []
+     "outputs": [],
+     "prompt_number": 2
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Creating a Client registers the parallel magics `%px`, `%%px`, `%pxresult`, `pxconfig`, and `%autopx`.  \n",
       "These magics are initially associated with a DirectView always associated with all currently registered engines."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Now we can execute code remotely with `%px`:"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px a=5"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px print a"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px a"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "with dv.sync_imports():\n",
       "    import sys"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px print >> sys.stderr, \"ERROR\""
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "You don't have to wait for results.  The `%pxconfig` magic lets you change the default blocking/targets for the `%px` magics:"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxconfig --noblock"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px import time\n",
       "%px time.sleep(5)\n",
       "%px time.time()"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "But you will notice that this didn't output the result of the last command.\n",
       "For this, we have `%pxresult`, which displays the output of the latest request:"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxresult"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Remember, an IPython engine is IPython, so you can do magics remotely as well!"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxconfig --block\n",
       "%px %pylab inline"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "`%%px` can also be used as a cell magic, for submitting whole blocks.\n",
       "This one acceps `--block` and `--noblock` flags to specify\n",
       "the blocking behavior, though the default is unchanged.\n"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "dv.scatter('id', dv.targets, flatten=True)\n",
       "dv['stride'] = len(dv)"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%%px --noblock\n",
       "x = linspace(0,pi,1000)\n",
       "for n in range(id,12, stride):\n",
       "    print n\n",
       "    plt.plot(x,sin(n*x))\n",
       "plt.title(\"Plot %i\" % id)"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxresult"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "It also lets you choose some amount of the grouping of the outputs with `--group-outputs`:\n",
       "\n",
       "The choices are:\n",
       "\n",
       "* `engine` - all of an engine's output is collected together\n",
       "* `type` - where stdout of each engine is grouped, etc. (the default)\n",
       "* `order` - same as `type`, but individual displaypub outputs are interleaved.\n",
       "  That is, it will output the first plot from each engine, then the second from each,\n",
       "  etc."
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%%px --group-outputs=engine\n",
       "x = linspace(0,pi,1000)\n",
       "for n in range(id+1,12, stride):\n",
       "    print n\n",
       "    plt.figure()\n",
       "    plt.plot(x,sin(n*x))\n",
       "    plt.title(\"Plot %i\" % n)"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "When you specify 'order', then individual display outputs (e.g. plots) will be interleaved.\n",
       "\n",
       "`%pxresult` takes the same output-ordering arguments as `%%px`, \n",
       "so you can view the previous result in a variety of different ways with a few sequential calls to `%pxresult`:"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxresult --group-outputs=order"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "heading",
      "level": 2,
      "metadata": {},
      "source": [
       "Single-engine views"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "When a DirectView has a single target, the output is a bit simpler (no prefixes on stdout/err, etc.):"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "def generate_output():\n",
       "    \"\"\"function for testing output\n",
       "    \n",
       "    publishes two outputs of each type, and returns something\n",
       "    \"\"\"\n",
       "    \n",
       "    import sys,os\n",
       "    from IPython.core.display import display, HTML, Math\n",
       "    \n",
       "    print \"stdout\"\n",
       "    print >> sys.stderr, \"stderr\"\n",
       "    \n",
       "    display(HTML(\"<b>HTML</b>\"))\n",
       "    \n",
       "    print \"stdout2\"\n",
       "    print >> sys.stderr, \"stderr2\"\n",
       "    \n",
       "    display(Math(r\"\\alpha=\\beta\"))\n",
       "    \n",
       "    return os.getpid()\n",
       "\n",
       "dv['generate_output'] = generate_output"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "You can also have more than one set of parallel magics registered at a time.\n",
       "\n",
       "The `View.activate()` method takes a suffix argument, which is added to `'px'`."
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "e0 = rc[-1]\n",
       "e0.block = True\n",
       "e0.activate('0')"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px0 generate_output()"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%px generate_output()"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "As mentioned above, we can redisplay those same results with various grouping:"
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxresult --group-outputs order"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%pxresult --group-outputs engine"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "heading",
      "level": 2,
      "metadata": {},
      "source": [
       "Parallel Exceptions"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "When you raise exceptions with the parallel exception,\n",
       "the CompositeError raised locally will display your remote traceback."
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%%px\n",
       "from numpy.random import random\n",
       "A = random((100,100,'invalid shape'))"
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
      "cell_type": "heading",
      "level": 2,
      "metadata": {},
      "source": [
       "Remote Cell Magics"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Remember, Engines are IPython too, so the cell that is run remotely by %%px can in turn use a cell magic."
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
       "%%px\n",
       "%%timeit\n",
       "from numpy.random import random\n",
       "from numpy.linalg import norm\n",
       "A = random((100,100))\n",
       "norm(A, 2)    "
      ],
      "language": "python",
      "metadata": {},
      "outputs": []
     },
     {
+     "cell_type": "heading",
+     "level": 2,
+     "metadata": {},
+     "source": [
+      "Local Execution"
+     ]
+    },
+    {
+     "cell_type": "markdown",
+     "metadata": {},
+     "source": [
+      "As of IPython 0.14, you can instruct `%%px` to also execute the cell locally.\n",
+      "This is useful for interactive definitions,\n",
+      "or if you want to load a data source everywhere,\n",
+      "not just on the engines."
+     ]
+    },
+    {
      "cell_type": "code",
      "collapsed": false,
-     "input": [],
+     "input": [
+      "%%px --local\n",
+      "thispid = os.getpid()\n",
+      "print thispid"
+     ],
      "language": "python",
      "metadata": {},
-     "outputs": []
+     "outputs": [],
+     "prompt_number": 5
     }
    ],
    "metadata": {}
   }
  ]
 }
  No newline at end of file