examples/IPython Kernel/Script Magics.ipynb

{
 "metadata": {
  "name": "Script Magics"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Running Scripts from IPython"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "IPython has a `%%script` cell magic, which lets you run a cell in\n",
      "a subprocess of any interpreter on your system, such as: bash, ruby, perl, zsh, R, etc.\n",
      "\n",
      "It can even be a script of your own, which expects input on stdin."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import sys"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Basic usage"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To use it, simply pass a path or shell command to the program you want to run on the `%%script` line,\n",
      "and the rest of the cell will be run by that script, and stdout/err from the subprocess are captured and displayed."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%script python\n",
      "import sys\n",
      "print 'hello from Python %s' % sys.version"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello from Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) \n",
        "[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]\n"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%script python3\n",
      "import sys\n",
      "print('hello from Python: %s' % sys.version)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello from Python: 3.2.3 (v3.2.3:3d0686d90f55, Apr 10 2012, 11:25:50) \n",
        "[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]\n"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "IPython also creates aliases for a few common interpreters, such as bash, ruby, perl, etc.\n",
      "\n",
      "These are all equivalent to `%%script <name>`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%ruby\n",
      "puts \"Hello from Ruby #{RUBY_VERSION}\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Hello from Ruby 1.8.7\n"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "echo \"hello from $BASH\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello from /usr/local/bin/bash\n"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Capturing output"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can also capture stdout/err from these subprocesses into Python variables, instead of letting them go directly to stdout/err"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "echo \"hi, stdout\"\n",
      "echo \"hello, stderr\" >&2\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hi, stdout\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stderr",
       "text": [
        "hello, stderr\n"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash --out output --err error\n",
      "echo \"hi, stdout\"\n",
      "echo \"hello, stderr\" >&2"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(error)\n",
      "print(output)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello, stderr\n",
        "\n",
        "hi, stdout\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Background Scripts"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "These scripts can be run in the background, by adding the `--bg` flag.\n",
      "\n",
      "When you do this, output is discarded unless you use the `--out/err`\n",
      "flags to store output as above."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%ruby --bg --out ruby_lines\n",
      "for n in 1...10\n",
      "    sleep 1\n",
      "    puts \"line #{n}\"\n",
      "    STDOUT.flush\n",
      "end"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Starting job # 0 in a separate thread.\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When you do store output of a background thread, these are the stdout/err *pipes*,\n",
      "rather than the text of the output."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ruby_lines"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "<open file '<fdopen>', mode 'rb' at 0x10a4be660>"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(ruby_lines.read())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "line 1\n",
        "line 2\n",
        "line 3\n",
        "line 4\n",
        "line 5\n",
        "line 6\n",
        "line 7\n",
        "line 8\n",
        "line 9\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Arguments to subcommand"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can pass arguments the subcommand as well,\n",
      "such as  this example instructing Python to use integer division from Python 3:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%script python -Qnew\n",
      "print 1/3"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0.333333333333\n"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can really specify *any* program for `%%script`,\n",
      "for instance here is a 'program' that echos the lines of stdin, with delays between each line."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%script --bg --out bashout bash -c \"while read line; do echo $line; sleep 1; done\"\n",
      "line 1\n",
      "line 2\n",
      "line 3\n",
      "line 4\n",
      "line 5\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Starting job # 2 in a separate thread.\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Remember, since the output of a background script is just the stdout pipe,\n",
      "you can read it as lines become available:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import time\n",
      "tic = time.time()\n",
      "line = True\n",
      "while True:\n",
      "    line = bashout.readline()\n",
      "    if not line:\n",
      "        break\n",
      "    sys.stdout.write(\"%.1fs: %s\" %(time.time()-tic, line))\n",
      "    sys.stdout.flush()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0.0s: line 1\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "1.0s: line 2\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "2.0s: line 3\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "3.0s: line 4\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "4.0s: line 5\n"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Configuring the default ScriptMagics"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The list of aliased script magics is configurable.\n",
      "\n",
      "The default is to pick from a few common interpreters, and use them if found, but you can specify your own in ipython_config.py:\n",
      "\n",
      "    c.ScriptMagics.scripts = ['R', 'pypy', 'myprogram']\n",
      "\n",
      "And if any of these programs do not apear on your default PATH, then you would also need to specify their location with:\n",
      "\n",
      "    c.ScriptMagics.script_paths = {'myprogram': '/opt/path/to/myprogram'}"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}