docs/examples/notebooks/Script Magics.ipynb

{
 "metadata": {
  "name": "Script Magics"
 },
 "nbformat": 3,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "source": [
      "Running Scripts from IPython"
     ]
    },
    {
     "cell_type": "markdown",
     "source": [
      "IPython has a `%%script` cell magic, which lets you run a cell in",
      "a subprocess of any interpreter on your system, such as: bash, ruby, perl, zsh, R, etc.",
      "",
      "It can even be a script of your own, which expects input on stdin."
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "import sys"
     ],
     "language": "python",
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "source": [
      "To use it, simply pass a path or shell command to the program you want to run on the `%%script` line,",
      "and the rest of the cell will be run by that script, and stdout/err from the subprocess are captured and displayed."
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "%%script python",
      "import sys",
      "print 'hello from Python %s' % sys.version"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello from Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) ",
        "[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]",
        ""
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "input": [
      "%%script python3",
      "import sys",
      "print('hello from Python: %s' % sys.version)"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello from Python: 3.2.3 (v3.2.3:3d0686d90f55, Apr 10 2012, 11:25:50) ",
        "[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]",
        ""
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "source": [
      "IPython also creates aliases for a few common interpreters, such as bash, ruby, perl, etc.",
      "",
      "These are all equivalent to `%%script <name>`"
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "%%ruby",
      "puts \"Hello from Ruby #{RUBY_VERSION}\""
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Hello from Ruby 1.8.7",
        ""
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "input": [
      "%%bash",
      "echo \"hello from $BASH\""
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello from /usr/local/bin/bash",
        ""
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "heading",
     "level": 2,
     "source": [
      "Capturing output"
     ]
    },
    {
     "cell_type": "markdown",
     "source": [
      "You can also capture stdout/err from these subprocesses into Python variables, instead of letting them go directly to stdout/err"
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "%%bash",
      "echo \"hi, stdout\"",
      "echo \"hello, stderr\" >&2",
      ""
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hi, stdout",
        ""
       ]
      },
      {
       "output_type": "stream",
       "stream": "stderr",
       "text": [
        "hello, stderr",
        ""
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "code",
     "input": [
      "%%bash --out output --err error",
      "echo \"hi, stdout\"",
      "echo \"hello, stderr\" >&2"
     ],
     "language": "python",
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "input": [
      "print error",
      "print output"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "hello, stderr",
        "",
        "hi, stdout",
        "",
        ""
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "heading",
     "level": 2,
     "source": [
      "Background Scripts"
     ]
    },
    {
     "cell_type": "markdown",
     "source": [
      "These scripts can be run in the background, by adding the `--bg` flag.",
      "",
      "When you do this, output is discarded unless you use the `--out/err`",
      "flags to store output as above."
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "%%ruby --bg --out ruby_lines",
      "for n in 1...10",
      "    sleep 1",
      "    puts \"line #{n}\"",
      "    STDOUT.flush",
      "end"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Starting job # 0 in a separate thread.",
        ""
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "source": [
      "When you do store output of a background thread, these are the stdout/err *pipes*,",
      "rather than the text of the output."
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "ruby_lines"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "<open file '<fdopen>', mode 'rb' at 0x10dc651e0>"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "input": [
      "print ruby_lines.read()"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "line 1",
        "line 2",
        "line 3",
        "line 4",
        "line 5",
        "line 6",
        "line 7",
        "line 8",
        "line 9",
        "",
        ""
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "heading",
     "level": 2,
     "source": [
      "Arguments to subcommand"
     ]
    },
    {
     "cell_type": "markdown",
     "source": [
      "You can pass arguments the subcommand as well,",
      "such as  this example instructing Python to use integer division from Python 3:"
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "%%script python -Qnew",
      "print 1/3"
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0.333333333333",
        ""
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "source": [
      "You can really specify *any* program for `%%script`,",
      "for instance here is a 'program' that echos the lines of stdin, with delays between each line."
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "%%script --bg --out bashout bash -c \"while read line; do echo $line; sleep 1; done\"",
      "line 1",
      "line 2",
      "line 3",
      "line 4",
      "line 5",
      ""
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Starting job # 2 in a separate thread.",
        ""
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "source": [
      "Remember, since the output of a background script is just the stdout pipe,",
      "you can read it as lines become available:"
     ]
    },
    {
     "cell_type": "code",
     "input": [
      "import time",
      "tic = time.time()",
      "line = True",
      "while True:",
      "    line = bashout.readline()",
      "    if not line:",
      "        break",
      "    sys.stdout.write(\"%.1fs: %s\" %(time.time()-tic, line))",
      "    sys.stdout.flush()",
      ""
     ],
     "language": "python",
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0.0s: line 1",
        ""
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "1.0s: line 2",
        ""
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "2.0s: line 3",
        ""
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "3.0s: line 4",
        ""
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "4.0s: line 5",
        ""
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "heading",
     "level": 2,
     "source": [
      "Configuring the default ScriptMagics"
     ]
    },
    {
     "cell_type": "markdown",
     "source": [
      "The list of aliased script magics is configurable.",
      "",
      "The default is to pick from a few common interpreters, and use them if found, but you can specify your own in ipython_config.py:",
      "",
      "    c.ScriptMagics.scripts = ['R', 'pypy', 'myprogram']",
      "",
      "And if any of these programs do not apear on your default PATH, then you would also need to specify their location with:",
      "",
      "    c.ScriptMagics.script_paths = {'myprogram': '/opt/path/to/myprogram'}"
     ]
    }
   ]
  }
 ]
}