{
"metadata": {
"name": "Script Magics"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Running Scripts from IPython"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"IPython has a `%%script` cell magic, which lets you run a cell in\n",
"a subprocess of any interpreter on your system, such as: bash, ruby, perl, zsh, R, etc.\n",
"\n",
"It can even be a script of your own, which expects input on stdin."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import sys"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 1
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Basic usage"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To use it, simply pass a path or shell command to the program you want to run on the `%%script` line,\n",
"and the rest of the cell will be run by that script, and stdout/err from the subprocess are captured and displayed."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%script python\n",
"import sys\n",
"print 'hello from Python %s' % sys.version"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"hello from Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) \n",
"[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]\n"
]
}
],
"prompt_number": 2
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%script python3\n",
"import sys\n",
"print('hello from Python: %s' % sys.version)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"hello from Python: 3.2.3 (v3.2.3:3d0686d90f55, Apr 10 2012, 11:25:50) \n",
"[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]\n"
]
}
],
"prompt_number": 3
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"IPython also creates aliases for a few common interpreters, such as bash, ruby, perl, etc.\n",
"\n",
"These are all equivalent to `%%script <name>`"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%ruby\n",
"puts \"Hello from Ruby #{RUBY_VERSION}\""
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Hello from Ruby 1.8.7\n"
]
}
],
"prompt_number": 4
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"echo \"hello from $BASH\""
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"hello from /usr/local/bin/bash\n"
]
}
],
"prompt_number": 5
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Capturing output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also capture stdout/err from these subprocesses into Python variables, instead of letting them go directly to stdout/err"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash\n",
"echo \"hi, stdout\"\n",
"echo \"hello, stderr\" >&2\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"hi, stdout\n"
]
},
{
"output_type": "stream",
"stream": "stderr",
"text": [
"hello, stderr\n"
]
}
],
"prompt_number": 6
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%bash --out output --err error\n",
"echo \"hi, stdout\"\n",
"echo \"hello, stderr\" >&2"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 7
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(error)\n",
"print(output)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"hello, stderr\n",
"\n",
"hi, stdout\n",
"\n"
]
}
],
"prompt_number": 8
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Background Scripts"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These scripts can be run in the background, by adding the `--bg` flag.\n",
"\n",
"When you do this, output is discarded unless you use the `--out/err`\n",
"flags to store output as above."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%ruby --bg --out ruby_lines\n",
"for n in 1...10\n",
" sleep 1\n",
" puts \"line #{n}\"\n",
" STDOUT.flush\n",
"end"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Starting job # 0 in a separate thread.\n"
]
}
],
"prompt_number": 9
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When you do store output of a background thread, these are the stdout/err *pipes*,\n",
"rather than the text of the output."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"ruby_lines"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 10,
"text": [
"<open file '<fdopen>', mode 'rb' at 0x10a4be660>"
]
}
],
"prompt_number": 10
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"print(ruby_lines.read())"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"line 1\n",
"line 2\n",
"line 3\n",
"line 4\n",
"line 5\n",
"line 6\n",
"line 7\n",
"line 8\n",
"line 9\n",
"\n"
]
}
],
"prompt_number": 11
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Arguments to subcommand"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can pass arguments the subcommand as well,\n",
"such as this example instructing Python to use integer division from Python 3:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%script python -Qnew\n",
"print 1/3"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"0.333333333333\n"
]
}
],
"prompt_number": 12
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can really specify *any* program for `%%script`,\n",
"for instance here is a 'program' that echos the lines of stdin, with delays between each line."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%script --bg --out bashout bash -c \"while read line; do echo $line; sleep 1; done\"\n",
"line 1\n",
"line 2\n",
"line 3\n",
"line 4\n",
"line 5\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Starting job # 2 in a separate thread.\n"
]
}
],
"prompt_number": 13
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember, since the output of a background script is just the stdout pipe,\n",
"you can read it as lines become available:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import time\n",
"tic = time.time()\n",
"line = True\n",
"while True:\n",
" line = bashout.readline()\n",
" if not line:\n",
" break\n",
" sys.stdout.write(\"%.1fs: %s\" %(time.time()-tic, line))\n",
" sys.stdout.flush()\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"0.0s: line 1\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"1.0s: line 2\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"2.0s: line 3\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"3.0s: line 4\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"4.0s: line 5\n"
]
}
],
"prompt_number": 14
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Configuring the default ScriptMagics"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The list of aliased script magics is configurable.\n",
"\n",
"The default is to pick from a few common interpreters, and use them if found, but you can specify your own in ipython_config.py:\n",
"\n",
" c.ScriptMagics.scripts = ['R', 'pypy', 'myprogram']\n",
"\n",
"And if any of these programs do not apear on your default PATH, then you would also need to specify their location with:\n",
"\n",
" c.ScriptMagics.script_paths = {'myprogram': '/opt/path/to/myprogram'}"
]
}
],
"metadata": {}
}
]
}