##// END OF EJS Templates
Add a notebook illustrating the backgroundjobs library.
Fernando Perez -
Show More
@@ -0,0 +1,171 b''
1 {
2 "worksheets": [
3 {
4 "cells": [
5 {
6 "source": "# Simple interactive bacgkround jobs with IPython\n\nWe start by loading the `backgroundjobs` library and defining a few trivial functions to illustrate things with.",
7 "cell_type": "markdown"
8 },
9 {
10 "cell_type": "code",
11 "language": "python",
12 "outputs": [],
13 "collapsed": false,
14 "prompt_number": 15,
15 "input": "from IPython.lib import backgroundjobs as bg\n\nimport sys\nimport time\n\ndef sleepfunc(interval=2, *a, **kw):\n args = dict(interval=interval,\n args=a,\n kwargs=kw)\n time.sleep(interval)\n return args\n\ndef diefunc(interval=2, *a, **kw):\n time.sleep(interval)\n raise Exception(\"Dead job with interval %s\" % interval)\n\ndef printfunc(interval=1, reps=5):\n for n in range(reps):\n time.sleep(interval)\n print 'In the background...', n\n sys.stdout.flush()\n print 'All done!'\n sys.stdout.flush()"
16 },
17 {
18 "source": "Now, we can create a job manager (called simply `jobs`) and use it to submit new jobs.\n<br>\nRun the cell below and wait a few seconds for the whole thing to finish, until you see the \"All done!\" printout.",
19 "cell_type": "markdown"
20 },
21 {
22 "cell_type": "code",
23 "language": "python",
24 "outputs": [
25 {
26 "output_type": "stream",
27 "stream": "stdout",
28 "text": "Starting job # 0 in a separate thread.\nStarting job # 2 in a separate thread.\nStarting job # 3 in a separate thread.\nStarting job # 4 in a separate thread.\nStarting job # 5 in a separate thread.\n"
29 },
30 {
31 "output_type": "stream",
32 "stream": "stdout",
33 "text": "In the background... 0\n"
34 },
35 {
36 "output_type": "stream",
37 "stream": "stdout",
38 "text": "In the background... 1\n"
39 },
40 {
41 "output_type": "stream",
42 "stream": "stdout",
43 "text": "In the background... 2\n"
44 },
45 {
46 "output_type": "stream",
47 "stream": "stdout",
48 "text": "All done!\n"
49 }
50 ],
51 "collapsed": false,
52 "prompt_number": 28,
53 "input": "jobs = bg.BackgroundJobManager()\n\n# Start a few jobs, the first one will have ID # 0\njobs.new(sleepfunc, 4)\njobs.new(sleepfunc, kw={'reps':2})\njobs.new('printfunc(1,3)')\n\n# This makes a couple of jobs which will die. Let's keep a reference to\n# them for easier traceback reporting later\ndiejob1 = jobs.new(diefunc, 1)\ndiejob2 = jobs.new(diefunc, 2)"
54 },
55 {
56 "source": "You can check the status of your jobs at any time:",
57 "cell_type": "markdown"
58 },
59 {
60 "cell_type": "code",
61 "language": "python",
62 "outputs": [
63 {
64 "output_type": "stream",
65 "stream": "stdout",
66 "text": "Completed jobs:\n0 : &lt;function sleepfunc at 0x30c8500&gt;\n2 : &lt;function sleepfunc at 0x30c8500&gt;\n3 : printfunc(1,3)\n\nDead jobs:\n4 : &lt;function diefunc at 0x30df758&gt;\n5 : &lt;function diefunc at 0x30df758&gt;\n\n"
67 }
68 ],
69 "collapsed": false,
70 "prompt_number": 29,
71 "input": "jobs.status()"
72 },
73 {
74 "source": "For any completed job, you can get its result easily:",
75 "cell_type": "markdown"
76 },
77 {
78 "cell_type": "code",
79 "language": "python",
80 "outputs": [
81 {
82 "output_type": "pyout",
83 "prompt_number": 31,
84 "text": "{&apos;args&apos;: (), &apos;interval&apos;: 4, &apos;kwargs&apos;: {}}"
85 }
86 ],
87 "collapsed": false,
88 "prompt_number": 31,
89 "input": "jobs[0].result"
90 },
91 {
92 "source": "You can get the traceback of any dead job. Run the line\nbelow again interactively until it prints a traceback (check the status\nof the job):\n",
93 "cell_type": "markdown"
94 },
95 {
96 "cell_type": "code",
97 "language": "python",
98 "outputs": [
99 {
100 "output_type": "stream",
101 "stream": "stdout",
102 "text": "Status of diejob1: Dead (Exception), call jobs.traceback() for details\n<span class=\"ansired\">---------------------------------------------------------------------------</span>\n<span class=\"ansired\">Exception</span> Traceback (most recent call last)\n<span class=\"ansigreen\">/home/fperez/usr/lib/python2.6/site-packages/IPython/lib/backgroundjobs.py</span> in <span class=\"ansicyan\">call</span><span class=\"ansiblue\">(self)</span>\n<span class=\"ansigreen\"> 462</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 463</span> <span class=\"ansigreen\">def</span> call<span class=\"ansiyellow\">(</span>self<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">--&gt; 464</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">return</span> self<span class=\"ansiyellow\">.</span>func<span class=\"ansiyellow\">(</span><span class=\"ansiyellow\">*</span>self<span class=\"ansiyellow\">.</span>args<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>self<span class=\"ansiyellow\">.</span>kwargs<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansigreen\">/home/fperez/ipython/ipython/docs/examples/lib/&lt;ipython-input-15-54795a097787&gt;</span> in <span class=\"ansicyan\">diefunc</span><span class=\"ansiblue\">(interval, *a, **kw)</span>\n<span class=\"ansigreen\"> 14</span> <span class=\"ansigreen\">def</span> diefunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">2</span><span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">*</span>a<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>kw<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 15</span> time<span class=\"ansiyellow\">.</span>sleep<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">---&gt; 16</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">raise</span> Exception<span class=\"ansiyellow\">(</span><span class=\"ansiblue\">&quot;Dead job with interval %s&quot;</span> <span class=\"ansiyellow\">%</span> interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 17</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 18</span> <span class=\"ansigreen\">def</span> printfunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">1</span><span class=\"ansiyellow\">,</span> reps<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">5</span><span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansired\">Exception</span>: Dead job with interval 1\n"
103 }
104 ],
105 "collapsed": false,
106 "prompt_number": 34,
107 "input": "print \"Status of diejob1:\", diejob1.status\ndiejob1.traceback() # jobs.traceback(4) would also work here, with the job number"
108 },
109 {
110 "source": "This will print all tracebacks for all dead jobs:",
111 "cell_type": "markdown"
112 },
113 {
114 "cell_type": "code",
115 "language": "python",
116 "outputs": [
117 {
118 "output_type": "stream",
119 "stream": "stdout",
120 "text": "Traceback for: &lt;BackgroundJob #4: &lt;function diefunc at 0x30df758&gt;&gt;\n<span class=\"ansired\">---------------------------------------------------------------------------</span>\n<span class=\"ansired\">Exception</span> Traceback (most recent call last)\n<span class=\"ansigreen\">/home/fperez/usr/lib/python2.6/site-packages/IPython/lib/backgroundjobs.py</span> in <span class=\"ansicyan\">call</span><span class=\"ansiblue\">(self)</span>\n<span class=\"ansigreen\"> 462</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 463</span> <span class=\"ansigreen\">def</span> call<span class=\"ansiyellow\">(</span>self<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">--&gt; 464</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">return</span> self<span class=\"ansiyellow\">.</span>func<span class=\"ansiyellow\">(</span><span class=\"ansiyellow\">*</span>self<span class=\"ansiyellow\">.</span>args<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>self<span class=\"ansiyellow\">.</span>kwargs<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansigreen\">/home/fperez/ipython/ipython/docs/examples/lib/&lt;ipython-input-15-54795a097787&gt;</span> in <span class=\"ansicyan\">diefunc</span><span class=\"ansiblue\">(interval, *a, **kw)</span>\n<span class=\"ansigreen\"> 14</span> <span class=\"ansigreen\">def</span> diefunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">2</span><span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">*</span>a<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>kw<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 15</span> time<span class=\"ansiyellow\">.</span>sleep<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">---&gt; 16</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">raise</span> Exception<span class=\"ansiyellow\">(</span><span class=\"ansiblue\">&quot;Dead job with interval %s&quot;</span> <span class=\"ansiyellow\">%</span> interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 17</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 18</span> <span class=\"ansigreen\">def</span> printfunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">1</span><span class=\"ansiyellow\">,</span> reps<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">5</span><span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansired\">Exception</span>: Dead job with interval 1\n\nTraceback for: &lt;BackgroundJob #5: &lt;function diefunc at 0x30df758&gt;&gt;\n<span class=\"ansired\">---------------------------------------------------------------------------</span>\n<span class=\"ansired\">Exception</span> Traceback (most recent call last)\n<span class=\"ansigreen\">/home/fperez/usr/lib/python2.6/site-packages/IPython/lib/backgroundjobs.py</span> in <span class=\"ansicyan\">call</span><span class=\"ansiblue\">(self)</span>\n<span class=\"ansigreen\"> 462</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 463</span> <span class=\"ansigreen\">def</span> call<span class=\"ansiyellow\">(</span>self<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">--&gt; 464</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">return</span> self<span class=\"ansiyellow\">.</span>func<span class=\"ansiyellow\">(</span><span class=\"ansiyellow\">*</span>self<span class=\"ansiyellow\">.</span>args<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>self<span class=\"ansiyellow\">.</span>kwargs<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansigreen\">/home/fperez/ipython/ipython/docs/examples/lib/&lt;ipython-input-15-54795a097787&gt;</span> in <span class=\"ansicyan\">diefunc</span><span class=\"ansiblue\">(interval, *a, **kw)</span>\n<span class=\"ansigreen\"> 14</span> <span class=\"ansigreen\">def</span> diefunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">2</span><span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">*</span>a<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>kw<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 15</span> time<span class=\"ansiyellow\">.</span>sleep<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">---&gt; 16</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">raise</span> Exception<span class=\"ansiyellow\">(</span><span class=\"ansiblue\">&quot;Dead job with interval %s&quot;</span> <span class=\"ansiyellow\">%</span> interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 17</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 18</span> <span class=\"ansigreen\">def</span> printfunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">1</span><span class=\"ansiyellow\">,</span> reps<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">5</span><span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansired\">Exception</span>: Dead job with interval 2\n\n"
121 }
122 ],
123 "collapsed": false,
124 "prompt_number": 33,
125 "input": "jobs.traceback()"
126 },
127 {
128 "source": "The job manager can be flushed of all completed jobs at any time:",
129 "cell_type": "markdown"
130 },
131 {
132 "cell_type": "code",
133 "language": "python",
134 "outputs": [
135 {
136 "output_type": "stream",
137 "stream": "stdout",
138 "text": "No jobs to flush.\n"
139 }
140 ],
141 "collapsed": false,
142 "prompt_number": 25,
143 "input": "jobs.flush()"
144 },
145 {
146 "source": "After that, the status is simply empty:",
147 "cell_type": "markdown"
148 },
149 {
150 "cell_type": "code",
151 "language": "python",
152 "outputs": [],
153 "collapsed": true,
154 "prompt_number": 27,
155 "input": "jobs.status()"
156 },
157 {
158 "input": "",
159 "cell_type": "code",
160 "collapsed": true,
161 "language": "python",
162 "outputs": []
163 }
164 ]
165 }
166 ],
167 "metadata": {
168 "name": "BackgroundJobs"
169 },
170 "nbformat": 2
171 } No newline at end of file
General Comments 0
You need to be logged in to leave comments. Login now