Show More
@@ -0,0 +1,171 b'' | |||||
|
1 | { | |||
|
2 | "worksheets": [ | |||
|
3 | { | |||
|
4 | "cells": [ | |||
|
5 | { | |||
|
6 | "source": "# Simple interactive bacgkround jobs with IPython\n\nWe start by loading the `backgroundjobs` library and defining a few trivial functions to illustrate things with.", | |||
|
7 | "cell_type": "markdown" | |||
|
8 | }, | |||
|
9 | { | |||
|
10 | "cell_type": "code", | |||
|
11 | "language": "python", | |||
|
12 | "outputs": [], | |||
|
13 | "collapsed": false, | |||
|
14 | "prompt_number": 15, | |||
|
15 | "input": "from IPython.lib import backgroundjobs as bg\n\nimport sys\nimport time\n\ndef sleepfunc(interval=2, *a, **kw):\n args = dict(interval=interval,\n args=a,\n kwargs=kw)\n time.sleep(interval)\n return args\n\ndef diefunc(interval=2, *a, **kw):\n time.sleep(interval)\n raise Exception(\"Dead job with interval %s\" % interval)\n\ndef printfunc(interval=1, reps=5):\n for n in range(reps):\n time.sleep(interval)\n print 'In the background...', n\n sys.stdout.flush()\n print 'All done!'\n sys.stdout.flush()" | |||
|
16 | }, | |||
|
17 | { | |||
|
18 | "source": "Now, we can create a job manager (called simply `jobs`) and use it to submit new jobs.\n<br>\nRun the cell below and wait a few seconds for the whole thing to finish, until you see the \"All done!\" printout.", | |||
|
19 | "cell_type": "markdown" | |||
|
20 | }, | |||
|
21 | { | |||
|
22 | "cell_type": "code", | |||
|
23 | "language": "python", | |||
|
24 | "outputs": [ | |||
|
25 | { | |||
|
26 | "output_type": "stream", | |||
|
27 | "stream": "stdout", | |||
|
28 | "text": "Starting job # 0 in a separate thread.\nStarting job # 2 in a separate thread.\nStarting job # 3 in a separate thread.\nStarting job # 4 in a separate thread.\nStarting job # 5 in a separate thread.\n" | |||
|
29 | }, | |||
|
30 | { | |||
|
31 | "output_type": "stream", | |||
|
32 | "stream": "stdout", | |||
|
33 | "text": "In the background... 0\n" | |||
|
34 | }, | |||
|
35 | { | |||
|
36 | "output_type": "stream", | |||
|
37 | "stream": "stdout", | |||
|
38 | "text": "In the background... 1\n" | |||
|
39 | }, | |||
|
40 | { | |||
|
41 | "output_type": "stream", | |||
|
42 | "stream": "stdout", | |||
|
43 | "text": "In the background... 2\n" | |||
|
44 | }, | |||
|
45 | { | |||
|
46 | "output_type": "stream", | |||
|
47 | "stream": "stdout", | |||
|
48 | "text": "All done!\n" | |||
|
49 | } | |||
|
50 | ], | |||
|
51 | "collapsed": false, | |||
|
52 | "prompt_number": 28, | |||
|
53 | "input": "jobs = bg.BackgroundJobManager()\n\n# Start a few jobs, the first one will have ID # 0\njobs.new(sleepfunc, 4)\njobs.new(sleepfunc, kw={'reps':2})\njobs.new('printfunc(1,3)')\n\n# This makes a couple of jobs which will die. Let's keep a reference to\n# them for easier traceback reporting later\ndiejob1 = jobs.new(diefunc, 1)\ndiejob2 = jobs.new(diefunc, 2)" | |||
|
54 | }, | |||
|
55 | { | |||
|
56 | "source": "You can check the status of your jobs at any time:", | |||
|
57 | "cell_type": "markdown" | |||
|
58 | }, | |||
|
59 | { | |||
|
60 | "cell_type": "code", | |||
|
61 | "language": "python", | |||
|
62 | "outputs": [ | |||
|
63 | { | |||
|
64 | "output_type": "stream", | |||
|
65 | "stream": "stdout", | |||
|
66 | "text": "Completed jobs:\n0 : <function sleepfunc at 0x30c8500>\n2 : <function sleepfunc at 0x30c8500>\n3 : printfunc(1,3)\n\nDead jobs:\n4 : <function diefunc at 0x30df758>\n5 : <function diefunc at 0x30df758>\n\n" | |||
|
67 | } | |||
|
68 | ], | |||
|
69 | "collapsed": false, | |||
|
70 | "prompt_number": 29, | |||
|
71 | "input": "jobs.status()" | |||
|
72 | }, | |||
|
73 | { | |||
|
74 | "source": "For any completed job, you can get its result easily:", | |||
|
75 | "cell_type": "markdown" | |||
|
76 | }, | |||
|
77 | { | |||
|
78 | "cell_type": "code", | |||
|
79 | "language": "python", | |||
|
80 | "outputs": [ | |||
|
81 | { | |||
|
82 | "output_type": "pyout", | |||
|
83 | "prompt_number": 31, | |||
|
84 | "text": "{'args': (), 'interval': 4, 'kwargs': {}}" | |||
|
85 | } | |||
|
86 | ], | |||
|
87 | "collapsed": false, | |||
|
88 | "prompt_number": 31, | |||
|
89 | "input": "jobs[0].result" | |||
|
90 | }, | |||
|
91 | { | |||
|
92 | "source": "You can get the traceback of any dead job. Run the line\nbelow again interactively until it prints a traceback (check the status\nof the job):\n", | |||
|
93 | "cell_type": "markdown" | |||
|
94 | }, | |||
|
95 | { | |||
|
96 | "cell_type": "code", | |||
|
97 | "language": "python", | |||
|
98 | "outputs": [ | |||
|
99 | { | |||
|
100 | "output_type": "stream", | |||
|
101 | "stream": "stdout", | |||
|
102 | "text": "Status of diejob1: Dead (Exception), call jobs.traceback() for details\n<span class=\"ansired\">---------------------------------------------------------------------------</span>\n<span class=\"ansired\">Exception</span> Traceback (most recent call last)\n<span class=\"ansigreen\">/home/fperez/usr/lib/python2.6/site-packages/IPython/lib/backgroundjobs.py</span> in <span class=\"ansicyan\">call</span><span class=\"ansiblue\">(self)</span>\n<span class=\"ansigreen\"> 462</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 463</span> <span class=\"ansigreen\">def</span> call<span class=\"ansiyellow\">(</span>self<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">--> 464</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">return</span> self<span class=\"ansiyellow\">.</span>func<span class=\"ansiyellow\">(</span><span class=\"ansiyellow\">*</span>self<span class=\"ansiyellow\">.</span>args<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>self<span class=\"ansiyellow\">.</span>kwargs<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansigreen\">/home/fperez/ipython/ipython/docs/examples/lib/<ipython-input-15-54795a097787></span> in <span class=\"ansicyan\">diefunc</span><span class=\"ansiblue\">(interval, *a, **kw)</span>\n<span class=\"ansigreen\"> 14</span> <span class=\"ansigreen\">def</span> diefunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">2</span><span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">*</span>a<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>kw<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 15</span> time<span class=\"ansiyellow\">.</span>sleep<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">---> 16</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">raise</span> Exception<span class=\"ansiyellow\">(</span><span class=\"ansiblue\">"Dead job with interval %s"</span> <span class=\"ansiyellow\">%</span> interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 17</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 18</span> <span class=\"ansigreen\">def</span> printfunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">1</span><span class=\"ansiyellow\">,</span> reps<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">5</span><span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansired\">Exception</span>: Dead job with interval 1\n" | |||
|
103 | } | |||
|
104 | ], | |||
|
105 | "collapsed": false, | |||
|
106 | "prompt_number": 34, | |||
|
107 | "input": "print \"Status of diejob1:\", diejob1.status\ndiejob1.traceback() # jobs.traceback(4) would also work here, with the job number" | |||
|
108 | }, | |||
|
109 | { | |||
|
110 | "source": "This will print all tracebacks for all dead jobs:", | |||
|
111 | "cell_type": "markdown" | |||
|
112 | }, | |||
|
113 | { | |||
|
114 | "cell_type": "code", | |||
|
115 | "language": "python", | |||
|
116 | "outputs": [ | |||
|
117 | { | |||
|
118 | "output_type": "stream", | |||
|
119 | "stream": "stdout", | |||
|
120 | "text": "Traceback for: <BackgroundJob #4: <function diefunc at 0x30df758>>\n<span class=\"ansired\">---------------------------------------------------------------------------</span>\n<span class=\"ansired\">Exception</span> Traceback (most recent call last)\n<span class=\"ansigreen\">/home/fperez/usr/lib/python2.6/site-packages/IPython/lib/backgroundjobs.py</span> in <span class=\"ansicyan\">call</span><span class=\"ansiblue\">(self)</span>\n<span class=\"ansigreen\"> 462</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 463</span> <span class=\"ansigreen\">def</span> call<span class=\"ansiyellow\">(</span>self<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">--> 464</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">return</span> self<span class=\"ansiyellow\">.</span>func<span class=\"ansiyellow\">(</span><span class=\"ansiyellow\">*</span>self<span class=\"ansiyellow\">.</span>args<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>self<span class=\"ansiyellow\">.</span>kwargs<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansigreen\">/home/fperez/ipython/ipython/docs/examples/lib/<ipython-input-15-54795a097787></span> in <span class=\"ansicyan\">diefunc</span><span class=\"ansiblue\">(interval, *a, **kw)</span>\n<span class=\"ansigreen\"> 14</span> <span class=\"ansigreen\">def</span> diefunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">2</span><span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">*</span>a<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>kw<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 15</span> time<span class=\"ansiyellow\">.</span>sleep<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">---> 16</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">raise</span> Exception<span class=\"ansiyellow\">(</span><span class=\"ansiblue\">"Dead job with interval %s"</span> <span class=\"ansiyellow\">%</span> interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 17</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 18</span> <span class=\"ansigreen\">def</span> printfunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">1</span><span class=\"ansiyellow\">,</span> reps<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">5</span><span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansired\">Exception</span>: Dead job with interval 1\n\nTraceback for: <BackgroundJob #5: <function diefunc at 0x30df758>>\n<span class=\"ansired\">---------------------------------------------------------------------------</span>\n<span class=\"ansired\">Exception</span> Traceback (most recent call last)\n<span class=\"ansigreen\">/home/fperez/usr/lib/python2.6/site-packages/IPython/lib/backgroundjobs.py</span> in <span class=\"ansicyan\">call</span><span class=\"ansiblue\">(self)</span>\n<span class=\"ansigreen\"> 462</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 463</span> <span class=\"ansigreen\">def</span> call<span class=\"ansiyellow\">(</span>self<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">--> 464</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">return</span> self<span class=\"ansiyellow\">.</span>func<span class=\"ansiyellow\">(</span><span class=\"ansiyellow\">*</span>self<span class=\"ansiyellow\">.</span>args<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>self<span class=\"ansiyellow\">.</span>kwargs<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansigreen\">/home/fperez/ipython/ipython/docs/examples/lib/<ipython-input-15-54795a097787></span> in <span class=\"ansicyan\">diefunc</span><span class=\"ansiblue\">(interval, *a, **kw)</span>\n<span class=\"ansigreen\"> 14</span> <span class=\"ansigreen\">def</span> diefunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">2</span><span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">*</span>a<span class=\"ansiyellow\">,</span> <span class=\"ansiyellow\">**</span>kw<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 15</span> time<span class=\"ansiyellow\">.</span>sleep<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\">---> 16</span><span class=\"ansiyellow\"> </span><span class=\"ansigreen\">raise</span> Exception<span class=\"ansiyellow\">(</span><span class=\"ansiblue\">"Dead job with interval %s"</span> <span class=\"ansiyellow\">%</span> interval<span class=\"ansiyellow\">)</span><span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 17</span> <span class=\"ansiyellow\"></span>\n<span class=\"ansigreen\"> 18</span> <span class=\"ansigreen\">def</span> printfunc<span class=\"ansiyellow\">(</span>interval<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">1</span><span class=\"ansiyellow\">,</span> reps<span class=\"ansiyellow\">=</span><span class=\"ansicyan\">5</span><span class=\"ansiyellow\">)</span><span class=\"ansiyellow\">:</span><span class=\"ansiyellow\"></span>\n\n<span class=\"ansired\">Exception</span>: Dead job with interval 2\n\n" | |||
|
121 | } | |||
|
122 | ], | |||
|
123 | "collapsed": false, | |||
|
124 | "prompt_number": 33, | |||
|
125 | "input": "jobs.traceback()" | |||
|
126 | }, | |||
|
127 | { | |||
|
128 | "source": "The job manager can be flushed of all completed jobs at any time:", | |||
|
129 | "cell_type": "markdown" | |||
|
130 | }, | |||
|
131 | { | |||
|
132 | "cell_type": "code", | |||
|
133 | "language": "python", | |||
|
134 | "outputs": [ | |||
|
135 | { | |||
|
136 | "output_type": "stream", | |||
|
137 | "stream": "stdout", | |||
|
138 | "text": "No jobs to flush.\n" | |||
|
139 | } | |||
|
140 | ], | |||
|
141 | "collapsed": false, | |||
|
142 | "prompt_number": 25, | |||
|
143 | "input": "jobs.flush()" | |||
|
144 | }, | |||
|
145 | { | |||
|
146 | "source": "After that, the status is simply empty:", | |||
|
147 | "cell_type": "markdown" | |||
|
148 | }, | |||
|
149 | { | |||
|
150 | "cell_type": "code", | |||
|
151 | "language": "python", | |||
|
152 | "outputs": [], | |||
|
153 | "collapsed": true, | |||
|
154 | "prompt_number": 27, | |||
|
155 | "input": "jobs.status()" | |||
|
156 | }, | |||
|
157 | { | |||
|
158 | "input": "", | |||
|
159 | "cell_type": "code", | |||
|
160 | "collapsed": true, | |||
|
161 | "language": "python", | |||
|
162 | "outputs": [] | |||
|
163 | } | |||
|
164 | ] | |||
|
165 | } | |||
|
166 | ], | |||
|
167 | "metadata": { | |||
|
168 | "name": "BackgroundJobs" | |||
|
169 | }, | |||
|
170 | "nbformat": 2 | |||
|
171 | } No newline at end of file |
General Comments 0
You need to be logged in to leave comments.
Login now