##// END OF EJS Templates
a line from the itermapresult.py file was missing
timo -
Show More
@@ -1,116 +1,116 b''
1 1 .. _parallel_asyncresult:
2 2
3 3 ======================
4 4 The AsyncResult object
5 5 ======================
6 6
7 7 In non-blocking mode, :meth:`apply` submits the command to be executed and
8 8 then returns a :class:`~.AsyncResult` object immediately. The
9 9 AsyncResult object gives you a way of getting a result at a later
10 10 time through its :meth:`get` method, but it also collects metadata
11 11 on execution.
12 12
13 13
14 14 Beyond multiprocessing's AsyncResult
15 15 ====================================
16 16
17 17 .. Note::
18 18
19 19 The :class:`~.AsyncResult` object provides a superset of the interface in
20 20 :py:class:`multiprocessing.pool.AsyncResult`. See the
21 21 `official Python documentation <http://docs.python.org/library/multiprocessing#multiprocessing.pool.AsyncResult>`_
22 22 for more on the basics of this interface.
23 23
24 24 Our AsyncResult objects add a number of convenient features for working with
25 25 parallel results, beyond what is provided by the original AsyncResult.
26 26
27 27
28 28 get_dict
29 29 --------
30 30
31 31 First, is :meth:`.AsyncResult.get_dict`, which pulls results as a dictionary
32 32 keyed by engine_id, rather than a flat list. This is useful for quickly
33 33 coordinating or distributing information about all of the engines.
34 34
35 35 As an example, here is a quick call that gives every engine a dict showing
36 36 the PID of every other engine:
37 37
38 38 .. sourcecode:: ipython
39 39
40 40 In [10]: ar = rc[:].apply_async(os.getpid)
41 41 In [11]: pids = ar.get_dict()
42 42 In [12]: rc[:]['pid_map'] = pids
43 43
44 44 This trick is particularly useful when setting up inter-engine communication,
45 45 as in IPython's :file:`examples/parallel/interengine` examples.
46 46
47 47
48 48 Metadata
49 49 ========
50 50
51 51 IPython.parallel tracks some metadata about the tasks, which is stored
52 52 in the :attr:`.Client.metadata` dict. The AsyncResult object gives you an
53 53 interface for this information as well, including timestamps stdout/err,
54 54 and engine IDs.
55 55
56 56
57 57 Timing
58 58 ------
59 59
60 60 IPython tracks various timestamps as :py:class:`.datetime` objects,
61 61 and the AsyncResult object has a few properties that turn these into useful
62 62 times (in seconds as floats).
63 63
64 64 For use while the tasks are still pending:
65 65
66 66 * :attr:`ar.elapsed` is just the elapsed seconds since submission, for use
67 67 before the AsyncResult is complete.
68 68 * :attr:`ar.progress` is the number of tasks that have completed. Fractional progress
69 69 would be::
70 70
71 71 1.0 * ar.progress / len(ar)
72 72
73 73 * :meth:`AsyncResult.wait_interactive` will wait for the result to finish, but
74 74 print out status updates on progress and elapsed time while it waits.
75 75
76 76 For use after the tasks are done:
77 77
78 78 * :attr:`ar.serial_time` is the sum of the computation time of all of the tasks
79 79 done in parallel.
80 80 * :attr:`ar.wall_time` is the time between the first task submitted and last result
81 81 received. This is the actual cost of computation, including IPython overhead.
82 82
83 83
84 84 .. note::
85 85
86 86 wall_time is only precise if the Client is waiting for results when
87 87 the task finished, because the `received` timestamp is made when the result is
88 88 unpacked by the Client, triggered by the :meth:`~Client.spin` call. If you
89 89 are doing work in the Client, and not waiting/spinning, then `received` might
90 90 be artificially high.
91 91
92 92 An often interesting metric is the time it actually cost to do the work in parallel
93 93 relative to the serial computation, and this can be given simply with
94 94
95 95 .. sourcecode:: python
96 96
97 97 speedup = ar.serial_time / ar.wall_time
98 98
99 99
100 100 Map results are iterable!
101 101 =========================
102 102
103 103 When an AsyncResult object has multiple results (e.g. the :class:`~AsyncMapResult`
104 104 object), you can actually iterate through them, and act on the results as they arrive:
105 105
106 106 .. literalinclude:: ../../examples/parallel/itermapresult.py
107 107 :language: python
108 :lines: 20-66
108 :lines: 20-67
109 109
110 110 .. seealso::
111 111
112 112 When AsyncResult or the AsyncMapResult don't provide what you need (for instance,
113 113 handling individual results as they arrive, but with metadata), you can always
114 114 just split the original result's ``msg_ids`` attribute, and handle them as you like.
115 115
116 116 For an example of this, see :file:`docs/examples/parallel/customresult.py`
General Comments 0
You need to be logged in to leave comments. Login now