##// END OF EJS Templates
msg spec 5.0
MinRK -
Show More
@@ -0,0 +1,70 b''
1 .. _execution_semantics:
2
3 Execution semantics in the IPython kernel
4 =========================================
5
6 The execution of use code consists of the following phases:
7
8 1. Fire the ``pre_execute`` event.
9 2. Fire the ``pre_run_cell`` event unless silent is True.
10 3. Execute the ``code`` field, see below for details.
11 4. If execution succeeds, expressions in ``user_expressions`` are computed.
12 This ensures that any error in the expressions don't affect the main code execution.
13 5. Fire the post_execute eventCall any method registered with :meth:`register_post_execute`.
14
15 .. warning::
16
17 The API for running code before/after the main code block is likely to
18 change soon. Both the ``pre_runcode_hook`` and the
19 :meth:`register_post_execute` are susceptible to modification, as we find a
20 consistent model for both.
21
22 To understand how the ``code`` field is executed, one must know that Python
23 code can be compiled in one of three modes (controlled by the ``mode`` argument
24 to the :func:`compile` builtin):
25
26 *single*
27 Valid for a single interactive statement (though the source can contain
28 multiple lines, such as a for loop). When compiled in this mode, the
29 generated bytecode contains special instructions that trigger the calling of
30 :func:`sys.displayhook` for any expression in the block that returns a value.
31 This means that a single statement can actually produce multiple calls to
32 :func:`sys.displayhook`, if for example it contains a loop where each
33 iteration computes an unassigned expression would generate 10 calls::
34
35 for i in range(10):
36 i**2
37
38 *exec*
39 An arbitrary amount of source code, this is how modules are compiled.
40 :func:`sys.displayhook` is *never* implicitly called.
41
42 *eval*
43 A single expression that returns a value. :func:`sys.displayhook` is *never*
44 implicitly called.
45
46
47 The ``code`` field is split into individual blocks each of which is valid for
48 execution in 'single' mode, and then:
49
50 - If there is only a single block: it is executed in 'single' mode.
51
52 - If there is more than one block:
53
54 * if the last one is a single line long, run all but the last in 'exec' mode
55 and the very last one in 'single' mode. This makes it easy to type simple
56 expressions at the end to see computed values.
57
58 * if the last one is no more than two lines long, run all but the last in
59 'exec' mode and the very last one in 'single' mode. This makes it easy to
60 type simple expressions at the end to see computed values. - otherwise
61 (last one is also multiline), run all in 'exec' mode
62
63 * otherwise (last one is also multiline), run all in 'exec' mode as a single
64 unit.
65
66
67 Errors in any registered post_execute functions are reported,
68 and the failing function is removed from the post_execution set so that it does
69 not continue triggering failures.
70
@@ -1,28 +1,29 b''
1 1 .. _developer_guide:
2 2
3 3 =========================
4 4 IPython developer's guide
5 5 =========================
6 6
7 7 This are two categories of developer focused documentation:
8 8
9 9 1. Documentation for developers of *IPython itself*.
10 10 2. Documentation for developers of third party tools and libraries
11 11 that use IPython.
12 12
13 13 This part of our documentation only contains information in the second category.
14 14
15 15 Developers interested in working on IPython itself should consult
16 16 our `developer information <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
17 17 on the IPython GitHub wiki.
18 18
19 19 .. toctree::
20 20 :maxdepth: 1
21 21
22 22 messaging
23 execution
23 24 parallel_messages
24 25 parallel_connections
25 26 lexer
26 27 pycompat
27 28 config
28 29 inputhook_app
This diff has been collapsed as it changes many lines, (522 lines changed) Show them Hide them
@@ -1,1178 +1,1052 b''
1 1 .. _messaging:
2 2
3 3 ======================
4 4 Messaging in IPython
5 5 ======================
6 6
7 7
8 8 Versioning
9 9 ==========
10 10
11 11 The IPython message specification is versioned independently of IPython.
12 The current version of the specification is 4.1.
12 The current version of the specification is 5.0.0.
13 13
14 14
15 15 Introduction
16 16 ============
17 17
18 18 This document explains the basic communications design and messaging
19 19 specification for how the various IPython objects interact over a network
20 20 transport. The current implementation uses the ZeroMQ_ library for messaging
21 21 within and between hosts.
22 22
23 23 .. Note::
24 24
25 25 This document should be considered the authoritative description of the
26 26 IPython messaging protocol, and all developers are strongly encouraged to
27 27 keep it updated as the implementation evolves, so that we have a single
28 28 common reference for all protocol details.
29 29
30 30 The basic design is explained in the following diagram:
31 31
32 32 .. image:: figs/frontend-kernel.png
33 33 :width: 450px
34 34 :alt: IPython kernel/frontend messaging architecture.
35 35 :align: center
36 36 :target: ../_images/frontend-kernel.png
37 37
38 38 A single kernel can be simultaneously connected to one or more frontends. The
39 39 kernel has three sockets that serve the following functions:
40 40
41 1. stdin: this ROUTER socket is connected to all frontends, and it allows
42 the kernel to request input from the active frontend when :func:`raw_input` is called.
43 The frontend that executed the code has a DEALER socket that acts as a 'virtual keyboard'
44 for the kernel while this communication is happening (illustrated in the
45 figure by the black outline around the central keyboard). In practice,
46 frontends may display such kernel requests using a special input widget or
47 otherwise indicating that the user is to type input for the kernel instead
48 of normal commands in the frontend.
49
50 2. Shell: this single ROUTER socket allows multiple incoming connections from
41 1. Shell: this single ROUTER socket allows multiple incoming connections from
51 42 frontends, and this is the socket where requests for code execution, object
52 43 information, prompts, etc. are made to the kernel by any frontend. The
53 44 communication on this socket is a sequence of request/reply actions from
54 45 each frontend and the kernel.
55 46
56 3. IOPub: this socket is the 'broadcast channel' where the kernel publishes all
47 2. IOPub: this socket is the 'broadcast channel' where the kernel publishes all
57 48 side effects (stdout, stderr, etc.) as well as the requests coming from any
58 49 client over the shell socket and its own requests on the stdin socket. There
59 50 are a number of actions in Python which generate side effects: :func:`print`
60 51 writes to ``sys.stdout``, errors generate tracebacks, etc. Additionally, in
61 52 a multi-client scenario, we want all frontends to be able to know what each
62 53 other has sent to the kernel (this can be useful in collaborative scenarios,
63 54 for example). This socket allows both side effects and the information
64 55 about communications taking place with one client over the shell channel
65 56 to be made available to all clients in a uniform manner.
66 57
58 3. stdin: this ROUTER socket is connected to all frontends, and it allows
59 the kernel to request input from the active frontend when :func:`raw_input` is called.
60 The frontend that executed the code has a DEALER socket that acts as a 'virtual keyboard'
61 for the kernel while this communication is happening (illustrated in the
62 figure by the black outline around the central keyboard). In practice,
63 frontends may display such kernel requests using a special input widget or
64 otherwise indicating that the user is to type input for the kernel instead
65 of normal commands in the frontend.
66
67 67 All messages are tagged with enough information (details below) for clients
68 68 to know which messages come from their own interaction with the kernel and
69 69 which ones are from other clients, so they can display each type
70 70 appropriately.
71 71
72 4. Control: This channel is identical to Shell, but operates on a separate socket,
73 to allow important messages to avoid queueing behind execution requests (e.g. shutdown or abort).
74
72 75 The actual format of the messages allowed on each of these channels is
73 76 specified below. Messages are dicts of dicts with string keys and values that
74 77 are reasonably representable in JSON. Our current implementation uses JSON
75 78 explicitly as its message format, but this shouldn't be considered a permanent
76 79 feature. As we've discovered that JSON has non-trivial performance issues due
77 80 to excessive copying, we may in the future move to a pure pickle-based raw
78 81 message format. However, it should be possible to easily convert from the raw
79 82 objects to JSON, since we may have non-python clients (e.g. a web frontend).
80 83 As long as it's easy to make a JSON version of the objects that is a faithful
81 84 representation of all the data, we can communicate with such clients.
82 85
83 86 .. Note::
84 87
85 88 Not all of these have yet been fully fleshed out, but the key ones are, see
86 89 kernel and frontend files for actual implementation details.
87 90
88 91 General Message Format
89 92 ======================
90 93
91 94 A message is defined by the following four-dictionary structure::
92 95
93 96 {
94 97 # The message header contains a pair of unique identifiers for the
95 98 # originating session and the actual message id, in addition to the
96 99 # username for the process that generated the message. This is useful in
97 100 # collaborative settings where multiple users may be interacting with the
98 101 # same kernel simultaneously, so that frontends can label the various
99 102 # messages in a meaningful way.
100 103 'header' : {
101 104 'msg_id' : uuid,
102 105 'username' : str,
103 106 'session' : uuid,
104 107 # All recognized message type strings are listed below.
105 108 'msg_type' : str,
106 109 # the message protocol version
107 110 'version' : '5.0.0',
108 111 },
109 112
110 113 # In a chain of messages, the header from the parent is copied so that
111 114 # clients can track where messages come from.
112 115 'parent_header' : dict,
113 116
114 117 # Any metadata associated with the message.
115 118 'metadata' : dict,
116 119
117 120 # The actual content of the message must be a dict, whose structure
118 121 # depends on the message type.
119 122 'content' : dict,
120 123 }
121 124
125 .. versionchanged:: 5.0.0
126
127 ``version`` key added to the header.
128
122 129 The Wire Protocol
123 130 =================
124 131
125 132
126 133 This message format exists at a high level,
127 134 but does not describe the actual *implementation* at the wire level in zeromq.
128 135 The canonical implementation of the message spec is our :class:`~IPython.kernel.zmq.session.Session` class.
129 136
130 137 .. note::
131 138
132 139 This section should only be relevant to non-Python consumers of the protocol.
133 140 Python consumers should simply import and use IPython's own implementation of the wire protocol
134 141 in the :class:`IPython.kernel.zmq.session.Session` object.
135 142
136 143 Every message is serialized to a sequence of at least six blobs of bytes:
137 144
138 145 .. sourcecode:: python
139 146
140 147 [
141 148 b'u-u-i-d', # zmq identity(ies)
142 149 b'<IDS|MSG>', # delimiter
143 150 b'baddad42', # HMAC signature
144 151 b'{header}', # serialized header dict
145 152 b'{parent_header}', # serialized parent header dict
146 153 b'{metadata}', # serialized metadata dict
147 154 b'{content}, # serialized content dict
148 155 b'blob', # extra raw data buffer(s)
149 156 ...
150 157 ]
151 158
152 159 The front of the message is the ZeroMQ routing prefix,
153 160 which can be zero or more socket identities.
154 161 This is every piece of the message prior to the delimiter key ``<IDS|MSG>``.
155 162 In the case of IOPub, there should be just one prefix component,
156 163 which is the topic for IOPub subscribers, e.g. ``execute_result``, ``display_data``.
157 164
158 165 .. note::
159 166
160 167 In most cases, the IOPub topics are irrelevant and completely ignored,
161 168 because frontends just subscribe to all topics.
162 169 The convention used in the IPython kernel is to use the msg_type as the topic,
163 170 and possibly extra information about the message, e.g. ``execute_result`` or ``stream.stdout``
164 171
165 172 After the delimiter is the `HMAC`_ signature of the message, used for authentication.
166 173 If authentication is disabled, this should be an empty string.
167 174 By default, the hashing function used for computing these signatures is sha256.
168 175
169 176 .. _HMAC: http://en.wikipedia.org/wiki/HMAC
170 177
171 178 .. note::
172 179
173 180 To disable authentication and signature checking,
174 181 set the `key` field of a connection file to an empty string.
175 182
176 183 The signature is the HMAC hex digest of the concatenation of:
177 184
178 185 - A shared key (typically the ``key`` field of a connection file)
179 186 - The serialized header dict
180 187 - The serialized parent header dict
181 188 - The serialized metadata dict
182 189 - The serialized content dict
183 190
184 191 In Python, this is implemented via:
185 192
186 193 .. sourcecode:: python
187 194
188 195 # once:
189 196 digester = HMAC(key, digestmod=hashlib.sha256)
190 197
191 198 # for each message
192 199 d = digester.copy()
193 200 for serialized_dict in (header, parent, metadata, content):
194 201 d.update(serialized_dict)
195 202 signature = d.hexdigest()
196 203
197 204 After the signature is the actual message, always in four frames of bytes.
198 205 The four dictionaries that compose a message are serialized separately,
199 206 in the order of header, parent header, metadata, and content.
200 207 These can be serialized by any function that turns a dict into bytes.
201 208 The default and most common serialization is JSON, but msgpack and pickle
202 209 are common alternatives.
203 210
204 211 After the serialized dicts are zero to many raw data buffers,
205 212 which can be used by message types that support binary data (mainly apply and data_pub).
206 213
207 214
208 215 Python functional API
209 216 =====================
210 217
211 218 As messages are dicts, they map naturally to a ``func(**kw)`` call form. We
212 219 should develop, at a few key points, functional forms of all the requests that
213 220 take arguments in this manner and automatically construct the necessary dict
214 221 for sending.
215 222
216 223 In addition, the Python implementation of the message specification extends
217 224 messages upon deserialization to the following form for convenience::
218 225
219 226 {
220 227 'header' : dict,
221 228 # The msg's unique identifier and type are always stored in the header,
222 229 # but the Python implementation copies them to the top level.
223 230 'msg_id' : uuid,
224 231 'msg_type' : str,
225 232 'parent_header' : dict,
226 233 'content' : dict,
227 234 'metadata' : dict,
228 235 }
229 236
230 237 All messages sent to or received by any IPython process should have this
231 238 extended structure.
232 239
233 240
234 241 Messages on the shell ROUTER/DEALER sockets
235 242 ===========================================
236 243
237 244 .. _execute:
238 245
239 246 Execute
240 247 -------
241 248
242 249 This message type is used by frontends to ask the kernel to execute code on
243 250 behalf of the user, in a namespace reserved to the user's variables (and thus
244 251 separate from the kernel's own internal code and variables).
245 252
246 253 Message type: ``execute_request``::
247 254
248 255 content = {
249 256 # Source code to be executed by the kernel, one or more lines.
250 257 'code' : str,
251 258
252 259 # A boolean flag which, if True, signals the kernel to execute
253 # this code as quietly as possible. This means that the kernel
254 # will compile the code with 'exec' instead of 'single' (so
255 # sys.displayhook will not fire), forces store_history to be False,
260 # this code as quietly as possible.
261 # silent=True forces store_history to be False,
256 262 # and will *not*:
257 # - broadcast exceptions on the PUB socket
258 # - do any logging
259 #
263 # - broadcast output on the IOPUB channel
264 # - have an execute_result
260 265 # The default is False.
261 266 'silent' : bool,
262 267
263 268 # A boolean flag which, if True, signals the kernel to populate history
264 269 # The default is True if silent is False. If silent is True, store_history
265 270 # is forced to be False.
266 271 'store_history' : bool,
267 272
268 # Similarly, a dict mapping names to expressions to be evaluated in the
273 # A dict mapping names to expressions to be evaluated in the
269 274 # user's dict. The rich display-data representation of each will be evaluated after execution.
270 275 # See the display_data content for the structure of the representation data.
271 276 'user_expressions' : dict,
272 277
273 278 # Some frontends do not support stdin requests.
274 279 # If raw_input is called from code executed from such a frontend,
275 280 # a StdinNotImplementedError will be raised.
276 281 'allow_stdin' : True,
277 282 }
278 283
279 The ``code`` field contains a single string (possibly multiline). The kernel
280 is responsible for splitting this into one or more independent execution blocks
281 and deciding whether to compile these in 'single' or 'exec' mode (see below for
282 detailed execution semantics).
284 .. versionchanged:: 5.0.0
285
286 ``user_variables`` removed, because it is redundant with user_expressions.
287
288 The ``code`` field contains a single string (possibly multiline) to be executed.
283 289
284 The ``user_expressions`` fields deserve a detailed explanation. In the past, IPython had
290 The ``user_expressions`` field deserves a detailed explanation. In the past, IPython had
285 291 the notion of a prompt string that allowed arbitrary code to be evaluated, and
286 292 this was put to good use by many in creating prompts that displayed system
287 293 status, path information, and even more esoteric uses like remote instrument
288 294 status acquired over the network. But now that IPython has a clean separation
289 295 between the kernel and the clients, the kernel has no prompt knowledge; prompts
290 296 are a frontend feature, and it should be even possible for different
291 297 frontends to display different prompts while interacting with the same kernel.
292
293 The kernel provides the ability to retrieve data from the user's namespace
294 after the execution of the main ``code``, thanks to two fields in the
295 ``execute_request`` message:
296
297 - ``user_expressions``: For more complex expressions that require function
298 evaluations, a dict can be provided with string keys and arbitrary python
299 expressions as values. The return message will contain also a dict with the
300 same keys and the rich representations of the evaluated expressions as value.
301
302 With this information, frontends can display any status information they wish
303 in the form that best suits each frontend (a status line, a popup, inline for a
304 terminal, etc).
305
306 .. Note::
307
308 In order to obtain the current execution counter for the purposes of
309 displaying input prompts, frontends simply make an execution request with an
310 empty code string and ``silent=True``.
311
312 Execution semantics
313 ~~~~~~~~~~~~~~~~~~~
314
315 When the silent flag is false, the execution of use code consists of the
316 following phases (in silent mode, only the ``code`` field is executed):
317
318 1. Run the ``pre_runcode_hook``.
319
320 2. Execute the ``code`` field, see below for details.
321
322 3. If #2 succeeds, expressions in ``user_expressions`` are computed.
323 This ensures that any error in the expressions don't affect the main code execution.
324
325 4. Call any method registered with :meth:`register_post_execute`.
326
327 .. warning::
328
329 The API for running code before/after the main code block is likely to
330 change soon. Both the ``pre_runcode_hook`` and the
331 :meth:`register_post_execute` are susceptible to modification, as we find a
332 consistent model for both.
333
334 To understand how the ``code`` field is executed, one must know that Python
335 code can be compiled in one of three modes (controlled by the ``mode`` argument
336 to the :func:`compile` builtin):
337
338 *single*
339 Valid for a single interactive statement (though the source can contain
340 multiple lines, such as a for loop). When compiled in this mode, the
341 generated bytecode contains special instructions that trigger the calling of
342 :func:`sys.displayhook` for any expression in the block that returns a value.
343 This means that a single statement can actually produce multiple calls to
344 :func:`sys.displayhook`, if for example it contains a loop where each
345 iteration computes an unassigned expression would generate 10 calls::
346
347 for i in range(10):
348 i**2
349
350 *exec*
351 An arbitrary amount of source code, this is how modules are compiled.
352 :func:`sys.displayhook` is *never* implicitly called.
353
354 *eval*
355 A single expression that returns a value. :func:`sys.displayhook` is *never*
356 implicitly called.
357
358
359 The ``code`` field is split into individual blocks each of which is valid for
360 execution in 'single' mode, and then:
361
362 - If there is only a single block: it is executed in 'single' mode.
363
364 - If there is more than one block:
365
366 * if the last one is a single line long, run all but the last in 'exec' mode
367 and the very last one in 'single' mode. This makes it easy to type simple
368 expressions at the end to see computed values.
369
370 * if the last one is no more than two lines long, run all but the last in
371 'exec' mode and the very last one in 'single' mode. This makes it easy to
372 type simple expressions at the end to see computed values. - otherwise
373 (last one is also multiline), run all in 'exec' mode
374
375 * otherwise (last one is also multiline), run all in 'exec' mode as a single
376 unit.
298 ``user_expressions`` can be used to retrieve this information.
377 299
378 300 Any error in evaluating any expression in ``user_expressions`` will result in
379 301 only that key containing a standard error message, of the form::
380 302
381 303 {
382 304 'status' : 'error',
383 305 'ename' : 'NameError',
384 306 'evalue' : 'foo',
385 307 'traceback' : ...
386 308 }
387 309
388 Errors in any registered post_execute functions are also reported,
389 and the failing function is removed from the post_execution set so that it does
390 not continue triggering failures.
310 .. Note::
311
312 In order to obtain the current execution counter for the purposes of
313 displaying input prompts, frontends may make an execution request with an
314 empty code string and ``silent=True``.
391 315
392 316 Upon completion of the execution request, the kernel *always* sends a reply,
393 317 with a status code indicating what happened and additional data depending on
394 318 the outcome. See :ref:`below <execution_results>` for the possible return
395 319 codes and associated data.
396 320
321 .. seealso::
322
323 :ref:`execution_semantics`
397 324
398 325 .. _execution_counter:
399 326
400 327 Execution counter (prompt number)
401 328 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
402 329
403 The kernel has a single, monotonically increasing counter of all execution
404 requests that are made with ``store_history=True``. This counter is used to populate
405 the ``In[n]``, ``Out[n]`` and ``_n`` variables, so clients will likely want to
406 display it in some form to the user, which will typically (but not necessarily)
407 be done in the prompts. The value of this counter will be returned as the
408 ``execution_count`` field of all ``execute_reply`` and ``pyin`` messages.
330 The kernel should have a single, monotonically increasing counter of all execution
331 requests that are made with ``store_history=True``. This counter is used to populate
332 the ``In[n]`` and ``Out[n]`` prompts. The value of this counter will be returned as the
333 ``execution_count`` field of all ``execute_reply`` and ``execute_input`` messages.
409 334
410 335 .. _execution_results:
411 336
412 337 Execution results
413 338 ~~~~~~~~~~~~~~~~~
414 339
415 340 Message type: ``execute_reply``::
416 341
417 342 content = {
418 343 # One of: 'ok' OR 'error' OR 'abort'
419 344 'status' : str,
420 345
421 346 # The global kernel counter that increases by one with each request that
422 347 # stores history. This will typically be used by clients to display
423 348 # prompt numbers to the user. If the request did not store history, this will
424 349 # be the current value of the counter in the kernel.
425 350 'execution_count' : int,
426 351 }
427 352
428 353 When status is 'ok', the following extra fields are present::
429 354
430 355 {
431 356 # 'payload' will be a list of payload dicts.
432 357 # Each execution payload is a dict with string keys that may have been
433 358 # produced by the code being executed. It is retrieved by the kernel at
434 359 # the end of the execution and sent back to the front end, which can take
435 360 # action on it as needed.
436 361 # The only requirement of each payload dict is that it have a 'source' key,
437 362 # which is a string classifying the payload (e.g. 'pager').
438 363 'payload' : list(dict),
439 364
440 365 # Results for the user_expressions.
441 366 'user_expressions' : dict,
442 367 }
443 368
369 .. versionchanged:: 5.0.0
370
371 ``user_variables`` is removed, use user_expressions instead.
372
444 373 .. admonition:: Execution payloads
445 374
446 375 The notion of an 'execution payload' is different from a return value of a
447 376 given set of code, which normally is just displayed on the execute_result stream
448 377 through the PUB socket. The idea of a payload is to allow special types of
449 378 code, typically magics, to populate a data container in the IPython kernel
450 379 that will be shipped back to the caller via this channel. The kernel
451 380 has an API for this in the PayloadManager::
452 381
453 382 ip.payload_manager.write_payload(payload_dict)
454 383
455 384 which appends a dictionary to the list of payloads.
456 385
457 386 The payload API is not yet stabilized,
458 387 and should probably not be supported by non-Python kernels at this time.
459 388 In such cases, the payload list should always be empty.
460 389
461 390
462 391 When status is 'error', the following extra fields are present::
463 392
464 393 {
465 394 'ename' : str, # Exception name, as a string
466 395 'evalue' : str, # Exception value, as a string
467 396
468 397 # The traceback will contain a list of frames, represented each as a
469 398 # string. For now we'll stick to the existing design of ultraTB, which
470 399 # controls exception level of detail statefully. But eventually we'll
471 400 # want to grow into a model where more information is collected and
472 401 # packed into the traceback object, with clients deciding how little or
473 402 # how much of it to unpack. But for now, let's start with a simple list
474 403 # of strings, since that requires only minimal changes to ultratb as
475 404 # written.
476 405 'traceback' : list,
477 406 }
478 407
479 408
480 409 When status is 'abort', there are for now no additional data fields. This
481 410 happens when the kernel was interrupted by a signal.
482 411
483 412
484 Object information
485 ------------------
413 Introspection
414 -------------
486 415
487 One of IPython's most used capabilities is the introspection of Python objects
488 in the user's namespace, typically invoked via the ``?`` and ``??`` characters
489 (which in reality are shorthands for the ``%pinfo`` magic). This is used often
490 enough that it warrants an explicit message type, especially because frontends
491 may want to get object information in response to user keystrokes (like Tab or
492 F1) besides from the user explicitly typing code like ``x??``.
416 Code can be inspected to show useful information to the user.
417 It is up to the Kernel to decide what information should be displayed, and its formatting.
493 418
494 Message type: ``object_info_request``::
419 Message type: ``inspect_request``::
495 420
496 421 content = {
497 # The (possibly dotted) name of the object to be searched in all
498 # relevant namespaces
499 'oname' : str,
422 # The code context in which introspection is requested
423 # this may be up to an entire multiline cell.
424 'code' : str,
425
426 # The cursor position within 'code' (in unicode characters) where inspection is requested
427 'cursor_pos' : int,
500 428
501 # The level of detail desired. The default (0) is equivalent to typing
429 # The level of detail desired. In IPython, the default (0) is equivalent to typing
502 430 # 'x?' at the prompt, 1 is equivalent to 'x??'.
503 'detail_level' : int,
431 # The difference is up to kernels, but in IPython level 1 includes the source code
432 # if available.
433 'detail_level' : 0 or 1,
504 434 }
505 435
506 The returned information will be a dictionary with keys very similar to the
507 field names that IPython prints at the terminal.
508
509 Message type: ``object_info_reply``::
436 .. versionchanged:: 5.0.0
437
438 ``object_info_request`` renamed to ``inspect_request``.
439
440 .. versionchanged:: 5.0.0
441
442 ``name`` key replaced with ``code`` and ``cursor_pos``,
443 moving the lexing responsibility to the kernel.
444
445 The reply is a mime-bundle, like a `display_data`_ message,
446 which should be a formatted representation of information about the context.
447 In the notebook, this is used to show tooltips over function calls, etc.
448
449 Message type: ``inspect_reply``::
510 450
511 451 content = {
512 # The name the object was requested under
513 'name' : str,
514
515 # Boolean flag indicating whether the named object was found or not. If
516 # it's false, all other fields will be empty.
517 'found' : bool,
518
519 # Flags for magics and system aliases
520 'ismagic' : bool,
521 'isalias' : bool,
522
523 # The name of the namespace where the object was found ('builtin',
524 # 'magics', 'alias', 'interactive', etc.)
525 'namespace' : str,
526
527 # The type name will be type.__name__ for normal Python objects, but it
528 # can also be a string like 'Magic function' or 'System alias'
529 'type_name' : str,
530
531 # The string form of the object, possibly truncated for length if
532 # detail_level is 0
533 'string_form' : str,
534
535 # For objects with a __class__ attribute this will be set
536 'base_class' : str,
537
538 # For objects with a __len__ attribute this will be set
539 'length' : int,
540
541 # If the object is a function, class or method whose file we can find,
542 # we give its full path
543 'file' : str,
544
545 # For pure Python callable objects, we can reconstruct the object
546 # definition line which provides its call signature. For convenience this
547 # is returned as a single 'definition' field, but below the raw parts that
548 # compose it are also returned as the argspec field.
549 'definition' : str,
550
551 # The individual parts that together form the definition string. Clients
552 # with rich display capabilities may use this to provide a richer and more
553 # precise representation of the definition line (e.g. by highlighting
554 # arguments based on the user's cursor position). For non-callable
555 # objects, this field is empty.
556 'argspec' : { # The names of all the arguments
557 args : list,
558 # The name of the varargs (*args), if any
559 varargs : str,
560 # The name of the varkw (**kw), if any
561 varkw : str,
562 # The values (as strings) of all default arguments. Note
563 # that these must be matched *in reverse* with the 'args'
564 # list above, since the first positional args have no default
565 # value at all.
566 defaults : list,
567 },
568
569 # For instances, provide the constructor signature (the definition of
570 # the __init__ method):
571 'init_definition' : str,
572
573 # Docstrings: for any object (function, method, module, package) with a
574 # docstring, we show it. But in addition, we may provide additional
575 # docstrings. For example, for instances we will show the constructor
576 # and class docstrings as well, if available.
577 'docstring' : str,
578
579 # For instances, provide the constructor and class docstrings
580 'init_docstring' : str,
581 'class_docstring' : str,
582
583 # If it's a callable object whose call method has a separate docstring and
584 # definition line:
585 'call_def' : str,
586 'call_docstring' : str,
587
588 # If detail_level was 1, we also try to find the source code that
589 # defines the object, if possible. The string 'None' will indicate
590 # that no source was found.
591 'source' : str,
452 # 'ok' if the request succeeded or 'error', with error information as in all other replies.
453 'status' : 'ok',
454
455 # data can be empty if nothing is found
456 'data' : dict,
457 'metadata' : dict,
592 458 }
593 459
594
595 Complete
596 --------
460 .. versionchanged:: 5.0.0
461
462 ``object_info_reply`` renamed to ``inspect_reply``.
463
464 .. versionchanged:: 5.0.0
465
466 Reply is changed from structured data to a mime bundle, allowing formatting decisions to be made by the kernel.
467
468 Completion
469 ----------
597 470
598 471 Message type: ``complete_request``::
599 472
600 473 content = {
601 # The text to be completed, such as 'a.is'
602 # this may be an empty string if the frontend does not do any lexing,
603 # in which case the kernel must figure out the completion
604 # based on 'line' and 'cursor_pos'.
605 'text' : str,
606
607 # The full line, such as 'print a.is'. This allows completers to
608 # make decisions that may require information about more than just the
609 # current word.
610 'line' : str,
611
612 # The entire block of text where the line is. This may be useful in the
613 # case of multiline completions where more context may be needed. Note: if
614 # in practice this field proves unnecessary, remove it to lighten the
615 # messages.
616
617 'block' : str or null/None,
618
619 # The position of the cursor where the user hit 'TAB' on the line.
474 # The code context in which completion is requested
475 # this may be up to an entire multiline cell, such as
476 # 'foo = a.isal'
477 'code' : str,
478
479 # The cursor position within 'code' (in unicode characters) where completion is requested
620 480 'cursor_pos' : int,
621 481 }
622 482
483 .. versionchanged:: 5.0.0
484
485 ``line``, ``block``, and ``text`` keys are removed in favor of a single ``code`` for context.
486 Lexing is up to the kernel.
487
488
623 489 Message type: ``complete_reply``::
624 490
625 491 content = {
626 492 # The list of all matches to the completion request, such as
627 493 # ['a.isalnum', 'a.isalpha'] for the above example.
628 494 'matches' : list,
629 495
630 # the substring of the matched text
631 # this is typically the common prefix of the matches,
632 # and the text that is already in the block that would be replaced by the full completion.
633 # This would be 'a.is' in the above example.
634 'matched_text' : str,
496 # The range of text that should be replaced by the above matches when a completion is accepted.
497 # typically cursor_end is the same as cursor_pos in the request.
498 'cursor_start' : int,
499 'cursor_end' : int,
500
501 # Information that frontend plugins might use for extra display information about completions.
502 'metadata' : dict,
635 503
636 504 # status should be 'ok' unless an exception was raised during the request,
637 505 # in which case it should be 'error', along with the usual error message content
638 506 # in other messages.
639 507 'status' : 'ok'
640 508 }
641 509
642
510 .. versionchanged:: 5.0.0
511
512 - ``matched_text`` is removed in favor of ``cursor_start`` and ``cursor_end``.
513 - ``metadata`` is added for extended information.
514
515
643 516 History
644 517 -------
645 518
646 519 For clients to explicitly request history from a kernel. The kernel has all
647 520 the actual execution history stored in a single location, so clients can
648 521 request it from the kernel when needed.
649 522
650 523 Message type: ``history_request``::
651 524
652 525 content = {
653 526
654 527 # If True, also return output history in the resulting dict.
655 528 'output' : bool,
656 529
657 530 # If True, return the raw input history, else the transformed input.
658 531 'raw' : bool,
659 532
660 533 # So far, this can be 'range', 'tail' or 'search'.
661 534 'hist_access_type' : str,
662 535
663 536 # If hist_access_type is 'range', get a range of input cells. session can
664 537 # be a positive session number, or a negative number to count back from
665 538 # the current session.
666 539 'session' : int,
667 540 # start and stop are line numbers within that session.
668 541 'start' : int,
669 542 'stop' : int,
670 543
671 544 # If hist_access_type is 'tail' or 'search', get the last n cells.
672 545 'n' : int,
673 546
674 547 # If hist_access_type is 'search', get cells matching the specified glob
675 548 # pattern (with * and ? as wildcards).
676 549 'pattern' : str,
677 550
678 551 # If hist_access_type is 'search' and unique is true, do not
679 552 # include duplicated history. Default is false.
680 553 'unique' : bool,
681 554
682 555 }
683 556
684 557 .. versionadded:: 4.0
685 558 The key ``unique`` for ``history_request``.
686 559
687 560 Message type: ``history_reply``::
688 561
689 562 content = {
690 563 # A list of 3 tuples, either:
691 564 # (session, line_number, input) or
692 565 # (session, line_number, (input, output)),
693 566 # depending on whether output was False or True, respectively.
694 567 'history' : list,
695 568 }
696 569
697 570
698 571 Connect
699 572 -------
700 573
701 574 When a client connects to the request/reply socket of the kernel, it can issue
702 575 a connect request to get basic information about the kernel, such as the ports
703 576 the other ZeroMQ sockets are listening on. This allows clients to only have
704 577 to know about a single port (the shell channel) to connect to a kernel.
705 578
706 579 Message type: ``connect_request``::
707 580
708 581 content = {
709 582 }
710 583
711 584 Message type: ``connect_reply``::
712 585
713 586 content = {
714 587 'shell_port' : int, # The port the shell ROUTER socket is listening on.
715 588 'iopub_port' : int, # The port the PUB socket is listening on.
716 589 'stdin_port' : int, # The port the stdin ROUTER socket is listening on.
717 590 'hb_port' : int, # The port the heartbeat socket is listening on.
718 591 }
719 592
720 593
721 594 Kernel info
722 595 -----------
723 596
724 597 If a client needs to know information about the kernel, it can
725 598 make a request of the kernel's information.
726 599 This message can be used to fetch core information of the
727 600 kernel, including language (e.g., Python), language version number and
728 601 IPython version number, and the IPython message spec version number.
729 602
730 603 Message type: ``kernel_info_request``::
731 604
732 605 content = {
733 606 }
734 607
735 608 Message type: ``kernel_info_reply``::
736 609
737 610 content = {
738 # Version of messaging protocol (mandatory).
611 # Version of messaging protocol.
739 612 # The first integer indicates major version. It is incremented when
740 613 # there is any backward incompatible change.
741 614 # The second integer indicates minor version. It is incremented when
742 615 # there is any backward compatible change.
743 616 'protocol_version': 'X.Y.Z',
744 617
745 # IPython version number (optional).
746 # Non-python kernel backend may not have this version number.
747 # could be '2.0.0-dev' for development version
748 'ipython_version': 'X.Y.Z',
618 # The kernel implementation name
619 # (e.g. 'ipython' for the IPython kernel)
620 'implementation': str,
621
622 # Implementation version number.
623 # The version number of the kernel's implementation
624 # (e.g. IPython.__version__ for the IPython kernel)
625 'implementation_version': 'X.Y.Z',
749 626
750 # Language version number (mandatory).
627 # Programming language in which kernel is implemented.
628 # Kernel included in IPython returns 'python'.
629 'language': str,
630
631 # Language version number.
751 632 # It is Python version number (e.g., '2.7.3') for the kernel
752 633 # included in IPython.
753 634 'language_version': 'X.Y.Z',
754 635
755 # Programming language in which kernel is implemented (mandatory).
756 # Kernel included in IPython returns 'python'.
757 'language': str,
636 # A banner of information about the kernel,
637 # which may be desplayed in console environments.
638 'banner' : str,
758 639 }
759 640
760 .. versionchanged:: 5.0
641 .. versionchanged:: 5.0.0
642
643 Versions changed from lists of integers to strings.
644
645 .. versionchanged:: 5.0.0
646
647 ``ipython_version`` is removed.
761 648
762 In protocol version 4.0, versions were given as lists of numbers,
763 not version strings.
649 .. versionchanged:: 5.0.0
650
651 ``implementation``, ``implementation_version``, and ``banner`` keys are added.
764 652
765 653
766 654 Kernel shutdown
767 655 ---------------
768 656
769 657 The clients can request the kernel to shut itself down; this is used in
770 658 multiple cases:
771 659
772 660 - when the user chooses to close the client application via a menu or window
773 661 control.
774 662 - when the user types 'exit' or 'quit' (or their uppercase magic equivalents).
775 663 - when the user chooses a GUI method (like the 'Ctrl-C' shortcut in the
776 664 IPythonQt client) to force a kernel restart to get a clean kernel without
777 665 losing client-side state like history or inlined figures.
778 666
779 667 The client sends a shutdown request to the kernel, and once it receives the
780 668 reply message (which is otherwise empty), it can assume that the kernel has
781 669 completed shutdown safely.
782 670
783 671 Upon their own shutdown, client applications will typically execute a last
784 672 minute sanity check and forcefully terminate any kernel that is still alive, to
785 673 avoid leaving stray processes in the user's machine.
786 674
787 675 Message type: ``shutdown_request``::
788 676
789 677 content = {
790 678 'restart' : bool # whether the shutdown is final, or precedes a restart
791 679 }
792 680
793 681 Message type: ``shutdown_reply``::
794 682
795 683 content = {
796 684 'restart' : bool # whether the shutdown is final, or precedes a restart
797 685 }
798 686
799 687 .. Note::
800 688
801 689 When the clients detect a dead kernel thanks to inactivity on the heartbeat
802 690 socket, they simply send a forceful process termination signal, since a dead
803 691 process is unlikely to respond in any useful way to messages.
804 692
805 693
806 694 Messages on the PUB/SUB socket
807 695 ==============================
808 696
809 697 Streams (stdout, stderr, etc)
810 698 ------------------------------
811 699
812 700 Message type: ``stream``::
813 701
814 702 content = {
815 703 # The name of the stream is one of 'stdout', 'stderr'
816 704 'name' : str,
817 705
818 706 # The data is an arbitrary string to be written to that stream
819 707 'data' : str,
820 708 }
821 709
822 710 Display Data
823 711 ------------
824 712
825 713 This type of message is used to bring back data that should be displayed (text,
826 714 html, svg, etc.) in the frontends. This data is published to all frontends.
827 715 Each message can have multiple representations of the data; it is up to the
828 716 frontend to decide which to use and how. A single message should contain all
829 717 possible representations of the same information. Each representation should
830 718 be a JSON'able data structure, and should be a valid MIME type.
831 719
832 720 Some questions remain about this design:
833 721
834 722 * Do we use this message type for execute_result/displayhook? Probably not, because
835 723 the displayhook also has to handle the Out prompt display. On the other hand
836 we could put that information into the metadata secion.
724 we could put that information into the metadata section.
725
726 .. _display_data:
837 727
838 728 Message type: ``display_data``::
839 729
840 730 content = {
841 731
842 732 # Who create the data
843 733 'source' : str,
844 734
845 735 # The data dict contains key/value pairs, where the keys are MIME
846 736 # types and the values are the raw data of the representation in that
847 737 # format.
848 738 'data' : dict,
849 739
850 740 # Any metadata that describes the data
851 741 'metadata' : dict
852 742 }
853 743
854 744
855 745 The ``metadata`` contains any metadata that describes the output.
856 746 Global keys are assumed to apply to the output as a whole.
857 747 The ``metadata`` dict can also contain mime-type keys, which will be sub-dictionaries,
858 748 which are interpreted as applying only to output of that type.
859 749 Third parties should put any data they write into a single dict
860 750 with a reasonably unique name to avoid conflicts.
861 751
862 752 The only metadata keys currently defined in IPython are the width and height
863 753 of images::
864 754
865 'metadata' : {
755 metadata = {
866 756 'image/png' : {
867 757 'width': 640,
868 758 'height': 480
869 759 }
870 760 }
871 761
872 762
763 .. versionchanged:: 5.0.0
764
765 `application/json` data should be unpacked JSON data,
766 not double-serialized as a JSON string.
767
768
873 769 Raw Data Publication
874 770 --------------------
875 771
876 772 ``display_data`` lets you publish *representations* of data, such as images and html.
877 773 This ``data_pub`` message lets you publish *actual raw data*, sent via message buffers.
878 774
879 775 data_pub messages are constructed via the :func:`IPython.lib.datapub.publish_data` function:
880 776
881 777 .. sourcecode:: python
882 778
883 779 from IPython.kernel.zmq.datapub import publish_data
884 780 ns = dict(x=my_array)
885 781 publish_data(ns)
886 782
887 783
888 784 Message type: ``data_pub``::
889 785
890 786 content = {
891 787 # the keys of the data dict, after it has been unserialized
892 keys = ['a', 'b']
788 'keys' : ['a', 'b']
893 789 }
894 790 # the namespace dict will be serialized in the message buffers,
895 791 # which will have a length of at least one
896 buffers = ['pdict', ...]
792 buffers = [b'pdict', ...]
897 793
898 794
899 795 The interpretation of a sequence of data_pub messages for a given parent request should be
900 796 to update a single namespace with subsequent results.
901 797
902 798 .. note::
903 799
904 800 No frontends directly handle data_pub messages at this time.
905 801 It is currently only used by the client/engines in :mod:`IPython.parallel`,
906 802 where engines may publish *data* to the Client,
907 803 of which the Client can then publish *representations* via ``display_data``
908 804 to various frontends.
909 805
910 Python inputs
911 -------------
806 Code inputs
807 -----------
912 808
913 809 To let all frontends know what code is being executed at any given time, these
914 810 messages contain a re-broadcast of the ``code`` portion of an
915 811 :ref:`execute_request <execute>`, along with the :ref:`execution_count
916 812 <execution_counter>`.
917 813
918 Message type: ``pyin``::
814 Message type: ``execute_input``::
919 815
920 816 content = {
921 817 'code' : str, # Source code to be executed, one or more lines
922 818
923 819 # The counter for this execution is also provided so that clients can
924 820 # display it, since IPython automatically creates variables called _iN
925 821 # (for input prompt In[N]).
926 822 'execution_count' : int
927 823 }
928 824
929 Python outputs
930 --------------
825 .. versionchanged:: 5.0.0
826
827 ``pyin`` is renamed to ``execute_input``.
828
931 829
932 When Python produces output from code that has been compiled in with the
933 'single' flag to :func:`compile`, any expression that produces a value (such as
934 ``1+1``) is passed to ``sys.displayhook``, which is a callable that can do with
935 this value whatever it wants. The default behavior of ``sys.displayhook`` in
936 the Python interactive prompt is to print to ``sys.stdout`` the :func:`repr` of
937 the value as long as it is not ``None`` (which isn't printed at all). In our
938 case, the kernel instantiates as ``sys.displayhook`` an object which has
939 similar behavior, but which instead of printing to stdout, broadcasts these
940 values as ``execute_result`` messages for clients to display appropriately.
941
942 IPython's displayhook can handle multiple simultaneous formats depending on its
943 configuration. The default pretty-printed repr text is always given with the
944 ``data`` entry in this message. Any other formats are provided in the
945 ``extra_formats`` list. Frontends are free to display any or all of these
946 according to its capabilities. ``extra_formats`` list contains 3-tuples of an ID
947 string, a type string, and the data. The ID is unique to the formatter
948 implementation that created the data. Frontends will typically ignore the ID
949 unless if it has requested a particular formatter. The type string tells the
950 frontend how to interpret the data. It is often, but not always a MIME type.
951 Frontends should ignore types that it does not understand. The data itself is
830 Execution results
831 -----------------
832
833 Results of an execution are published as an ``execute_result``.
834 These are identical to `display_data`_ messages, with the addition of an ``execution_count`` key.
835
836 Results can have multiple simultaneous formats depending on its
837 configuration. A plain text representation should always be provided
838 in the ``text/plain`` mime-type. Frontends are free to display any or all of these
839 according to its capabilities.
840 Frontends should ignore mime-types they do not understand. The data itself is
952 841 any JSON object and depends on the format. It is often, but not always a string.
953 842
954 843 Message type: ``execute_result``::
955 844
956 845 content = {
957 846
958 847 # The counter for this execution is also provided so that clients can
959 848 # display it, since IPython automatically creates variables called _N
960 849 # (for prompt N).
961 850 'execution_count' : int,
962
851
963 852 # data and metadata are identical to a display_data message.
964 853 # the object being displayed is that passed to the display hook,
965 854 # i.e. the *result* of the execution.
966 855 'data' : dict,
967 856 'metadata' : dict,
968 857 }
969
970 Python errors
971 -------------
858
859 Execution errors
860 ----------------
972 861
973 862 When an error occurs during code execution
974 863
975 864 Message type: ``error``::
976 865
977 866 content = {
978 867 # Similar content to the execute_reply messages for the 'error' case,
979 868 # except the 'status' field is omitted.
980 869 }
981 870
871 .. versionchanged:: 5.0.0
872
873 ``pyerr`` renamed to ``error``
874
982 875 Kernel status
983 876 -------------
984 877
985 878 This message type is used by frontends to monitor the status of the kernel.
986 879
987 880 Message type: ``status``::
988 881
989 882 content = {
990 883 # When the kernel starts to execute code, it will enter the 'busy'
991 884 # state and when it finishes, it will enter the 'idle' state.
992 885 # The kernel will publish state 'starting' exactly once at process startup.
993 886 execution_state : ('busy', 'idle', 'starting')
994 887 }
995 888
996 889 Clear output
997 890 ------------
998 891
999 892 This message type is used to clear the output that is visible on the frontend.
1000 893
1001 894 Message type: ``clear_output``::
1002 895
1003 896 content = {
1004 897
1005 898 # Wait to clear the output until new output is available. Clears the
1006 899 # existing output immediately before the new output is displayed.
1007 900 # Useful for creating simple animations with minimal flickering.
1008 901 'wait' : bool,
1009 902 }
1010 903
1011 904 .. versionchanged:: 4.1
1012 905
1013 'stdout', 'stderr', and 'display' boolean keys for selective clearing are removed,
1014 and 'wait' is added.
906 ``stdout``, ``stderr``, and ``display`` boolean keys for selective clearing are removed,
907 and ``wait`` is added.
1015 908 The selective clearing keys are ignored in v4 and the default behavior remains the same,
1016 909 so v4 clear_output messages will be safely handled by a v4.1 frontend.
1017 910
1018 911
1019 912 Messages on the stdin ROUTER/DEALER sockets
1020 913 ===========================================
1021 914
1022 915 This is a socket where the request/reply pattern goes in the opposite direction:
1023 916 from the kernel to a *single* frontend, and its purpose is to allow
1024 917 ``raw_input`` and similar operations that read from ``sys.stdin`` on the kernel
1025 918 to be fulfilled by the client. The request should be made to the frontend that
1026 919 made the execution request that prompted ``raw_input`` to be called. For now we
1027 920 will keep these messages as simple as possible, since they only mean to convey
1028 921 the ``raw_input(prompt)`` call.
1029 922
1030 923 Message type: ``input_request``::
1031 924
1032 content = { 'prompt' : str, 'password' : bool }
925 content = {
926 # the text to show at the prompt
927 'prompt' : str,
928 # Is the request for a password?
929 # If so, the frontend shouldn't echo input.
930 'password' : bool
931 }
1033 932
1034 933 Message type: ``input_reply``::
1035 934
1036 935 content = { 'value' : str }
1037 936
1038 937
1039 938 When ``password`` is True, the frontend should not echo the input as it is entered.
1040 939
1041 .. versionchanged:: 5.0
940 .. versionchanged:: 5.0.0
1042 941
1043 ``password`` key added in msg spec 5.0.
942 ``password`` key added.
1044 943
1045 944 .. note::
1046 945
1047 946 The stdin socket of the client is required to have the same zmq IDENTITY
1048 947 as the client's shell socket.
1049 948 Because of this, the ``input_request`` must be sent with the same IDENTITY
1050 949 routing prefix as the ``execute_reply`` in order for the frontend to receive
1051 950 the message.
1052 951
1053 952 .. note::
1054 953
1055 954 We do not explicitly try to forward the raw ``sys.stdin`` object, because in
1056 955 practice the kernel should behave like an interactive program. When a
1057 956 program is opened on the console, the keyboard effectively takes over the
1058 957 ``stdin`` file descriptor, and it can't be used for raw reading anymore.
1059 958 Since the IPython kernel effectively behaves like a console program (albeit
1060 959 one whose "keyboard" is actually living in a separate process and
1061 960 transported over the zmq connection), raw ``stdin`` isn't expected to be
1062 961 available.
1063 962
1064
963
1065 964 Heartbeat for kernels
1066 965 =====================
1067 966
1068 Initially we had considered using messages like those above over ZMQ for a
1069 kernel 'heartbeat' (a way to detect quickly and reliably whether a kernel is
1070 alive at all, even if it may be busy executing user code). But this has the
1071 problem that if the kernel is locked inside extension code, it wouldn't execute
1072 the python heartbeat code. But it turns out that we can implement a basic
1073 heartbeat with pure ZMQ, without using any Python messaging at all.
1074
1075 The monitor sends out a single zmq message (right now, it is a str of the
1076 monitor's lifetime in seconds), and gets the same message right back, prefixed
1077 with the zmq identity of the DEALER socket in the heartbeat process. This can be
1078 a uuid, or even a full message, but there doesn't seem to be a need for packing
1079 up a message when the sender and receiver are the exact same Python object.
1080
1081 The model is this::
967 Clients send ping messages on a REQ socket, which are echoed right back
968 from the Kernel's REP socket. These are simple bytestrings, not full JSON messages described above.
1082 969
1083 monitor.send(str(self.lifetime)) # '1.2345678910'
1084
1085 and the monitor receives some number of messages of the form::
1086
1087 ['uuid-abcd-dead-beef', '1.2345678910']
1088
1089 where the first part is the zmq.IDENTITY of the heart's DEALER on the engine, and
1090 the rest is the message sent by the monitor. No Python code ever has any
1091 access to the message between the monitor's send, and the monitor's recv.
1092 970
1093 971 Custom Messages
1094 972 ===============
1095 973
1096 974 .. versionadded:: 4.1
1097 975
1098 976 IPython 2.0 (msgspec v4.1) adds a messaging system for developers to add their own objects with Frontend
1099 977 and Kernel-side components, and allow them to communicate with each other.
1100 978 To do this, IPython adds a notion of a ``Comm``, which exists on both sides,
1101 979 and can communicate in either direction.
1102 980
1103 981 These messages are fully symmetrical - both the Kernel and the Frontend can send each message,
1104 982 and no messages expect a reply.
1105 983 The Kernel listens for these messages on the Shell channel,
1106 984 and the Frontend listens for them on the IOPub channel.
1107 985
1108 986 Opening a Comm
1109 987 --------------
1110 988
1111 989 Opening a Comm produces a ``comm_open`` message, to be sent to the other side::
1112 990
1113 991 {
1114 992 'comm_id' : 'u-u-i-d',
1115 993 'target_name' : 'my_comm',
1116 994 'data' : {}
1117 995 }
1118 996
1119 997 Every Comm has an ID and a target name.
1120 998 The code handling the message on the receiving side is responsible for maintaining a mapping
1121 999 of target_name keys to constructors.
1122 1000 After a ``comm_open`` message has been sent,
1123 1001 there should be a corresponding Comm instance on both sides.
1124 1002 The ``data`` key is always a dict and can be any extra JSON information used in initialization of the comm.
1125 1003
1126 1004 If the ``target_name`` key is not found on the receiving side,
1127 1005 then it should immediately reply with a ``comm_close`` message to avoid an inconsistent state.
1128 1006
1129 1007 Comm Messages
1130 1008 -------------
1131 1009
1132 1010 Comm messages are one-way communications to update comm state,
1133 1011 used for synchronizing widget state, or simply requesting actions of a comm's counterpart.
1134 1012
1135 1013 Essentially, each comm pair defines their own message specification implemented inside the ``data`` dict.
1136 1014
1137 1015 There are no expected replies (of course, one side can send another ``comm_msg`` in reply).
1138 1016
1139 1017 Message type: ``comm_msg``::
1140 1018
1141 1019 {
1142 1020 'comm_id' : 'u-u-i-d',
1143 1021 'data' : {}
1144 1022 }
1145 1023
1146 1024 Tearing Down Comms
1147 1025 ------------------
1148 1026
1149 1027 Since comms live on both sides, when a comm is destroyed the other side must be notified.
1150 1028 This is done with a ``comm_close`` message.
1151 1029
1152 1030 Message type: ``comm_close``::
1153 1031
1154 1032 {
1155 1033 'comm_id' : 'u-u-i-d',
1156 1034 'data' : {}
1157 1035 }
1158 1036
1159 1037 Output Side Effects
1160 1038 -------------------
1161 1039
1162 1040 Since comm messages can execute arbitrary user code,
1163 1041 handlers should set the parent header and publish status busy / idle,
1164 1042 just like an execute request.
1165 1043
1166 1044
1167 ToDo
1168 ====
1045 To Do
1046 =====
1169 1047
1170 1048 Missing things include:
1171 1049
1172 1050 * Important: finish thinking through the payload concept and API.
1173 1051
1174 * Important: ensure that we have a good solution for magics like %edit. It's
1175 likely that with the payload concept we can build a full solution, but not
1176 100% clear yet.
1177
1178 1052 .. include:: ../links.txt
General Comments 0
You need to be logged in to leave comments. Login now