##// END OF EJS Templates
move images out of _static folder into _images...
Julian Taylor -
Show More
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_dashboard.png to docs/source/_images/ipy_013_dashboard.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_dashboard_cluster.png to docs/source/_images/ipy_013_dashboard_cluster.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_cythonmagic.png to docs/source/_images/ipy_013_notebook_cythonmagic.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_long_out.png to docs/source/_images/ipy_013_notebook_long_out.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_octavemagic.png to docs/source/_images/ipy_013_notebook_octavemagic.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_rmagic.png to docs/source/_images/ipy_013_notebook_rmagic.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_script_cells.png to docs/source/_images/ipy_013_notebook_script_cells.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_spectrogram.png to docs/source/_images/ipy_013_notebook_spectrogram.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_notebook_tooltip.png to docs/source/_images/ipy_013_notebook_tooltip.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_par_tb.png to docs/source/_images/ipy_013_par_tb.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_qtconsole_baboon.png to docs/source/_images/ipy_013_qtconsole_baboon.png
1 NO CONTENT: file renamed from docs/source/_static/ipy_013_qtconsole_completer.png to docs/source/_images/ipy_013_qtconsole_completer.png
1 NO CONTENT: file renamed from docs/source/_static/ms_visual_studio.png to docs/source/_images/ms_visual_studio.png
1 NO CONTENT: file renamed from docs/source/_static/notebook_specgram.png to docs/source/_images/notebook_specgram.png
1 NO CONTENT: file renamed from docs/source/_static/qtconsole.png to docs/source/_images/qtconsole.png
1 NO CONTENT: file renamed from docs/source/_static/qtconsole_tabbed.png to docs/source/_images/qtconsole_tabbed.png
@@ -1,606 +1,606
1 1 .. _qtconsole:
2 2
3 3 =========================
4 4 A Qt Console for IPython
5 5 =========================
6 6
7 7 We now have a version of IPython, using the new two-process :ref:`ZeroMQ Kernel
8 8 <ipythonzmq>`, running in a PyQt_ GUI. This is a very lightweight widget that
9 9 largely feels like a terminal, but provides a number of enhancements only
10 10 possible in a GUI, such as inline figures, proper multiline editing with syntax
11 11 highlighting, graphical calltips, and much more.
12 12
13 .. figure:: ../_static/qtconsole.png
13 .. figure:: ../../_images/qtconsole.png
14 14 :width: 400px
15 15 :alt: IPython Qt console with embedded plots
16 16 :align: center
17 :target: ../_static/qtconsole.png
17 :target: ../_images/qtconsole.png
18 18
19 19 The Qt console for IPython, using inline matplotlib plots.
20 20
21 21 To get acquainted with the Qt console, type `%guiref` to see a quick
22 22 introduction of its main features.
23 23
24 24 The Qt frontend has hand-coded emacs-style bindings for text navigation. This
25 25 is not yet configurable.
26 26
27 27 .. tip::
28 28
29 29 Since the Qt console tries hard to behave like a terminal, by default it
30 30 immediately executes single lines of input that are complete. If you want
31 31 to force multiline input, hit :kbd:`Ctrl-Enter` at the end of the first line
32 32 instead of :kbd:`Enter`, and it will open a new line for input. At any
33 33 point in a multiline block, you can force its execution (without having to
34 34 go to the bottom) with :kbd:`Shift-Enter`.
35 35
36 36 ``%load``
37 37 =========
38 38
39 39 The new ``%load`` magic (previously ``%loadpy``) takes any script, and pastes
40 40 its contents as your next input, so you can edit it before executing. The
41 41 script may be on your machine, but you can also specify an history range, or a
42 42 url, and it will download the script from the web. This is particularly useful
43 43 for playing with examples from documentation, such as matplotlib.
44 44
45 45 .. sourcecode:: ipython
46 46
47 47 In [6]: %load http://matplotlib.sourceforge.net/plot_directive/mpl_examples/mplot3d/contour3d_demo.py
48 48
49 49 In [7]: from mpl_toolkits.mplot3d import axes3d
50 50 ...: import matplotlib.pyplot as plt
51 51 ...:
52 52 ...: fig = plt.figure()
53 53 ...: ax = fig.add_subplot(111, projection='3d')
54 54 ...: X, Y, Z = axes3d.get_test_data(0.05)
55 55 ...: cset = ax.contour(X, Y, Z)
56 56 ...: ax.clabel(cset, fontsize=9, inline=1)
57 57 ...:
58 58 ...: plt.show()
59 59
60 60 Pylab
61 61 =====
62 62
63 63 One of the most exciting features of the new console is embedded matplotlib
64 64 figures. You can use any standard matplotlib GUI backend
65 65 to draw the figures, and since there is now a two-process model, there is no
66 66 longer a conflict between user input and the drawing eventloop.
67 67
68 68 .. image:: figs/besselj.png
69 69 :width: 519px
70 70
71 71 .. display:
72 72
73 73 :func:`display`
74 74 ***************
75 75
76 76 An additional function, :func:`display`, will be added to the global namespace
77 77 if you specify the ``--pylab`` option at the command line. The IPython display
78 78 system provides a mechanism for specifying PNG or SVG (and more)
79 79 representations of objects for GUI frontends. By default, IPython registers
80 80 convenient PNG and SVG renderers for matplotlib figures, so you can embed them
81 81 in your document by calling :func:`display` on one or more of them. This is
82 82 especially useful for saving_ your work.
83 83
84 84 .. sourcecode:: ipython
85 85
86 86 In [5]: plot(range(5)) # plots in the matplotlib window
87 87
88 88 In [6]: display(gcf()) # embeds the current figure in the qtconsole
89 89
90 90 In [7]: display(*getfigs()) # embeds all active figures in the qtconsole
91 91
92 92 If you have a reference to a matplotlib figure object, you can always display
93 93 that specific figure:
94 94
95 95 .. sourcecode:: ipython
96 96
97 97 In [1]: f = figure()
98 98
99 99 In [2]: plot(rand(100))
100 100 Out[2]: [<matplotlib.lines.Line2D at 0x7fc6ac03dd90>]
101 101
102 102 In [3]: display(f)
103 103
104 104 # Plot is shown here
105 105
106 106 In [4]: title('A title')
107 107 Out[4]: <matplotlib.text.Text at 0x7fc6ac023450>
108 108
109 109 In [5]: display(f)
110 110
111 111 # Updated plot with title is shown here.
112 112
113 113 .. _inline:
114 114
115 115 ``--pylab=inline``
116 116 ******************
117 117
118 118 If you want to have all of your figures embedded in your session, instead of
119 119 calling :func:`display`, you can specify ``--pylab=inline`` when you start the
120 120 console, and each time you make a plot, it will show up in your document, as if
121 121 you had called :func:`display(fig)`.
122 122
123 123 The inline backend can use either SVG or PNG figures (PNG being the default).
124 124 To switch between them, set the ``InlineBackend.figure_format`` configurable
125 125 in a config file, or via the ``%config`` magic:
126 126
127 127 .. sourcecode:: ipython
128 128
129 129 In [10]: %config InlineBackend.figure_format = 'svg'
130 130
131 131 .. note::
132 132
133 133 Changing the inline figure format also affects calls to :func:`display` above,
134 134 even if you are not using the inline backend for all figures.
135 135
136 136 By default, IPython closes all figures at the completion of each execution. This means you
137 137 don't have to manually close figures, which is less convenient when figures aren't attached
138 138 to windows with an obvious close button. It also means that the first matplotlib call in
139 139 each cell will always create a new figure:
140 140
141 141 .. sourcecode:: ipython
142 142
143 143 In [11]: plot(range(100))
144 144 <single-line plot>
145 145
146 146 In [12]: plot([1,3,2])
147 147 <another single-line plot>
148 148
149 149
150 150 However, it does prevent the list of active figures surviving from one input cell to the
151 151 next, so if you want to continue working with a figure, you must hold on to a reference to
152 152 it:
153 153
154 154 .. sourcecode:: ipython
155 155
156 156 In [11]: fig = gcf()
157 157 ....: fig.plot(rand(100))
158 158 <plot>
159 159 In [12]: fig.title('Random Title')
160 160 <redraw plot with title>
161 161
162 162 This behavior is controlled by the :attr:`InlineBackend.close_figures` configurable, and
163 163 if you set it to False, via %config or config file, then IPython will *not* close figures,
164 164 and tools like :func:`gcf`, :func:`gca`, :func:`getfigs` will behave the same as they
165 165 do with other backends. You will, however, have to manually close figures:
166 166
167 167 .. sourcecode:: ipython
168 168
169 169 # close all active figures:
170 170 In [13]: [ fig.close() for fig in getfigs() ]
171 171
172 172
173 173
174 174 .. _saving:
175 175
176 176 Saving and Printing
177 177 ===================
178 178
179 179 IPythonQt has the ability to save your current session, as either HTML or
180 180 XHTML. If you have been using :func:`display` or inline_ pylab, your figures
181 181 will be PNG in HTML, or inlined as SVG in XHTML. PNG images have the option to
182 182 be either in an external folder, as in many browsers' "Webpage, Complete"
183 183 option, or inlined as well, for a larger, but more portable file.
184 184
185 185 .. note::
186 186
187 187 Export to SVG+XHTML requires that you are using SVG figures, which is *not*
188 188 the default. To switch the inline figure format to use SVG during an active
189 189 session, do:
190 190
191 191 .. sourcecode:: ipython
192 192
193 193 In [10]: %config InlineBackend.figure_format = 'svg'
194 194
195 195 Or, you can add the same line (c.Inline... instead of %config Inline...) to
196 196 your config files.
197 197
198 198 This will only affect figures plotted after making this call
199 199
200 200
201 201 The widget also exposes the ability to print directly, via the default print
202 202 shortcut or context menu.
203 203
204 204
205 205 .. Note::
206 206
207 207 Saving is only available to richtext Qt widgets, which are used by default,
208 208 but if you pass the ``--plain`` flag, saving will not be available to you.
209 209
210 210
211 211 See these examples of :download:`png/html<figs/jn.html>` and
212 212 :download:`svg/xhtml <figs/jn.xhtml>` output. Note that syntax highlighting
213 213 does not survive export. This is a known issue, and is being investigated.
214 214
215 215
216 216 Colors and Highlighting
217 217 =======================
218 218
219 219 Terminal IPython has always had some coloring, but never syntax
220 220 highlighting. There are a few simple color choices, specified by the ``colors``
221 221 flag or ``%colors`` magic:
222 222
223 223 * LightBG for light backgrounds
224 224 * Linux for dark backgrounds
225 225 * NoColor for a simple colorless terminal
226 226
227 227 The Qt widget has full support for the ``colors`` flag used in the terminal shell.
228 228
229 229 The Qt widget, however, has full syntax highlighting as you type, handled by
230 230 the `pygments`_ library. The ``style`` argument exposes access to any style by
231 231 name that can be found by pygments, and there are several already
232 232 installed. The ``colors`` argument, if unspecified, will be guessed based on
233 233 the chosen style. Similarly, there are default styles associated with each
234 234 ``colors`` option.
235 235
236 236
237 237 Screenshot of ``ipython qtconsole --colors=linux``, which uses the 'monokai'
238 238 theme by default:
239 239
240 240 .. image:: figs/colors_dark.png
241 241 :width: 627px
242 242
243 243 .. Note::
244 244
245 245 Calling ``ipython qtconsole -h`` will show all the style names that
246 246 pygments can find on your system.
247 247
248 248 You can also pass the filename of a custom CSS stylesheet, if you want to do
249 249 your own coloring, via the ``stylesheet`` argument. The default LightBG
250 250 stylesheet:
251 251
252 252 .. sourcecode:: css
253 253
254 254 QPlainTextEdit, QTextEdit { background-color: white;
255 255 color: black ;
256 256 selection-background-color: #ccc}
257 257 .error { color: red; }
258 258 .in-prompt { color: navy; }
259 259 .in-prompt-number { font-weight: bold; }
260 260 .out-prompt { color: darkred; }
261 261 .out-prompt-number { font-weight: bold; }
262 262 /* .inverted is used to highlight selected completion */
263 263 .inverted { background-color: black ; color: white; }
264 264
265 265 Fonts
266 266 =====
267 267
268 268 The QtConsole has configurable via the ConsoleWidget. To change these, set the
269 269 ``font_family`` or ``font_size`` traits of the ConsoleWidget. For instance, to
270 270 use 9pt Anonymous Pro::
271 271
272 272 $> ipython qtconsole --ConsoleWidget.font_family="Anonymous Pro" --ConsoleWidget.font_size=9
273 273
274 274 Process Management
275 275 ==================
276 276
277 277 With the two-process ZMQ model, the frontend does not block input during
278 278 execution. This means that actions can be taken by the frontend while the
279 279 Kernel is executing, or even after it crashes. The most basic such command is
280 280 via 'Ctrl-.', which restarts the kernel. This can be done in the middle of a
281 281 blocking execution. The frontend can also know, via a heartbeat mechanism, that
282 282 the kernel has died. This means that the frontend can safely restart the
283 283 kernel.
284 284
285 285 .. _multiple_consoles:
286 286
287 287 Multiple Consoles
288 288 *****************
289 289
290 290 Since the Kernel listens on the network, multiple frontends can connect to it.
291 291 These do not have to all be qt frontends - any IPython frontend can connect and
292 292 run code. When you start ipython qtconsole, there will be an output line,
293 293 like::
294 294
295 295 [IPKernelApp] To connect another client to this kernel, use:
296 296 [IPKernelApp] --existing kernel-12345.json
297 297
298 298 Other frontends can connect to your kernel, and share in the execution. This is
299 299 great for collaboration. The ``--existing`` flag means connect to a kernel
300 300 that already exists. Starting other consoles
301 301 with that flag will not try to start their own kernel, but rather connect to
302 302 yours. :file:`kernel-12345.json` is a small JSON file with the ip, port, and
303 303 authentication information necessary to connect to your kernel. By default, this file
304 304 will be in your default profile's security directory. If it is somewhere else,
305 305 the output line will print the full path of the connection file, rather than
306 306 just its filename.
307 307
308 308 If you need to find the connection info to send, and don't know where your connection file
309 309 lives, there are a couple of ways to get it. If you are already running an IPython console
310 310 connected to the kernel, you can use the ``%connect_info`` magic to display the information
311 311 necessary to connect another frontend to the kernel.
312 312
313 313 .. sourcecode:: ipython
314 314
315 315 In [2]: %connect_info
316 316 {
317 317 "stdin_port":50255,
318 318 "ip":"127.0.0.1",
319 319 "hb_port":50256,
320 320 "key":"70be6f0f-1564-4218-8cda-31be40a4d6aa",
321 321 "shell_port":50253,
322 322 "iopub_port":50254
323 323 }
324 324
325 325 Paste the above JSON into a file, and connect with:
326 326 $> ipython <app> --existing <file>
327 327 or, if you are local, you can connect with just:
328 328 $> ipython <app> --existing kernel-12345.json
329 329 or even just:
330 330 $> ipython <app> --existing
331 331 if this is the most recent IPython session you have started.
332 332
333 333 Otherwise, you can find a connection file by name (and optionally profile) with
334 334 :func:`IPython.lib.kernel.find_connection_file`:
335 335
336 336 .. sourcecode:: bash
337 337
338 338 $> python -c "from IPython.lib.kernel import find_connection_file;\
339 339 print find_connection_file('kernel-12345.json')"
340 340 /home/you/.ipython/profile_default/security/kernel-12345.json
341 341
342 342 And if you are using a particular IPython profile:
343 343
344 344 .. sourcecode:: bash
345 345
346 346 $> python -c "from IPython.lib.kernel import find_connection_file;\
347 347 print find_connection_file('kernel-12345.json', profile='foo')"
348 348 /home/you/.ipython/profile_foo/security/kernel-12345.json
349 349
350 350 You can even launch a standalone kernel, and connect and disconnect Qt Consoles
351 351 from various machines. This lets you keep the same running IPython session
352 352 on your work machine (with matplotlib plots and everything), logging in from home,
353 353 cafés, etc.::
354 354
355 355 $> ipython kernel
356 356 [IPKernelApp] To connect another client to this kernel, use:
357 357 [IPKernelApp] --existing kernel-12345.json
358 358
359 359 This is actually exactly the same as the subprocess launched by the qtconsole, so
360 360 all the information about connecting to a standalone kernel is identical to that
361 361 of connecting to the kernel attached to a running console.
362 362
363 363 .. _kernel_security:
364 364
365 365 Security
366 366 --------
367 367
368 368 .. warning::
369 369
370 370 Since the ZMQ code currently has no encryption, listening on an
371 371 external-facing IP is dangerous. You are giving any computer that can see
372 372 you on the network the ability to connect to your kernel, and view your traffic.
373 373 Read the rest of this section before listening on external ports
374 374 or running an IPython kernel on a shared machine.
375 375
376 376 By default (for security reasons), the kernel only listens on localhost, so you
377 377 can only connect multiple frontends to the kernel from your local machine. You
378 378 can specify to listen on an external interface by specifying the ``ip``
379 379 argument::
380 380
381 381 $> ipython qtconsole --ip=192.168.1.123
382 382
383 383 If you specify the ip as 0.0.0.0 or '*', that means all interfaces, so any
384 384 computer that can see yours on the network can connect to the kernel.
385 385
386 386 Messages are not encrypted, so users with access to the ports your kernel is using will be
387 387 able to see any output of the kernel. They will **NOT** be able to issue shell commands as
388 388 you due to message signatures, which are enabled by default as of IPython 0.12.
389 389
390 390 .. warning::
391 391
392 392 If you disable message signatures, then any user with access to the ports your
393 393 kernel is listening on can issue arbitrary code as you. **DO NOT** disable message
394 394 signatures unless you have a lot of trust in your environment.
395 395
396 396 The one security feature IPython does provide is protection from unauthorized execution.
397 397 IPython's messaging system will sign messages with HMAC digests using a shared-key. The key
398 398 is never sent over the network, it is only used to generate a unique hash for each message,
399 399 based on its content. When IPython receives a message, it will check that the digest
400 400 matches, and discard the message. You can use any file that only you have access to to
401 401 generate this key, but the default is just to generate a new UUID. You can generate a random
402 402 private key with::
403 403
404 404 # generate 1024b of random data, and store in a file only you can read:
405 405 # (assumes IPYTHONDIR is defined, otherwise use your IPython directory)
406 406 $> python -c "import os; print os.urandom(128).encode('base64')" > $IPYTHONDIR/sessionkey
407 407 $> chmod 600 $IPYTHONDIR/sessionkey
408 408
409 409 The *contents* of this file will be stored in the JSON connection file, so that file
410 410 contains everything you need to connect to and use a kernel.
411 411
412 412 To use this generated key, simply specify the ``Session.keyfile`` configurable
413 413 in :file:`ipython_config.py` or at the command-line, as in::
414 414
415 415 # instruct IPython to sign messages with that key, instead of a new UUID
416 416 $> ipython qtconsole --Session.keyfile=$IPYTHONDIR/sessionkey
417 417
418 418 .. _ssh_tunnels:
419 419
420 420 SSH Tunnels
421 421 -----------
422 422
423 423 Sometimes you want to connect to machines across the internet, or just across
424 424 a LAN that either doesn't permit open ports or you don't trust the other
425 425 machines on the network. To do this, you can use SSH tunnels. SSH tunnels
426 426 are a way to securely forward ports on your local machine to ports on another
427 427 machine, to which you have SSH access.
428 428
429 429 In simple cases, IPython's tools can forward ports over ssh by simply adding the
430 430 ``--ssh=remote`` argument to the usual ``--existing...`` set of flags for connecting
431 431 to a running kernel, after copying the JSON connection file (or its contents) to
432 432 the second computer.
433 433
434 434 .. warning::
435 435
436 436 Using SSH tunnels does *not* increase localhost security. In fact, when
437 437 tunneling from one machine to another *both* machines have open
438 438 ports on localhost available for connections to the kernel.
439 439
440 440 There are two primary models for using SSH tunnels with IPython. The first
441 441 is to have the Kernel listen only on localhost, and connect to it from
442 442 another machine on the same LAN.
443 443
444 444 First, let's start a kernel on machine **worker**, listening only
445 445 on loopback::
446 446
447 447 user@worker $> ipython kernel
448 448 [IPKernelApp] To connect another client to this kernel, use:
449 449 [IPKernelApp] --existing kernel-12345.json
450 450
451 451 In this case, the IP that you would connect
452 452 to would still be 127.0.0.1, but you want to specify the additional ``--ssh`` argument
453 453 with the hostname of the kernel (in this example, it's 'worker')::
454 454
455 455 user@client $> ipython qtconsole --ssh=worker --existing /path/to/kernel-12345.json
456 456
457 457 Which will write a new connection file with the forwarded ports, so you can reuse them::
458 458
459 459 [IPythonQtConsoleApp] To connect another client via this tunnel, use:
460 460 [IPythonQtConsoleApp] --existing kernel-12345-ssh.json
461 461
462 462 Note again that this opens ports on the *client* machine that point to your kernel.
463 463
464 464 .. note::
465 465
466 466 the ssh argument is simply passed to openssh, so it can be fully specified ``user@host:port``
467 467 but it will also respect your aliases, etc. in :file:`.ssh/config` if you have any.
468 468
469 469 The second pattern is for connecting to a machine behind a firewall across the internet
470 470 (or otherwise wide network). This time, we have a machine **login** that you have ssh access
471 471 to, which can see **kernel**, but **client** is on another network. The important difference
472 472 now is that **client** can see **login**, but *not* **worker**. So we need to forward ports from
473 473 client to worker *via* login. This means that the kernel must be started listening
474 474 on external interfaces, so that its ports are visible to `login`::
475 475
476 476 user@worker $> ipython kernel --ip=0.0.0.0
477 477 [IPKernelApp] To connect another client to this kernel, use:
478 478 [IPKernelApp] --existing kernel-12345.json
479 479
480 480 Which we can connect to from the client with::
481 481
482 482 user@client $> ipython qtconsole --ssh=login --ip=192.168.1.123 --existing /path/to/kernel-12345.json
483 483
484 484 .. note::
485 485
486 486 The IP here is the address of worker as seen from *login*, and need only be specified if
487 487 the kernel used the ambiguous 0.0.0.0 (all interfaces) address. If it had used
488 488 192.168.1.123 to start with, it would not be needed.
489 489
490 490
491 491 Manual SSH tunnels
492 492 ------------------
493 493
494 494 It's possible that IPython's ssh helper functions won't work for you, for various
495 495 reasons. You can still connect to remote machines, as long as you set up the tunnels
496 496 yourself. The basic format of forwarding a local port to a remote one is::
497 497
498 498 [client] $> ssh <server> <localport>:<remoteip>:<remoteport> -f -N
499 499
500 500 This will forward local connections to **localport** on client to **remoteip:remoteport**
501 501 *via* **server**. Note that remoteip is interpreted relative to *server*, not the client.
502 502 So if you have direct ssh access to the machine to which you want to forward connections,
503 503 then the server *is* the remote machine, and remoteip should be server's IP as seen from the
504 504 server itself, i.e. 127.0.0.1. Thus, to forward local port 12345 to remote port 54321 on
505 505 a machine you can see, do::
506 506
507 507 [client] $> ssh machine 12345:127.0.0.1:54321 -f -N
508 508
509 509 But if your target is actually on a LAN at 192.168.1.123, behind another machine called **login**,
510 510 then you would do::
511 511
512 512 [client] $> ssh login 12345:192.168.1.16:54321 -f -N
513 513
514 514 The ``-f -N`` on the end are flags that tell ssh to run in the background,
515 515 and don't actually run any commands beyond creating the tunnel.
516 516
517 517 .. seealso::
518 518
519 519 A short discussion of ssh tunnels: http://www.revsys.com/writings/quicktips/ssh-tunnel.html
520 520
521 521
522 522
523 523 Stopping Kernels and Consoles
524 524 *****************************
525 525
526 526 Since there can be many consoles per kernel, the shutdown mechanism and dialog
527 527 are probably more complicated than you are used to. Since you don't always want
528 528 to shutdown a kernel when you close a window, you are given the option to just
529 529 close the console window or also close the Kernel and *all other windows*. Note
530 530 that this only refers to all other *local* windows, as remote Consoles are not
531 531 allowed to shutdown the kernel, and shutdowns do not close Remote consoles (to
532 532 allow for saving, etc.).
533 533
534 534 Rules:
535 535
536 536 * Restarting the kernel automatically clears all *local* Consoles, and prompts remote
537 537 Consoles about the reset.
538 538 * Shutdown closes all *local* Consoles, and notifies remotes that
539 539 the Kernel has been shutdown.
540 540 * Remote Consoles may not restart or shutdown the kernel.
541 541
542 542 Qt and the QtConsole
543 543 ====================
544 544
545 545 An important part of working with the QtConsole when you are writing your own
546 546 Qt code is to remember that user code (in the kernel) is *not* in the same
547 547 process as the frontend. This means that there is not necessarily any Qt code
548 548 running in the kernel, and under most normal circumstances there isn't. If,
549 549 however, you specify ``--pylab=qt`` at the command-line, then there *will* be a
550 550 :class:`QCoreApplication` instance running in the kernel process along with
551 551 user-code. To get a reference to this application, do:
552 552
553 553 .. sourcecode:: python
554 554
555 555 from PyQt4 import QtCore
556 556 app = QtCore.QCoreApplication.instance()
557 557 # app will be None if there is no such instance
558 558
559 559 A common problem listed in the PyQt4 Gotchas_ is the fact that Python's garbage
560 560 collection will destroy Qt objects (Windows, etc.) once there is no longer a
561 561 Python reference to them, so you have to hold on to them. For instance, in:
562 562
563 563 .. sourcecode:: python
564 564
565 565 def make_window():
566 566 win = QtGui.QMainWindow()
567 567
568 568 def make_and_return_window():
569 569 win = QtGui.QMainWindow()
570 570 return win
571 571
572 572 :func:`make_window` will never draw a window, because garbage collection will
573 573 destroy it before it is drawn, whereas :func:`make_and_return_window` lets the
574 574 caller decide when the window object should be destroyed. If, as a developer,
575 575 you know that you always want your objects to last as long as the process, you
576 576 can attach them to the QApplication instance itself:
577 577
578 578 .. sourcecode:: python
579 579
580 580 # do this just once:
581 581 app = QtCore.QCoreApplication.instance()
582 582 app.references = set()
583 583 # then when you create Windows, add them to the set
584 584 def make_window():
585 585 win = QtGui.QMainWindow()
586 586 app.references.add(win)
587 587
588 588 Now the QApplication itself holds a reference to ``win``, so it will never be
589 589 garbage collected until the application itself is destroyed.
590 590
591 591 .. _Gotchas: http://www.riverbankcomputing.co.uk/static/Docs/PyQt4/html/gotchas.html#garbage-collection
592 592
593 593 Regressions
594 594 ===========
595 595
596 596 There are some features, where the qt console lags behind the Terminal
597 597 frontend:
598 598
599 599 * !cmd input: Due to our use of pexpect, we cannot pass input to subprocesses
600 600 launched using the '!' escape, so you should never call a command that
601 601 requires interactive input. For such cases, use the terminal IPython. This
602 602 will not be fixed, as abandoning pexpect would significantly degrade the
603 603 console experience.
604 604
605 605 .. _PyQt: http://www.riverbankcomputing.co.uk/software/pyqt/download
606 606 .. _pygments: http://pygments.org/
@@ -1,765 +1,765
1 1 =============
2 2 0.11 Series
3 3 =============
4 4
5 5 Release 0.11
6 6 ============
7 7
8 8 IPython 0.11 is a *major* overhaul of IPython, two years in the making. Most
9 9 of the code base has been rewritten or at least reorganized, breaking backward
10 10 compatibility with several APIs in previous versions. It is the first major
11 11 release in two years, and probably the most significant change to IPython since
12 12 its inception. We plan to have a relatively quick succession of releases, as
13 13 people discover new bugs and regressions. Once we iron out any significant
14 14 bugs in this process and settle down the new APIs, this series will become
15 15 IPython 1.0. We encourage feedback now on the core APIs, which we hope to
16 16 maintain stable during the 1.0 series.
17 17
18 18 Since the internal APIs have changed so much, projects using IPython as a
19 19 library (as opposed to end-users of the application) are the most likely to
20 20 encounter regressions or changes that break their existing use patterns. We
21 21 will make every effort to provide updated versions of the APIs to facilitate
22 22 the transition, and we encourage you to contact us on the `development mailing
23 23 list`__ with questions and feedback.
24 24
25 25 .. __: http://mail.scipy.org/mailman/listinfo/ipython-dev
26 26
27 27 Chris Fonnesbeck recently wrote an `excellent post`__ that highlights some of
28 28 our major new features, with examples and screenshots. We encourage you to
29 29 read it as it provides an illustrated, high-level overview complementing the
30 30 detailed feature breakdown in this document.
31 31
32 32 .. __: http://fonnesbeck.calepin.co/innovations-in-ipython.html
33 33
34 34 A quick summary of the major changes (see below for details):
35 35
36 36 * **Standalone Qt console**: a new rich console has been added to IPython,
37 37 started with `ipython qtconsole`. In this application we have tried to
38 38 retain the feel of a terminal for fast and efficient workflows, while adding
39 39 many features that a line-oriented terminal simply can not support, such as
40 40 inline figures, full multiline editing with syntax highlighting, graphical
41 41 tooltips for function calls and much more. This development was sponsored by
42 42 `Enthought Inc.`__. See :ref:`below <qtconsole_011>` for details.
43 43
44 44 .. __: http://enthought.com
45 45
46 46 * **High-level parallel computing with ZeroMQ**. Using the same architecture
47 47 that our Qt console is based on, we have completely rewritten our high-level
48 48 parallel computing machinery that in prior versions used the Twisted
49 49 networking framework. While this change will require users to update their
50 50 codes, the improvements in performance, memory control and internal
51 51 consistency across our codebase convinced us it was a price worth paying. We
52 52 have tried to explain how to best proceed with this update, and will be happy
53 53 to answer questions that may arise. A full tutorial describing these
54 54 features `was presented at SciPy'11`__, more details :ref:`below
55 55 <parallel_011>`.
56 56
57 57 .. __: http://minrk.github.com/scipy-tutorial-2011
58 58
59 59 * **New model for GUI/plotting support in the terminal**. Now instead of the
60 60 various `-Xthread` flags we had before, GUI support is provided without the
61 61 use of any threads, by directly integrating GUI event loops with Python's
62 62 `PyOS_InputHook` API. A new command-line flag `--gui` controls GUI support,
63 63 and it can also be enabled after IPython startup via the new `%gui` magic.
64 64 This requires some changes if you want to execute GUI-using scripts inside
65 65 IPython, see :ref:`the GUI support section <gui_support>` for more details.
66 66
67 67 * **A two-process architecture.** The Qt console is the first use of a new
68 68 model that splits IPython between a kernel process where code is executed and
69 69 a client that handles user interaction. We plan on also providing terminal
70 70 and web-browser based clients using this infrastructure in future releases.
71 71 This model allows multiple clients to interact with an IPython process
72 72 through a :ref:`well-documented messaging protocol <messaging>` using the
73 73 ZeroMQ networking library.
74 74
75 75 * **Refactoring.** the entire codebase has been refactored, in order to make it
76 76 more modular and easier to contribute to. IPython has traditionally been a
77 77 hard project to participate because the old codebase was very monolithic. We
78 78 hope this (ongoing) restructuring will make it easier for new developers to
79 79 join us.
80 80
81 81 * **Vim integration**. Vim can be configured to seamlessly control an IPython
82 82 kernel, see the files in :file:`docs/examples/vim` for the full details.
83 83 This work was done by Paul Ivanov, who prepared a nice `video
84 84 demonstration`__ of the features it provides.
85 85
86 86 .. __: http://pirsquared.org/blog/2011/07/28/vim-ipython/
87 87
88 88 * **Integration into Microsoft Visual Studio**. Thanks to the work of the
89 89 Microsoft `Python Tools for Visual Studio`__ team, this version of IPython
90 90 has been integrated into Microsoft Visual Studio's Python tools open source
91 91 plug-in. `Details below`_
92 92
93 93 .. __: http://pytools.codeplex.com
94 94 .. _details below: ms_visual_studio_011_
95 95
96 96 * **Improved unicode support**. We closed many bugs related to unicode input.
97 97
98 98 * **Python 3**. IPython now runs on Python 3.x. See :ref:`python3_011` for
99 99 details.
100 100
101 101 * **New profile model**. Profiles are now directories that contain all relevant
102 102 information for that session, and thus better isolate IPython use-cases.
103 103
104 104 * **SQLite storage for history**. All history is now stored in a SQLite
105 105 database, providing support for multiple simultaneous sessions that won't
106 106 clobber each other as well as the ability to perform queries on all stored
107 107 data.
108 108
109 109 * **New configuration system**. All parts of IPython are now configured via a
110 110 mechanism inspired by the Enthought Traits library. Any configurable element
111 111 can have its attributes set either via files that now use real Python syntax
112 112 or from the command-line.
113 113
114 114 * **Pasting of code with prompts**. IPython now intelligently strips out input
115 115 prompts , be they plain Python ones (``>>>`` and ``...``) or IPython ones
116 116 (``In [N]:`` and ``...:``). More details :ref:`here <pasting_with_prompts>`.
117 117
118 118
119 119 Authors and support
120 120 -------------------
121 121
122 122 Over 60 separate authors have contributed to this release, see :ref:`below
123 123 <credits_011>` for a full list. In particular, we want to highlight the
124 124 extremely active participation of two new core team members: Evan Patterson
125 125 implemented the Qt console, and Thomas Kluyver started with our Python 3 port
126 126 and by now has made major contributions to just about every area of IPython.
127 127
128 128 We are also grateful for the support we have received during this development
129 129 cycle from several institutions:
130 130
131 131 - `Enthought Inc`__ funded the development of our new Qt console, an effort that
132 132 required developing major pieces of underlying infrastructure, which now
133 133 power not only the Qt console but also our new parallel machinery. We'd like
134 134 to thank Eric Jones and Travis Oliphant for their support, as well as Ilan
135 135 Schnell for his tireless work integrating and testing IPython in the
136 136 `Enthought Python Distribution`_.
137 137
138 138 .. __: http://enthought.com
139 139 .. _Enthought Python Distribution: http://www.enthought.com/products/epd.php
140 140
141 141 - Nipy/NIH: funding via the `NiPy project`__ (NIH grant 5R01MH081909-02) helped
142 142 us jumpstart the development of this series by restructuring the entire
143 143 codebase two years ago in a way that would make modular development and
144 144 testing more approachable. Without this initial groundwork, all the new
145 145 features we have added would have been impossible to develop.
146 146
147 147 .. __: http://nipy.org
148 148
149 149 - Sage/NSF: funding via the grant `Sage: Unifying Mathematical Software for
150 150 Scientists, Engineers, and Mathematicians`__ (NSF grant DMS-1015114)
151 151 supported a meeting in spring 2011 of several of the core IPython developers
152 152 where major progress was made integrating the last key pieces leading to this
153 153 release.
154 154
155 155 .. __: http://modular.math.washington.edu/grants/compmath09
156 156
157 157 - Microsoft's team working on `Python Tools for Visual Studio`__ developed the
158 158 integraton of IPython into the Python plugin for Visual Studio 2010.
159 159
160 160 .. __: http://pytools.codeplex.com
161 161
162 162 - Google Summer of Code: in 2010, we had two students developing prototypes of
163 163 the new machinery that is now maturing in this release: `Omar Zapata`_ and
164 164 `Gerardo Gutiérrez`_.
165 165
166 166 .. _Omar Zapata: http://ipythonzmq.blogspot.com/2010/08/ipython-zmq-status.html
167 167 .. _Gerardo Gutiérrez: http://ipythonqt.blogspot.com/2010/04/ipython-qt-interface-gsoc-2010-proposal.html>
168 168
169 169
170 170 Development summary: moving to Git and Github
171 171 ---------------------------------------------
172 172
173 173 In April 2010, after `one breakage too many with bzr`__, we decided to move our
174 174 entire development process to Git and Github.com. This has proven to be one of
175 175 the best decisions in the project's history, as the combination of git and
176 176 github have made us far, far more productive than we could be with our previous
177 177 tools. We first converted our bzr repo to a git one without losing history,
178 178 and a few weeks later ported all open Launchpad bugs to github issues with
179 179 their comments mostly intact (modulo some formatting changes). This ensured a
180 180 smooth transition where no development history or submitted bugs were lost.
181 181 Feel free to use our little Launchpad to Github issues `porting script`_ if you
182 182 need to make a similar transition.
183 183
184 184 .. __: http://mail.scipy.org/pipermail/ipython-dev/2010-April/005944.html
185 185 .. _porting script: https://gist.github.com/835577
186 186
187 187 These simple statistics show how much work has been done on the new release, by
188 188 comparing the current code to the last point it had in common with the 0.10
189 189 series. A huge diff and ~2200 commits make up this cycle::
190 190
191 191 git diff $(git merge-base 0.10.2 HEAD) | wc -l
192 192 288019
193 193
194 194 git log $(git merge-base 0.10.2 HEAD)..HEAD --oneline | wc -l
195 195 2200
196 196
197 197 Since our move to github, 511 issues were closed, 226 of which were pull
198 198 requests and 285 regular issues (:ref:`a full list with links
199 199 <issues_list_011>` is available for those interested in the details). Github's
200 200 pull requests are a fantastic mechanism for reviewing code and building a
201 201 shared ownership of the project, and we are making enthusiastic use of it.
202 202
203 203 .. Note::
204 204
205 205 This undercounts the number of issues closed in this development cycle,
206 206 since we only moved to github for issue tracking in May 2010, but we have no
207 207 way of collecting statistics on the number of issues closed in the old
208 208 Launchpad bug tracker prior to that.
209 209
210 210
211 211 .. _qtconsole_011:
212 212
213 213 Qt Console
214 214 ----------
215 215
216 216 IPython now ships with a Qt application that feels very much like a terminal,
217 217 but is in fact a rich GUI that runs an IPython client but supports inline
218 218 figures, saving sessions to PDF and HTML, multiline editing with syntax
219 219 highlighting, graphical calltips and much more:
220 220
221 .. figure:: ../_static/qtconsole.png
221 .. figure:: ../_images/qtconsole.png
222 222 :width: 400px
223 223 :alt: IPython Qt console with embedded plots
224 224 :align: center
225 :target: ../_static/qtconsole.png
225 :target: ../_images/qtconsole.png
226 226
227 227 The Qt console for IPython, using inline matplotlib plots.
228 228
229 229 We hope that many projects will embed this widget, which we've kept
230 230 deliberately very lightweight, into their own environments. In the future we
231 231 may also offer a slightly more featureful application (with menus and other GUI
232 232 elements), but we remain committed to always shipping this easy to embed
233 233 widget.
234 234
235 235 See the :ref:`Qt console section <qtconsole>` of the docs for a detailed
236 236 description of the console's features and use.
237 237
238 238
239 239 .. _parallel_011:
240 240
241 241 High-level parallel computing with ZeroMQ
242 242 -----------------------------------------
243 243
244 244 We have completely rewritten the Twisted-based code for high-level parallel
245 245 computing to work atop our new ZeroMQ architecture. While we realize this will
246 246 break compatibility for a number of users, we hope to make the transition as
247 247 easy as possible with our docs, and we are convinced the change is worth it.
248 248 ZeroMQ provides us with much tighter control over memory, higher performance,
249 249 and its communications are impervious to the Python Global Interpreter Lock
250 250 because they take place in a system-level C++ thread. The impact of the GIL in
251 251 our previous code was something we could simply not work around, given that
252 252 Twisted is itself a Python library. So while Twisted is a very capable
253 253 framework, we think ZeroMQ fits our needs much better and we hope you will find
254 254 the change to be a significant improvement in the long run.
255 255
256 256 Our manual contains :ref:`a full description of how to use IPython for parallel
257 257 computing <parallel_overview>`, and the `tutorial`__ presented by Min
258 258 Ragan-Kelley at the SciPy 2011 conference provides a hands-on complement to the
259 259 reference docs.
260 260
261 261 .. __: http://minrk.github.com/scipy-tutorial-2011
262 262
263 263
264 264 Refactoring
265 265 -----------
266 266
267 267 As of this release, a signifiant portion of IPython has been refactored. This
268 268 refactoring is founded on a number of new abstractions. The main new classes
269 269 that implement these abstractions are:
270 270
271 271 * :class:`IPython.utils.traitlets.HasTraits`.
272 272 * :class:`IPython.config.configurable.Configurable`.
273 273 * :class:`IPython.config.application.Application`.
274 274 * :class:`IPython.config.loader.ConfigLoader`.
275 275 * :class:`IPython.config.loader.Config`
276 276
277 277 We are still in the process of writing developer focused documentation about
278 278 these classes, but for now our :ref:`configuration documentation
279 279 <config_overview>` contains a high level overview of the concepts that these
280 280 classes express.
281 281
282 282 The biggest user-visible change is likely the move to using the config system
283 283 to determine the command-line arguments for IPython applications. The benefit
284 284 of this is that *all* configurable values in IPython are exposed on the
285 285 command-line, but the syntax for specifying values has changed. The gist is
286 286 that assigning values is pure Python assignment. Simple flags exist for
287 287 commonly used options, these are always prefixed with '--'.
288 288
289 289 The IPython command-line help has the details of all the options (via
290 290 ``ipythyon --help``), but a simple example should clarify things; the ``pylab``
291 291 flag can be used to start in pylab mode with the qt4 backend::
292 292
293 293 ipython --pylab=qt
294 294
295 295 which is equivalent to using the fully qualified form::
296 296
297 297 ipython --TerminalIPythonApp.pylab=qt
298 298
299 299 The long-form options can be listed via ``ipython --help-all``.
300 300
301 301
302 302 ZeroMQ architecture
303 303 -------------------
304 304
305 305 There is a new GUI framework for IPython, based on a client-server model in
306 306 which multiple clients can communicate with one IPython kernel, using the
307 307 ZeroMQ messaging framework. There is already a Qt console client, which can
308 308 be started by calling ``ipython qtconsole``. The protocol is :ref:`documented
309 309 <messaging>`.
310 310
311 311 The parallel computing framework has also been rewritten using ZMQ. The
312 312 protocol is described :ref:`here <parallel_messages>`, and the code is in the
313 313 new :mod:`IPython.parallel` module.
314 314
315 315 .. _python3_011:
316 316
317 317 Python 3 support
318 318 ----------------
319 319
320 320 A Python 3 version of IPython has been prepared. For the time being, this is
321 321 maintained separately and updated from the main codebase. Its code can be found
322 322 `here <https://github.com/ipython/ipython-py3k>`_. The parallel computing
323 323 components are not perfect on Python3, but most functionality appears to be
324 324 working. As this work is evolving quickly, the best place to find updated
325 325 information about it is our `Python 3 wiki page`__.
326 326
327 327 .. __: http://wiki.ipython.org/index.php?title=Python_3
328 328
329 329
330 330 Unicode
331 331 -------
332 332
333 333 Entering non-ascii characters in unicode literals (``u"€ø"``) now works
334 334 properly on all platforms. However, entering these in byte/string literals
335 335 (``"€ø"``) will not work as expected on Windows (or any platform where the
336 336 terminal encoding is not UTF-8, as it typically is for Linux & Mac OS X). You
337 337 can use escape sequences (``"\xe9\x82"``) to get bytes above 128, or use
338 338 unicode literals and encode them. This is a limitation of Python 2 which we
339 339 cannot easily work around.
340 340
341 341 .. _ms_visual_studio_011:
342 342
343 343 Integration with Microsoft Visual Studio
344 344 ----------------------------------------
345 345
346 346 IPython can be used as the interactive shell in the `Python plugin for
347 347 Microsoft Visual Studio`__, as seen here:
348 348
349 .. figure:: ../_static/ms_visual_studio.png
349 .. figure:: ../_images/ms_visual_studio.png
350 350 :width: 500px
351 351 :alt: IPython console embedded in Microsoft Visual Studio.
352 352 :align: center
353 :target: ../_static/ms_visual_studio.png
353 :target: ../_images/ms_visual_studio.png
354 354
355 355 IPython console embedded in Microsoft Visual Studio.
356 356
357 357 The Microsoft team developing this currently has a release candidate out using
358 358 IPython 0.11. We will continue to collaborate with them to ensure that as they
359 359 approach their final release date, the integration with IPython remains smooth.
360 360 We'd like to thank Dino Viehland and Shahrokh Mortazavi for the work they have
361 361 done towards this feature, as well as Wenming Ye for his support of our WinHPC
362 362 capabilities.
363 363
364 364 .. __: http://pytools.codeplex.com
365 365
366 366
367 367 Additional new features
368 368 -----------------------
369 369
370 370 * Added ``Bytes`` traitlet, removing ``Str``. All 'string' traitlets should
371 371 either be ``Unicode`` if a real string, or ``Bytes`` if a C-string. This
372 372 removes ambiguity and helps the Python 3 transition.
373 373
374 374 * New magic ``%loadpy`` loads a python file from disk or web URL into
375 375 the current input buffer.
376 376
377 377 * New magic ``%pastebin`` for sharing code via the 'Lodge it' pastebin.
378 378
379 379 * New magic ``%precision`` for controlling float and numpy pretty printing.
380 380
381 381 * IPython applications initiate logging, so any object can gain access to
382 382 a the logger of the currently running Application with:
383 383
384 384 .. sourcecode:: python
385 385
386 386 from IPython.config.application import Application
387 387 logger = Application.instance().log
388 388
389 389 * You can now get help on an object halfway through typing a command. For
390 390 instance, typing ``a = zip?`` shows the details of :func:`zip`. It also
391 391 leaves the command at the next prompt so you can carry on with it.
392 392
393 393 * The input history is now written to an SQLite database. The API for
394 394 retrieving items from the history has also been redesigned.
395 395
396 396 * The :mod:`IPython.extensions.pretty` extension has been moved out of
397 397 quarantine and fully updated to the new extension API.
398 398
399 399 * New magics for loading/unloading/reloading extensions have been added:
400 400 ``%load_ext``, ``%unload_ext`` and ``%reload_ext``.
401 401
402 402 * The configuration system and configuration files are brand new. See the
403 403 configuration system :ref:`documentation <config_index>` for more details.
404 404
405 405 * The :class:`~IPython.core.interactiveshell.InteractiveShell` class is now a
406 406 :class:`~IPython.config.configurable.Configurable` subclass and has traitlets
407 407 that determine the defaults and runtime environment. The ``__init__`` method
408 408 has also been refactored so this class can be instantiated and run without
409 409 the old :mod:`ipmaker` module.
410 410
411 411 * The methods of :class:`~IPython.core.interactiveshell.InteractiveShell` have
412 412 been organized into sections to make it easier to turn more sections
413 413 of functionality into components.
414 414
415 415 * The embedded shell has been refactored into a truly standalone subclass of
416 416 :class:`InteractiveShell` called :class:`InteractiveShellEmbed`. All
417 417 embedding logic has been taken out of the base class and put into the
418 418 embedded subclass.
419 419
420 420 * Added methods of :class:`~IPython.core.interactiveshell.InteractiveShell` to
421 421 help it cleanup after itself. The :meth:`cleanup` method controls this. We
422 422 couldn't do this in :meth:`__del__` because we have cycles in our object
423 423 graph that prevent it from being called.
424 424
425 425 * Created a new module :mod:`IPython.utils.importstring` for resolving
426 426 strings like ``foo.bar.Bar`` to the actual class.
427 427
428 428 * Completely refactored the :mod:`IPython.core.prefilter` module into
429 429 :class:`~IPython.config.configurable.Configurable` subclasses. Added a new
430 430 layer into the prefilter system, called "transformations" that all new
431 431 prefilter logic should use (rather than the older "checker/handler"
432 432 approach).
433 433
434 434 * Aliases are now components (:mod:`IPython.core.alias`).
435 435
436 436 * New top level :func:`~IPython.frontend.terminal.embed.embed` function that can
437 437 be called to embed IPython at any place in user's code. On the first call it
438 438 will create an :class:`~IPython.frontend.terminal.embed.InteractiveShellEmbed`
439 439 instance and call it. In later calls, it just calls the previously created
440 440 :class:`~IPython.frontend.terminal.embed.InteractiveShellEmbed`.
441 441
442 442 * Created a configuration system (:mod:`IPython.config.configurable`) that is
443 443 based on :mod:`IPython.utils.traitlets`. Configurables are arranged into a
444 444 runtime containment tree (not inheritance) that i) automatically propagates
445 445 configuration information and ii) allows singletons to discover each other in
446 446 a loosely coupled manner. In the future all parts of IPython will be
447 447 subclasses of :class:`~IPython.config.configurable.Configurable`. All IPython
448 448 developers should become familiar with the config system.
449 449
450 450 * Created a new :class:`~IPython.config.loader.Config` for holding
451 451 configuration information. This is a dict like class with a few extras: i)
452 452 it supports attribute style access, ii) it has a merge function that merges
453 453 two :class:`~IPython.config.loader.Config` instances recursively and iii) it
454 454 will automatically create sub-:class:`~IPython.config.loader.Config`
455 455 instances for attributes that start with an uppercase character.
456 456
457 457 * Created new configuration loaders in :mod:`IPython.config.loader`. These
458 458 loaders provide a unified loading interface for all configuration
459 459 information including command line arguments and configuration files. We
460 460 have two default implementations based on :mod:`argparse` and plain python
461 461 files. These are used to implement the new configuration system.
462 462
463 463 * Created a top-level :class:`Application` class in
464 464 :mod:`IPython.core.application` that is designed to encapsulate the starting
465 465 of any basic Python program. An application loads and merges all the
466 466 configuration objects, constructs the main application, configures and
467 467 initiates logging, and creates and configures any :class:`Configurable`
468 468 instances and then starts the application running. An extended
469 469 :class:`BaseIPythonApplication` class adds logic for handling the
470 470 IPython directory as well as profiles, and all IPython entry points
471 471 extend it.
472 472
473 473 * The :class:`Type` and :class:`Instance` traitlets now handle classes given
474 474 as strings, like ``foo.bar.Bar``. This is needed for forward declarations.
475 475 But, this was implemented in a careful way so that string to class
476 476 resolution is done at a single point, when the parent
477 477 :class:`~IPython.utils.traitlets.HasTraitlets` is instantiated.
478 478
479 479 * :mod:`IPython.utils.ipstruct` has been refactored to be a subclass of
480 480 dict. It also now has full docstrings and doctests.
481 481
482 482 * Created a Traits like implementation in :mod:`IPython.utils.traitlets`. This
483 483 is a pure Python, lightweight version of a library that is similar to
484 484 Enthought's Traits project, but has no dependencies on Enthought's code. We
485 485 are using this for validation, defaults and notification in our new component
486 486 system. Although it is not 100% API compatible with Enthought's Traits, we
487 487 plan on moving in this direction so that eventually our implementation could
488 488 be replaced by a (yet to exist) pure Python version of Enthought Traits.
489 489
490 490 * Added a new module :mod:`IPython.lib.inputhook` to manage the integration
491 491 with GUI event loops using `PyOS_InputHook`. See the docstrings in this
492 492 module or the main IPython docs for details.
493 493
494 494 * For users, GUI event loop integration is now handled through the new
495 495 :command:`%gui` magic command. Type ``%gui?`` at an IPython prompt for
496 496 documentation.
497 497
498 498 * For developers :mod:`IPython.lib.inputhook` provides a simple interface
499 499 for managing the event loops in their interactive GUI applications.
500 500 Examples can be found in our :file:`docs/examples/lib` directory.
501 501
502 502 Backwards incompatible changes
503 503 ------------------------------
504 504
505 505 * The Twisted-based :mod:`IPython.kernel` has been removed, and completely
506 506 rewritten as :mod:`IPython.parallel`, using ZeroMQ.
507 507
508 508 * Profiles are now directories. Instead of a profile being a single config file,
509 509 profiles are now self-contained directories. By default, profiles get their
510 510 own IPython history, log files, and everything. To create a new profile, do
511 511 ``ipython profile create <name>``.
512 512
513 513 * All IPython applications have been rewritten to use
514 514 :class:`~IPython.config.loader.KeyValueConfigLoader`. This means that
515 515 command-line options have changed. Now, all configurable values are accessible
516 516 from the command-line with the same syntax as in a configuration file.
517 517
518 518 * The command line options ``-wthread``, ``-qthread`` and
519 519 ``-gthread`` have been removed. Use ``--gui=wx``, ``--gui=qt``, ``--gui=gtk``
520 520 instead.
521 521
522 522 * The extension loading functions have been renamed to
523 523 :func:`load_ipython_extension` and :func:`unload_ipython_extension`.
524 524
525 525 * :class:`~IPython.core.interactiveshell.InteractiveShell` no longer takes an
526 526 ``embedded`` argument. Instead just use the
527 527 :class:`~IPython.core.interactiveshell.InteractiveShellEmbed` class.
528 528
529 529 * ``__IPYTHON__`` is no longer injected into ``__builtin__``.
530 530
531 531 * :meth:`Struct.__init__` no longer takes `None` as its first argument. It
532 532 must be a :class:`dict` or :class:`Struct`.
533 533
534 534 * :meth:`~IPython.core.interactiveshell.InteractiveShell.ipmagic` has been
535 535 renamed :meth:`~IPython.core.interactiveshell.InteractiveShell.magic.`
536 536
537 537 * The functions :func:`ipmagic` and :func:`ipalias` have been removed from
538 538 :mod:`__builtins__`.
539 539
540 540 * The references to the global
541 541 :class:`~IPython.core.interactivehell.InteractiveShell` instance (``_ip``, and
542 542 ``__IP``) have been removed from the user's namespace. They are replaced by a
543 543 new function called :func:`get_ipython` that returns the current
544 544 :class:`~IPython.core.interactiveshell.InteractiveShell` instance. This
545 545 function is injected into the user's namespace and is now the main way of
546 546 accessing the running IPython.
547 547
548 548 * Old style configuration files :file:`ipythonrc` and :file:`ipy_user_conf.py`
549 549 are no longer supported. Users should migrate there configuration files to
550 550 the new format described :ref:`here <config_overview>` and :ref:`here
551 551 <configuring_ipython>`.
552 552
553 553 * The old IPython extension API that relied on :func:`ipapi` has been
554 554 completely removed. The new extension API is described :ref:`here
555 555 <configuring_ipython>`.
556 556
557 557 * Support for ``qt3`` has been dropped. Users who need this should use
558 558 previous versions of IPython.
559 559
560 560 * Removed :mod:`shellglobals` as it was obsolete.
561 561
562 562 * Removed all the threaded shells in :mod:`IPython.core.shell`. These are no
563 563 longer needed because of the new capabilities in
564 564 :mod:`IPython.lib.inputhook`.
565 565
566 566 * New top-level sub-packages have been created: :mod:`IPython.core`,
567 567 :mod:`IPython.lib`, :mod:`IPython.utils`, :mod:`IPython.deathrow`,
568 568 :mod:`IPython.quarantine`. All existing top-level modules have been
569 569 moved to appropriate sub-packages. All internal import statements
570 570 have been updated and tests have been added. The build system (setup.py
571 571 and friends) have been updated. See :ref:`this section <module_reorg>` of the
572 572 documentation for descriptions of these new sub-packages.
573 573
574 574 * :mod:`IPython.ipapi` has been moved to :mod:`IPython.core.ipapi`.
575 575 :mod:`IPython.Shell` and :mod:`IPython.iplib` have been split and removed as
576 576 part of the refactor.
577 577
578 578 * :mod:`Extensions` has been moved to :mod:`extensions` and all existing
579 579 extensions have been moved to either :mod:`IPython.quarantine` or
580 580 :mod:`IPython.deathrow`. :mod:`IPython.quarantine` contains modules that we
581 581 plan on keeping but that need to be updated. :mod:`IPython.deathrow` contains
582 582 modules that are either dead or that should be maintained as third party
583 583 libraries. More details about this can be found :ref:`here <module_reorg>`.
584 584
585 585 * Previous IPython GUIs in :mod:`IPython.frontend` and :mod:`IPython.gui` are
586 586 likely broken, and have been removed to :mod:`IPython.deathrow` because of the
587 587 refactoring in the core. With proper updates, these should still work.
588 588
589 589
590 590 Known Regressions
591 591 -----------------
592 592
593 593 We do our best to improve IPython, but there are some known regressions in 0.11
594 594 relative to 0.10.2. First of all, there are features that have yet to be
595 595 ported to the new APIs, and in order to ensure that all of the installed code
596 596 runs for our users, we have moved them to two separate directories in the
597 597 source distribution, `quarantine` and `deathrow`. Finally, we have some other
598 598 miscellaneous regressions that we hope to fix as soon as possible. We now
599 599 describe all of these in more detail.
600 600
601 601 Quarantine
602 602 ~~~~~~~~~~
603 603
604 604 These are tools and extensions that we consider relatively easy to update to
605 605 the new classes and APIs, but that we simply haven't had time for. Any user
606 606 who is interested in one of these is encouraged to help us by porting it and
607 607 submitting a pull request on our `development site`_.
608 608
609 609 .. _development site: http://github.com/ipython/ipython
610 610
611 611 Currently, the quarantine directory contains::
612 612
613 613 clearcmd.py ipy_fsops.py ipy_signals.py
614 614 envpersist.py ipy_gnuglobal.py ipy_synchronize_with.py
615 615 ext_rescapture.py ipy_greedycompleter.py ipy_system_conf.py
616 616 InterpreterExec.py ipy_jot.py ipy_which.py
617 617 ipy_app_completers.py ipy_lookfor.py ipy_winpdb.py
618 618 ipy_autoreload.py ipy_profile_doctest.py ipy_workdir.py
619 619 ipy_completers.py ipy_pydb.py jobctrl.py
620 620 ipy_editors.py ipy_rehashdir.py ledit.py
621 621 ipy_exportdb.py ipy_render.py pspersistence.py
622 622 ipy_extutil.py ipy_server.py win32clip.py
623 623
624 624 Deathrow
625 625 ~~~~~~~~
626 626
627 627 These packages may be harder to update or make most sense as third-party
628 628 libraries. Some of them are completely obsolete and have been already replaced
629 629 by better functionality (we simply haven't had the time to carefully weed them
630 630 out so they are kept here for now). Others simply require fixes to code that
631 631 the current core team may not be familiar with. If a tool you were used to is
632 632 included here, we encourage you to contact the dev list and we can discuss
633 633 whether it makes sense to keep it in IPython (if it can be maintained).
634 634
635 635 Currently, the deathrow directory contains::
636 636
637 637 astyle.py ipy_defaults.py ipy_vimserver.py
638 638 dtutils.py ipy_kitcfg.py numeric_formats.py
639 639 Gnuplot2.py ipy_legacy.py numutils.py
640 640 GnuplotInteractive.py ipy_p4.py outputtrap.py
641 641 GnuplotRuntime.py ipy_profile_none.py PhysicalQInput.py
642 642 ibrowse.py ipy_profile_numpy.py PhysicalQInteractive.py
643 643 igrid.py ipy_profile_scipy.py quitter.py*
644 644 ipipe.py ipy_profile_sh.py scitedirector.py
645 645 iplib.py ipy_profile_zope.py Shell.py
646 646 ipy_constants.py ipy_traits_completer.py twshell.py
647 647
648 648
649 649 Other regressions
650 650 ~~~~~~~~~~~~~~~~~
651 651
652 652 * The machinery that adds functionality to the 'sh' profile for using IPython
653 653 as your system shell has not been updated to use the new APIs. As a result,
654 654 only the aesthetic (prompt) changes are still implemented. We intend to fix
655 655 this by 0.12. Tracked as issue 547_.
656 656
657 657 .. _547: https://github.com/ipython/ipython/issues/547
658 658
659 659 * The installation of scripts on Windows was broken without setuptools, so we
660 660 now depend on setuptools on Windows. We hope to fix setuptools-less
661 661 installation, and then remove the setuptools dependency. Issue 539_.
662 662
663 663 .. _539: https://github.com/ipython/ipython/issues/539
664 664
665 665 * The directory history `_dh` is not saved between sessions. Issue 634_.
666 666
667 667 .. _634: https://github.com/ipython/ipython/issues/634
668 668
669 669
670 670 Removed Features
671 671 ----------------
672 672
673 673 As part of the updating of IPython, we have removed a few features for the
674 674 purposes of cleaning up the codebase and interfaces. These removals are
675 675 permanent, but for any item listed below, equivalent functionality is
676 676 available.
677 677
678 678 * The magics Exit and Quit have been dropped as ways to exit IPython. Instead,
679 679 the lowercase forms of both work either as a bare name (``exit``) or a
680 680 function call (``exit()``). You can assign these to other names using
681 681 exec_lines in the config file.
682 682
683 683
684 684 .. _credits_011:
685 685
686 686 Credits
687 687 -------
688 688
689 689 Many users and developers contributed code, features, bug reports and ideas to
690 690 this release. Please do not hesitate in contacting us if we've failed to
691 691 acknowledge your contribution here. In particular, for this release we have
692 692 contribution from the following people, a mix of new and regular names (in
693 693 alphabetical order by first name):
694 694
695 695 * Aenugu Sai Kiran Reddy <saikrn08-at-gmail.com>
696 696 * andy wilson <wilson.andrew.j+github-at-gmail.com>
697 697 * Antonio Cuni <antocuni>
698 698 * Barry Wark <barrywark-at-gmail.com>
699 699 * Beetoju Anuradha <anu.beethoju-at-gmail.com>
700 700 * Benjamin Ragan-Kelley <minrk-at-Mercury.local>
701 701 * Brad Reisfeld
702 702 * Brian E. Granger <ellisonbg-at-gmail.com>
703 703 * Christoph Gohlke <cgohlke-at-uci.edu>
704 704 * Cody Precord
705 705 * dan.milstein
706 706 * Darren Dale <dsdale24-at-gmail.com>
707 707 * Dav Clark <davclark-at-berkeley.edu>
708 708 * David Warde-Farley <wardefar-at-iro.umontreal.ca>
709 709 * epatters <ejpatters-at-gmail.com>
710 710 * epatters <epatters-at-caltech.edu>
711 711 * epatters <epatters-at-enthought.com>
712 712 * Eric Firing <efiring-at-hawaii.edu>
713 713 * Erik Tollerud <erik.tollerud-at-gmail.com>
714 714 * Evan Patterson <epatters-at-enthought.com>
715 715 * Fernando Perez <Fernando.Perez-at-berkeley.edu>
716 716 * Gael Varoquaux <gael.varoquaux-at-normalesup.org>
717 717 * Gerardo <muzgash-at-Muzpelheim>
718 718 * Jason Grout <jason.grout-at-drake.edu>
719 719 * John Hunter <jdh2358-at-gmail.com>
720 720 * Jens Hedegaard Nielsen <jenshnielsen-at-gmail.com>
721 721 * Johann Cohen-Tanugi <johann.cohentanugi-at-gmail.com>
722 722 * Jörgen Stenarson <jorgen.stenarson-at-bostream.nu>
723 723 * Justin Riley <justin.t.riley-at-gmail.com>
724 724 * Kiorky
725 725 * Laurent Dufrechou <laurent.dufrechou-at-gmail.com>
726 726 * Luis Pedro Coelho <lpc-at-cmu.edu>
727 727 * Mani chandra <mchandra-at-iitk.ac.in>
728 728 * Mark E. Smith
729 729 * Mark Voorhies <mark.voorhies-at-ucsf.edu>
730 730 * Martin Spacek <git-at-mspacek.mm.st>
731 731 * Michael Droettboom <mdroe-at-stsci.edu>
732 732 * MinRK <benjaminrk-at-gmail.com>
733 733 * muzuiget <muzuiget-at-gmail.com>
734 734 * Nick Tarleton <nick-at-quixey.com>
735 735 * Nicolas Rougier <Nicolas.rougier-at-inria.fr>
736 736 * Omar Andres Zapata Mesa <andresete.chaos-at-gmail.com>
737 737 * Paul Ivanov <pivanov314-at-gmail.com>
738 738 * Pauli Virtanen <pauli.virtanen-at-iki.fi>
739 739 * Prabhu Ramachandran
740 740 * Ramana <sramana9-at-gmail.com>
741 741 * Robert Kern <robert.kern-at-gmail.com>
742 742 * Sathesh Chandra <satheshchandra88-at-gmail.com>
743 743 * Satrajit Ghosh <satra-at-mit.edu>
744 744 * Sebastian Busch
745 745 * Skipper Seabold <jsseabold-at-gmail.com>
746 746 * Stefan van der Walt <bzr-at-mentat.za.net>
747 747 * Stephan Peijnik <debian-at-sp.or.at>
748 748 * Steven Bethard
749 749 * Thomas Kluyver <takowl-at-gmail.com>
750 750 * Thomas Spura <tomspur-at-fedoraproject.org>
751 751 * Tom Fetherston <tfetherston-at-aol.com>
752 752 * Tom MacWright
753 753 * tzanko
754 754 * vankayala sowjanya <hai.sowjanya-at-gmail.com>
755 755 * Vivian De Smedt <vds2212-at-VIVIAN>
756 756 * Ville M. Vainio <vivainio-at-gmail.com>
757 757 * Vishal Vatsa <vishal.vatsa-at-gmail.com>
758 758 * Vishnu S G <sgvishnu777-at-gmail.com>
759 759 * Walter Doerwald <walter-at-livinglogic.de>
760 760
761 761 .. note::
762 762
763 763 This list was generated with the output of
764 764 ``git log dev-0.11 HEAD --format='* %aN <%aE>' | sed 's/@/\-at\-/' | sed 's/<>//' | sort -u``
765 765 after some cleanup. If you should be on this list, please add yourself.
@@ -1,370 +1,370
1 1 =============
2 2 0.12 Series
3 3 =============
4 4
5 5 Release 0.12.1
6 6 ==============
7 7
8 8 IPython 0.12.1 is a bugfix release of 0.12, pulling only bugfixes and minor
9 9 cleanup from 0.13, timed for the Ubuntu 12.04 LTS release.
10 10
11 11 See the :ref:`list of fixed issues <issues_list_012>` for specific backported issues.
12 12
13 13
14 14 Release 0.12
15 15 ============
16 16
17 17 IPython 0.12 contains several major new features, as well as a large amount of
18 18 bug and regression fixes. The 0.11 release brought with it a lot of new
19 19 functionality and major refactorings of the codebase; by and large this has
20 20 proven to be a success as the number of contributions to the project has
21 21 increased dramatically, proving that the code is now much more approachable.
22 22 But in the refactoring inevitably some bugs were introduced, and we have also
23 23 squashed many of those as well as recovered some functionality that had been
24 24 temporarily disabled due to the API changes.
25 25
26 26 The following major new features appear in this version.
27 27
28 28
29 29 An interactive browser-based Notebook with rich media support
30 30 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
31 31
32 32 A powerful new interface puts IPython in your browser. You can start it with
33 33 the command ``ipython notebook``:
34 34
35 .. figure:: ../_static/notebook_specgram.png
35 .. figure:: ../_images/notebook_specgram.png
36 36 :width: 400px
37 37 :alt: The IPython notebook with embedded text, code, math and figures.
38 38 :align: center
39 :target: ../_static/notebook_specgram.png
39 :target: ../_images/notebook_specgram.png
40 40
41 41 The new IPython notebook showing text, mathematical expressions in LaTeX,
42 42 code, results and embedded figures created with Matplotlib.
43 43
44 44 This new interface maintains all the features of IPython you are used to, as it
45 45 is a new client that communicates with the same IPython kernels used by the
46 46 terminal and Qt console. But the web notebook provides for a different
47 47 workflow where you can integrate, along with code execution, also text,
48 48 mathematical expressions, graphics, video, and virtually any content that a
49 49 modern browser is capable of displaying.
50 50
51 51 You can save your work sessions as documents that retain all these elements and
52 52 which can be version controlled, emailed to colleagues or saved as HTML or PDF
53 53 files for printing or publishing statically on the web. The internal storage
54 54 format is a JSON file that can be easily manipulated for manual exporting to
55 55 other formats.
56 56
57 57 This Notebook is a major milestone for IPython, as for years we have tried to
58 58 build this kind of system. We were inspired originally by the excellent
59 59 implementation in Mathematica, we made a number of attempts using older
60 60 technologies in earlier Summer of Code projects in 2005 (both students and
61 61 Robert Kern developed early prototypes), and in recent years we have seen the
62 62 excellent implementation offered by the `Sage <http://sagemath.org>` system.
63 63 But we continued to work on something that would be consistent with the rest of
64 64 IPython's design, and it is clear now that the effort was worth it: based on
65 65 the ZeroMQ communications architecture introduced in version 0.11, the notebook
66 66 can now retain 100% of the features of the real IPython. But it can also
67 67 provide the rich media support and high quality Javascript libraries that were
68 68 not available in browsers even one or two years ago (such as high-quality
69 69 mathematical rendering or built-in video).
70 70
71 71 The notebook has too many useful and important features to describe in these
72 72 release notes; our documentation now contains a directory called
73 73 ``examples/notebooks`` with several notebooks that illustrate various aspects
74 74 of the system. You should start by reading those named
75 75 ``00_notebook_tour.ipynb`` and ``01_notebook_introduction.ipynb`` first, and
76 76 then can proceed to read the others in any order you want.
77 77
78 78 To start the notebook server, go to a directory containing the notebooks you
79 79 want to open (or where you want to create new ones) and type::
80 80
81 81 ipython notebook
82 82
83 83 You can see all the relevant options with::
84 84
85 85 ipython notebook --help
86 86 ipython notebook --help-all # even more
87 87
88 88 and just like the Qt console, you can start the notebook server with pylab
89 89 support by using::
90 90
91 91 ipython notebook --pylab
92 92
93 93 for floating matplotlib windows or::
94 94
95 95 ipython notebook --pylab inline
96 96
97 97 for plotting support with automatically inlined figures. Note that it is now
98 98 possible also to activate pylab support at runtime via ``%pylab``, so you do
99 99 not need to make this decision when starting the server.
100 100
101 101 See :ref:`the Notebook docs <htmlnotebook>` for technical details.
102 102
103 103 .. _two_process_console:
104 104
105 105 Two-process terminal console
106 106 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
107 107
108 108 Based on the same architecture as the notebook and the Qt console, we also have
109 109 now a terminal-based console that can connect to an external IPython kernel
110 110 (the same kernels used by the Qt console or the notebook, in fact). While this
111 111 client behaves almost identically to the usual IPython terminal application,
112 112 this capability can be very useful to attach an interactive console to an
113 113 existing kernel that was started externally. It lets you use the interactive
114 114 ``%debug`` facilities in a notebook, for example (the web browser can't
115 115 interact directly with the debugger) or debug a third-party code where you may
116 116 have embedded an IPython kernel.
117 117
118 118 This is also something that we have wanted for a long time, and which is a
119 119 culmination (as a team effort) of the work started last year during the 2010
120 120 Google Summer of Code project.
121 121
122 122 Tabbed QtConsole
123 123 ~~~~~~~~~~~~~~~~
124 124
125 125 The QtConsole now supports starting multiple kernels in tabs, and has a
126 126 menubar, so it looks and behaves more like a real application. Keyboard
127 127 enthusiasts can disable the menubar with ctrl-shift-M (:ghpull:`887`).
128 128
129 .. figure:: ../_static/qtconsole_tabbed.png
129 .. figure:: ../_images/qtconsole_tabbed.png
130 130 :width: 400px
131 131 :alt: Tabbed IPython Qt console with embedded plots and menus.
132 132 :align: center
133 :target: ../_static/qtconsole_tabbed.png
133 :target: ../_images/qtconsole_tabbed.png
134 134
135 135 The improved Qt console for IPython, now with tabs to control multiple
136 136 kernels and full menu support.
137 137
138 138
139 139 Full Python 3 compatibility
140 140 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
141 141
142 142 IPython can now be installed from a single codebase on Python 2 and
143 143 Python 3. The installation process for Python 3 automatically runs 2to3. The
144 144 same 'default' profile is now used for Python 2 and 3 (the previous version had
145 145 a separate 'python3' profile).
146 146
147 147 Standalone Kernel
148 148 ~~~~~~~~~~~~~~~~~
149 149
150 150 The ``ipython kernel`` subcommand has been added, to allow starting a
151 151 standalone kernel, that can be used with various frontends. You can then later
152 152 connect a Qt console or a terminal console to this kernel by typing e.g.::
153 153
154 154 ipython qtconsole --existing
155 155
156 156 if it's the only one running, or by passing explicitly the connection
157 157 parameters (printed by the kernel at startup).
158 158
159 159
160 160 PyPy support
161 161 ~~~~~~~~~~~~
162 162
163 163 The terminal interface to IPython now runs under `PyPy <http://pypy.org/>`_.
164 164 We will continue to monitor PyPy's progress, and hopefully before long at least
165 165 we'll be able to also run the notebook. The Qt console may take longer, as Qt
166 166 is a very complex set of bindings to a huge C++ library, and that is currently
167 167 the area where PyPy still lags most behind. But for everyday interactive use
168 168 at the terminal, with this release and PyPy 1.7, things seem to work quite well
169 169 from our admittedly limited testing.
170 170
171 171
172 172 Other important new features
173 173 ----------------------------
174 174
175 175 * **SSH Tunnels**: In 0.11, the :mod:`IPython.parallel` Client could tunnel its
176 176 connections to the Controller via ssh. Now, the QtConsole :ref:`supports
177 177 <ssh_tunnels>` ssh tunneling, as do parallel engines.
178 178
179 179 * **relaxed command-line parsing**: 0.11 was released with overly-strict
180 180 command-line parsing, preventing the ability to specify arguments with spaces,
181 181 e.g. ``ipython --pylab qt`` or ``ipython -c "print 'hi'"``. This has
182 182 been fixed, by using argparse. The new parsing is a strict superset of 0.11, so
183 183 any commands in 0.11 should still work in 0.12.
184 184
185 185 * **HistoryAccessor**: The :class:`~IPython.core.history.HistoryManager` class
186 186 for interacting with your IPython SQLite history database has been split,
187 187 adding a parent :class:`~IPython.core.history.HistoryAccessor` class, so that
188 188 users can write code to access and search their IPython history without being
189 189 in an IPython session (:ghpull:`824`).
190 190
191 191 * **kernel %gui and %pylab**: The ``%gui`` and ``%pylab`` magics have been
192 192 restored to the IPython kernel (e.g. in the qtconsole or notebook). This
193 193 allows activation of pylab-mode, or eventloop integration after starting the
194 194 kernel, which was unavailable in 0.11. Unlike in the terminal, this can be
195 195 set only once, and cannot be changed.
196 196
197 197 * **%config**: A new ``%config`` magic has been added, giving easy access to the
198 198 IPython configuration system at runtime (:ghpull:`923`).
199 199
200 200 * **Multiline History**: Multiline readline history has been restored to the
201 201 Terminal frontend by default (:ghpull:`838`).
202 202
203 203 * **%store**: The ``%store`` magic from earlier versions has been updated and
204 204 re-enabled (:ref:`extensions_storemagic`; :ghpull:`1029`). To autorestore
205 205 stored variables on startup, specify ``c.StoreMagic.autorestore = True`` in
206 206 :file:`ipython_config.py`.
207 207
208 208
209 209 Major Bugs fixed
210 210 ----------------
211 211
212 212 In this cycle, we have :ref:`closed over 500 issues <issues_list_012>`, but a
213 213 few major ones merit special mention:
214 214
215 215 * Simple configuration errors should no longer crash IPython. In 0.11, errors
216 216 in config files, as well as invalid trait values, could crash IPython. Now,
217 217 such errors are reported, and help is displayed.
218 218
219 219 * Certain SyntaxErrors no longer crash IPython (e.g. just typing keywords, such
220 220 as ``return``, ``break``, etc.). See :ghissue:`704`.
221 221
222 222 * IPython path utils, such as :func:`~IPython.utils.path.get_ipython_dir` now
223 223 check for write permissions, so IPython should function on systems where the
224 224 default path resolution might point to a read-only location, such as
225 225 ``HOMESHARE`` on Windows (:ghissue:`669`).
226 226
227 227 * :func:`raw_input` now works in the kernel when multiple frontends are in
228 228 use. The request will be sent to the frontend that made the request, and an
229 229 exception is raised if that frontend does not support stdin requests
230 230 (e.g. the notebook) (:ghissue:`673`).
231 231
232 232 * :mod:`zmq` version detection no longer uses simple lexicographical comparison
233 233 to check minimum version, which prevents 0.11 from working with pyzmq-2.1.10
234 234 (:ghpull:`758`).
235 235
236 236 * A bug in PySide < 1.0.7 caused crashes on OSX when tooltips were shown
237 237 (:ghissue:`711`). these tooltips are now disabled on old PySide
238 238 (:ghpull:`963`).
239 239
240 240 * IPython no longer crashes when started on recent versions of Python 3 in
241 241 Windows (:ghissue:`737`).
242 242
243 243 * Instances of classes defined interactively can now be pickled (:ghissue:`29`;
244 244 :ghpull:`648`). Note that pickling saves a reference to the class definition,
245 245 so unpickling the instances will only work where the class has been defined.
246 246
247 247
248 248 Backwards incompatible changes
249 249 ------------------------------
250 250
251 251 * IPython connection information is no longer specified via ip/port directly,
252 252 rather via json connection files. These files are stored in the security
253 253 directory, and enable us to turn on HMAC message authentication by default,
254 254 significantly improving the security of kernels. Various utility functions
255 255 have been added to :mod:`IPython.lib.kernel`, for easier connecting to existing
256 256 kernels.
257 257
258 258 * :class:`~IPython.zmq.kernelmanager.KernelManager` now has one ip, and several
259 259 port traits, rather than several ip/port pair ``_addr`` traits. This better
260 260 matches the rest of the code, where the ip cannot not be set separately for
261 261 each channel.
262 262
263 263 * Custom prompts are now configured using a new class,
264 264 :class:`~IPython.core.prompts.PromptManager`, which has traits for
265 265 :attr:`in_template`, :attr:`in2_template` (the ``...:`` continuation prompt),
266 266 :attr:`out_template` and :attr:`rewrite_template`. This uses Python's string
267 267 formatting system, so you can use ``{time}`` and ``{cwd}``, although we have
268 268 preserved the abbreviations from previous versions, e.g. ``\#`` (prompt number)
269 269 and ``\w`` (working directory). For the list of available fields, refer to the
270 270 source of :file:`IPython/core/prompts.py`.
271 271
272 272 * The class inheritance of the Launchers in
273 273 :mod:`IPython.parallel.apps.launcher` used by ipcluster has changed, so that
274 274 trait names are more consistent across batch systems. This may require a few
275 275 renames in your config files, if you customized the command-line args for
276 276 launching controllers and engines. The configurable names have also been
277 277 changed to be clearer that they point to class names, and can now be
278 278 specified by name only, rather than requiring the full import path of each
279 279 class, e.g.::
280 280
281 281 IPClusterEngines.engine_launcher = 'IPython.parallel.apps.launcher.MPIExecEngineSetLauncher'
282 282 IPClusterStart.controller_launcher = 'IPython.parallel.apps.launcher.SSHControllerLauncher'
283 283
284 284 would now be specified as::
285 285
286 286 IPClusterEngines.engine_launcher_class = 'MPI'
287 287 IPClusterStart.controller_launcher_class = 'SSH'
288 288
289 289 The full path will still work, and is necessary for using custom launchers
290 290 not in IPython's launcher module.
291 291
292 292 Further, MPIExec launcher names are now prefixed with just MPI, to better match
293 293 other batch launchers, and be generally more intuitive. The MPIExec names are
294 294 deprecated, but continue to work.
295 295
296 296 * For embedding a shell, note that the parameters ``user_global_ns`` and
297 297 ``global_ns`` have been deprectated in favour of ``user_module`` and
298 298 ``module`` respsectively. The new parameters expect a module-like object,
299 299 rather than a namespace dict. The old parameters remain for backwards
300 300 compatibility, although ``user_global_ns`` is now ignored. The ``user_ns``
301 301 parameter works the same way as before, and calling
302 302 :func:`~IPython.frontend.terminal.embed.embed` with no arguments still works
303 303 as before.
304 304
305 305
306 306 Development summary and credits
307 307 -------------------------------
308 308
309 309 The previous version (IPython 0.11) was released on July 31 2011, so this
310 310 release cycle was roughly 4 1/2 months long, we closed a total of 515 issues,
311 311 257 pull requests and 258 regular issues (a :ref:`detailed list
312 312 <issues_list_012>` is available).
313 313
314 314 Many users and developers contributed code, features, bug reports and ideas to
315 315 this release. Please do not hesitate in contacting us if we've failed to
316 316 acknowledge your contribution here. In particular, for this release we have
317 317 had commits from the following 45 contributors, a mix of new and regular names
318 318 (in alphabetical order by first name):
319 319
320 320 * Alcides <alcides-at-do-not-span-me.com>
321 321 * Ben Edwards <bedwards-at-cs.unm.edu>
322 322 * Benjamin Ragan-Kelley <benjaminrk-at-gmail.com>
323 323 * Benjamin Thyreau <benjamin.thyreau-at-gmail.com>
324 324 * Bernardo B. Marques <bernardo.fire-at-gmail.com>
325 325 * Bernard Paulus <bprecyclebin-at-gmail.com>
326 326 * Bradley M. Froehle <brad.froehle-at-gmail.com>
327 327 * Brian E. Granger <ellisonbg-at-gmail.com>
328 328 * Christian Boos <cboos-at-bct-technology.com>
329 329 * Daniel Velkov <danielv-at-mylife.com>
330 330 * Erik Tollerud <erik.tollerud-at-gmail.com>
331 331 * Evan Patterson <epatters-at-enthought.com>
332 332 * Felix Werner <Felix.Werner-at-kit.edu>
333 333 * Fernando Perez <Fernando.Perez-at-berkeley.edu>
334 334 * Gabriel <g2p.code-at-gmail.com>
335 335 * Grahame Bowland <grahame-at-angrygoats.net>
336 336 * Hannes Schulz <schulz-at-ais.uni-bonn.de>
337 337 * Jens Hedegaard Nielsen <jenshnielsen-at-gmail.com>
338 338 * Jonathan March <jmarch-at-enthought.com>
339 339 * Jörgen Stenarson <jorgen.stenarson-at-bostream.nu>
340 340 * Julian Taylor <jtaylor.debian-at-googlemail.com>
341 341 * Kefu Chai <tchaikov-at-gmail.com>
342 342 * macgyver <neil.rabinowitz-at-merton.ox.ac.uk>
343 343 * Matt Cottingham <matt.cottingham-at-gmail.com>
344 344 * Matthew Brett <matthew.brett-at-gmail.com>
345 345 * Matthias BUSSONNIER <bussonniermatthias-at-gmail.com>
346 346 * Michael Droettboom <mdboom-at-gmail.com>
347 347 * Nicolas Rougier <Nicolas.Rougier-at-inria.fr>
348 348 * Olivier Verdier <olivier.verdier-at-gmail.com>
349 349 * Omar Andres Zapata Mesa <andresete.chaos-at-gmail.com>
350 350 * Pablo Winant <pablo.winant-at-gmail.com>
351 351 * Paul Ivanov <pivanov314-at-gmail.com>
352 352 * Pauli Virtanen <pav-at-iki.fi>
353 353 * Pete Aykroyd <aykroyd-at-gmail.com>
354 354 * Prabhu Ramachandran <prabhu-at-enthought.com>
355 355 * Puneeth Chaganti <punchagan-at-gmail.com>
356 356 * Robert Kern <robert.kern-at-gmail.com>
357 357 * Satrajit Ghosh <satra-at-mit.edu>
358 358 * Stefan van der Walt <stefan-at-sun.ac.za>
359 359 * Szabolcs Horvát <szhorvat-at-gmail.com>
360 360 * Thomas Kluyver <takowl-at-gmail.com>
361 361 * Thomas Spura <thomas.spura-at-gmail.com>
362 362 * Timo Paulssen <timonator-at-perpetuum-immobile.de>
363 363 * Valentin Haenel <valentin.haenel-at-gmx.de>
364 364 * Yaroslav Halchenko <debian-at-onerussian.com>
365 365
366 366 .. note::
367 367
368 368 This list was generated with the output of
369 369 ``git log rel-0.11..HEAD --format='* %aN <%aE>' | sed 's/@/\-at\-/' | sed 's/<>//' | sort -u``
370 370 after some cleanup. If you should be on this list, please add yourself.
@@ -1,673 +1,673
1 1 =============
2 2 0.13 Series
3 3 =============
4 4
5 5 Release 0.13
6 6 ============
7 7
8 8 IPython 0.13 contains several major new features, as well as a large amount of
9 9 bug and regression fixes. The previous version (0.12) was released on December
10 10 19 2011, and in this development cycle we had:
11 11
12 12 - ~6 months of work.
13 13 - 373 pull requests merged.
14 14 - 742 issues closed (non-pull requests).
15 15 - contributions from 62 authors.
16 16 - 1760 commits.
17 17 - a diff of 114226 lines.
18 18
19 19 The amount of work included in this release is so large, that we can only cover
20 20 here the main highlights; please see our :ref:`detailed release statistics
21 21 <issues_list_013>` for links to every issue and pull request closed on GitHub
22 22 as well as a full list of individual contributors.
23 23
24 24
25 25 Major Notebook improvements: new user interface and more
26 26 --------------------------------------------------------
27 27
28 28 The IPython Notebook, which has proven since its release to be wildly popular,
29 29 has seen a massive amount of work in this release cycle, leading to a
30 30 significantly improved user experience as well as many new features.
31 31
32 32 The first user-visible change is a reorganization of the user interface; the
33 33 left panel has been removed and was replaced by a real menu system and a
34 34 toolbar with icons. Both the toolbar and the header above the menu can be
35 35 collapsed to leave an unobstructed working area:
36 36
37 .. image:: ../_static/ipy_013_notebook_spectrogram.png
37 .. image:: ../_images/ipy_013_notebook_spectrogram.png
38 38 :width: 460px
39 39 :alt: New user interface for Notebook
40 40 :align: center
41 :target: ../_static/ipy_013_notebook_spectrogram.png
41 :target: ../_images/ipy_013_notebook_spectrogram.png
42 42
43 43 The notebook handles very long outputs much better than before (this was a
44 44 serious usability issue when running processes that generated massive amounts
45 45 of output). Now, in the presence of outputs longer than ~100 lines, the
46 46 notebook will automatically collapse to a scrollable area and the entire left
47 47 part of this area controls the display: one click in this area will expand the
48 48 output region completely, and a double-click will hide it completely. This
49 49 figure shows both the scrolled and hidden modes:
50 50
51 .. image:: ../_static/ipy_013_notebook_long_out.png
51 .. image:: ../_images/ipy_013_notebook_long_out.png
52 52 :width: 460px
53 53 :alt: Scrolling and hiding of long output in the notebook.
54 54 :align: center
55 :target: ../_static/ipy_013_notebook_long_out.png
55 :target: ../_images/ipy_013_notebook_long_out.png
56 56
57 57 .. note::
58 58
59 59 The auto-folding of long outputs is disabled in Firefox due to bugs in its
60 60 scrolling behavior. See :ghpull:`2047` for details.
61 61
62 62 Uploading notebooks to the dashboard is now easier: in addition to drag and
63 63 drop (which can be finicky sometimes), you can now click on the upload text and
64 64 use a regular file dialog box to select notebooks to upload. Furthermore, the
65 65 notebook dashboard now auto-refreshes its contents and offers buttons to shut
66 66 down any running kernels (:ghpull:`1739`):
67 67
68 .. image:: ../_static/ipy_013_dashboard.png
68 .. image:: ../_images/ipy_013_dashboard.png
69 69 :width: 460px
70 70 :alt: Improved dashboard
71 71 :align: center
72 :target: ../_static/ipy_013_dashboard.png
72 :target: ../_images/ipy_013_dashboard.png
73 73
74 74
75 75 Cluster management
76 76 ~~~~~~~~~~~~~~~~~~
77 77
78 78 The notebook dashboard can now also start and stop clusters, thansk to a new
79 79 tab in the dashboard user interface:
80 80
81 .. image:: ../_static/ipy_013_dashboard_cluster.png
81 .. image:: ../_images/ipy_013_dashboard_cluster.png
82 82 :width: 460px
83 83 :alt: Cluster management from the notebook dashboard
84 84 :align: center
85 :target: ../_static/ipy_013_dashboard_cluster.png
85 :target: ../_images/ipy_013_dashboard_cluster.png
86 86
87 87 This interface allows, for each profile you have configured, to start and stop
88 88 a cluster (and optionally override the default number of engines corresponding
89 89 to that configuration). While this hides all error reporting, once you have a
90 90 configuration that you know works smoothly, it is a very convenient interface
91 91 for controlling your parallel resources.
92 92
93 93
94 94 New notebook format
95 95 ~~~~~~~~~~~~~~~~~~~
96 96
97 97 The notebooks saved now use version 3 of our format, which supports heading
98 98 levels as well as the concept of 'raw' text cells that are not rendered as
99 99 Markdown. These will be useful with converters_ we are developing, to pass raw
100 100 markup (say LaTeX). That conversion code is still under heavy development and
101 101 not quite ready for prime time, but we welcome help on this front so that we
102 102 can merge it for full production use as soon as possible.
103 103
104 104 .. _converters: https://github.com/ipython/nbconvert
105 105
106 106 .. note::
107 107
108 108 v3 notebooks can *not* be read by older versions of IPython, but we provide
109 109 a `simple script`_ that you can use in case you need to export a v3
110 110 notebook to share with a v2 user.
111 111
112 112 .. _simple script: https://gist.github.com/1935808
113 113
114 114
115 115 JavaScript refactoring
116 116 ~~~~~~~~~~~~~~~~~~~~~~
117 117
118 118 All the client-side JavaScript has been decoupled to ease reuse of parts of the
119 119 machinery without having to build a full-blown notebook. This will make it much
120 120 easier to communicate with an IPython kernel from existing web pages and to
121 121 integrate single cells into other sites, without loading the full notebook
122 122 document-like UI. :ghpull:`1711`.
123 123
124 124 This refactoring also enables the possibility of writing dynamic javascript
125 125 widgets that are returned from Python code and that present an interactive view
126 126 to the user, with callbacks in Javascript executing calls to the Kernel. This
127 127 will enable many interactive elements to be added by users in notebooks.
128 128
129 129 An example of this capability has been provided as a proof of concept in
130 130 :file:`docs/examples/widgets` that lets you directly communicate with one or more
131 131 parallel engines, acting as a mini-console for parallel debugging and
132 132 introspection.
133 133
134 134
135 135 Improved tooltips
136 136 ~~~~~~~~~~~~~~~~~
137 137
138 138 The object tooltips have gained some new functionality. By pressing tab several
139 139 times, you can expand them to see more of a docstring, keep them visible as you
140 140 fill in a function's parameters, or transfer the information to the pager at the
141 141 bottom of the screen. For the details, look at the example notebook
142 142 :file:`01_notebook_introduction.ipynb`.
143 143
144 .. figure:: ../_static/ipy_013_notebook_tooltip.png
144 .. figure:: ../_images/ipy_013_notebook_tooltip.png
145 145 :width: 460px
146 146 :alt: Improved tooltips in the notebook.
147 147 :align: center
148 :target: ../_static/ipy_013_notebook_tooltip.png
148 :target: ../_images/ipy_013_notebook_tooltip.png
149 149
150 150 The new notebook tooltips.
151 151
152 152 Other improvements to the Notebook
153 153 ----------------------------------
154 154
155 155 These are some other notable small improvements to the notebook, in addition to
156 156 many bug fixes and minor changes to add polish and robustness throughout:
157 157
158 158 * The notebook pager (the area at the bottom) is now resizeable by dragging its
159 159 divider handle, a feature that had been requested many times by just about
160 160 anyone who had used the notebook system. :ghpull:`1705`.
161 161
162 162 * It is now possible to open notebooks directly from the command line; for
163 163 example: ``ipython notebook path/`` will automatically set ``path/`` as the
164 164 notebook directory, and ``ipython notebook path/foo.ipynb`` will further
165 165 start with the ``foo.ipynb`` notebook opened. :ghpull:`1686`.
166 166
167 167 * If a notebook directory is specified with ``--notebook-dir`` (or with the
168 168 corresponding configuration flag ``NotebookManager.notebook_dir``), all
169 169 kernels start in this directory.
170 170
171 171 * Fix codemirror clearing of cells with ``Ctrl-Z``; :ghpull:`1965`.
172 172
173 173 * Text (markdown) cells now line wrap correctly in the notebook, making them
174 174 much easier to edit :ghpull:`1330`.
175 175
176 176 * PNG and JPEG figures returned from plots can be interactively resized in the
177 177 notebook, by dragging them from their lower left corner. :ghpull:`1832`.
178 178
179 179 * Clear ``In []`` prompt numbers on "Clear All Output". For more
180 180 version-control-friendly ``.ipynb`` files, we now strip all prompt numbers
181 181 when doing a "Clear all output". This reduces the amount of noise in
182 182 commit-to-commit diffs that would otherwise show the (highly variable) prompt
183 183 number changes. :ghpull:`1621`.
184 184
185 185 * The notebook server now requires *two* consecutive ``Ctrl-C`` within 5
186 186 seconds (or an interactive confirmation) to terminate operation. This makes
187 187 it less likely that you will accidentally kill a long-running server by
188 188 typing ``Ctrl-C`` in the wrong terminal. :ghpull:`1609`.
189 189
190 190 * Using ``Ctrl-S`` (or ``Cmd-S`` on a Mac) actually saves the notebook rather
191 191 than providing the fairly useless browser html save dialog. :ghpull:`1334`.
192 192
193 193 * Allow accessing local files from the notebook (in urls), by serving any local
194 194 file as the url ``files/<relativepath>``. This makes it possible to, for
195 195 example, embed local images in a notebook. :ghpull:`1211`.
196 196
197 197
198 198 Cell magics
199 199 -----------
200 200
201 201 We have completely refactored the magic system, finally moving the magic
202 202 objects to standalone, independent objects instead of being the mixin class
203 203 we'd had since the beginning of IPython (:ghpull:`1732`). Now, a separate base
204 204 class is provided in :class:`IPython.core.magic.Magics` that users can subclass
205 205 to create their own magics. Decorators are also provided to create magics from
206 206 simple functions without the need for object orientation. Please see the
207 207 :ref:`magic` docs for further details.
208 208
209 209 All builtin magics now exist in a few subclasses that group together related
210 210 functionality, and the new :mod:`IPython.core.magics` package has been created
211 211 to organize this into smaller files.
212 212
213 213 This cleanup was the last major piece of deep refactoring needed from the
214 214 original 2001 codebase.
215 215
216 216 We have also introduced a new type of magic function, prefixed with `%%`
217 217 instead of `%`, which operates at the whole-cell level. A cell magic receives
218 218 two arguments: the line it is called on (like a line magic) and the body of the
219 219 cell below it.
220 220
221 221 Cell magics are most natural in the notebook, but they also work in the
222 222 terminal and qt console, with the usual approach of using a blank line to
223 223 signal cell termination.
224 224
225 225 For example, to time the execution of several statements::
226 226
227 227 %%timeit x = 0 # setup
228 228 for i in range(100000):
229 229 x += i**2
230 230
231 231 This is particularly useful to integrate code in another language, and cell
232 232 magics already exist for shell scripts, Cython, R and Octave. Using ``%%script
233 233 /usr/bin/foo``, you can run a cell in any interpreter that accepts code via
234 234 stdin.
235 235
236 236 Another handy cell magic makes it easy to write short text files: ``%%file
237 237 ~/save/to/here.txt``.
238 238
239 239 The following cell magics are now included by default; all those that use
240 240 special interpreters (Perl, Ruby, bash, etc.) assume you have the requisite
241 241 interpreter installed:
242 242
243 243 * ``%%!``: run cell body with the underlying OS shell; this is similar to
244 244 prefixing every line in the cell with ``!``.
245 245
246 246 * ``%%bash``: run cell body under bash.
247 247
248 248 * ``%%capture``: capture the output of the code in the cell (and stderr as
249 249 well). Useful to run codes that produce too much output that you don't even
250 250 want scrolled.
251 251
252 252 * ``%%file``: save cell body as a file.
253 253
254 254 * ``%%perl``: run cell body using Perl.
255 255
256 256 * ``%%prun``: run cell body with profiler (cell extension of ``%prun``).
257 257
258 258 * ``%%python3``: run cell body using Python 3.
259 259
260 260 * ``%%ruby``: run cell body using Ruby.
261 261
262 262 * ``%%script``: run cell body with the script specified in the first line.
263 263
264 264 * ``%%sh``: run cell body using sh.
265 265
266 266 * ``%%sx``: run cell with system shell and capture process output (cell
267 267 extension of ``%sx``).
268 268
269 269 * ``%%system``: run cell with system shell (``%%!`` is an alias to this).
270 270
271 271 * ``%%timeit``: time the execution of the cell (extension of ``%timeit``).
272 272
273 273 This is what some of the script-related magics look like in action:
274 274
275 .. image:: ../_static/ipy_013_notebook_script_cells.png
275 .. image:: ../_images/ipy_013_notebook_script_cells.png
276 276 :width: 460px
277 277 :alt: Cluster management from the notebook dashboard
278 278 :align: center
279 :target: ../_static/ipy_013_notebook_script_cells.png
279 :target: ../_images/ipy_013_notebook_script_cells.png
280 280
281 281 In addition, we have also a number of :ref:`extensions <extensions_overview>`
282 282 that provide specialized magics. These typically require additional software
283 283 to run and must be manually loaded via ``%load_ext <extension name>``, but are
284 284 extremely useful. The following extensions are provided:
285 285
286 286 **Cython magics** (extension :ref:`cythonmagic <extensions_cythonmagic>`)
287 287 This extension provides magics to automatically build and compile Python
288 288 extension modules using the Cython_ language. You must install Cython
289 289 separately, as well as a C compiler, for this to work. The examples
290 290 directory in the source distribution ships with a full notebook
291 291 demonstrating these capabilities:
292 292
293 .. image:: ../_static/ipy_013_notebook_cythonmagic.png
293 .. image:: ../_images/ipy_013_notebook_cythonmagic.png
294 294 :width: 460px
295 295 :alt: Cython magic
296 296 :align: center
297 :target: ../_static/ipy_013_notebook_cythonmagic.png
297 :target: ../_images/ipy_013_notebook_cythonmagic.png
298 298
299 299 .. _cython: http://cython.org
300 300
301 301 **Octave magics** (extension :ref:`octavemagic <extensions_octavemagic>`)
302 302 This extension provides several magics that support calling code written in
303 303 the Octave_ language for numerical computing. You can execute single-lines
304 304 or whole blocks of Octave code, capture both output and figures inline
305 305 (just like matplotlib plots), and have variables automatically converted
306 306 between the two languages. To use this extension, you must have Octave
307 307 installed as well as the oct2py_ package. The examples
308 308 directory in the source distribution ships with a full notebook
309 309 demonstrating these capabilities:
310 310
311 .. image:: ../_static/ipy_013_notebook_octavemagic.png
311 .. image:: ../_images/ipy_013_notebook_octavemagic.png
312 312 :width: 460px
313 313 :alt: Octave magic
314 314 :align: center
315 :target: ../_static/ipy_013_notebook_octavemagic.png
315 :target: ../_images/ipy_013_notebook_octavemagic.png
316 316
317 317 .. _octave: http://www.gnu.org/software/octave
318 318 .. _oct2py: http://pypi.python.org/pypi/oct2py
319 319
320 320 **R magics** (extension :ref:`rmagic <extensions_rmagic>`)
321 321 This extension provides several magics that support calling code written in
322 322 the R_ language for statistical data analysis. You can execute
323 323 single-lines or whole blocks of R code, capture both output and figures
324 324 inline (just like matplotlib plots), and have variables automatically
325 325 converted between the two languages. To use this extension, you must have
326 326 R installed as well as the rpy2_ package that bridges Python and R. The
327 327 examples directory in the source distribution ships with a full notebook
328 328 demonstrating these capabilities:
329 329
330 .. image:: ../_static/ipy_013_notebook_rmagic.png
330 .. image:: ../_images/ipy_013_notebook_rmagic.png
331 331 :width: 460px
332 332 :alt: R magic
333 333 :align: center
334 :target: ../_static/ipy_013_notebook_rmagic.png
334 :target: ../_images/ipy_013_notebook_rmagic.png
335 335
336 336 .. _R: http://www.r-project.org
337 337 .. _rpy2: http://rpy.sourceforge.net/rpy2.html
338 338
339 339
340 340 Tab completer improvements
341 341 --------------------------
342 342
343 343 Useful tab-completion based on live inspection of objects is one of the most
344 344 popular features of IPython. To make this process even more user-friendly, the
345 345 completers of both the Qt console and the Notebook have been reworked.
346 346
347 347 The Qt console comes with a new ncurses-like tab completer, activated by
348 348 default, which lets you cycle through the available completions by pressing tab,
349 349 or select a completion with the arrow keys (:ghpull:`1851`).
350 350
351 .. figure:: ../_static/ipy_013_qtconsole_completer.png
351 .. figure:: ../_images/ipy_013_qtconsole_completer.png
352 352 :width: 460px
353 353 :alt: ncurses-like completer, with highlighted selection.
354 354 :align: center
355 :target: ../_static/ipy_013_qtconsole_completer.png
355 :target: ../_images/ipy_013_qtconsole_completer.png
356 356
357 357 The new improved Qt console's ncurses-like completer allows to easily
358 358 navigate thought long list of completions.
359 359
360 360 In the notebook, completions are now sourced both from object introspection and
361 361 analysis of surrounding code, so limited completions can be offered for
362 362 variables defined in the current cell, or while the kernel is busy
363 363 (:ghpull:`1711`).
364 364
365 365
366 366 We have implemented a new configurable flag to control tab completion on
367 367 modules that provide the ``__all__`` attribute::
368 368
369 369 IPCompleter.limit_to__all__= Boolean
370 370
371 371 This instructs the completer to honor ``__all__`` for the completion.
372 372 Specifically, when completing on ``object.<tab>``, if True: only those names
373 373 in ``obj.__all__`` will be included. When False [default]: the ``__all__``
374 374 attribute is ignored. :ghpull:`1529`.
375 375
376 376
377 377 Improvements to the Qt console
378 378 ------------------------------
379 379
380 380 The Qt console continues to receive improvements and refinements, despite the
381 381 fact that it is by now a fairly mature and robust component. Lots of small
382 382 polish has gone into it, here are a few highlights:
383 383
384 384 * A number of changes were made to the underlying code for easier integration
385 385 into other projects such as Spyder_ (:ghpull:`2007`, :ghpull:`2024`).
386 386
387 387 * Improved menus with a new Magic menu that is organized by magic groups (this
388 388 was made possible by the reorganization of the magic system
389 389 internals). :ghpull:`1782`.
390 390
391 391 * Allow for restarting kernels without clearing the qtconsole, while leaving a
392 392 visible indication that the kernel has restarted. :ghpull:`1681`.
393 393
394 394 * Allow the native display of jpeg images in the qtconsole. :ghpull:`1643`.
395 395
396 396 .. _spyder: https://code.google.com/p/spyderlib
397 397
398 398
399 399
400 400 Parallel
401 401 --------
402 402
403 403 The parallel tools have been improved and fine-tuned on multiple fronts. Now,
404 404 the creation of an :class:`IPython.parallel.Client` object automatically
405 405 activates a line and cell magic function ``px`` that sends its code to all the
406 406 engines. Further magics can be easily created with the :meth:`.Client.activate`
407 407 method, to conveniently execute code on any subset of engines. :ghpull:`1893`.
408 408
409 409 The ``%%px`` cell magic can also be given an optional targets argument, as well
410 410 as a ``--out`` argument for storing its output.
411 411
412 412 A new magic has also been added, ``%pxconfig``, that lets you configure various
413 413 defaults of the parallel magics. As usual, type ``%pxconfig?`` for details.
414 414
415 415 The exception reporting in parallel contexts has been improved to be easier to
416 416 read. Now, IPython directly reports the remote exceptions without showing any
417 417 of the internal execution parts:
418 418
419 .. image:: ../_static/ipy_013_par_tb.png
419 .. image:: ../_images/ipy_013_par_tb.png
420 420 :width: 460px
421 421 :alt: Improved parallel exceptions.
422 422 :align: center
423 :target: ../_static/ipy_013_par_tb.png
423 :target: ../_images/ipy_013_par_tb.png
424 424
425 425 The parallel tools now default to using ``NoDB`` as the storage backend for
426 426 intermediate results. This means that the default usage case will have a
427 427 significantly reduced memory footprint, though certain advanced features are
428 428 not available with this backend. For more details, see :ref:`parallel_db`.
429 429
430 430 The parallel magics now display all output, so you can do parallel plotting or
431 431 other actions with complex display. The ``px`` magic has now both line and cell
432 432 modes, and in cell mode finer control has been added about how to collate
433 433 output from multiple engines. :ghpull:`1768`.
434 434
435 435 There have also been incremental improvements to the SSH launchers:
436 436
437 437 * add to_send/fetch steps for moving connection files around.
438 438
439 439 * add SSHProxyEngineSetLauncher, for invoking to `ipcluster engines` on a
440 440 remote host. This can be used to start a set of engines via PBS/SGE/MPI
441 441 *remotely*.
442 442
443 443 This makes the SSHLauncher usable on machines without shared filesystems.
444 444
445 445 A number of 'sugar' methods/properties were added to AsyncResult that are
446 446 quite useful (:ghpull:`1548`) for everday work:
447 447
448 448 * ``ar.wall_time`` = received - submitted
449 449 * ``ar.serial_time`` = sum of serial computation time
450 450 * ``ar.elapsed`` = time since submission (wall_time if done)
451 451 * ``ar.progress`` = (int) number of sub-tasks that have completed
452 452 * ``len(ar)`` = # of tasks
453 453 * ``ar.wait_interactive()``: prints progress
454 454
455 455 Added :meth:`.Client.spin_thread` / :meth:`~.Client.stop_spin_thread` for
456 456 running spin in a background thread, to keep zmq queue clear. This can be used
457 457 to ensure that timing information is as accurate as possible (at the cost of
458 458 having a background thread active).
459 459
460 460 Set TaskScheduler.hwm default to 1 instead of 0. 1 has more
461 461 predictable/intuitive behavior, if often slower, and thus a more logical
462 462 default. Users whose workloads require maximum throughput and are largely
463 463 homogeneous in time per task can make the optimization themselves, but now the
464 464 behavior will be less surprising to new users. :ghpull:`1294`.
465 465
466 466
467 467 Kernel/Engine unification
468 468 -------------------------
469 469
470 470 This is mostly work 'under the hood', but it is actually a *major* achievement
471 471 for the project that has deep implications in the long term: at last, we have
472 472 unified the main object that executes as the user's interactive shell (which we
473 473 refer to as the *IPython kernel*) with the objects that run in all the worker
474 474 nodes of the parallel computing facilities (the *IPython engines*). Ever since
475 475 the first implementation of IPython's parallel code back in 2006, we had wanted
476 476 to have these two roles be played by the same machinery, but a number of
477 477 technical reasons had prevented that from being true.
478 478
479 479 In this release we have now merged them, and this has a number of important
480 480 consequences:
481 481
482 482 * It is now possible to connect any of our clients (qtconsole or terminal
483 483 console) to any individual parallel engine, with the *exact* behavior of
484 484 working at a 'regular' IPython console/qtconsole. This makes debugging,
485 485 plotting, etc. in parallel scenarios vastly easier.
486 486
487 487 * Parallel engines can always execute arbitrary 'IPython code', that is, code
488 488 that has magics, shell extensions, etc. In combination with the ``%%px``
489 489 magics, it is thus extremely natural for example to send to all engines a
490 490 block of Cython or R code to be executed via the new Cython and R magics. For
491 491 example, this snippet would send the R block to all active engines in a
492 492 cluster::
493 493
494 494 %%px
495 495 %%R
496 496 ... R code goes here
497 497
498 498 * It is possible to embed not only an interactive shell with the
499 499 :func:`IPython.embed` call as always, but now you can also embed a *kernel*
500 500 with :func:`IPython.embed_kernel()`. Embedding an IPython kernel in an
501 501 application is useful when you want to use :func:`IPython.embed` but don't
502 502 have a terminal attached on stdin and stdout.
503 503
504 504 * The new :func:`IPython.parallel.bind_kernel` allows you to promote Engines to
505 505 listening Kernels, and connect QtConsoles to an Engine and debug it
506 506 directly.
507 507
508 508 In addition, having a single core object through our entire architecture also
509 509 makes the project conceptually cleaner, easier to maintain and more robust.
510 510 This took a lot of work to get in place, but we are thrilled to have this major
511 511 piece of architecture finally where we'd always wanted it to be.
512 512
513 513
514 514 Official Public API
515 515 -------------------
516 516
517 517 We have begun organizing our API for easier public use, with an eye towards an
518 518 official IPython 1.0 release which will firmly maintain this API compatible for
519 519 its entire lifecycle. There is now an :mod:`IPython.display` module that
520 520 aggregates all display routines, and the :mod:`IPython.config` namespace has
521 521 all public configuration tools. We will continue improving our public API
522 522 layout so that users only need to import names one level deeper than the main
523 523 ``IPython`` package to access all public namespaces.
524 524
525 525
526 526 IPython notebook file icons
527 527 ---------------------------
528 528
529 529 The directory ``docs/resources`` in the source distribution contains SVG and
530 530 PNG versions of our file icons, as well as an ``Info.plist.example`` file with
531 531 instructions to install them on Mac OSX. This is a first draft of our icons,
532 532 and we encourage contributions from users with graphic talent to improve them
533 533 in the future:
534 534
535 535 .. image:: ../../resources/ipynb_icon_128x128.png
536 536 :alt: IPython notebook file icon.
537 537
538 538
539 539 New top-level `locate` command
540 540 ------------------------------
541 541
542 542 Add `locate` entry points; these would be useful for quickly locating IPython
543 543 directories and profiles from other (non-Python) applications. :ghpull:`1762`.
544 544
545 545 Examples::
546 546
547 547 $> ipython locate
548 548 /Users/me/.ipython
549 549
550 550 $> ipython locate profile foo
551 551 /Users/me/.ipython/profile_foo
552 552
553 553 $> ipython locate profile
554 554 /Users/me/.ipython/profile_default
555 555
556 556 $> ipython locate profile dne
557 557 [ProfileLocate] Profile u'dne' not found.
558 558
559 559
560 560 Other new features and improvements
561 561 -----------------------------------
562 562
563 563 * **%install_ext**: A new magic function to install an IPython extension from
564 564 a URL. E.g. ``%install_ext
565 565 https://bitbucket.org/birkenfeld/ipython-physics/raw/default/physics.py``.
566 566
567 567 * The ``%loadpy`` magic is no longer restricted to Python files, and has been
568 568 renamed ``%load``. The old name remains as an alias.
569 569
570 570 * New command line arguments will help external programs find IPython folders:
571 571 ``ipython locate`` finds the user's IPython directory, and ``ipython locate
572 572 profile foo`` finds the folder for the 'foo' profile (if it exists).
573 573
574 574 * The :envvar:`IPYTHON_DIR` environment variable, introduced in the Great
575 575 Reorganization of 0.11 and existing only in versions 0.11-0.13, has been
576 576 deprecated. As described in :ghpull:`1167`, the complexity and confusion of
577 577 migrating to this variable is not worth the aesthetic improvement. Please use
578 578 the historical :envvar:`IPYTHONDIR` environment variable instead.
579 579
580 580 * The default value of *interactivity* passed from
581 581 :meth:`~IPython.core.interactiveshell.InteractiveShell.run_cell` to
582 582 :meth:`~IPython.core.interactiveshell.InteractiveShell.run_ast_nodes`
583 583 is now configurable.
584 584
585 585 * New ``%alias_magic`` function to conveniently create aliases of existing
586 586 magics, if you prefer to have shorter names for personal use.
587 587
588 588 * We ship unminified versions of the JavaScript libraries we use, to better
589 589 comply with Debian's packaging policies.
590 590
591 591 * Simplify the information presented by ``obj?/obj??`` to eliminate a few
592 592 redundant fields when possible. :ghpull:`2038`.
593 593
594 594 * Improved continuous integration for IPython. We now have automated test runs
595 595 on `Shining Panda <https://jenkins.shiningpanda.com/ipython>`_ and `Travis-CI
596 596 <http://travis-ci.org/#!/ipython/ipython>`_, as well as `Tox support
597 597 <http://tox.testrun.org>`_.
598 598
599 599 * The `vim-ipython`_ functionality (externally developed) has been updated to
600 600 the latest version.
601 601
602 602 .. _vim-ipython: https://github.com/ivanov/vim-ipython
603 603
604 604 * The ``%save`` magic now has a ``-f`` flag to force overwriting, which makes
605 605 it much more usable in the notebook where it is not possible to reply to
606 606 interactive questions from the kernel. :ghpull:`1937`.
607 607
608 608 * Use dvipng to format sympy.Matrix, enabling display of matrices in the Qt
609 609 console with the sympy printing extension. :ghpull:`1861`.
610 610
611 611 * Our messaging protocol now has a reasonable test suite, helping ensure that
612 612 we don't accidentally deviate from the spec and possibly break third-party
613 613 applications that may have been using it. We encourage users to contribute
614 614 more stringent tests to this part of the test suite. :ghpull:`1627`.
615 615
616 616 * Use LaTeX to display, on output, various built-in types with the SymPy
617 617 printing extension. :ghpull:`1399`.
618 618
619 619 * Add Gtk3 event loop integration and example. :ghpull:`1588`.
620 620
621 621 * ``clear_output`` improvements, which allow things like progress bars and other
622 622 simple animations to work well in the notebook (:ghpull:`1563`):
623 623
624 624 * `clear_output()` clears the line, even in terminal IPython, the QtConsole
625 625 and plain Python as well, by printing `\r` to streams.
626 626
627 627 * `clear_output()` avoids the flicker in the notebook by adding a delay,
628 628 and firing immediately upon the next actual display message.
629 629
630 630 * `display_javascript` hides its `output_area` element, so using display to
631 631 run a bunch of javascript doesn't result in ever-growing vertical space.
632 632
633 633 * Add simple support for running inside a virtualenv. While this doesn't
634 634 supplant proper installation (as users should do), it helps ad-hoc calling of
635 635 IPython from inside a virtualenv. :ghpull:`1388`.
636 636
637 637
638 638 Major Bugs fixed
639 639 ----------------
640 640
641 641 In this cycle, we have :ref:`closed over 740 issues <issues_list_013>`, but a
642 642 few major ones merit special mention:
643 643
644 644 * The ``%pastebin`` magic has been updated to point to gist.github.com, since
645 645 unfortunately http://paste.pocoo.org has closed down. We also added a -d flag
646 646 for the user to provide a gist description string. :ghpull:`1670`.
647 647
648 648 * Fix ``%paste`` that would reject certain valid inputs. :ghpull:`1258`.
649 649
650 650 * Fix sending and receiving of Numpy structured arrays (those with composite
651 651 dtypes, often used as recarrays). :ghpull:`2034`.
652 652
653 653 * Reconnect when the websocket connection closes unexpectedly. :ghpull:`1577`.
654 654
655 655 * Fix truncated representation of objects in the debugger by showing at least
656 656 80 characters' worth of information. :ghpull:`1793`.
657 657
658 658 * Fix logger to be Unicode-aware: logging could crash ipython if there was
659 659 unicode in the input. :ghpull:`1792`.
660 660
661 661 * Fix images missing from XML/SVG export in the Qt console. :ghpull:`1449`.
662 662
663 663 * Fix deepreload on Python 3. :ghpull:`1625`, as well as having a much cleaner
664 664 and more robust implementation of deepreload in general. :ghpull:`1457`.
665 665
666 666
667 667 Backwards incompatible changes
668 668 ------------------------------
669 669
670 670 * The exception :exc:`IPython.core.error.TryNext` previously accepted
671 671 arguments and keyword arguments to be passed to the next implementation
672 672 of the hook. This feature was removed as it made error message propagation
673 673 difficult and violated the principle of loose coupling.
General Comments 0
You need to be logged in to leave comments. Login now