##// END OF EJS Templates
Add 0.13 release notes - in progress.
Fernando Perez -
Show More
This diff has been collapsed as it changes many lines, (577 lines changed) Show them Hide them
@@ -0,0 +1,577
1 =============
2 0.13 Series
3 =============
4
5 Release 0.13
6 ============
7
8 IPython 0.13 contains several major new features, as well as a large amount of
9 bug and regression fixes. The previous version (0.12) was released on December
10 19 2011, so this release cycle was roughly 6 months long, during which we
11 closed a total of 373 pull requests and 742 issues, with contributions from 62
12 authors comprising over 1740 commits.
13
14 The amount of work included in this release is so large, that we can only cover
15 here the main highlights; please see our :ref:`detailed release statistics
16 <issues_list_013>` for links to every issue and pull request closed on github
17
18
19 Major Notebook improvements: new user interface and more
20 --------------------------------------------------------
21
22 The IPython Notebook, which has proven since its release to be wildly popular,
23 has seen a massive amount of work in this release cycle, leading to a
24 significantly improved user experience as well as many new features.
25
26 The first user-visible change is a reorganization of the user interface; the
27 left panel has been removed and was replaced by a real menu system and a
28 toolbar with icons. Both the toolbar and the header above the menu can be
29 collapsed to leave an unobstructed working area:
30
31 .. image:: ../_static/ipy_013_notebook_spectrogram.png
32 :width: 400px
33 :alt: New user interface for Notebook
34 :align: center
35 :target: ../_static/ipy_013_notebook_spectrogram.png
36
37 The notebook handles very long outputs much better than before (this was a
38 serious usability issue when running processes that generated massive amounts
39 of output). Now, in the presence of outputs longer than ~100 lines, the
40 notebook will automatically collapse to a scrollable area and the entire left
41 part of this area controls the display: one click in this area will expand the
42 output region completely, and a double-click will hide it completely. This
43 figure shows both the scrolled and hidden modes:
44
45 .. image:: ../_static/ipy_013_notebook_long_out.png
46 :width: 400px
47 :alt: Scrolling and hiding of long output in the notebook.
48 :align: center
49 :target: ../_static/ipy_013_notebook_long_out.png
50
51 .. note::
52
53 The auto-folding of long outputs is disabled in Firefox due to bugs in its
54 scrolling behavior. See :ghpull:`2047` for details.
55
56 Uploading notebooks to the dashboard is now easier: in addition to drag and
57 drop (which can be finicky sometimes), you can now click on the upload text and
58 use a regular file dialog box to select notebooks to upload. Furthermore, the
59 notebook dashboard now auto-refreshes its contents and offers buttons to shut
60 down any running kernels (:ghpull:`1739`):
61
62 .. image:: ../_static/ipy_013_dashboard.png
63 :width: 400px
64 :alt: Improved dashboard
65 :align: center
66 :target: ../_static/ipy_013_dashboard.png
67
68
69 Cluster management
70 ~~~~~~~~~~~~~~~~~~
71
72 The notebook dashboard can now also start and stop clusters, and you can
73 override the number of engines started. There is a new tab in the dashboard
74 user interface:
75
76 .. image:: ../_static/ipy_013_dashboard_cluster.png
77 :width: 400px
78 :alt: Cluster management from the notebook dashboard
79 :align: center
80 :target: ../_static/ipy_013_dashboard_cluster.png
81
82 This tab allows you, for each profile you have configured, to start and stop a
83 cluster (and optionally override the default number of engines corresponding to
84 that configuration). While this hides all error reporting, once you have a
85 configuration that you know works smoothly, it is a very convenient interface
86 for controlling your parallel resources.
87
88
89 New notebook format
90 ~~~~~~~~~~~~~~~~~~~
91
92 The notebooks saved now use version 3 of our format, which supports heading
93 levels as well as the concept of 'raw' text cells that are not rendered as
94 Markdown. These will be useful with converters_ we are developing, to pass raw
95 markup (say LaTeX). That conversion code is still under heavy development and
96 not quite ready for prime time, but we welcome help on this front so that we
97 can merge it for full production use as soon as possible.
98
99 .. _converters: https://github.com/ipython/nbconvert
100
101 .. note::
102
103 v3 notebooks can *not* be read by older versions of IPython, but we provide
104 a `simple script`__ that you can use in case you need to export a v3
105 notebook to share with a v2 user.
106
107 .. _simple script: https://gist.github.com/1935808
108
109
110 JavaScript refactoring
111 ~~~~~~~~~~~~~~~~~~~~~~
112
113 All the client-side JavaScript has been decoupled to ease reuse of parts of the
114 machinery without having to build a full-blown notebook. This will make it much
115 easier to communicate with an IPython kernel from existing web pages and to
116 integrate single cells into other sites, without loading the full notebook
117 document-like UI. :ghpull:`1711`.
118
119 This refactoring also enables the possibility of writing dynamic javascript
120 widgets that are returned from Python code and that present an interactive view
121 to the user, with callbacks in Javascript executing calls to the Kernel. This
122 will enable many interactive elements to be added by users in notebooks.
123
124 An example of this capability has been provided as a proof of concept in
125 `docs/examples/widgets` that lets you directly communicate with one or more
126 parallel engines, acting as a mini-console for parallel debugging and
127 introspection.
128
129
130 Improved tooltips
131 ~~~~~~~~~~~~~~~~~
132
133 The object tooltips have gained some new functionality. By pressing tab several
134 times, you can expand them to see more of a docstring, keep them visible as you
135 fill in a function's parameters, or transfer the information to the pager at the
136 bottom of the screen. For the details, look at the example notebook
137 :file:`01_notebook_introduction.ipynb`.
138
139 .. figure:: ../_static/ipy_013_notebook_tooltip.png
140 :width: 400px
141 :alt: Improved tooltips in the notebook.
142 :align: center
143 :target: ../_static/ipy_013_notebook_tooltip.png
144
145 The new notebook tooltips.
146
147 Other improvements to the Notebook
148 ----------------------------------
149
150 * The notebook pager (the area at the bottom) is now resizeable by dragging its
151 divider handle, a feature that had been requested many times by just about
152 anyone who had used the notebook system. :ghpull:`1705`.
153
154 * If a notebook directory is specified with ``--notebook-dir`` (or with the
155 corresponding configuration flag ``NotebookManager.notebook_dir``), all
156 kernels start in this directory.
157
158 * It is now possible to open notebooks directly from the command line; for
159 example: ``ipython notebook path/`` will automatically set ``path/`` as the
160 notebook directory, and ``ipython notebook path/foo.ipynb`` will further
161 start with the ``foo.ipynb`` notebook opened. :ghpull:`1686`.
162
163 * Fix codemirror clearing of cells with ``Ctrl-Z``; :ghpull:`1965`.
164
165 * Text (markdown) cells now line wrap correctly in the notebook, making them
166 much easier to edit :ghpull:`1330`.
167
168 * PNG and JPEG figures returned from plots can be interactively resized in the
169 notebook, by dragging them from their lower left corner. :ghpull:`1832`.
170
171 * Clear In[] prompt numbers on "Clear All Output". For more
172 version-control-friendly `.ipynb` files, this strips the `In[]` prompt
173 numbers when doing a "Clear all output". This reduces the amount of noise in
174 commit-to-commit diffs that would otherwise show the (highly variable) prompt
175 number changes. :ghpull:`1621`.
176
177 * The notebook server now requires *two* consecutive ``Ctrl-C`` to stop within 5
178 seconds (or an interactive confirmation). This makes it less likely that you
179 will accidentally kill a long-running server by typing ``Ctrl-C`` in the
180 wrong terminal. :ghpull:`1609`.
181
182 * Using ``Ctrl-S`` (or ``Cmd-S`` on a Mac) actually saves the notebook rather
183 than providing the fairly useless browser html save dialog. :ghpull:`1334`.
184
185 * Allow accessing local files from the notebook (in urls), by serving any local
186 file as the url ``files/<relativepath``. This makes it possible to, for
187 example, embed local images in a notebook. :ghpull:`1211`.
188
189
190 Cell magics
191 -----------
192
193 We have completely refactored the magic system, finally moving the magic
194 objects to standalone, independent objects instead of being the mixin class
195 we'd had since the beginning of IPython (:ghpull:`1732`). Now, a separate base
196 class is provided in :class:`IPython.core.magic.Magics` that users can subclass
197 to create their own magics. Decorators are also provided to create magics from
198 simple functions without the need for object orientation.
199
200 All builtin magics now exist in a few subclasses that group together related
201 functionality, and the new :mod:`IPython.core.magics` package has been created
202 to organize this into smaller files.
203
204 This cleanup was the last major piece of deep refactoring needed from the
205 original 2001 codebase.
206
207 We have also introduced a new type of magic function, prefixed with `%%`
208 instead of `%`, which operates at the cell level. A cell magic receives two
209 arguments: the line it is called on (like a line magic) and the body of the
210 cell below it.
211
212 Cell magics are most natural in the notebook, but they also work in the
213 terminal and qt console, with the usual approach of using a blank line to
214 signal cell termination.
215
216 For example, to time the execution of several statements::
217
218 %%timeit x = 0 # setup
219 for i in range(100000):
220 x += i**2
221
222 This is particularly useful to integrate code in another language, and cell
223 magics already exist for shell scripts, Cython, R and Octave. Using ``%%script
224 /usr/bin/foo``, you can run a cell in any interpreter that accepts code via
225 stdin.
226
227 Another handy cell magic makes it easy to write short text files: ``%%file
228 ~/save/to/here.txt``.
229
230 The following cell magics are now included by default; all those that use
231 special interpreters (Perl, Ruby, bash, etc., assume you have the requisite
232 interpreter installed):
233
234 * ``%%!``: run cell body with the underlying OS shell; this is similar to
235 prefixing every line in the cell with ``!``.
236
237 * ``%%bash``: run cell body under bash.
238
239 * ``%%capture``: capture the output of the code in the cell (and stderr as
240 well). Useful to run codes that produce too much output that you don't even
241 want scrolled.
242
243 * ``%%file``: save cell body as a file.
244
245 * ``%%perl``: run cell body using Perl.
246
247 * ``%%prun``: run cell body with profiler (cell extension of ``%prun``).
248
249 * ``%%python3``: run cell body using Python 3.
250
251 * ``%%ruby``: run cell body using Ruby.
252
253 * ``%%script``: run cell body with the script specified in the first line.
254
255 * ``%%sh``: run cell body using sh.
256
257 * ``%%sx``: capture cell output running the code with the system shell (cell
258 extension of ``%sx``).
259
260 * ``%%system``: run cell with system shell (similar to ``%%!``)
261
262 * ``%%timeit``: time the execution of the cell (extension of ``%timeit``).
263
264 This is what some of the script-related magics look like in action:
265
266 .. image:: ../_static/ipy_013_notebook_script_cells.png
267 :width: 400px
268 :alt: Cluster management from the notebook dashboard
269 :align: center
270 :target: ../_static/ipy_013_notebook_script_cells.png
271
272 In addition, we have also a number of :ref:`extensions <extensions_overview>`
273 that provide specialized magics. These typically require additional software
274 to run and must be manually loaded via ``%load_ext <extension name>``, but are
275 extremely useful. In particular, we want to highlight a few:
276
277
278
279
280 Tab completer improvements
281 --------------------------
282
283 Useful tab-completion based on live inspection of objects is one of the most
284 popular features of IPython. To make this process even more user-friendly, the
285 completers of both the Qt console and the Notebook have been reworked.
286
287 The Qt console comes with a new ncurses-like tab completer, activated by
288 default, which lets you cycle through the available completions by pressing tab,
289 or select a completion with the arrow keys (:ghpull:`1851`).
290
291 .. figure:: ../_static/ipy_013_qtconsole_completer.png
292 :width: 400px
293 :alt: ncurses-like completer, with highlighted selection.
294 :align: center
295 :target: ../_static/ipy_013_qtconsole_completer.png
296
297 The new improved Qt console's ncurses-like completer allows to easily
298 navigate thought long list of completions.
299
300 In the notebook, completions are now sourced both from object introspection and
301 analysis of surrounding code, so limited completions can be offered for
302 variables defined in the current cell, or while the kernel is busy
303 (:ghpull:`1711`).
304
305
306 We have implemented a new configurable flag to control tab completion on
307 modules that provide the ``__all__`` attribute::
308
309 IPCompleter.limit_to__all__= Boolean.
310
311 This instructs the completer to honor ``__all__`` for the completion.
312 Specifically, when completing on ``object.<tab>``, if True: only those names
313 in ``obj.__all__`` will be included. When False [default]: the ``__all__``
314 attribute is ignored. :ghpull:`1529`.
315
316
317 Improvements to the Qt console
318 ------------------------------
319
320 * changes for easier integration into other projects such as Spyder_.
321
322 .. _spyder: https://code.google.com/p/spyderlib
323
324 * Improved menus with a new Magic menu that is organized by magic groups (this
325 was made possible by the reorganization of the magic system
326 internals). :ghpull:`1782`.
327
328 * Allow for restarting kernels without clearing the qtconsole, while leaving a
329 visible indication that the kernel has restarted. :ghpull:`1681`.
330
331 * Allow the native display of jpeg image in the qtconsole. :ghpull:`1643`.
332
333
334 Parallel
335 --------
336
337 The parallel tools have been improved and fine-tuned on multiple fronts. Now,
338 the creation of an :class:`IPython.parallel.Client` object automatically
339 activates a line and cell magic function ``px`` that sends its code to all the
340 engines. Further magics can be easily created with the :meth:`Client.activate`
341 method, to conveniently execute code on any subset of engines. :ghpull:`1893`.
342
343 The ``%%px`` cell magic can also be given an optional targets argument, as well
344 as a ``--out`` argument for storing its output.
345
346 A new magic has also been added, ``%pxconfig``, that lets you configure various
347 defaults of the parallel magics. As usual, type ``%pxconfig?`` for details.
348
349 The exception reporting in parallel contexts has been improved to be easier to
350 read. Now, IPython directly reports the remote exceptions without showing any
351 of the internal execution parts:
352
353 .. image:: ../_static/ipy_013_par_tb.png
354 :width: 400px
355 :alt: Improved parallel exceptions.
356 :align: center
357 :target: ../_static/ipy_013_par_tb.png
358
359
360 The parallel tools now default to using ``NoDB`` as the storage backend for
361 intermediate results. This means that the default usage case will have a
362 significantly reduced memory footprint, though certain advanced features are
363 not available with this backend. For more details, see :ref:`parallel_db`.
364
365 The parallel magics now display all output, so you can do parallel plotting or
366 other actions with complex display. The ``px`` magic has now both line and cell
367 modes, and in cell mode finer control has been added about how to collate
368 output from multiple engines. :ghpull:`1768`.
369
370 Incremental improvements to SSH launchers:
371
372 * add to_send/fetch steps for moving connection files around.
373
374 * add SSHProxyEngineSetLauncher, for invoking to `ipcluster engines` on a
375 remote host. This can be used to start a set of engines via PBS/SGE/MPI
376 *remotely*.
377
378 This makes the SSHLauncher usable on machines without shared filesystems.
379
380 When sending files, the destination directory must *already exist* - that is,
381 `ipython profile create` may be necessary on the remote system, before the
382 security dir exists for putting the connection file the first
383 time. :ghpull:`1634`.
384
385 Add sugar methods/properties to AsyncResult that are generically useful
386 (:ghpull:`1548`):
387
388 * ``ar.wall_time`` = received - submitted
389 * ``ar.serial_time`` = sum of serial computation time
390 * ``ar.elapsed`` = time since submission (wall_time if done)
391 * ``ar.progress`` = (int) number of sub-tasks that have completed
392 * ``len(ar)`` = # of tasks
393 * ``ar.wait_interactive()``: prints progress
394
395 Added ``Client.spin_thread(interval)`` / ``stop_spin_thread()`` for running
396 spin in a background thread, to keep zmq queue clear. This can be used to
397 ensure that timing information is as accurate as possible (at the cost of
398 having a background thread active).
399
400 Set TaskScheduler.hwm default to 1 instead of 0. 1 has more
401 predictable/intuitive behavior, if often slower, and thus a more logical
402 default. Users whose workloads require maximum throughput and are largely
403 homogeneous in time per task can make the optimization themselves, but now the
404 behavior will be less surprising to new users. :ghpull:`1294`.
405
406
407 Kernel/Engine unification
408 -------------------------
409
410 :func:`IPython.parallel.bind_kernel`
411
412 Add IPython.embed_kernel() as a public API.
413 Embedding an IPython kernel in an application is useful when you want to use
414 IPython.embed() but don't have a terminal attached on stdin and stdout. 1640
415
416
417
418 Official Public API
419 -------------------
420
421 We have begun organizing our API for easier public use, with an eye towards an
422 official IPython 1.0 release which will firmly maintain this API compatible for
423 its entire lifecycle. There is now an :mod:`IPython.display` module that
424 aggregates all display routines, and the :mod:`IPython.config` namespace has
425 all public configuration tools. We will continue improving our public API
426 layout so that users only need to import names one level deeper than the main
427 ``IPython`` package to access all public namespaces.
428
429
430 IPython notebook file icons
431 ---------------------------
432
433 The directory ``docs/resources`` in the source distribution contains SVG and
434 PNG versions of our file icons, as well as an ``Info.plist.example`` file with
435 instructions to install them on Mac OSX. This is a first draft of our icons,
436 and we encourage contributions from users with graphic talent to improve them
437 in the future:
438
439 .. image:: ../../resources/ipynb_icon_128x128.png
440 :alt: IPython notebook file icon.
441
442
443 New top-level `locate` command
444 ------------------------------
445
446 Add `locate` entry points; these would be useful for quickly locating IPython
447 directories and profiles from other (non-Python) applications. :ghpull:`1762`.
448
449 Examples::
450
451 $> ipython locate
452 /Users/me/.ipython
453
454 $> ipython locate profile foo
455 /Users/me/.ipython/profile_foo
456
457 $> ipython locate profile
458 /Users/me/.ipython/profile_default
459
460 $> ipython locate profile dne
461 [ProfileLocate] Profile u'dne' not found.
462
463
464 Other new features and improvements
465 -----------------------------------
466
467 * **%install_ext**: A new magic function to install an IPython extension from
468 a URL. E.g. ``%install_ext
469 https://bitbucket.org/birkenfeld/ipython-physics/raw/default/physics.py``.
470
471 * The ``%loadpy`` magic is no longer restricted to Python files, and has been
472 renamed ``%load``. The old name remains as an alias.
473
474 * New command line arguments will help external programs find IPython folders:
475 ``ipython locate`` finds the user's IPython directory, and ``ipython locate
476 profile foo`` finds the folder for the 'foo' profile (if it exists).
477
478 * The :envvar:`IPYTHON_DIR` environment variable, introduced in the Great
479 Reorganization of 0.11 and existing only in versions 0.11-0.13, has been
480 deprecated. As described in :ghpull:`1167`, the complexity and confusion of
481 migrating to this variable is not worth the aesthetic improvement. Please use
482 the historical :envvar:`IPYTHONDIR` environment variable instead.
483
484 * The default value of *interactivity* passed from
485 :meth:`~IPython.core.interactiveshell.InteractiveShell.run_cell` to
486 :meth:`~IPython.core.interactiveshell.InteractiveShell.run_ast_nodes`
487 is now configurable.
488
489 * New ``%alias_magic`` function to conveniently create aliases of existing
490 magics, if you prefer to have shorter names for personal use.
491
492 * We ship unminified versions of the JavaScript libraries we use, to better
493 comply with Debian's packaging policies.
494
495 * Simplify the information presented by ``obj?/obj??`` to eliminate a few
496 redundant fields when possible. :ghpull:`2038`.
497
498 * Improved continuous integration for IPython. We now have automated test runs
499 on `Shining Panda <https://jenkins.shiningpanda.com/ipython>`_ and `Travis-CI
500 <http://travis-ci.org/#!/ipython/ipython>`_, as well as `Tox support
501 <http://tox.testrun.org>`_.
502
503 * The `vim-ipython`_ functionality (externally developed) has been updated to
504 the latest version.
505
506 .. _vim-ipython: https://github.com/ivanov/vim-ipython
507
508 * The ``%save`` magic now has a ``-f`` flag to force overwriting, which makes
509 it much more usable in the notebook where it is not possible to reply to
510 interactive questions from the kernel. :ghpull:`1937`.
511
512 * Use dvipng to format sympy.Matrix, enabling display of matrices in the Qt
513 console with the sympy printing extension. :ghpull:`1861`.
514
515 * Our messaging protocol now has a reasonable test suite, helping ensure that
516 we don't accidentally deviate from the spec and possibly break third-party
517 applications that may have been using it. We encourage users to contribute
518 more stringent tests to this part of the test suite. :ghpull:`1627`.
519
520 * Use LaTeX to display, on output, various built-in types with the SymPy
521 printing extension. :ghpull:`1399`.
522
523 * Add Gtk3 event loop integration and example. :ghpull:`1588`.
524
525 * ``clear_output`` improvements, which allow things like progress bars and other
526 simple animations to work well in the notebook (:ghpull:`1563`):
527
528 * `clear_output()` clears the line, even in terminal IPython, the QtConsole
529 and plain Python as well, by printing `\r` to streams.
530
531 * `clear_output()` avoids the flicker in the notebook by adding a delay,
532 and firing immediately upon the next actual display message.
533
534 * `display_javascript` hides its `output_area` element, so using display to
535 run a bunch of javascript doesn't result in ever-growing vertical space.
536
537 * Add simple support for running inside a virtualenv. While this doesn't
538 supplant proper installation (as users should do), it helps ad-hoc calling of
539 IPython from inside a virtualenv. :ghpull:`1388`.
540
541
542 Major Bugs fixed
543 ----------------
544
545 In this cycle, we have :ref:`closed over 740 issues <issues_list_013>`, but a
546 few major ones merit special mention:
547
548 * The ``%pastebin`` magic has been updated to point to gist.github.com, since
549 unfortunately http://paste.pocoo.org has closed down. We also added a -d flag
550 for the user to provide a gist description string. :ghpull:`1670`.
551
552 * Fix ``%paste`` that would reject certain valid inputs. :ghpull:`1258`.
553
554 * Fix sending and receiving of Numpy structured arrays (those with composite
555 dtypes, often used as recarrays). :ghpull:`2034`.
556
557 * Reconnect when the websocket connection closes unexpectedly. :ghpull:`1577`.
558
559 * Fix truncated representation of objects in the debugger by showing at least
560 80 characters' worth of information. :ghpull:`1793`.
561
562 * Fix logger to be Unicode-aware: logging could crash ipython if there was
563 unicode in the input. :ghpull:`1792`.
564
565 * Fix images missing from XML/SVG export in the Qt console. :ghpull:`1449`.
566
567 * Fix deepreload on Python 3. :ghpull:`1625`, as well as having a much cleaner
568 and more robust implementation of deepreload in general. :ghpull:`1457`.
569
570
571 Backwards incompatible changes
572 ------------------------------
573
574 * The exception :exc:`IPython.core.error.TryNext` previously accepted
575 arguments and keyword arguments to be passed to the next implementation
576 of the hook. This feature was removed as it made error message propagation
577 difficult and violated the principle of loose coupling.
General Comments 0
You need to be logged in to leave comments. Login now