docs/source/whatsnew/development.rst

=====================
 Development version
=====================

This document describes in-flight development work.

.. warning::

    Please do not edit this file by hand (doing so will likely cause merge
    conflicts for other Pull Requests). Instead, create a new file in the
    `docs/source/whatsnew/pr` folder

Select Notebook Name When Renaming a Notebook
---------------------------------------------

The default notebook name is Untitled. It's unlikely you want to keep this name
or part of it when naming your notebook. Instead, IPython will select the text
in the input field so the user can easily type over the name and change it.

clear_output changes
--------------------

* There is no longer a 500ms delay when calling ``clear_output``.
* The ability to clear stderr and stdout individually was removed.
* A new ``wait`` flag that prevents ``clear_output`` from being executed until new
  output is available.  This eliminates animation flickering by allowing the
  user to double buffer the output.
* The output div height is remembered when the ``wait=True`` flag is used.

Extending Configurable Containers
---------------------------------

Some configurable traits are containers (list, dict, set)
Config objects now support calling ``extend``, ``update``, ``insert``, etc.
on traits in config files, which will ultimately result in calling
those methods on the original object.

The effect being that you can now add to containers without having to copy/paste
the initial value::

    c = get_config()
    c.InlineBackend.rc.update({ 'figure.figsize' : (6, 4) })

Single codebase Python 3 support
--------------------------------

IPython previously supported Python 3 by running 2to3 during setup. We
have now switched to a single codebase which runs natively on Python 2.7
and 3.3.

For notes on how to maintain this, see :doc:`/development/pycompat`.

changes to hidden namespace on startup
--------------------------------------

Previously, all names declared in code run at startup
(startup files, ``ipython -i script.py``, etc.)
were added to the hidden namespace, which hides the names from tools like ``%whos``.
There are two changes to this behavior:

1. Scripts run on the command-line ``ipython -i script.py``now behave the same as if they were
   passed to ``%run``, so their variables are never hidden.
2. A boolean config flag ``InteractiveShellApp.hide_initial_ns`` has been added to optionally
   disable the hidden behavior altogether. The default behavior is unchanged.

Using dill to expand serialization support
------------------------------------------

adds :func:`~IPython.utils.pickleutil.use_dill` for allowing
dill to extend serialization support in :mod:`IPython.parallel` (closures, etc.).
Also adds :meth:`DirectView.use_dill` convenience method for enabling dill
locally and on all engines with one call.

New IPython Console Lexer
-------------------------

The IPython console lexer has been rewritten and now supports tracebacks
and customized input/output prompts. An entire suite of lexers is now
available at :mod:`IPython.nbconvert.utils.lexers`. These include:

IPythonLexer & IPython3Lexer
  Lexers for pure IPython (python + magic/shell commands)

IPythonPartialTracebackLexer & IPythonTracebackLexer
  Supports 2.x and 3.x via the keyword `python3`. The partial traceback
  lexer reads everything but the Python code appearing in a traceback.
  The full lexer combines the partial lexer with an IPython lexer.

IPythonConsoleLexer
  A lexer for IPython console sessions, with support for tracebacks.
  Supports 2.x and 3.x via the keyword `python3`.

IPyLexer
  A friendly lexer which examines the first line of text and from it,
  decides whether to use an IPython lexer or an IPython console lexer.
  Supports 2.x and 3.x via the keyword `python3`.

Previously, the :class:`IPythonConsoleLexer` class was available at
:mod:`IPython.sphinxext.ipython_console_hightlight`.  It was inserted
into Pygments' list of available lexers under the name `ipython`.  It should
be mentioned that this name is inaccurate, since an IPython console session
is not the same as IPython code (which itself is a superset of the Python
language).

Now, the Sphinx extension inserts two console lexers into Pygments' list of
available lexers. Both are IPyLexer instances under the names: `ipython` and
`ipython3`. Although the names can be confusing (as mentioned above), their
continued use is, in part, to maintain backwards compatibility and to
aid typical usage. If a project needs to make Pygments aware of more than just
the IPyLexer class, then one should not make the IPyLexer class available under
the name `ipython` and use `ipy` or some other non-conflicting value.

Code blocks such as:

.. code-block:: rst

    .. code-block:: ipython

        In [1]: 2**2
    Out[1]: 4

    will continue to work as before, but now, they will also properly highlight
tracebacks.  For pure IPython code, the same lexer will also work:

.. code-block:: rst

    .. code-block:: ipython

        x = ''.join(map(str, range(10)))
    !echo $x

    Since the first line of the block did not begin with a standard IPython console
prompt, the entire block is assumed to consist of IPython code instead.

DisplayFormatter changes
------------------------

There was no official way to query or remove callbacks in the Formatter API.
To remedy this, the following methods are added to :class:`BaseFormatter`:

- ``lookup(instance)`` - return appropriate callback or a given object
- ``lookup_by_type(type_or_str)`` - return appropriate callback for a given type or ``'mod.name'`` type string
- ``pop(type_or_str)`` - remove a type (by type or string).
  Pass a second argument to avoid KeyError (like dict).

All of the above methods raise a KeyError if no match is found.

And the following methods are changed:

- ``for_type(type_or_str)`` - behaves the same as before, only adding support for ``'mod.name'``
  type strings in addition to plain types. This removes the need for ``for_type_by_name()``,
  but it remains for backward compatibility.

Notebook Widgets
----------------

Available in the new `IPython.html.widgets` namespace, widgets provide an easy
way for IPython notebook users to display GUI controls in the IPython notebook.
IPython comes with bundle of built-in widgets and also the ability for users
to define their own widgets.  A widget is displayed in the front-end using
using a view.  For example, a FloatRangeWidget can be displayed using a
FloatSliderView (which is the default if no view is specified when displaying
the widget).  IPython also comes with a bundle of views and the ability for the
user to define custom views.  One widget can be displayed multiple times, in on
or more cells, using one or more views.  All views will automatically remain in
sync with the widget which is accessible in the back-end.

The widget layer provides an MVC-like architecture on top of the comm layer.
It's useful for widgets that can be expressed via a list of properties.
Widgets work by synchronizing IPython traitlet models in the back-end with
backbone models in the front-end. The widget layer automatically handles

* delta compression (only sending the state information that has changed)
* wiring the message callbacks to the correct cells automatically
* inter-view synchronization (handled by backbone)
* message throttling (to avoid flooding the kernel)
* parent/child relationships between views (which one can override to specify custom parent/child relationships)
* ability to manipulate the widget view's DOM from python using CSS, $().addClass, and $().removeClass methods

Signing Notebooks
-----------------

To prevent untrusted code from executing on users' behalf when notebooks open,
we have added a signature to the notebook, stored in metadata.

For more information, see :ref:`signing_notebooks`.

Dashboard "Running" Tab
-----------------------

The dashboard now has a "Running" tab which shows all of the running
notebooks.

Other changes
-------------

* `%%capture` cell magic now captures the rich display output, not just
  stdout/stderr

* In notebook, Showing tooltip on tab has been disables to avoid conflict with
  completion, Shift-Tab could still be used to invoke tooltip when inside
  function signature and/or on selection.

* ``object_info_request`` as been replaced by ``object_info`` for consistency in the javascript API.
  ``object_info`` as a simpler interface to register callback that is incompatible with ``object_info_request``.

* Previous versions of IPython on Linux would use the XDG config directory,
  creating :file:`~/.config/ipython` by default. We have decided to go
  back to :file:`~/.ipython` for consistency among systems. IPython will
  issue a warning if it finds the XDG location, and will move it to the new
  location if there isn't already a directory there.

* Equations, images and tables are now centered in Markdown cells.
* Multiline equations are now centered in output areas; single line equations
  remain left justified.

* IPython config objects can be loaded from and serialized to JSON.
  JSON config file have the same base name as their ``.py`` counterpart,
  and will be loaded with higher priority if found.

* bash completion updated with support for all ipython subcommands and flags, including nbconvert

* ``ipython history trim``: added ``--keep=<N>`` as an alias for the more verbose
  ``--HistoryTrim.keep=<N>``
* new ``ipython history clear`` subcommand, which is the same as the newly supported
  ``ipython history trim --keep=0``

* You can now run notebooks in an interactive session via ``%run notebook.ipynb``.

* Print preview is back in the notebook menus, along with options to
  download the open notebook in various formats. This is powered by
  nbconvert.

* :exc:`~IPython.nbconvert.utils.pandoc.PandocMissing` exceptions will be
  raised if Pandoc is unavailable, and warnings will be printed if the version
  found is too old. The recommended Pandoc version for use with nbconvert is
  1.12.1.

* The InlineBackend.figure_format flag now supports JPEG output if PIL/Pillow is available.

* Input transformers (see :doc:`/config/inputtransforms`) may now raise
  :exc:`SyntaxError` if they determine that input is invalid. The input
  transformation machinery in IPython will handle displaying the exception to
  the user and resetting state.

* Calling ``container.show()`` on javascript display is deprecated and will
  trigger errors on future IPython notebook versions. ``container`` now show
  itself as soon as non-empty

* Added ``InlineBackend.print_figure_kwargs`` to allow passing keyword arguments
  to matplotlib's ``Canvas.print_figure``. This can be used to change the value of
  ``bbox_inches``, which is 'tight' by default, or set the quality of JPEG figures.

* A new callback system has been introduced. For details, see :doc:`/config/callbacks`.

.. DO NOT EDIT THIS LINE BEFORE RELEASE. FEATURE INSERTION POINT.

Backwards incompatible changes
------------------------------

* Python 2.6 and 3.2 are no longer supported: the minimum required
  Python versions are now 2.7 and 3.3.
* The Transformer classes have been renamed to Preprocessor in nbconvert and
  their `call` methods for them have been renamed to `preprocess`.
* The `call` methods of nbconvert post-processsors have been renamed to
  `postprocess`.

* The module ``IPython.core.fakemodule`` has been removed.

* The alias system has been reimplemented to use magic functions. There should be little
  visible difference while automagics are enabled, as they are by default, but parts of the
  :class:`~IPython.core.alias.AliasManager` API have been removed.

* We fixed an issue with switching between matplotlib inline and GUI backends,
  but the fix requires matplotlib 1.1 or newer.  So from now on, we consider
  matplotlib 1.1 to be the minimally supported version for IPython. Older
  versions for the most part will work, but we make no guarantees about it.

* The :command:`pycolor` command has been removed. We recommend the much more capable
  :command:`pygmentize` command from the `Pygments <http://pygments.org/>`_ project.
  If you need to keep the exact output of :command:`pycolor`, you can still use
  ``python -m IPython.utils.PyColorize foo.py``.

* :mod:`IPython.lib.irunner` and its command-line entry point have been removed.
  It had fallen out of use long ago.

* The ``input_prefilter`` hook has been removed, as it was never
  actually used by the code. The input transformer system offers much
  more powerful APIs to work with input code. See
  :doc:`/config/inputtransforms` for details.

* :class:`IPython.core.inputsplitter.IPythonInputSplitter` no longer has a method
  ``source_raw_reset()``, but gains :meth:`~IPython.core.inputsplitter.IPythonInputSplitter.raw_reset`
  instead. Use of ``source_raw_reset`` can be replaced with::

      raw = isp.source_raw
      transformed = isp.source_reset()

* The Azure notebook manager was removed as it was no longer compatible with the notebook storage scheme

.. DO NOT EDIT THIS LINE BEFORE RELEASE. INCOMPAT INSERTION POINT.

Simplifying configurable URLs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- base_kernel_url configurable is removed
- websocket_url configurable is removed
- base_project_url is renamed to base_url (base_project_url is kept as a deprecated alias, for now)