Improve release notes and config details

IPython/core/interactiveshell.py

@@ -250,7 +250,7 b' class InteractiveShell(SingletonConfigurable):'
     ).tag(config=True)
     colors = CaselessStrEnum(('Neutral', 'NoColor','LightBG','Linux'),
                              default_value='Neutral',
-        help="Set the color scheme (NoColor, Linux, or LightBG)."
+        help="Set the color scheme (NoColor, Neutral, Linux, or LightBG)."
     ).tag(config=True)
     colors_force = Bool(False, help=
         """

docs/source/config/details.rst

@@ -2,89 +2,85 b''
 Specific config details
 =======================
 
+.. _custom_prompts:
+
+Custom Prompts
+==============
+
+.. versionchanged:: 5.0
+
+From IPython 5, prompts are produced as a list of Pygments tokens, which are
+tuples of (token_type, text). You can customise prompts by writing a method
+which generates a list of tokens.
+
+There are four kinds of prompt:
+
+* The **in** prompt is shown before the first line of input
+  (default like ``In [1]:``).
+* The **continuation** prompt is shown before further lines of input
+  (default like ``...:``).
+* The **rewrite** prompt is shown to highlight how special syntax has been
+  interpreted (default like ``----->``).
+* The **out** prompt is shown before the result from evaluating the input
+  (default like ``Out[1]:``).
+
+Custom prompts are supplied together as a class. If you want to customise only
+some of the prompts, inherit from :class:`IPython.terminal.prompts.Prompts`,
+which defines the defaults. The required interface is like this:
+
+.. class:: MyPrompts(shell)
+
+   Prompt style definition. *shell* is a reference to the
+   :class:`~.TerminalInteractiveShell` instance.
+
+   .. method:: in_prompt_tokens(cli=None)
+               continuation_prompt_tokens(self, cli=None, width=None)
+               rewrite_prompt_tokens()
+               out_prompt_tokens()
+
+      Return the respective prompts as lists of ``(token_type, text)`` tuples.
+
+      For continuation prompts, *width* is an integer representing the width of
+      the prompt area in terminal columns.
+
+      *cli*, where used, is the prompt_toolkit ``CommandLineInterface`` instance.
+      This is mainly for compatibility with the API prompt_toolkit expects.
+
+Inside IPython or in a startup script, you can use a custom prompts class
+by setting ``get_ipython().prompts`` to an *instance* of the class.
+In configuration, ``TerminalInteractiveShell.prompts_class`` may be set to
+either the class object, or a string of its full importable name.
+
 .. _termcolour:
 
 Terminal Colors
 ===============
 
-The default IPython configuration has most bells and whistles turned on
-(they're pretty safe). But there's one that may cause problems on some
-systems: the use of color on screen for displaying information. This is
-very useful, since IPython can show prompts and exception tracebacks
-with various colors, display syntax-highlighted source code, and in
-general make it easier to visually parse information.
-
-The following terminals seem to handle the color sequences fine:
-
-    * Linux main text console, KDE Konsole, Gnome Terminal, E-term,
-      rxvt, xterm.
-    * CDE terminal (tested under Solaris). This one boldfaces light colors.
-    * (X)Emacs buffers. See the :ref:`emacs` section for more details on
-      using IPython with (X)Emacs.
-    * A Windows (XP/2k) CygWin shell. Although some users have reported
-      problems; it is not clear whether there is an issue for everyone
-      or only under specific configurations. If you have full color
-      support under cygwin, please post to the IPython mailing list so
-      this issue can be resolved for all users.
-
-These have shown problems:
-
-    * Windows command prompt in WinXP/2k logged into a Linux machine via
-      telnet or ssh.
-    * Windows native command prompt in WinXP/2k, without Gary Bishop's
-      extensions. Once Gary's readline library is installed, the normal
-      WinXP/2k command prompt works perfectly.
-
-IPython uses colors for various groups of things that may be
-controlled by different configuration options: prompts, tracebacks, "as
-you type" in the terminal, and the object introspection system which
-passes large sets of data through a pager. There are various way to
-change the colors.
-
-We can distinguish the coloration into 2 main categories:
-
-- The one that affects only the terminal client.
-- The ones that also affect clients connected through the Jupyter
-  protocol.
-
-Traceback, debugger, and pager are highlighted kernel-side so they fall
-into the second category. For historical reasons they are often
-governed by a ``colors`` attribute or configuration option that can
-take one of 3 case insensitive values: ``NoColors``, ``Linux`` and
-``LightBG``.
-
-Colors that affect only the terminal client are governed mainly by
-``TerminalInteractiveShell.highlight_style`` taking the name of a
-``Pygments`` style.
-
-As of IPython 5.0 the color configuration works as follows:
-
-  - by default, ``TerminalInteractiveShell.highlight_style`` is set to
-    ``legacy`` which **trys to** emulate the colors of IPython pre 5.0
-    and respect the ``.color`` configuration option.
-    The emulation is an approximation of the current version of Pygments
-    (2.1) and only supports extended ANSI escape sequence, hence the
-    theme cannot adapt to your terminal custom mapping if you have
-    one.
-
-    The last extra difference being that the "as you type" coloration
-    is present using the theme "default" if `color` is `LightBG`, and
-    using the theme "monokai" if `Linux`.
-
-  - if ``TerminalInteractiveShell.highlight_style`` is set to any other
-    themes, this theme is used for "as you type" highlighting. The
-    prompt highlighting is then governed by
-    ``--TerminalInteractiveShell.highlighting_style_overrides``
-
-As a summary, by default IPython 5.0 should mostly behave unchanged
-from IPython 4.x and before. Use
-``TerminalInteractiveShell.highlight_style`` and
-``--TerminalInteractiveShell.highlighting_style_overrides`` for extra
-flexibility.
-
-With default configuration `--colors=[nocolors|linux|ightbg]` as well
-as the `%colors` magic should behave identically as before.
+.. versionchanged:: 5.0
+
+There are two main configuration options controlling colours.
+
+``InteractiveShell.colors`` sets the colour of tracebacks and object info (the
+output from e.g. ``zip?``). It may also affect other things if the option below
+is set to ``'legacy'``. It has four case-insensitive values:
+``'nocolor', 'neutral', 'linux', 'lightbg'``. The default is *neutral*, which
+should be legible on either dark or light terminal backgrounds. *linux* is
+optimised for dark backgrounds and *lightbg* for light ones.
+
+``TerminalInteractiveShell.highlight_style`` determines prompt colours and syntax
+highlighting. It takes the name of a Pygments style as a string, or the special
+value ``'legacy'`` to pick a style in accordance with ``InteractiveShell.colors``.
 
+You can see the Pygments styles available on your system by running::
+
+    import pygments
+    list(pygments.styles.get_all_styles())
+
+Additionally, ``TerminalInteractiveShell.highlight_style_overrides`` can override
+specific styles in the highlighting. It should be a dictionary mapping Pygments
+token types to strings defining the style. See `Pygments' documentation
+<http://pygments.org/docs/styles/#creating-own-styles>`__ for the language used
+to define styles.
 
 Colors in the pager
 -------------------
@@ -98,9 +94,6 b' To configure your default pager to allow these:'
    properly interpret control sequences, which is how color
    information is given to your terminal.
 
-
-
-
 .. _editors:
 
 Editor configuration

docs/source/whatsnew/version5.rst

@@ -5,10 +5,27 b''
 IPython 5.0
 ===========
 
-Released June, 2016
+Released July, 2016
 
-IPython 5.0 now uses ``prompt-toolkit`` for the command line interface, thus
-allowing real multi-line editing and syntactic coloration as you type.
+New terminal interface
+----------------------
+
+IPython 5 features a major upgrade to the terminal interface, bringing live
+syntax highlighting as you type, proper multiline editing and multiline paste,
+and tab completions that don't clutter up your history.
+
+.. image:: ../_images/ptshell_features.png
+    :alt: New terminal interface features
+    :align: center
+    :target: ../_images/ptshell_features.png
+
+These features are provided by the Python library `prompt_toolkit
+<http://python-prompt-toolkit.readthedocs.io/en/stable/>`__, which replaces
+``readline`` throughout our terminal interface.
+
+Relying on this pure-Python, cross platform module also makes it simpler to
+install IPython. We have removed dependencies on ``pyreadline`` for Windows and
+``gnureadline`` for Mac.
 
 
 When using IPython as a subprocess, like for emacs inferior-shell, IPython can
@@ -19,57 +36,44 b' disabled.'
 Backwards incompatible changes
 ------------------------------
 
-
-The `install_ext magic` function which was deprecated since 4.0 has now been deleted.
-You can still distribute and install extensions as packages on PyPI.
-
-Update IPython event triggering to ensure callback registration and
-unregistration will only affect the set of callbacks the *next* time that event is
-triggered. See :ghissue:`9447` and :ghpull:`9453`.
-
-This is a change to the existing semantics, wherein one callback registering a
-second callback when triggered for an event would previously be invoked for
-that same event.
-
-Integration with pydb has been removed since pydb development has been stopped
-since 2012, and pydb is not installable from PyPI.
-
-
-
-Replacement of readline in TerminalInteractiveShell and PDB
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-IPython 5.0 now uses ``prompt_toolkit``. The
-``IPython.terminal.interactiveshell.TerminalInteractiveShell`` now uses
-``prompt_toolkit``. It is an almost complete rewrite, so many settings have
-thus changed or disappeared. The class keep the same name to avoid breaking
-user configuration for the options with names that are unchanged.
-
-The usage of ``prompt_toolkit`` is accompanied by a complete removal of all
-code, using ``readline``. A particular effect of not using `readline` anymore
-is that `.inputrc` settings are note effective anymore. Options having similar
-effects have likely been replaced by a configuration option on IPython itself
-(e.g: vi input mode).
-
-The `PromptManager` class have been removed, and the prompt machinery simplified.
-See `TerminalInteractiveShell.prompts` configurable for how to setup your prompts.
-
-.. note::
-
-    During development and beta cycle, ``TerminalInteractiveShell`` was
-    temporarly moved to ``IPython.terminal.ptshell``.
-
-
-Most of the above remarks also affect `IPython.core.debugger.Pdb`, the `%debug`
-and `%pdb` magic which do not use readline anymore either.
-
-The color handling has been slightly changed and is now exposed,
-in particular the colors of prompts and as you type
-highlighting can be affected by :
-``TerminalInteractiveShell.highlight_style``. With default
-configuration the ``--colors`` flag and ``%colors`` magic behavior
-should be mostly unchanged. See the `colors <termcolour>`_ section of
-our documentation
+- The ``%install_ext`` magic function, deprecated since 4.0, has now been deleted.
+  You can distribute and install extensions as packages on PyPI.
+- Callbacks registered while an event is being handled will now only be called
+  for subsequent events; previously they could be called for the current event.
+  Similarly, callbacks removed while handling an event *will* always get that
+  event. See :ghissue:`9447` and :ghpull:`9453`.
+- Integration with pydb has been removed since pydb development has been stopped
+  since 2012, and pydb is not installable from PyPI.
+- The ``autoedit_syntax`` option has apparently been broken for many years.
+  It has been removed.
+
+New terminal interface
+~~~~~~~~~~~~~~~~~~~~~~
+
+The overhaul of the terminal interface will probably cause a range of minor issues;
+this is inevitable for such a significant change. These are some that we're aware of:
+
+IPython no longer uses readline configuration (``~/.inputrc``). We hope that
+the functionality you want (e.g. vi input mode) will be available by configuring
+IPython directly (see :doc:`/config/options/terminal`).
+If something's missing, please file an issue.
+
+The ``PromptManager`` class has been removed, and the prompt machinery simplified.
+See :ref:`custom_prompts` to customise prompts with the new machinery.
+
+:mod:`IPython.core.debugger` now provides a plainer interface.
+:mod:`IPython.terminal.debugger` contains the terminal debugger using
+prompt_toolkit.
+
+There are new options to configure the colours used in syntax highlighting.
+We have tried to integrate them with our classic  ``--colors`` option and
+``%colors`` magic, but there's a mismatch in possibilities, so some configurations
+may produce unexpected results. See :ref:`termcolour` for more information.
+
+The new interface is not compatible with Emacs 'inferior-shell' feature. To
+continue using this, add the ``--simple-prompt`` flag to the command Emacs
+runs. This flag disables most IPython features, relying on Emacs to provide
+things like tab completion.
 
 Provisional Changes
 -------------------
@@ -116,12 +120,6 b' widgets... As stated above this is nightly experimental feature with a lot of'
 it.
 
 
-Removed Feature
----------------
-
-- ``TerminalInteractiveShell.autoedit_syntax`` Has been broken for many years now
-  apparently. It has been removed.
-
 
 Deprecated Features
 -------------------