Describe the IPython situation in What's New docs.

docs/source/whatsnew/development.txt

@@ -1,298 +1,308 b''
 ================================================
 Development version
 ================================================
 
 The changes listed here are a brief summary of the substantial work on IPython
 since the 0.10.x release series. For more details, please consult the actual
 source.
 
 Main `ipython` branch
 =====================
 
 Refactoring
 -----------
 
 As of the 0.11 version of IPython, a signifiant portion of the core has been
 refactored.  This refactoring is founded on a number of new abstractions.
 The main new classes that implement these abstractions are:
 
 * :class:`IPython.utils.traitlets.HasTraits`.
 * :class:`IPython.config.configurable.Configurable`.
 * :class:`IPython.config.application.Application`.
 * :class:`IPython.config.loader.ConfigLoader`.
 * :class:`IPython.config.loader.Config`
 
 We are still in the process of writing developer focused documentation about
 these classes, but for now our :ref:`configuration documentation
 <config_overview>` contains a high level overview of the concepts that these
 classes express.
 
 The biggest user-visible change is likely the move to using the config system to
 determine the command-line arguments for IPython applications. The benefit of
 this is that *all* configurable values in IPython are exposed on the
 command-line, but the syntax for specifying values has changed. The gist is that
 assigning values is pure Python assignment, so there is always an '=', and never
 a leading '-', nor a space separating key from value. Flags exist, to set
 multiple values or boolean flags, and these are always prefixed with '--', and
 never take arguments.
 
 ZMQ architecture
 ----------------
 
 There is a new GUI framework for IPython, based on a client-server model in
 which multiple clients can communicate with one IPython kernel, using the
 ZeroMQ messaging framework. There is already a Qt console client, which can
 be started by calling ``ipython qtconsole``. The protocol is :ref:`documented
 <messaging>`.
 
 The parallel computing framework has also been rewritten using ZMQ. The
 protocol is described :ref:`here <parallel_messages>`, and the code is in the
 new :mod:`IPython.parallel` module.
 
 Python 3 support
 ----------------
 
 A Python 3 version of IPython has been prepared. For the time being, this is
 maintained separately and updated from the main codebase. Its code can be found
 `here <https://github.com/ipython/ipython-py3k>`_. Note that the parallel
 computing features do not yet work in Python 3.
 
+Unicode
+-------
+
+Entering non-ascii characters in unicode literals (``u"€ø"``) now works properly
+on all platforms. However, entering these in byte/string literals (``"€ø"``)
+will not work as expected on Windows (or any platform where the terminal encoding
+is not UTF-8, as it typically is for Linux & Mac OS X). You can use escape sequences
+(``"\xe9\x82"``) to get bytes above 128, or use unicode literals and encode
+them. This is a limitation of Python 2 which we cannot easily work around.
+
 New features
 ------------
 
 * Added ``Bytes`` traitlet, removing ``Str``.  All 'string' traitlets should
   either be ``Unicode`` if a real string, or ``Bytes`` if a C-string. This
   removes ambiguity and helps the Python 3 transition.
 
 * New magic ``%loadpy`` loads a python file from disk or web URL into
   the current input buffer.
 
 * New magic ``%pastebin`` for sharing code via the 'Lodge it' pastebin.
 
 * New magic ``%precision`` for controlling float and numpy pretty printing.
 
 * IPython applications initiate logging, so any object can gain access to
   a the logger of the currently running Application with:
 
 .. sourcecode:: python
 
     from IPython.config.application import Application
     logger = Application.instance().log
 
 * You can now get help on an object halfway through typing a command. For
   instance, typing ``a = zip?`` shows the details of :func:`zip`. It also
   leaves the command at the next prompt so you can carry on with it.
 
 * The input history is now written to an SQLite database. The API for
   retrieving items from the history has also been redesigned.
 
 * The :mod:`IPython.extensions.pretty` extension has been moved out of
   quarantine and fully updated to the new extension API.
 
 * New magics for loading/unloading/reloading extensions have been added:
   ``%load_ext``, ``%unload_ext`` and ``%reload_ext``.
 
 * The configuration system and configuration files are brand new. See the
   configuration system :ref:`documentation <config_index>` for more details.
 
 * The :class:`~IPython.core.interactiveshell.InteractiveShell` class is now a
   :class:`~IPython.config.configurable.Configurable` subclass and has traitlets that
   determine the defaults and runtime environment. The ``__init__`` method has
   also been refactored so this class can be instantiated and run without the
   old :mod:`ipmaker` module.
 
 * The methods of :class:`~IPython.core.interactiveshell.InteractiveShell` have
   been organized into sections to make it easier to turn more sections
   of functionality into components.
 
 * The embedded shell has been refactored into a truly standalone subclass of
   :class:`InteractiveShell` called :class:`InteractiveShellEmbed`.  All
   embedding logic has been taken out of the base class and put into the 
   embedded subclass.
 
 * Added methods of :class:`~IPython.core.interactiveshell.InteractiveShell` to
   help it cleanup after itself. The :meth:`cleanup` method controls this. We
   couldn't do this in :meth:`__del__` because we have cycles in our object
   graph that prevent it from being called.
 
 * Created a new module :mod:`IPython.utils.importstring` for resolving
   strings like ``foo.bar.Bar`` to the actual class.
 
 * Completely refactored the :mod:`IPython.core.prefilter` module into
   :class:`~IPython.config.configurable.Configurable` subclasses. Added a new layer
   into the prefilter system, called "transformations" that all new prefilter
   logic should use (rather than the older "checker/handler" approach).
 
 * Aliases are now components (:mod:`IPython.core.alias`).
 
 * We are now using an internally shipped version of
   :mod:`~IPython.external.argparse` to parse command line options for
   :command:`ipython`.
 
 * New top level :func:`~IPython.frontend.terminal.embed.embed` function that can
   be called to embed IPython at any place in user's code. One the first call it
   will create an :class:`~IPython.frontend.terminal.embed.InteractiveShellEmbed`
   instance and call it. In later calls, it just calls the previously created
   :class:`~IPython.frontend.terminal.embed.InteractiveShellEmbed`.
 
 * Created a configuration system (:mod:`IPython.config.configurable`) that is
   based on :mod:`IPython.utils.traitlets`. Configurables are arranged into a
   runtime containment tree (not inheritance) that i) automatically propagates
   configuration information and ii) allows singletons to discover each other in
   a loosely coupled manner. In the future all parts of IPython will be
   subclasses of :class:`~IPython.config.configurable.Configurable`. All IPython
   developers should become familiar with the config system.
 
 * Created a new :class:`~IPython.config.loader.Config` for holding
   configuration information. This is a dict like class with a few extras: i)
   it supports attribute style access, ii) it has a merge function that merges
   two :class:`~IPython.config.loader.Config` instances recursively and iii) it
   will automatically create sub-:class:`~IPython.config.loader.Config`
   instances for attributes that start with an uppercase character.
 
 * Created new configuration loaders in :mod:`IPython.config.loader`. These
   loaders provide a unified loading interface for all configuration
   information including command line arguments and configuration files. We
   have two default implementations based on :mod:`argparse` and plain python
   files.  These are used to implement the new configuration system.
 
 * Created a top-level :class:`Application` class in
   :mod:`IPython.core.application` that is designed to encapsulate the starting
   of any basic Python program. An application loads and merges all the
   configuration objects, constructs the main application, configures and
   initiates logging, and creates and configures any :class:`Configurable`
   instances and then starts the application running. An extended
   :class:`BaseIPythonApplication` class adds logic for handling the
   IPython directory as well as profiles, and all IPython entry points
   extend it.
 
 * The :class:`Type` and :class:`Instance` traitlets now handle classes given
   as strings, like ``foo.bar.Bar``. This is needed for forward declarations.
   But, this was implemented in a careful way so that string to class
   resolution is done at a single point, when the parent
   :class:`~IPython.utils.traitlets.HasTraitlets` is instantiated.
 
 * :mod:`IPython.utils.ipstruct` has been refactored to be a subclass of 
   dict.  It also now has full docstrings and doctests.
 * Created a Trait's like implementation in :mod:`IPython.utils.traitlets`.
   This is a pure Python, lightweight version of a library that is similar to
   :mod:`enthought.traits`.  We are using this for validation, defaults and
   notification in our new component system.  Although it is not API compatible
   with :mod:`enthought.traits`, we plan on moving in this direction so that
   eventually our implementation could be replaced by a (yet to exist) pure
   Python version of :mod:`enthought.traits`.
 
 * Added a new module :mod:`IPython.lib.inputhook` to manage the integration
   with GUI event loops using `PyOS_InputHook`.  See the docstrings in this
   module or the main IPython docs for details.
 
 * For users, GUI event loop integration is now handled through the new
   :command:`%gui` magic command.  Type ``%gui?`` at an IPython prompt for
   documentation.
 
 * For developers :mod:`IPython.lib.inputhook` provides a simple interface
   for managing the event loops in their interactive GUI applications.
   Examples can be found in our :file:`docs/examples/lib` directory.
 
 Backwards incompatible changes
 ------------------------------
 
 * The Twisted-based :mod:`IPython.kernel` has been removed, and completely
   rewritten as :mod:`IPython.parallel`, using ZeroMQ.
 
 * Profiles are now directories. Instead of a profile being a single config file,
   profiles are now self-contained directories. By default, profiles get their
   own IPython history, log files, and everything. To create a new profile, do
   ``ipython profile create <name>``.
 
 * All IPython applications have been rewritten to use
   :class:`~IPython.config.loader.KeyValueConfigLoader`. This means that
   command-line options have changed. Now, all configurable values are accessible
   from the command-line with the same syntax as in a configuration file.
 
 * The command line options ``-wthread``, ``-qthread`` and
   ``-gthread`` have been removed. Use ``gui=wx``, ``gui=qt``, ``gui=gtk``
   instead.
 
 * The extension loading functions have been renamed to
   :func:`load_ipython_extension` and :func:`unload_ipython_extension`.
 
 * :class:`~IPython.core.interactiveshell.InteractiveShell` no longer takes an
   ``embedded`` argument. Instead just use the
   :class:`~IPython.core.interactiveshell.InteractiveShellEmbed` class.
 
 * ``__IPYTHON__`` is no longer injected into ``__builtin__``.
 
 * :meth:`Struct.__init__` no longer takes `None` as its first argument.  It
   must be a :class:`dict` or :class:`Struct`.
 
 * :meth:`~IPython.core.interactiveshell.InteractiveShell.ipmagic` has been
   renamed :meth:`~IPython.core.interactiveshell.InteractiveShell.magic.`
 
 * The functions :func:`ipmagic` and :func:`ipalias` have been removed from
   :mod:`__builtins__`.
 
 * The references to the global
   :class:`~IPython.core.interactivehell.InteractiveShell` instance (``_ip``, and
   ``__IP``) have been removed from the user's namespace. They are replaced by a
   new function called :func:`get_ipython` that returns the current
   :class:`~IPython.core.interactiveshell.InteractiveShell` instance. This
   function is injected into the user's namespace and is now the main way of
   accessing the running IPython.
 
 * Old style configuration files :file:`ipythonrc` and :file:`ipy_user_conf.py`
   are no longer supported. Users should migrate there configuration files to
   the new format described :ref:`here <config_overview>` and :ref:`here
   <configuring_ipython>`.
 
 * The old IPython extension API that relied on :func:`ipapi` has been
   completely removed. The new extension API is described :ref:`here
   <configuring_ipython>`.
 
 * Support for ``qt3`` has been dropped.  Users who need this should use
   previous versions of IPython.
 
 * Removed :mod:`shellglobals` as it was obsolete.
 
 * Removed all the threaded shells in :mod:`IPython.core.shell`. These are no
   longer needed because of the new capabilities in
   :mod:`IPython.lib.inputhook`.
 
 * New top-level sub-packages have been created: :mod:`IPython.core`, 
   :mod:`IPython.lib`, :mod:`IPython.utils`, :mod:`IPython.deathrow`,
   :mod:`IPython.quarantine`.  All existing top-level modules have been
   moved to appropriate sub-packages.  All internal import statements
   have been updated and tests have been added.  The build system (setup.py
   and friends) have been updated.  See :ref:`this section <module_reorg>` of the
   documentation for descriptions of these new sub-packages.
 
 * :mod:`IPython.ipapi` has been moved to :mod:`IPython.core.ipapi`.
   :mod:`IPython.Shell` and :mod:`IPython.iplib` have been split and removed as
   part of the refactor.
 
 * :mod:`Extensions` has been moved to :mod:`extensions` and all existing
   extensions have been moved to either :mod:`IPython.quarantine` or
   :mod:`IPython.deathrow`. :mod:`IPython.quarantine` contains modules that we
   plan on keeping but that need to be updated. :mod:`IPython.deathrow`
   contains modules that are either dead or that should be maintained as third
   party libraries. More details about this can be found :ref:`here
   <module_reorg>`.
 
 * Previous IPython GUIs in :mod:`IPython.frontend` and :mod:`IPython.gui` are
   likely broken, and have been removed to :mod:`IPython.deathrow` because of the
   refactoring in the core. With proper updates, these should still work.
 
 
 Known Regressions
 -----------------
 
 We do our best to improve IPython, but there are some known regressions in 0.11 relative
 to 0.10.2.
 
 * The machinery that adds functionality to the 'sh' profile for using IPython as your
   system shell has not been updated to use the new APIs.  As a result, only the aesthetic
   (prompt) changes are still implemented. We intend to fix this by 0.12.
 
 * The installation of scripts on Windows was broken without setuptools, so we now
   depend on setuptools on Windows.  We hope to fix setuptools-less installation,
   and then remove the setuptools dependency.