Major work on the documentation.

* Everything is cleaned up.
* Outdated things have been re-written or removed.
* Full documentation about the new config system has been added.
* The developer guide has been re-organized and re-written where
  appropriate.

docs/source/config/editors.txt

@@ -0,0 +1,117 b''
+.. _editors:
+
+====================
+Editor configuration
+====================
+
+IPython can integrate with text editors in a number of different ways:
+
+* Editors (such as (X)Emacs [Emacs]_, vim [vim]_ and TextMate [TextMate]_) can
+  send code to IPython for execution.
+
+* IPython's ``%edit`` magic command can open an editor of choice to edit
+  a code block.
+
+The %edit command (and its alias %ed) will invoke the editor set in your
+environment as :envvar:`EDITOR`. If this variable is not set, it will default
+to vi under Linux/Unix and to notepad under Windows. You may want to set this
+variable properly and to a lightweight editor which doesn't take too long to
+start (that is, something other than a new instance of Emacs). This way you
+can edit multi-line code quickly and with the power of a real editor right
+inside IPython.
+
+You can also control the editor via the commmand-line option '-editor' or in
+your configuration file, by setting the :attr:`InteractiveShell.editor`
+configuration attribute.
+
+TextMate
+========
+
+Currently, TextMate support in IPython is broken.  It used to work well,
+but the code has been moved to :mod:`IPython.quarantine` until it is updated.
+
+vim configuration
+=================
+
+Currently, vim support in IPython is broken.  Like the TextMate code,
+the vim support code has been moved to :mod:`IPython.quarantine` until it
+is updated.
+
+.. _emacs:
+
+(X)Emacs
+========
+
+Editor
+======
+
+If you are a dedicated Emacs user, and want to use Emacs when IPython's
+``%edit`` magic command is called you should set up the Emacs server so that
+new requests are handled by the original process. This means that almost no
+time is spent in handling the request (assuming an Emacs process is already
+running). For this to work, you need to set your EDITOR environment variable
+to 'emacsclient'. The code below, supplied by Francois Pinard, can then be
+used in your :file:`.emacs` file to enable the server::
+
+    (defvar server-buffer-clients)
+    (when (and (fboundp 'server-start) (string-equal (getenv "TERM") 'xterm))
+      (server-start)
+      (defun fp-kill-server-with-buffer-routine ()
+        (and server-buffer-clients (server-done)))
+      (add-hook 'kill-buffer-hook 'fp-kill-server-with-buffer-routine))
+
+Thanks to the work of Alexander Schmolck and Prabhu Ramachandran,
+currently (X)Emacs and IPython get along very well in other ways.
+
+.. note::
+
+    You will need to use a recent enough version of :file:`python-mode.el`,
+    along with the file :file:`ipython.el`. You can check that the version you
+    have of :file:`python-mode.el` is new enough by either looking at the
+    revision number in the file itself, or asking for it in (X)Emacs via ``M-x
+    py-version``. Versions 4.68 and newer contain the necessary fixes for
+    proper IPython support.
+
+The file :file:`ipython.el` is included with the IPython distribution, in the
+directory :file:`docs/emacs`. Once you put these files in your Emacs path, all
+you need in your :file:`.emacs` file is::
+
+    (require 'ipython)
+
+This should give you full support for executing code snippets via
+IPython, opening IPython as your Python shell via ``C-c !``, etc.
+
+You can customize the arguments passed to the IPython instance at startup by
+setting the ``py-python-command-args`` variable.  For example, to start always
+in ``pylab`` mode with hardcoded light-background colors, you can use::
+
+    (setq py-python-command-args '("-pylab" "-colors" "LightBG"))
+
+If you happen to get garbage instead of colored prompts as described in
+the previous section, you may need to set also in your :file:`.emacs` file::
+
+    (setq ansi-color-for-comint-mode t)
+
+Notes on emacs support:
+
+* There is one caveat you should be aware of: you must start the IPython shell
+  before attempting to execute any code regions via ``C-c |``. Simply type
+  ``C-c !`` to start IPython before passing any code regions to the
+  interpreter, and you shouldn't experience any problems. This is due to a bug
+  in Python itself, which has been fixed for Python 2.3, but exists as of
+  Python 2.2.2 (reported as SF bug [ 737947 ]).
+
+* The (X)Emacs support is maintained by Alexander Schmolck, so all
+  comments/requests should be directed to him through the IPython mailing
+  lists.
+
+* This code is still somewhat experimental so it's a bit rough around the
+  edges (although in practice, it works quite well).
+
+* Be aware that if you customized ``py-python-command`` previously, this value
+  will override what :file:`ipython.el` does (because loading the customization
+  variables comes later).
+
+.. [Emacs] Emacs. http://www.gnu.org/software/emacs/
+.. [TextMate] TextMate: the missing editor.  http://macromates.com/
+.. [vim] vim. http://www.vim.org/

docs/source/config/ipython.txt

@@ -0,0 +1,134 b''
+.. _configuring_ipython:
+
+===========================================================
+Configuring the :command:`ipython` command line application
+===========================================================
+
+This section contains information about how to configure the
+:command:`ipython` command line application. See the :ref:`configuration
+overview <config_overview>` for a more general description of the
+configuration system and configuration file format.
+
+The default configuration file for the :command:`ipython` command line application
+is :file:`ipython_config.py`.  By setting the attributes in this file, you
+can configure the application.   A sample is provided in
+:mod:`IPython.config.default.ipython_config`.  Simply copy this file to your
+IPython directory to start using it.
+
+Most configuration attributes that this file accepts are associated with
+classes that are subclasses of :class:`~IPython.core.component.Component`.
+
+A few configuration attributes are not associated with a particular 
+:class:`~IPython.core.component.Component` subclass.  These are application
+wide configuration attributes and are stored in the ``Global`` 
+sub-configuration section.  We begin with a description of these 
+attributes.
+
+Global configuration
+====================
+
+Assuming that your configuration file has the following at the top::
+
+    c = get_config()
+
+the following attributes can be set in the ``Global`` section.
+
+:attr:`c.Global.display_banner`
+    A boolean that determined if the banner is printer when :command:`ipython`
+    is started.
+
+:attr:`c.Global.classic`
+    A boolean that determines if IPython starts in "classic" mode.  In this
+    mode, the prompts and everything mimic that of the normal :command:`python`
+    shell
+
+:attr:`c.Global.nosep`
+    A boolean that determines if there should be no blank lines between
+    prompts.
+
+:attr:`c.Global.log_level`
+    An integer that sets the detail of the logging level during the startup
+    of :command:`ipython`.  The default is 30 and the possible values are
+    (0, 10, 20, 30, 40, 50).  Higher is quieter and lower is more verbose.
+
+:attr:`c.Global.extensions`
+    A list of strings, each of which is an importable IPython extension. An
+    IPython extension is a regular Python module or package that has a
+    :func:`load_in_ipython(ip)` method. This method gets called when the
+    extension is loaded with the currently running
+    :class:`~IPython.core.iplib.InteractiveShell` as its only argument. You
+    can put your extensions anywhere they can be imported but we add the
+    :file:`extensions` subdirectory of the ipython directory to ``sys.path``
+    during extension loading, so you can put them there as well.  Extensions
+    are not executed in the user's interactive namespace and they must
+    be pure Python code.  Extensions are the recommended way of customizing
+    :command:`ipython`.
+
+:attr:`c.Global.exec_lines`
+    A list of strings, each of which is Python code that is run in the user's
+    namespace after IPython start. These lines can contain full IPython syntax
+    with magics, etc.
+
+:attr:`c.Global.exec_files`
+    A list of strings, each of which is the full pathname of a ``.py`` or
+    ``.ipy`` file that will be executed as IPython starts. These files are run
+    in IPython in the user's namespace. Files with a ``.py`` extension need to
+    be pure Python. Files with a ``.ipy`` extension can have custom IPython
+    syntax (magics, etc.). These files need to be in the cwd, the ipythondir
+    or be absolute paths.
+
+Classes that can be configured
+==============================
+
+The following classes can also be configured in the configuration file for
+:command:`ipython`:
+
+* :class:`~IPython.core.iplib.InteractiveShell`
+
+* :class:`~IPython.core.prefilter.PrefilterManager`
+
+* :class:`~IPython.core.alias.AliasManager`
+
+To see which attributes of these classes are configurable, please see the
+source code for these classes, the class docstrings or the sample
+configuration file :mod:`IPython.config.default.ipython_config`.
+
+Example
+=======
+
+For those who want to get a quick start, here is a sample
+:file:`ipython_config.py` that sets some of the common configuration
+attributes::
+
+    # sample ipython_config.py
+    c = get_config()
+
+    c.Global.display_banner = True
+    c.Global.log_level = 20
+    c.Global.extensions = [
+        'myextension'
+    ]
+    c.Global.exec_lines = [
+        'import numpy',
+        'import scipy'
+    ]
+    c.Global.exec_files = [
+        'mycode.py',
+        'fancy.ipy'
+    ]
+    c.InteractiveShell.autoindent = True
+    c.InteractiveShell.colors = 'LightBG'
+    c.InteractiveShell.confirm_exit = False
+    c.InteractiveShell.deep_reload = True
+    c.InteractiveShell.editor = 'nano'
+    c.InteractiveShell.prompt_in1 = 'In [\#]: '
+    c.InteractiveShell.prompt_in2 = '   .\D.: '
+    c.InteractiveShell.prompt_out = 'Out[\#]: '
+    c.InteractiveShell.prompts_pad_left = True
+    c.InteractiveShell.xmode = 'Context'
+
+    c.PrefilterManager.multi_line_specials = True
+
+    c.AliasManager.user_aliases = [
+     ('la', 'ls -al')
+    ]
  No newline at end of file

docs/source/config/overview.txt

@@ -0,0 +1,331 b''
+.. _config_overview:
+
+============================================
+Overview of the IPython configuration system
+============================================
+
+This section describes the IPython configuration system. Starting with version
+0.11, IPython has a completely new configuration system that is quite
+different from the older :file:`ipythonrc` or :file:`ipy_user_conf.py`
+approaches. The new configuration system was designed from scratch to address
+the particular configuration needs of IPython. While there are many
+other excellent configuration systems out there, we found that none of them
+met our requirements.
+
+.. warning::
+
+    If you are upgrading to version 0.11 of IPython, you will need to migrate
+    your old :file:`ipythonrc` or :file:`ipy_user_conf.py` configuration files
+    to the new system.  Read on for information on how to do this.
+
+The discussion that follows is focused on teaching user's how to configure
+IPython to their liking.  Developer's who want to know more about how they
+can enable their objects to take advantage of the configuration system
+should consult our :ref:`developer guide <developer_guide>`
+
+The main concepts
+=================
+
+There are a number of abstractions that the IPython configuration system uses.
+Each of these abstractions is represented by a Python class.
+
+Configuration object: :class:`~IPython.config.loader.Config`
+    A configuration object is a simple dictionary-like class that holds
+    configuration attributes and sub-configuration objects. These classes
+    support dotted attribute style access (``Foo.bar``) in addition to the
+    regular dictionary style access (``Foo['bar']``). Configuration objects
+    are smart. They know how to merge themselves with other configuration
+    objects and they automatically create sub-configuration objects.
+
+Application: :class:`~IPython.core.application.Application`
+    An application is a process that does a specific job. The most obvious
+    application is the :command:`ipython` command line program. Each
+    application reads a *single* configuration file and command line options
+    and then produces a master configuration object for the application. This
+    configuration object is then passed to the components that the application
+    creates. Components implement the actual logic of the application and know
+    how to configure themselves given the configuration object.
+
+Component: :class:`~IPython.core.component.Component`
+    A component is a regular Python class that serves as a base class for all
+    main classes in an application. The
+    :class:`~IPython.core.component.Component` base class is lightweight and
+    only does two main things.
+    
+    First, it keeps track of all instances of itself and provides an
+    interfaces for querying those instances. This enables components to get
+    references to other components, even though they are not "nearby" in the
+    runtime object graph.
+    
+    Second, it declares what class attributes are configurable and specifies
+    the default types and values of those attributes. This information is used
+    to automatically configure instances given the applications configuration
+    object.
+    
+    Developers create :class:`~IPython.core.component.Component` subclasses
+    that implement all of the logic in the application. Each of these
+    subclasses has its own configuration information that controls how
+    instances are created.
+
+Having described these main concepts, we can now state the main idea in our
+configuration system: *"configuration" allows the default values of class
+attributes to be controlled on a class by class basis*. Thus all instances of
+a given class are configured in the same way. Furthermore, if two instances
+need to be configured differently, they need to be instances of two different
+classes. While this model may seem a bit restrictive, we have found that it
+expresses most things that need to be configured extremely well.
+
+Now, we show what our configuration objects and files look like.
+
+Configuration objects and files
+===============================
+
+A configuration file is simply a pure Python file that sets the attributes
+of a global, pre-created configuration object.  This configuration object is a 
+:class:`~IPython.config.loader.Config` instance.  While in a configuration
+file, to get a reference to this object, simply call the :func:`get_config`
+function.  We inject this function into the global namespace that the 
+configuration file is executed in.
+
+Here is an example of a super simple configuration file that does nothing::
+
+    c = get_config()
+
+Once you get a reference to the configuration object, you simply set
+attributes on it.  All you have to know is:
+
+* The name of each attribute.
+* The type of each attribute.
+
+The answers to these two questions are provided by the various
+:class:`~IPython.core.component.Component` subclasses that an application
+uses.  Let's look at how this would work for a simple component subclass::
+
+    # Sample component that can be configured.
+    from IPython.core.component import Component
+    from IPython.utils.traitlets import Int, Float, Str, Bool
+    
+    class MyComponent(Component):
+        name = Str('defaultname', config=True)
+        ranking = Int(0, config=True)
+        value = Float(99.0)
+        # The rest of the class implementation would go here..
+
+In this example, we see that :class:`MyComponent` has three attributes, two
+of whom (``name``, ``ranking``) can be configured.  All of the attributes
+are given types and default values.  If a :class:`MyComponent` is instantiated,
+but not configured, these default values will be used.  But let's see how
+to configure this class in a configuration file::
+
+    # Sample config file
+    c = get_config()
+    
+    c.MyComponent.name = 'coolname'
+    c.MyComponent.ranking = 10
+
+After this configuration file is loaded, the values set in it will override
+the class defaults anytime a :class:`MyComponent` is created.  Furthermore,
+these attributes will be type checked and validated anytime they are set.
+This type checking is handled by the :mod:`IPython.utils.traitlets` module,
+which provides the :class:`Str`, :class:`Int` and :class:`Float` types.  In 
+addition to these traitlets, the :mod:`IPython.utils.traitlets` provides
+traitlets for a number of other types.
+
+.. note::
+
+    Underneath the hood, the :class:`Component` base class is a subclass of
+    :class:`IPython.utils.traitlets.HasTraitlets`. The
+    :mod:`IPython.utils.traitlets` module is a lightweight version of
+    :mod:`enthought.traits`. Our implementation is a pure Python subset
+    (mostly API compatible) of :mod:`enthought.traits` that does not have any
+    of the automatic GUI generation capabilities. Our plan is to achieve 100%
+    API compatibility to enable the actual :mod:`enthought.traits` to
+    eventually be used instead. Currently, we cannot use
+    :mod:`enthought.traits` as we are committed to the core of IPython being
+    pure Python.
+
+It should be very clear at this point what the naming convention is for 
+configuration attributes::
+
+    c.ClassName.attribute_name = attribute_value
+
+Here, ``ClassName`` is the name of the class whose configuration attribute you
+want to set, ``attribute_name`` is the name of the attribute you want to set
+and ``attribute_value`` the the value you want it to have. The ``ClassName``
+attribute of ``c`` is not the actual class, but instead is another
+:class:`~IPython.config.loader.Config` instance.
+
+.. note::
+
+    The careful reader may wonder how the ``ClassName`` (``MyComponent`` in
+    the above example) attribute of the configuration object ``c`` gets
+    created. These attributes are created on the fly by the
+    :class:`~IPython.config.loader.Config` instance, using a simple naming
+    convention. Any attribute of a :class:`~IPython.config.loader.Config`
+    instance whose name begins with an uppercase character is assumed to be a
+    sub-configuration and a new empty :class:`~IPython.config.loader.Config`
+    instance is dynamically created for that attribute. This allows deeply
+    hierarchical information created easily (``c.Foo.Bar.value``) on the 
+    fly.
+
+Configuration files inheritance
+===============================
+
+Let's say you want to have different configuration files for various purposes.
+Our configuration system makes it easy for one configuration file to inherit
+the information in another configuration file. The :func:`load_subconfig`
+command can be used in a configuration file for this purpose. Here is a simple
+example that loads all of the values from the file :file:`base_config.py`::
+
+    # base_config.py
+    c = get_config()
+    c.MyComponent.name = 'coolname'
+    c.MyComponent.ranking = 100
+
+into the configuration file :file:`main_config.py`::
+
+    # main_config.py
+    c = get_config()
+    
+    # Load everything from base_config.py
+    load_subconfig('base_config.py')
+    
+    # Now override one of the values
+    c.MyComponent.name = 'bettername'
+
+In a situation like this the :func:`load_subconfig` makes sure that the
+search path for sub-configuration files is inherited from that of the parent.
+Thus, you can typically put the two in the same directory and everything will
+just work.
+
+Class based configuration inheritance
+=====================================
+
+There is another aspect of configuration where inheritance comes into play.
+Sometimes, your classes will have an inheritance hierarchy that you want
+to be reflected in the configuration system.  Here is a simple example::
+
+    from IPython.core.component import Component
+    from IPython.utils.traitlets import Int, Float, Str, Bool
+    
+    class Foo(Component):
+        name = Str('fooname', config=True)
+        value = Float(100.0, config=True)
+    
+    class Bar(Foo):
+        name = Str('barname', config=True)
+        othervalue = Int(0, config=True)
+
+Now, we can create a configuration file to configure instances of :class:`Foo`
+and :class:`Bar`::
+
+    # config file
+    c = get_config()
+    
+    c.Foo.name = 'bestname'
+    c.Bar.othervalue = 10
+
+This class hierarchy and configuration file accomplishes the following:
+
+* The default value for :attr:`Foo.name` and :attr:`Bar.name` will be
+  'bestname'.  Because :class:`Bar` is a :class:`Foo` subclass it also
+  picks up the configuration information for :class:`Foo`.
+* The default value for :attr:`Foo.value` and :attr:`Bar.value` will be
+  ``100.0``, which is the value specified as the class default.
+* The default value for :attr:`Bar.othervalue` will be 10 as set in the
+  configuration file.  Because :class:`Foo` is the parent of :class:`Bar`
+  it doesn't know anything about the :attr:`othervalue` attribute.
+
+Configuration file location
+===========================
+
+So where should you put your configuration files?  By default, all IPython
+applications look in the so called "IPython directory".  The location of
+this directory is determined by the following algorithm:
+
+* If the ``-ipythondir`` command line flag is given, its value is used.
+
+* If not, the value returned by :func:`IPython.utils.genutils.get_ipython_dir`
+  is used. This function will first look at the :envvar:`IPYTHONDIR`
+  environment variable and then default to the directory
+  :file:`$HOME/.ipythondir`.
+
+For most users, the default value will simply be something like 
+:file:`$HOME/.ipythondir`.
+
+Once the location of the IPython directory has been determined, you need to
+know what filename to use for the configuration file. The basic idea is that
+each application has its own default configuration filename. The default named
+used by the :command:`ipython` command line program is
+:file:`ipython_config.py`.  This value can be overriden by the ``-config_file``
+command line flag.  A sample :file:`ipython_config.py` file can be found
+in :mod:`IPython.config.default.ipython_config.py`.  Simple copy it to your
+IPython directory to begin using it.
+
+.. _Profiles:
+
+Profiles
+========
+
+A profile is simply a configuration file that follows a simple naming
+convention and can be loaded using a simplified syntax.  The idea is
+that users often want to maintain a set of configuration files for different
+purposes:  one for doing numerical computing with NumPy and SciPy and
+another for doing symbolic computing with SymPy.  Profiles make it easy
+to keep a separate configuration file for each of these purposes.
+
+Let's start by showing how a profile is used:
+
+.. code-block:: bash
+
+    $ ipython -p sympy
+
+This tells the :command:`ipython` command line program to get its
+configuration from the "sympy" profile. The search path for profiles is the
+same as that of regular configuration files. The only difference is that
+profiles are named in a special way. In the case above, the "sympy" profile
+would need to have the name :file:`ipython_config_sympy.py`.
+
+The general pattern is this: simply add ``_profilename`` to the end of the
+normal configuration file name. Then load the profile by adding ``-p
+profilename`` to your command line options.
+
+IPython ships with some sample profiles in :mod:`IPython.config.profile`.
+Simply copy these to your IPython directory to begin using them.
+
+Design requirements
+===================
+
+Here are the main requirements we wanted our configuration system to have:
+
+* Support for hierarchical configuration information.
+
+* Full integration with command line option parsers.  Often, you want to read
+  a configuration file, but then override some of the values with command line
+  options.  Our configuration system automates this process and allows each
+  command line option to be linked to a particular attribute in the 
+  configuration hierarchy that it will override.
+
+* Configuration files that are themselves valid Python code. This accomplishes
+  many things. First, it becomes possible to put logic in your configuration
+  files that sets attributes based on your operating system, network setup,
+  Python version, etc. Second, Python has a super simple syntax for accessing
+  hierarchical data structures, namely regular attribute access
+  (``Foo.Bar.Bam.name``). Third, using Python makes it easy for users to
+  import configuration attributes from one configuration file to another.
+  Forth, even though Python is dynamically typed, it does have types that can
+  be checked at runtime. Thus, a ``1`` in a config file is the integer '1',
+  while a ``'1'`` is a string.
+
+* A fully automated method for getting the configuration information to the
+  classes that need it at runtime. Writing code that walks a configuration
+  hierarchy to extract a particular attribute is painful. When you have
+  complex configuration information with hundreds of attributes, this makes
+  you want to cry.
+
+* Type checking and validation that doesn't require the entire configuration
+  hierarchy to be specified statically before runtime. Python is a very
+  dynamic language and you don't always know everything that needs to be
+  configured when a program starts.
+
+

IPython/config/default/ipython_config.py

@@ -1,148 +1,148 b''
 # Get the config being loaded so we can set attributes on it
 c = get_config()
 
 #-----------------------------------------------------------------------------
 # Global options
 #-----------------------------------------------------------------------------
 
 # c.Global.display_banner = True
 
 # c.Global.classic = False
 
 # c.Global.nosep = True
 
 # Set this to determine the detail of what is logged at startup.
 # The default is 30 and possible values are 0,10,20,30,40,50.
 c.Global.log_level = 20
 
 # This should be a list of importable Python modules that have an
 # load_in_ipython(ip) method.  This method gets called when the extension
 # is loaded.  You can put your extensions anywhere they can be imported
 # but we add the extensions subdir of the ipython directory to sys.path
 # during extension loading, so you can put them there as well.
 # c.Global.extensions = [
 #     'myextension'
 # ]
 
 # These lines are run in IPython in the user's namespace after extensions 
 # are loaded.  They can contain full IPython syntax with magics etc.
 # c.Global.exec_lines = [
 #     'import numpy',
 #     'a = 10; b = 20',
 #     '1/0'
 # ]
 
 # These files are run in IPython in the user's namespace.  Files with a .py
 # extension need to be pure Python.  Files with a .ipy extension can have
 # custom IPython syntax (like magics, etc.).  
 # These files need to be in the cwd, the ipythondir or be absolute paths.
 # c.Global.exec_files = [
 #     'mycode.py',
 #     'fancy.ipy'
 # ]
 
 #-----------------------------------------------------------------------------
 # InteractiveShell options
 #-----------------------------------------------------------------------------
 
 # c.InteractiveShell.autocall = 1
 
 # c.InteractiveShell.autoedit_syntax = False
 
 # c.InteractiveShell.autoindent = True
 
 # c.InteractiveShell.automagic = False
 
 # c.InteractiveShell.banner1 = 'This if for overriding the default IPython banner'
  
 # c.InteractiveShell.banner2 = "This is for extra banner text"
 
 # c.InteractiveShell.cache_size = 1000
 
 # c.InteractiveShell.colors = 'LightBG'
 
 # c.InteractiveShell.color_info = True
 
 # c.InteractiveShell.confirm_exit = True
 
 # c.InteractiveShell.deep_reload = False
 
 # c.InteractiveShell.editor = 'nano'
 
 # c.InteractiveShell.logstart = True
 
 # c.InteractiveShell.logfile = 'ipython_log.py'
 
 # c.InteractiveShell.logappend = 'mylog.py'
 
 # c.InteractiveShell.object_info_string_level = 0
 
 # c.InteractiveShell.pager = 'less'
 
 # c.InteractiveShell.pdb = False
 
 # c.InteractiveShell.pprint = True
 
 # c.InteractiveShell.prompt_in1 = 'In [\#]: '
 # c.InteractiveShell.prompt_in2 = '   .\D.: '
 # c.InteractiveShell.prompt_out = 'Out[\#]: '
 # c.InteractiveShell.prompts_pad_left = True
 
 # c.InteractiveShell.quiet = False
 
 # Readline 
 # c.InteractiveShell.readline_use = True
 
 # c.InteractiveShell.readline_parse_and_bind = [
 #     'tab: complete',
 #     '"\C-l": possible-completions',
 #     'set show-all-if-ambiguous on',
 #     '"\C-o": tab-insert',
 #     '"\M-i": "    "',
 #     '"\M-o": "\d\d\d\d"',
 #     '"\M-I": "\d\d\d\d"',
 #     '"\C-r": reverse-search-history',
 #     '"\C-s": forward-search-history',
 #     '"\C-p": history-search-backward',
 #     '"\C-n": history-search-forward',
 #     '"\e[A": history-search-backward',
 #     '"\e[B": history-search-forward',
 #     '"\C-k": kill-line',
 #     '"\C-u": unix-line-discard',
 # ]
 # c.InteractiveShell.readline_remove_delims = '-/~'
 # c.InteractiveShell.readline_merge_completions = True
 # c.InteractiveShell.readline_omit_names = 0
 
 # c.InteractiveShell.screen_length = 0
 
 # c.InteractiveShell.separate_in = '\n'
 # c.InteractiveShell.separate_out = ''
 # c.InteractiveShell.separate_out2 = ''
 
 # c.InteractiveShell.system_header = "IPython system call: "
 
 # c.InteractiveShell.system_verbose = True
 
 # c.InteractiveShell.term_title = False
 
 # c.InteractiveShell.wildcards_case_sensitive = True
 
 # c.InteractiveShell.xmode = 'Context'
 
 #-----------------------------------------------------------------------------
 # PrefilterManager options
 #-----------------------------------------------------------------------------
 
 # c.PrefilterManager.multi_line_specials = True
 
 #-----------------------------------------------------------------------------
 # AliasManager options
 #-----------------------------------------------------------------------------
 
-# Do this to enable all defaults
+# Do this to disable all defaults
 # c.AliasManager.default_aliases = []
 
 # c.AliasManager.user_aliases = [
 #     ('foo', 'echo Hi')
 # ]
  No newline at end of file

IPython/core/component.py

@@ -1,325 +1,325 b''
 #!/usr/bin/env python
 # encoding: utf-8
 """
 A lightweight component system for IPython.
 
 Authors:
 
 * Brian Granger
 * Fernando Perez
 """
 
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2008-2009  The IPython Development Team
 #
 #  Distributed under the terms of the BSD License.  The full license is in
 #  the file COPYING, distributed as part of this software.
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Imports
 #-----------------------------------------------------------------------------
 
 from copy import deepcopy
 import datetime
 from weakref import WeakValueDictionary
 
 from IPython.utils.importstring import import_item
 from IPython.config.loader import Config
 from IPython.utils.traitlets import (
     HasTraitlets, TraitletError, MetaHasTraitlets, Instance, This
 )
 
 
 #-----------------------------------------------------------------------------
 # Helper classes for Components
 #-----------------------------------------------------------------------------
 
 
 class ComponentError(Exception):
     pass
 
 class MetaComponentTracker(type):
     """A metaclass that tracks instances of Components and its subclasses."""
 
     def __init__(cls, name, bases, d):
         super(MetaComponentTracker, cls).__init__(name, bases, d)
         cls.__instance_refs = WeakValueDictionary()
         cls.__numcreated = 0
 
     def __call__(cls, *args, **kw):
         """Called when a class is called (instantiated)!!!
         
         When a Component or subclass is instantiated, this is called and
         the instance is saved in a WeakValueDictionary for tracking.
         """
         instance = cls.__new__(cls, *args, **kw)
 
         # Register the instance before __init__ is called so get_instances 
         # works inside __init__ methods!
         indices = cls.register_instance(instance)
 
         # This is in a try/except because of the __init__ method fails, the
         # instance is discarded and shouldn't be tracked.
         try:
             if isinstance(instance, cls):
                 cls.__init__(instance, *args, **kw)
         except:
             # Unregister the instance because __init__ failed!
             cls.unregister_instances(indices)
             raise
         else:
             return instance
 
     def register_instance(cls, instance):
         """Register instance with cls and its subclasses."""
         # indices is a list of the keys used to register the instance
         # with.  This list is needed if the instance needs to be unregistered.
         indices = []
         for c in cls.__mro__:
             if issubclass(cls, c) and issubclass(c, Component):
                 c.__numcreated += 1
                 indices.append(c.__numcreated)
                 c.__instance_refs[c.__numcreated] = instance
             else:
                 break
         return indices
 
     def unregister_instances(cls, indices):
         """Unregister instance with cls and its subclasses."""
         for c, index in zip(cls.__mro__, indices):
             try:
                 del c.__instance_refs[index]
             except KeyError:
                 pass
 
     def clear_instances(cls):
         """Clear all instances tracked by cls."""
         cls.__instance_refs.clear()
         cls.__numcreated = 0
 
     def get_instances(cls, name=None, root=None, klass=None):
         """Get all instances of cls and its subclasses.
 
         Parameters
         ----------
         name : str
             Limit to components with this name.
         root : Component or subclass
             Limit to components having this root.
         klass : class or str
             Limits to instances of the class or its subclasses.  If a str
             is given ut must be in the form 'foo.bar.MyClass'.  The str
             form of this argument is useful for forward declarations.
         """
         if klass is not None:
             if isinstance(klass, basestring):
                 klass = import_item(klass)
             # Limit search to instances of klass for performance
             if issubclass(klass, Component):
                 return klass.get_instances(name=name, root=root)
         instances = cls.__instance_refs.values()
         if name is not None:
             instances = [i for i in instances if i.name == name]
         if klass is not None:
             instances = [i for i in instances if isinstance(i, klass)]
         if root is not None:
             instances = [i for i in instances if i.root == root]
         return instances
 
     def get_instances_by_condition(cls, call, name=None, root=None,
                                    klass=None):
         """Get all instances of cls, i such that call(i)==True.
 
         This also takes the ``name`` and ``root`` and ``classname`` 
         arguments of :meth:`get_instance`
         """
         return [i for i in cls.get_instances(name, root, klass) if call(i)]
 
 
 def masquerade_as(instance, cls):
     """Let instance masquerade as an instance of cls.
 
     Sometimes, such as in testing code, it is useful to let a class
     masquerade as another.  Python, being duck typed, allows this by 
     default.  But, instances of components are tracked by their class type.
 
-    After calling this, cls.get_instances() will return ``instance``.  This
-    does not, however, cause isinstance(instance, cls) to return ``True``.
+    After calling this, ``cls.get_instances()`` will return ``instance``. This
+    does not, however, cause ``isinstance(instance, cls)`` to return ``True``.
 
     Parameters
     ----------
     instance : an instance of a Component or Component subclass
         The instance that will pretend to be a cls.
     cls : subclass of Component
         The Component subclass that instance will pretend to be.
     """
     cls.register_instance(instance)
 
 
 class ComponentNameGenerator(object):
     """A Singleton to generate unique component names."""
 
     def __init__(self, prefix):
         self.prefix = prefix
         self.i = 0
 
     def __call__(self):
         count = self.i
         self.i += 1
         return "%s%s" % (self.prefix, count)
 
 
 ComponentNameGenerator = ComponentNameGenerator('ipython.component')
 
 
 class MetaComponent(MetaHasTraitlets, MetaComponentTracker):
     pass
 
 
 #-----------------------------------------------------------------------------
 # Component implementation
 #-----------------------------------------------------------------------------
 
 
 class Component(HasTraitlets):
 
     __metaclass__ = MetaComponent
 
     # Traitlets are fun!
     config = Instance(Config,(),{})
     parent = This()
     root = This()
     created = None
 
     def __init__(self, parent, name=None, config=None):
         """Create a component given a parent and possibly and name and config.
 
         Parameters
         ----------
         parent : Component subclass
             The parent in the component graph.  The parent is used
             to get the root of the component graph.
         name : str
             The unique name of the component.  If empty, then a unique
             one will be autogenerated.
         config : Config
             If this is empty, self.config = parent.config, otherwise
             self.config = config and root.config is ignored.  This argument
             should only be used to *override* the automatic inheritance of 
             parent.config.  If a caller wants to modify parent.config 
             (not override), the caller should make a copy and change 
             attributes and then pass the copy to this argument.
         
         Notes
         -----
         Subclasses of Component must call the :meth:`__init__` method of
         :class:`Component` *before* doing anything else and using 
         :func:`super`::
         
             class MyComponent(Component):
                 def __init__(self, parent, name=None, config=None):
                     super(MyComponent, self).__init__(parent, name, config)
                     # Then any other code you need to finish initialization.
 
         This ensures that the :attr:`parent`, :attr:`name` and :attr:`config`
         attributes are handled properly.
         """
         super(Component, self).__init__()
         self._children = []
         if name is None:
             self.name = ComponentNameGenerator()
         else:
             self.name = name
         self.root = self # This is the default, it is set when parent is set
         self.parent = parent
         if config is not None:
             self.config = config
             # We used to deepcopy, but for now we are trying to just save
             # by reference.  This *could* have side effects as all components
             # will share config.
             # self.config = deepcopy(config)
         else:
             if self.parent is not None:
                 self.config = self.parent.config
                 # We used to deepcopy, but for now we are trying to just save
                 # by reference.  This *could* have side effects as all components
                 # will share config.
                 # self.config = deepcopy(self.parent.config)
 
         self.created = datetime.datetime.now()
 
     #-------------------------------------------------------------------------
     # Static traitlet notifiations
     #-------------------------------------------------------------------------
 
     def _parent_changed(self, name, old, new):
         if old is not None:
             old._remove_child(self)
         if new is not None:
             new._add_child(self)
 
         if new is None:
             self.root = self
         else:
             self.root = new.root
 
     def _root_changed(self, name, old, new):
         if self.parent is None:
             if not (new is self):
                 raise ComponentError("Root not self, but parent is None.")
         else:
             if not self.parent.root is new:
                 raise ComponentError("Error in setting the root attribute: "
                                      "root != parent.root")
 
     def _config_changed(self, name, old, new):
         """Update all the class traits having ``config=True`` as metadata.
 
         For any class traitlet with a ``config`` metadata attribute that is
         ``True``, we update the traitlet with the value of the corresponding
         config entry.
         """
         # Get all traitlets with a config metadata entry that is True
         traitlets = self.traitlets(config=True)
 
         # We auto-load config section for this class as well as any parent
         # classes that are Component subclasses.  This starts with Component
         # and works down the mro loading the config for each section.
         section_names = [cls.__name__ for cls in \
             reversed(self.__class__.__mro__) if 
             issubclass(cls, Component) and issubclass(self.__class__, cls)]
 
         for sname in section_names:
             # Don't do a blind getattr as that would cause the config to 
             # dynamically create the section with name self.__class__.__name__.
             if new._has_section(sname):
                 my_config = new[sname]
                 for k, v in traitlets.items():
                     try:
                         config_value = my_config[k]
                     except KeyError:
                         pass
                     else:
                         # print "Setting %s.%s from %s.%s=%r" % \
                         #     (self.__class__.__name__,k,sname,k,config_value)
                         setattr(self, k, config_value)
 
     @property
     def children(self):
         """A list of all my child components."""
         return self._children
 
     def _remove_child(self, child):
         """A private method for removing children components."""
         if child in self._children:
             index = self._children.index(child)
             del self._children[index]
 
     def _add_child(self, child):
         """A private method for adding children components."""
         if child not in self._children:
             self._children.append(child)
 
     def __repr__(self):
         return "<%s('%s')>" % (self.__class__.__name__, self.name)

IPython/core/prefilter.py

@@ -1,995 +1,995 b''
 #!/usr/bin/env python
 # encoding: utf-8
 """
 Prefiltering components.
 
 Prefilters transform user input before it is exec'd by Python.  These
 transforms are used to implement additional syntax such as !ls and %magic.
 
 Authors:
 
 * Brian Granger
 * Fernando Perez
 * Dan Milstein
 * Ville Vainio
 """
 
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2008-2009  The IPython Development Team
 #
 #  Distributed under the terms of the BSD License.  The full license is in
 #  the file COPYING, distributed as part of this software.
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Imports
 #-----------------------------------------------------------------------------
 
 import __builtin__
 import codeop
 import keyword
 import os
 import re
 import sys
 
 from IPython.core.alias import AliasManager
 from IPython.core.autocall import IPyAutocall
 from IPython.core.component import Component
 from IPython.core.splitinput import split_user_input
 from IPython.core.page import page
 
 from IPython.utils.traitlets import List, Int, Any, Str, CBool, Bool
 from IPython.utils.genutils import make_quoted_expr
 from IPython.utils.autoattr import auto_attr
 
 #-----------------------------------------------------------------------------
 # Global utilities, errors and constants
 #-----------------------------------------------------------------------------
 
 # Warning, these cannot be changed unless various regular expressions
 # are updated in a number of places.  Not great, but at least we told you.
 ESC_SHELL  = '!'
 ESC_SH_CAP = '!!'
 ESC_HELP   = '?'
 ESC_MAGIC  = '%'
 ESC_QUOTE  = ','
 ESC_QUOTE2 = ';'
 ESC_PAREN  = '/'
 
 
 class PrefilterError(Exception):
     pass
 
 
 # RegExp to identify potential function names
 re_fun_name = re.compile(r'[a-zA-Z_]([a-zA-Z0-9_.]*) *$')
 
 # RegExp to exclude strings with this start from autocalling.  In
 # particular, all binary operators should be excluded, so that if foo is
 # callable, foo OP bar doesn't become foo(OP bar), which is invalid.  The
 # characters '!=()' don't need to be checked for, as the checkPythonChars
 # routine explicitely does so, to catch direct calls and rebindings of
 # existing names.
 
 # Warning: the '-' HAS TO BE AT THE END of the first group, otherwise
 # it affects the rest of the group in square brackets.
 re_exclude_auto = re.compile(r'^[,&^\|\*/\+-]'
                              r'|^is |^not |^in |^and |^or ')
 
 # try to catch also methods for stuff in lists/tuples/dicts: off
 # (experimental). For this to work, the line_split regexp would need
 # to be modified so it wouldn't break things at '['. That line is
 # nasty enough that I shouldn't change it until I can test it _well_.
 #self.re_fun_name = re.compile (r'[a-zA-Z_]([a-zA-Z0-9_.\[\]]*) ?$')
 
 
 # Handler Check Utilities
 def is_shadowed(identifier, ip):
     """Is the given identifier defined in one of the namespaces which shadow
     the alias and magic namespaces?  Note that an identifier is different
     than ifun, because it can not contain a '.' character."""
     # This is much safer than calling ofind, which can change state
     return (identifier in ip.user_ns \
             or identifier in ip.internal_ns \
             or identifier in ip.ns_table['builtin'])
 
 
 #-----------------------------------------------------------------------------
 # The LineInfo class used throughout
 #-----------------------------------------------------------------------------
 
 
 class LineInfo(object):
     """A single line of input and associated info.
 
     Includes the following as properties: 
 
     line
       The original, raw line
     
     continue_prompt
       Is this line a continuation in a sequence of multiline input?
     
     pre
       The initial esc character or whitespace.
     
     pre_char
       The escape character(s) in pre or the empty string if there isn't one.
       Note that '!!' is a possible value for pre_char.  Otherwise it will
       always be a single character.
     
     pre_whitespace
       The leading whitespace from pre if it exists.  If there is a pre_char,
       this is just ''.
     
     ifun
       The 'function part', which is basically the maximal initial sequence
       of valid python identifiers and the '.' character.  This is what is
       checked for alias and magic transformations, used for auto-calling,
       etc.
     
     the_rest
       Everything else on the line.
     """
     def __init__(self, line, continue_prompt):
         self.line            = line
         self.continue_prompt = continue_prompt
         self.pre, self.ifun, self.the_rest = split_user_input(line)
 
         self.pre_char       = self.pre.strip()
         if self.pre_char:
             self.pre_whitespace = '' # No whitespace allowd before esc chars
         else: 
             self.pre_whitespace = self.pre
 
         self._oinfo = None
 
     def ofind(self, ip):
         """Do a full, attribute-walking lookup of the ifun in the various
         namespaces for the given IPython InteractiveShell instance.
 
         Return a dict with keys: found,obj,ospace,ismagic
 
         Note: can cause state changes because of calling getattr, but should
         only be run if autocall is on and if the line hasn't matched any
         other, less dangerous handlers.
 
         Does cache the results of the call, so can be called multiple times
         without worrying about *further* damaging state.
         """
         if not self._oinfo:
             self._oinfo = ip._ofind(self.ifun)
         return self._oinfo
 
     def __str__(self):                                                         
         return "Lineinfo [%s|%s|%s]" %(self.pre,self.ifun,self.the_rest) 
 
 
 #-----------------------------------------------------------------------------
 # Main Prefilter manager
 #-----------------------------------------------------------------------------
 
 
 class PrefilterManager(Component):
     """Main prefilter component.
 
     The IPython prefilter is run on all user input before it is run.  The
     prefilter consumes lines of input and produces transformed lines of 
     input.
 
     The iplementation consists of two phases:
 
     1. Transformers
     2. Checkers and handlers
 
     Over time, we plan on deprecating the checkers and handlers and doing
     everything in the transformers.
 
     The transformers are instances of :class:`PrefilterTransformer` and have
     a single method :meth:`transform` that takes a line and returns a
     transformed line.  The transformation can be accomplished using any
     tool, but our current ones use regular expressions for speed.  We also
     ship :mod:`pyparsing` in :mod:`IPython.external` for use in transformers.
 
     After all the transformers have been run, the line is fed to the checkers,
     which are instances of :class:`PrefilterChecker`.  The line is passed to
     the :meth:`check` method, which either returns `None` or a 
     :class:`PrefilterHandler` instance.  If `None` is returned, the other
     checkers are tried.  If an :class:`PrefilterHandler` instance is returned,
     the line is passed to the :meth:`handle` method of the returned
     handler and no further checkers are tried.
     
     Both transformers and checkers have a `priority` attribute, that determines
     the order in which they are called.  Smaller priorities are tried first.
 
     Both transformers and checkers also have `enabled` attribute, which is
     a boolean that determines if the instance is used.
 
     Users or developers can change the priority or enabled attribute of
     transformers or checkers, but they must call the :meth:`sort_checkers`
-    or :meth`sort_transformers` method after changing the priority.
+    or :meth:`sort_transformers` method after changing the priority.
     """
 
     multi_line_specials = CBool(True, config=True)
 
     def __init__(self, parent, config=None):
         super(PrefilterManager, self).__init__(parent, config=config)
         self.init_transformers()
         self.init_handlers()
         self.init_checkers()
 
     @auto_attr
     def shell(self):
         return Component.get_instances(
             root=self.root,
             klass='IPython.core.iplib.InteractiveShell')[0]
 
     #-------------------------------------------------------------------------
     # API for managing transformers
     #-------------------------------------------------------------------------
 
     def init_transformers(self):
         """Create the default transformers."""
         self._transformers = []
         for transformer_cls in _default_transformers:
             transformer_cls(self, config=self.config)
 
     def sort_transformers(self):
         """Sort the transformers by priority.
 
         This must be called after the priority of a transformer is changed.
         The :meth:`register_transformer` method calls this automatically.
         """
         self._transformers.sort(cmp=lambda x,y: x.priority-y.priority)
 
     @property
     def transformers(self):
         """Return a list of checkers, sorted by priority."""
         return self._transformers
 
     def register_transformer(self, transformer):
         """Register a transformer instance."""
         if transformer not in self._transformers:
             self._transformers.append(transformer)
             self.sort_transformers()
 
     def unregister_transformer(self, transformer):
         """Unregister a transformer instance."""
         if transformer in self._transformers:
             self._transformers.remove(transformer)
 
     #-------------------------------------------------------------------------
     # API for managing checkers
     #-------------------------------------------------------------------------
 
     def init_checkers(self):
         """Create the default checkers."""
         self._checkers = []
         for checker in _default_checkers:
             checker(self, config=self.config)
 
     def sort_checkers(self):
         """Sort the checkers by priority.
 
         This must be called after the priority of a checker is changed.
         The :meth:`register_checker` method calls this automatically.
         """
         self._checkers.sort(cmp=lambda x,y: x.priority-y.priority)
 
     @property
     def checkers(self):
         """Return a list of checkers, sorted by priority."""
         return self._checkers
 
     def register_checker(self, checker):
         """Register a checker instance."""
         if checker not in self._checkers:
             self._checkers.append(checker)
             self.sort_checkers()
 
     def unregister_checker(self, checker):
         """Unregister a checker instance."""
         if checker in self._checkers:
             self._checkers.remove(checker)
 
     #-------------------------------------------------------------------------
     # API for managing checkers
     #-------------------------------------------------------------------------
 
     def init_handlers(self):
         """Create the default handlers."""
         self._handlers = {}
         self._esc_handlers = {}
         for handler in _default_handlers:
             handler(self, config=self.config)
 
     @property
     def handlers(self):
         """Return a dict of all the handlers."""
         return self._handlers
 
     def register_handler(self, name, handler, esc_strings):
         """Register a handler instance by name with esc_strings."""
         self._handlers[name] = handler
         for esc_str in esc_strings:
             self._esc_handlers[esc_str] = handler
 
     def unregister_handler(self, name, handler, esc_strings):
         """Unregister a handler instance by name with esc_strings."""
         try:
             del self._handlers[name]
         except KeyError:
             pass
         for esc_str in esc_strings:
             h = self._esc_handlers.get(esc_str)
             if h is handler:
                 del self._esc_handlers[esc_str]
 
     def get_handler_by_name(self, name):
         """Get a handler by its name."""
         return self._handlers.get(name)
 
     def get_handler_by_esc(self, esc_str):
         """Get a handler by its escape string."""
         return self._esc_handlers.get(esc_str)
 
     #-------------------------------------------------------------------------
     # Main prefiltering API
     #-------------------------------------------------------------------------
 
     def prefilter_line_info(self, line_info):
         """Prefilter a line that has been converted to a LineInfo object.
 
         This implements the checker/handler part of the prefilter pipe.
         """
         # print "prefilter_line_info: ", line_info
         handler = self.find_handler(line_info)
         return handler.handle(line_info)
 
     def find_handler(self, line_info):
         """Find a handler for the line_info by trying checkers."""
         for checker in self.checkers:
             if checker.enabled:
                 handler = checker.check(line_info)
                 if handler:
                     return handler
         return self.get_handler_by_name('normal')
 
     def transform_line(self, line, continue_prompt):
         """Calls the enabled transformers in order of increasing priority."""
         for transformer in self.transformers:
             if transformer.enabled:
                 line = transformer.transform(line, continue_prompt)
         return line
 
     def prefilter_line(self, line, continue_prompt):
         """Prefilter a single input line as text.
 
         This method prefilters a single line of text by calling the
         transformers and then the checkers/handlers.
         """
 
         # print "prefilter_line: ", line, continue_prompt
         # All handlers *must* return a value, even if it's blank ('').
 
         # Lines are NOT logged here. Handlers should process the line as
         # needed, update the cache AND log it (so that the input cache array
         # stays synced).
 
         # save the line away in case we crash, so the post-mortem handler can
         # record it
         self.shell._last_input_line = line
 
         if not line:
             # Return immediately on purely empty lines, so that if the user
             # previously typed some whitespace that started a continuation
             # prompt, he can break out of that loop with just an empty line.
             # This is how the default python prompt works.
 
             # Only return if the accumulated input buffer was just whitespace!
             if ''.join(self.shell.buffer).isspace():
                 self.shell.buffer[:] = []
             return ''
 
         # At this point, we invoke our transformers.
         if not continue_prompt or (continue_prompt and self.multi_line_specials):
             line = self.transform_line(line, continue_prompt)
 
         # Now we compute line_info for the checkers and handlers
         line_info = LineInfo(line, continue_prompt)
         
         # the input history needs to track even empty lines
         stripped = line.strip()
 
         normal_handler = self.get_handler_by_name('normal')
         if not stripped:
             if not continue_prompt:
                 self.shell.outputcache.prompt_count -= 1
 
             return normal_handler.handle(line_info)
 
         # special handlers are only allowed for single line statements
         if continue_prompt and not self.multi_line_specials:
             return normal_handler.handle(line_info)
 
         prefiltered = self.prefilter_line_info(line_info)
         # print "prefiltered line: %r" % prefiltered
         return prefiltered
 
     def prefilter_lines(self, lines, continue_prompt):
         """Prefilter multiple input lines of text.
 
         This is the main entry point for prefiltering multiple lines of
         input.  This simply calls :meth:`prefilter_line` for each line of
         input.
 
         This covers cases where there are multiple lines in the user entry,
         which is the case when the user goes back to a multiline history
         entry and presses enter.
         """
         out = []
         for line in lines.rstrip('\n').split('\n'):
             out.append(self.prefilter_line(line, continue_prompt))
         return '\n'.join(out)
 
 
 #-----------------------------------------------------------------------------
 # Prefilter transformers
 #-----------------------------------------------------------------------------
 
 
 class PrefilterTransformer(Component):
     """Transform a line of user input."""
 
     priority = Int(100, config=True)
     shell = Any
     prefilter_manager = Any
     enabled = Bool(True, config=True)
 
     def __init__(self, parent, config=None):
         super(PrefilterTransformer, self).__init__(parent, config=config)
         self.prefilter_manager.register_transformer(self)
 
     @auto_attr
     def shell(self):
         return Component.get_instances(
             root=self.root,
             klass='IPython.core.iplib.InteractiveShell')[0]
 
     @auto_attr
     def prefilter_manager(self):
         return PrefilterManager.get_instances(root=self.root)[0]
 
     def transform(self, line, continue_prompt):
         """Transform a line, returning the new one."""
         return None
 
     def __repr__(self):
         return "<%s(priority=%r, enabled=%r)>" % (
             self.__class__.__name__, self.priority, self.enabled)
 
 
 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
                                r'\s*=\s*!(?P<cmd>.*)')
 
 
 class AssignSystemTransformer(PrefilterTransformer):
     """Handle the `files = !ls` syntax."""
 
     priority = Int(100, config=True)
 
     def transform(self, line, continue_prompt):
         m = _assign_system_re.match(line)
         if m is not None:
             cmd = m.group('cmd')
             lhs = m.group('lhs')
             expr = make_quoted_expr("sc -l =%s" % cmd)
             new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
             return new_line
         return line
 
 
 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
                                r'\s*=\s*%(?P<cmd>.*)')
 
 class AssignMagicTransformer(PrefilterTransformer):
     """Handle the `a = %who` syntax."""
 
     priority = Int(200, config=True)
 
     def transform(self, line, continue_prompt):
         m = _assign_magic_re.match(line)
         if m is not None:
             cmd = m.group('cmd')
             lhs = m.group('lhs')
             expr = make_quoted_expr(cmd)
             new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
             return new_line
         return line
 
 
 #-----------------------------------------------------------------------------
 # Prefilter checkers
 #-----------------------------------------------------------------------------
 
 
 class PrefilterChecker(Component):
     """Inspect an input line and return a handler for that line."""
 
     priority = Int(100, config=True)
     shell = Any
     prefilter_manager = Any
     enabled = Bool(True, config=True)
 
     def __init__(self, parent, config=None):
         super(PrefilterChecker, self).__init__(parent, config=config)
         self.prefilter_manager.register_checker(self)
 
     @auto_attr
     def shell(self):
         return Component.get_instances(
             root=self.root,
             klass='IPython.core.iplib.InteractiveShell')[0]
 
     @auto_attr
     def prefilter_manager(self):
         return PrefilterManager.get_instances(root=self.root)[0]
 
     def check(self, line_info):
         """Inspect line_info and return a handler instance or None."""
         return None
 
     def __repr__(self):
         return "<%s(priority=%r, enabled=%r)>" % (
             self.__class__.__name__, self.priority, self.enabled)
 
 
 class EmacsChecker(PrefilterChecker):
 
     priority = Int(100, config=True)
     enabled = Bool(False, config=True)
 
     def check(self, line_info):
         "Emacs ipython-mode tags certain input lines."
         if line_info.line.endswith('# PYTHON-MODE'):
             return self.prefilter_manager.get_handler_by_name('emacs')
         else:
             return None
 
 
 class ShellEscapeChecker(PrefilterChecker):
 
     priority = Int(200, config=True)
 
     def check(self, line_info):
         if line_info.line.lstrip().startswith(ESC_SHELL):
             return self.prefilter_manager.get_handler_by_name('shell')
 
 
 class IPyAutocallChecker(PrefilterChecker):
 
     priority = Int(300, config=True)
 
     def check(self, line_info):
         "Instances of IPyAutocall in user_ns get autocalled immediately"
         obj = self.shell.user_ns.get(line_info.ifun, None)
         if isinstance(obj, IPyAutocall):
             obj.set_ip(self.shell)
             return self.prefilter_manager.get_handler_by_name('auto')
         else:
             return None
 
 
 class MultiLineMagicChecker(PrefilterChecker):
 
     priority = Int(400, config=True)
 
     def check(self, line_info):
         "Allow ! and !! in multi-line statements if multi_line_specials is on"
         # Note that this one of the only places we check the first character of
         # ifun and *not* the pre_char.  Also note that the below test matches
         # both ! and !!.    
         if line_info.continue_prompt \
             and self.prefilter_manager.multi_line_specials:
                 if line_info.ifun.startswith(ESC_MAGIC):
                     return self.prefilter_manager.get_handler_by_name('magic')
         else:
             return None
 
 
 class EscCharsChecker(PrefilterChecker):
 
     priority = Int(500, config=True)
 
     def check(self, line_info):
         """Check for escape character and return either a handler to handle it,
         or None if there is no escape char."""
         if line_info.line[-1] == ESC_HELP \
                and line_info.pre_char != ESC_SHELL \
                and line_info.pre_char != ESC_SH_CAP:
             # the ? can be at the end, but *not* for either kind of shell escape,
             # because a ? can be a vaild final char in a shell cmd
             return self.prefilter_manager.get_handler_by_name('help')
         else:
             # This returns None like it should if no handler exists
             return self.prefilter_manager.get_handler_by_esc(line_info.pre_char)
 
 
 class AssignmentChecker(PrefilterChecker):
 
     priority = Int(600, config=True)
 
     def check(self, line_info):
         """Check to see if user is assigning to a var for the first time, in
         which case we want to avoid any sort of automagic / autocall games.
     
         This allows users to assign to either alias or magic names true python
         variables (the magic/alias systems always take second seat to true
         python code).  E.g. ls='hi', or ls,that=1,2"""
         if line_info.the_rest:
             if line_info.the_rest[0] in '=,':
                 return self.prefilter_manager.get_handler_by_name('normal')
         else:
             return None
 
 
 class AutoMagicChecker(PrefilterChecker):
 
     priority = Int(700, config=True)
 
     def check(self, line_info):
         """If the ifun is magic, and automagic is on, run it.  Note: normal,
         non-auto magic would already have been triggered via '%' in
         check_esc_chars. This just checks for automagic.  Also, before
         triggering the magic handler, make sure that there is nothing in the
         user namespace which could shadow it."""
         if not self.shell.automagic or not hasattr(self.shell,'magic_'+line_info.ifun):
             return None
 
         # We have a likely magic method.  Make sure we should actually call it.
         if line_info.continue_prompt and not self.shell.multi_line_specials:
             return None
 
         head = line_info.ifun.split('.',1)[0]
         if is_shadowed(head, self.shell):
             return None
 
         return self.prefilter_manager.get_handler_by_name('magic')
 
 
 class AliasChecker(PrefilterChecker):
 
     priority = Int(800, config=True)
 
     @auto_attr
     def alias_manager(self):
         return AliasManager.get_instances(root=self.root)[0]
 
     def check(self, line_info):
         "Check if the initital identifier on the line is an alias."
         # Note: aliases can not contain '.'
         head = line_info.ifun.split('.',1)[0]
         if line_info.ifun not in self.alias_manager \
                or head not in self.alias_manager \
                or is_shadowed(head, self.shell):
             return None
 
         return self.prefilter_manager.get_handler_by_name('alias')
 
 
 class PythonOpsChecker(PrefilterChecker):
 
     priority = Int(900, config=True)
 
     def check(self, line_info):
         """If the 'rest' of the line begins with a function call or pretty much
         any python operator, we should simply execute the line (regardless of
         whether or not there's a possible autocall expansion).  This avoids
         spurious (and very confusing) geattr() accesses."""
         if line_info.the_rest and line_info.the_rest[0] in '!=()<>,+*/%^&|':
             return self.prefilter_manager.get_handler_by_name('normal')
         else:
             return None
 
 
 class AutocallChecker(PrefilterChecker):
 
     priority = Int(1000, config=True)
 
     def check(self, line_info):
         "Check if the initial word/function is callable and autocall is on."
         if not self.shell.autocall:
             return None
 
         oinfo = line_info.ofind(self.shell) # This can mutate state via getattr
         if not oinfo['found']:
             return None
         
         if callable(oinfo['obj']) \
                and (not re_exclude_auto.match(line_info.the_rest)) \
                and re_fun_name.match(line_info.ifun):
             return self.prefilter_manager.get_handler_by_name('auto')
         else:
             return None
 
 
 #-----------------------------------------------------------------------------
 # Prefilter handlers
 #-----------------------------------------------------------------------------
 
 
 class PrefilterHandler(Component):
 
     handler_name = Str('normal')
     esc_strings = List([])
     shell = Any
     prefilter_manager = Any
 
     def __init__(self, parent, config=None):
         super(PrefilterHandler, self).__init__(parent, config=config)
         self.prefilter_manager.register_handler(
             self.handler_name,
             self,
             self.esc_strings
         )
 
     @auto_attr
     def shell(self):
         return Component.get_instances(
             root=self.root,
             klass='IPython.core.iplib.InteractiveShell')[0]
 
     @auto_attr
     def prefilter_manager(self):
         return PrefilterManager.get_instances(root=self.root)[0]
 
     def handle(self, line_info):
         # print "normal: ", line_info
         """Handle normal input lines. Use as a template for handlers."""
 
         # With autoindent on, we need some way to exit the input loop, and I
         # don't want to force the user to have to backspace all the way to
         # clear the line.  The rule will be in this case, that either two
         # lines of pure whitespace in a row, or a line of pure whitespace but
         # of a size different to the indent level, will exit the input loop.
         line = line_info.line
         continue_prompt = line_info.continue_prompt
 
         if (continue_prompt and self.shell.autoindent and line.isspace() and
             (0 < abs(len(line) - self.shell.indent_current_nsp) <= 2 or
              (self.shell.buffer[-1]).isspace() )):
             line = ''
 
         self.shell.log(line, line, continue_prompt)
         return line
 
     def __str__(self):
         return "<%s(name=%s)>" % (self.__class__.__name__, self.handler_name)
 
 
 class AliasHandler(PrefilterHandler):
 
     handler_name = Str('alias')
 
     @auto_attr
     def alias_manager(self):
         return AliasManager.get_instances(root=self.root)[0]
 
     def handle(self, line_info):
         """Handle alias input lines. """
         transformed = self.alias_manager.expand_aliases(line_info.ifun,line_info.the_rest)
         # pre is needed, because it carries the leading whitespace.  Otherwise
         # aliases won't work in indented sections.
         line_out = '%sget_ipython().system(%s)' % (line_info.pre_whitespace,
                                          make_quoted_expr(transformed))
         
         self.shell.log(line_info.line, line_out, line_info.continue_prompt)
         return line_out
 
 
 class ShellEscapeHandler(PrefilterHandler):
 
     handler_name = Str('shell')
     esc_strings = List([ESC_SHELL, ESC_SH_CAP])
 
     def handle(self, line_info):
         """Execute the line in a shell, empty return value"""
         magic_handler = self.prefilter_manager.get_handler_by_name('magic')
 
         line = line_info.line
         if line.lstrip().startswith(ESC_SH_CAP):
             # rewrite LineInfo's line, ifun and the_rest to properly hold the
             # call to %sx and the actual command to be executed, so
             # handle_magic can work correctly.  Note that this works even if
             # the line is indented, so it handles multi_line_specials
             # properly.
             new_rest = line.lstrip()[2:]
             line_info.line = '%ssx %s' % (ESC_MAGIC, new_rest)
             line_info.ifun = 'sx'
             line_info.the_rest = new_rest
             return magic_handler.handle(line_info)
         else:
             cmd = line.lstrip().lstrip(ESC_SHELL)
             line_out = '%sget_ipython().system(%s)' % (line_info.pre_whitespace,
                                              make_quoted_expr(cmd))
         # update cache/log and return
         self.shell.log(line, line_out, line_info.continue_prompt)
         return line_out
 
 
 class MagicHandler(PrefilterHandler):
 
     handler_name = Str('magic')
     esc_strings = List([ESC_MAGIC])
 
     def handle(self, line_info):
         """Execute magic functions."""
         ifun    = line_info.ifun
         the_rest = line_info.the_rest
         cmd = '%sget_ipython().magic(%s)' % (line_info.pre_whitespace,
                                    make_quoted_expr(ifun + " " + the_rest))
         self.shell.log(line_info.line, cmd, line_info.continue_prompt)
         return cmd
 
 
 class AutoHandler(PrefilterHandler):
 
     handler_name = Str('auto')
     esc_strings = List([ESC_PAREN, ESC_QUOTE, ESC_QUOTE2])
 
     def handle(self, line_info):
         """Hande lines which can be auto-executed, quoting if requested."""
         line    = line_info.line
         ifun    = line_info.ifun
         the_rest = line_info.the_rest
         pre     = line_info.pre
         continue_prompt = line_info.continue_prompt
         obj = line_info.ofind(self)['obj']
 
         #print 'pre <%s> ifun <%s> rest <%s>' % (pre,ifun,the_rest)  # dbg
 
         # This should only be active for single-line input!
         if continue_prompt:
             self.log(line,line,continue_prompt)
             return line
 
         force_auto = isinstance(obj, IPyAutocall)
         auto_rewrite = True
         
         if pre == ESC_QUOTE:
             # Auto-quote splitting on whitespace
             newcmd = '%s("%s")' % (ifun,'", "'.join(the_rest.split()) )
         elif pre == ESC_QUOTE2:
             # Auto-quote whole string
             newcmd = '%s("%s")' % (ifun,the_rest)
         elif pre == ESC_PAREN:
             newcmd = '%s(%s)' % (ifun,",".join(the_rest.split()))
         else:
             # Auto-paren.
             # We only apply it to argument-less calls if the autocall
             # parameter is set to 2.  We only need to check that autocall is <
             # 2, since this function isn't called unless it's at least 1.
             if not the_rest and (self.shell.autocall < 2) and not force_auto:
                 newcmd = '%s %s' % (ifun,the_rest)
                 auto_rewrite = False
             else:
                 if not force_auto and the_rest.startswith('['):
                     if hasattr(obj,'__getitem__'):
                         # Don't autocall in this case: item access for an object
                         # which is BOTH callable and implements __getitem__.
                         newcmd = '%s %s' % (ifun,the_rest)
                         auto_rewrite = False
                     else:
                         # if the object doesn't support [] access, go ahead and
                         # autocall
                         newcmd = '%s(%s)' % (ifun.rstrip(),the_rest)
                 elif the_rest.endswith(';'):
                     newcmd = '%s(%s);' % (ifun.rstrip(),the_rest[:-1])
                 else:
                     newcmd = '%s(%s)' % (ifun.rstrip(), the_rest)
 
         if auto_rewrite:
             rw = self.shell.outputcache.prompt1.auto_rewrite() + newcmd
             
             try:
                 # plain ascii works better w/ pyreadline, on some machines, so
                 # we use it and only print uncolored rewrite if we have unicode
                 rw = str(rw)
                 print >>Term.cout, rw
             except UnicodeEncodeError:
                 print "-------------->" + newcmd
             
         # log what is now valid Python, not the actual user input (without the
         # final newline)
         self.shell.log(line,newcmd,continue_prompt)
         return newcmd
 
 
 class HelpHandler(PrefilterHandler):
 
     handler_name = Str('help')
     esc_strings = List([ESC_HELP])
 
     def handle(self, line_info):
         """Try to get some help for the object.
 
         obj? or ?obj   -> basic information.
         obj?? or ??obj -> more details.
         """
         normal_handler = self.prefilter_manager.get_handler_by_name('normal')
         line = line_info.line
         # We need to make sure that we don't process lines which would be
         # otherwise valid python, such as "x=1 # what?"
         try:
             codeop.compile_command(line)
         except SyntaxError:
             # We should only handle as help stuff which is NOT valid syntax
             if line[0]==ESC_HELP:
                 line = line[1:]
             elif line[-1]==ESC_HELP:
                 line = line[:-1]
             self.shell.log(line, '#?'+line, line_info.continue_prompt)
             if line:
                 #print 'line:<%r>' % line  # dbg
                 self.shell.magic_pinfo(line)
             else:
                 page(self.shell.usage, screen_lines=self.shell.usable_screen_length)
             return '' # Empty string is needed here!
         except:
             raise
             # Pass any other exceptions through to the normal handler
             return normal_handler.handle(line_info)
         else:
             raise
             # If the code compiles ok, we should handle it normally
             return normal_handler.handle(line_info)
 
 
 class EmacsHandler(PrefilterHandler):
 
     handler_name = Str('emacs')
     esc_strings = List([])
 
     def handle(self, line_info):
         """Handle input lines marked by python-mode."""
 
         # Currently, nothing is done.  Later more functionality can be added
         # here if needed.
 
         # The input cache shouldn't be updated
         return line_info.line
 
 
 #-----------------------------------------------------------------------------
 # Defaults
 #-----------------------------------------------------------------------------
 
 
 _default_transformers = [
     AssignSystemTransformer,
     AssignMagicTransformer
 ]
 
 _default_checkers = [
     EmacsChecker,
     ShellEscapeChecker,
     IPyAutocallChecker,
     MultiLineMagicChecker,
     EscCharsChecker,
     AssignmentChecker,
     AutoMagicChecker,
     AliasChecker,
     PythonOpsChecker,
     AutocallChecker
 ]
 
 _default_handlers = [
     PrefilterHandler,
     AliasHandler,
     ShellEscapeHandler,
     MagicHandler,
     AutoHandler,
     HelpHandler,
     EmacsHandler
 ]
 

docs/source/_templates/layout.html

@@ -1,23 +1,23 b''
 {% extends "!layout.html" %}
 
 
 {% block rootrellink %}
         <li><a href="{{ pathto('index') }}">home</a>|&nbsp;</li>
         <li><a href="{{ pathto('search') }}">search</a>|&nbsp;</li>
-       <li><a href="{{ pathto('whatsnew/index') }}">what's new </a> &raquo;</li>
+       <li><a href="{{ pathto('index') }}">documentation </a> &raquo;</li>
 {% endblock %}
 
 
 {% block relbar1 %}
 
 <div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px">
 <a href="{{ pathto('index') }}"><img src="{{
 pathto("_static/logo.png", 1) }}" border="0" alt="IPython Documentation"/></a>
 </div>
 {{ super() }}
 {% endblock %}
 
 {# put the sidebar before the body #}
 {% block sidebar1 %}{{ sidebar() }}{% endblock %}
 {% block sidebar2 %}{% endblock %}
 

docs/source/config/index.txt

@@ -1,11 +1,11 b''
 ===============================
 Configuration and customization
 ===============================
 
 .. toctree::
    :maxdepth: 2
 
-   initial_config.txt
-   customization.txt
-   new_config.txt
-
+   overview.txt
+   ipython.txt
+   editors.txt
+   old.txt

docs/source/config/old.txt

@@ -1,250 +1,230 b''
 .. _initial config:
 
-=========================================
-Initial configuration of your environment
-=========================================
+=============================================================
+Outdated configuration information that might still be useful
+=============================================================
+
+.. warning::
+
+    All of the information in this file is outdated. Until the new
+    configuration system is better documented, this material is being kept.
 
 This section will help you set various things in your environment for
 your IPython sessions to be as efficient as possible. All of IPython's
 configuration information, along with several example files, is stored
 in a directory named by default $HOME/.ipython. You can change this by
 defining the environment variable IPYTHONDIR, or at runtime with the
 command line option -ipythondir.
 
 If all goes well, the first time you run IPython it should automatically create
 a user copy of the config directory for you, based on its builtin defaults. You
 can look at the files it creates to learn more about configuring the
 system. The main file you will modify to configure IPython's behavior is called
 ipythonrc (with a .ini extension under Windows), included for reference
 :ref:`here <ipythonrc>`. This file is very commented and has many variables you
 can change to suit your taste, you can find more details :ref:`here
 <customization>`. Here we discuss the basic things you will want to make sure
 things are working properly from the beginning.
 
-
-.. _accessing_help:
-
-Access to the Python help system
-================================
-
-This is true for Python in general (not just for IPython): you should have an
-environment variable called PYTHONDOCS pointing to the directory where your
-HTML Python documentation lives. In my system it's
-:file:`/usr/share/doc/python-doc/html`, check your local details or ask your
-systems administrator.
-
-This is the directory which holds the HTML version of the Python
-manuals. Unfortunately it seems that different Linux distributions
-package these files differently, so you may have to look around a bit.
-Below I show the contents of this directory on my system for reference::
-
-    [html]> ls
-    about.html  dist/  icons/      lib/           python2.5.devhelp.gz  whatsnew/
-    acks.html   doc/   index.html  mac/           ref/
-    api/        ext/   inst/       modindex.html  tut/
-
-You should really make sure this variable is correctly set so that
-Python's pydoc-based help system works. It is a powerful and convenient
-system with full access to the Python manuals and all modules accessible
-to you.
-
-Under Windows it seems that pydoc finds the documentation automatically,
-so no extra setup appears necessary.
-
-
-Editor
-======
-
-The %edit command (and its alias %ed) will invoke the editor set in your
-environment as EDITOR. If this variable is not set, it will default to
-vi under Linux/Unix and to notepad under Windows. You may want to set
-this variable properly and to a lightweight editor which doesn't take
-too long to start (that is, something other than a new instance of
-Emacs). This way you can edit multi-line code quickly and with the power
-of a real editor right inside IPython.
-
-If you are a dedicated Emacs user, you should set up the Emacs server so
-that new requests are handled by the original process. This means that
-almost no time is spent in handling the request (assuming an Emacs
-process is already running). For this to work, you need to set your
-EDITOR environment variable to 'emacsclient'. The code below, supplied
-by Francois Pinard, can then be used in your .emacs file to enable the
-server::
-
-    (defvar server-buffer-clients)
-    (when (and (fboundp 'server-start) (string-equal (getenv "TERM") 'xterm))
-      (server-start)
-      (defun fp-kill-server-with-buffer-routine ()
-        (and server-buffer-clients (server-done)))
-      (add-hook 'kill-buffer-hook 'fp-kill-server-with-buffer-routine))
-
-You can also set the value of this editor via the commmand-line option
-'-editor' or in your ipythonrc file. This is useful if you wish to use
-specifically for IPython an editor different from your typical default
-(and for Windows users who tend to use fewer environment variables).
-
-
 Color
 =====
 
 The default IPython configuration has most bells and whistles turned on
 (they're pretty safe). But there's one that may cause problems on some
 systems: the use of color on screen for displaying information. This is
 very useful, since IPython can show prompts and exception tracebacks
 with various colors, display syntax-highlighted source code, and in
 general make it easier to visually parse information.
 
 The following terminals seem to handle the color sequences fine:
 
     * Linux main text console, KDE Konsole, Gnome Terminal, E-term,
       rxvt, xterm.
     * CDE terminal (tested under Solaris). This one boldfaces light colors.
     * (X)Emacs buffers. See the emacs_ section for more details on
       using IPython with (X)Emacs.
     * A Windows (XP/2k) command prompt with pyreadline_.
     * A Windows (XP/2k) CygWin shell. Although some users have reported
       problems; it is not clear whether there is an issue for everyone
       or only under specific configurations. If you have full color
       support under cygwin, please post to the IPython mailing list so
       this issue can be resolved for all users.
 
 .. _pyreadline: https://code.launchpad.net/pyreadline
       
 These have shown problems:
 
     * Windows command prompt in WinXP/2k logged into a Linux machine via
       telnet or ssh.
     * Windows native command prompt in WinXP/2k, without Gary Bishop's
       extensions. Once Gary's readline library is installed, the normal
       WinXP/2k command prompt works perfectly.
 
 Currently the following color schemes are available:
 
     * NoColor: uses no color escapes at all (all escapes are empty '' ''
       strings). This 'scheme' is thus fully safe to use in any terminal.
     * Linux: works well in Linux console type environments: dark
       background with light fonts. It uses bright colors for
       information, so it is difficult to read if you have a light
       colored background.
     * LightBG: the basic colors are similar to those in the Linux scheme
       but darker. It is easy to read in terminals with light backgrounds.
 
 IPython uses colors for two main groups of things: prompts and
 tracebacks which are directly printed to the terminal, and the object
 introspection system which passes large sets of data through a pager.
 
-
 Input/Output prompts and exception tracebacks
 =============================================
 
 You can test whether the colored prompts and tracebacks work on your
 system interactively by typing '%colors Linux' at the prompt (use
 '%colors LightBG' if your terminal has a light background). If the input
 prompt shows garbage like::
 
     [0;32mIn [[1;32m1[0;32m]: [0;00m
 
 instead of (in color) something like::
 
     In [1]:
 
 this means that your terminal doesn't properly handle color escape
 sequences. You can go to a 'no color' mode by typing '%colors NoColor'.
 
 You can try using a different terminal emulator program (Emacs users,
 see below). To permanently set your color preferences, edit the file
 $HOME/.ipython/ipythonrc and set the colors option to the desired value.
 
 
 Object details (types, docstrings, source code, etc.)
 =====================================================
 
 IPython has a set of special functions for studying the objects you are working
 with, discussed in detail :ref:`here <dynamic_object_info>`. But this system
 relies on passing information which is longer than your screen through a data
 pager, such as the common Unix less and more programs. In order to be able to
 see this information in color, your pager needs to be properly configured. I
 strongly recommend using less instead of more, as it seems that more simply can
 not understand colored text correctly.
 
 In order to configure less as your default pager, do the following:
 
    1. Set the environment PAGER variable to less.
    2. Set the environment LESS variable to -r (plus any other options
       you always want to pass to less by default). This tells less to
       properly interpret control sequences, which is how color
       information is given to your terminal.
 
 For the bash shell, add to your ~/.bashrc file the lines::
 
     export PAGER=less
     export LESS=-r
 
 For the csh or tcsh shells, add to your ~/.cshrc file the lines::
 
     setenv PAGER less
     setenv LESS -r
     
 There is similar syntax for other Unix shells, look at your system
 documentation for details.
 
 If you are on a system which lacks proper data pagers (such as Windows),
 IPython will use a very limited builtin pager.
 
-.. _emacs:
-
-(X)Emacs configuration
-======================
-
-Thanks to the work of Alexander Schmolck and Prabhu Ramachandran,
-currently (X)Emacs and IPython get along very well.
-
-Important note: You will need to use a recent enough version of
-python-mode.el, along with the file ipython.el. You can check that the
-version you have of python-mode.el is new enough by either looking at
-the revision number in the file itself, or asking for it in (X)Emacs via
-M-x py-version. Versions 4.68 and newer contain the necessary fixes for
-proper IPython support.
-
-The file ipython.el is included with the IPython distribution, in the
-documentation directory (where this manual resides in PDF and HTML
-formats).
-
-Once you put these files in your Emacs path, all you need in your .emacs
-file is::
-
-    (require 'ipython)
-
-This should give you full support for executing code snippets via
-IPython, opening IPython as your Python shell via ``C-c !``, etc.
-
-You can customize the arguments passed to the IPython instance at startup by
-setting the ``py-python-command-args`` variable.  For example, to start always
-in ``pylab`` mode with hardcoded light-background colors, you can use::
-
-    (setq py-python-command-args '("-pylab" "-colors" "LightBG"))
-
-If you happen to get garbage instead of colored prompts as described in
-the previous section, you may need to set also in your .emacs file::
-
-    (setq ansi-color-for-comint-mode t)
-
-Notes:
+.. _Prompts:
+
+Fine-tuning your prompt
+=======================
+
+IPython's prompts can be customized using a syntax similar to that of
+the bash shell. Many of bash's escapes are supported, as well as a few
+additional ones. We list them below::
+
+    \#
+        the prompt/history count number. This escape is automatically
+        wrapped in the coloring codes for the currently active color scheme. 
+    \N
+        the 'naked' prompt/history count number: this is just the number
+        itself, without any coloring applied to it. This lets you produce
+        numbered prompts with your own colors. 
+    \D
+        the prompt/history count, with the actual digits replaced by dots.
+        Used mainly in continuation prompts (prompt_in2) 
+    \w
+        the current working directory 
+    \W
+        the basename of current working directory 
+    \Xn
+        where $n=0\ldots5.$ The current working directory, with $HOME
+        replaced by ~, and filtered out to contain only $n$ path elements 
+    \Yn
+        Similar to \Xn, but with the $n+1$ element included if it is ~ (this
+        is similar to the behavior of the %cn escapes in tcsh) 
+    \u
+        the username of the current user 
+    \$
+        if the effective UID is 0, a #, otherwise a $ 
+    \h
+        the hostname up to the first '.' 
+    \H
+        the hostname 
+    \n
+        a newline 
+    \r
+        a carriage return 
+    \v
+        IPython version string 
+
+In addition to these, ANSI color escapes can be insterted into the
+prompts, as \C_ColorName. The list of valid color names is: Black, Blue,
+Brown, Cyan, DarkGray, Green, LightBlue, LightCyan, LightGray,
+LightGreen, LightPurple, LightRed, NoColor, Normal, Purple, Red, White,
+Yellow.
+
+Finally, IPython supports the evaluation of arbitrary expressions in
+your prompt string. The prompt strings are evaluated through the syntax
+of PEP 215, but basically you can use $x.y to expand the value of x.y,
+and for more complicated expressions you can use braces: ${foo()+x} will
+call function foo and add to it the value of x, before putting the
+result into your prompt. For example, using
+prompt_in1 '${commands.getoutput("uptime")}\nIn [\#]: ' 
+will print the result of the uptime command on each prompt (assuming the
+commands module has been imported in your ipythonrc file).
+
+
+      Prompt examples
+
+The following options in an ipythonrc file will give you IPython's
+default prompts::
+
+    prompt_in1 'In [\#]:' 
+    prompt_in2 '   .\D.:' 
+    prompt_out 'Out[\#]:'
+
+which look like this::
+
+    In [1]: 1+2 
+    Out[1]: 3
+
+    In [2]: for i in (1,2,3): 
+       ...:    print i, 
+       ...: 
+    1 2 3
+
+These will give you a very colorful prompt with path information::
+
+    #prompt_in1 '\C_Red\u\C_Blue[\C_Cyan\Y1\C_Blue]\C_LightGreen\#>' 
+    prompt_in2 ' ..\D>' 
+    prompt_out '<\#>'
+
+which look like this::
+
+    fperez[~/ipython]1> 1+2 
+                    <1> 3 
+    fperez[~/ipython]2> for i in (1,2,3): 
+                   ...>     print i, 
+                   ...> 
+    1 2 3
 
-    * There is one caveat you should be aware of: you must start the
-      IPython shell before attempting to execute any code regions via
-      ``C-c |``. Simply type C-c ! to start IPython before passing any code
-      regions to the interpreter, and you shouldn't experience any
-      problems.
-      This is due to a bug in Python itself, which has been fixed for
-      Python 2.3, but exists as of Python 2.2.2 (reported as SF bug [
-      737947 ]).
-    * The (X)Emacs support is maintained by Alexander Schmolck, so all
-      comments/requests should be directed to him through the IPython
-      mailing lists.
-    * This code is still somewhat experimental so it's a bit rough
-      around the edges (although in practice, it works quite well).
-    * Be aware that if you customize py-python-command previously, this
-      value will override what ipython.el does (because loading the
-      customization variables comes later).
 

docs/source/development/coding_guide.txt

@@ -1,156 +1,78 b''
-==============
+============
  Coding guide
-==============
+============
 
-
-Coding conventions
-==================
+General coding conventions
+==========================
 
 In general, we'll try to follow the standard Python style conventions as
-described in Python's `PEP 8`_, the official Python Style Guide.
+described in Python's PEP 8 [PEP8]_, the official Python Style Guide.
 
-.. _PEP 8: http://www.python.org/peps/pep-0008.html
+Other general comments:
 
-Other comments:
-
-- In a large file, top level classes and functions should be separated by 2-3
+* In a large file, top level classes and functions should be separated by 2
   lines to make it easier to separate them visually.
 
-- Use 4 spaces for indentation, *never* use hard tabs.
+* Use 4 spaces for indentation, **never** use hard tabs.
 
-- Keep the ordering of methods the same in classes that have the same methods.
+* Keep the ordering of methods the same in classes that have the same methods.
   This is particularly true for classes that implement similar interfaces and
   for interfaces that are similar.
 
 Naming conventions
-------------------
+==================
 
-In terms of naming conventions, we'll follow the guidelines of PEP 8.  Some of
-the existing code doesn't honor this perfectly, but for all new IPython code
-(and much existing code is being refactored), we'll use:
+In terms of naming conventions, we'll follow the guidelines of PEP 8 [PEP8]_.
+Some of the existing code doesn't honor this perfectly, but for all new
+IPython code (and much existing code is being refactored), we'll use:
 
-- All ``lowercase`` module names.
+* All ``lowercase`` module names.
 
-- ``CamelCase`` for class names.
+* ``CamelCase`` for class names.
 
-- ``lowercase_with_underscores`` for methods, functions, variables and
+* ``lowercase_with_underscores`` for methods, functions, variables and
   attributes.
 
 This may be confusing as some of the existing codebase uses a different
 convention (``lowerCamelCase`` for methods and attributes).  Slowly, we will
 move IPython over to the new convention, providing shadow names for backward
 compatibility in public interfaces.
 
 There are, however, some important exceptions to these rules.  In some cases,
 IPython code will interface with packages (Twisted, Wx, Qt) that use other
 conventions.  At some level this makes it impossible to adhere to our own
 standards at all times.  In particular, when subclassing classes that use other
 naming conventions, you must follow their naming conventions.  To deal with
 cases like this, we propose the following policy:
 
-- If you are subclassing a class that uses different conventions, use its
-  naming conventions throughout your subclass.  Thus, if you are creating a
-  Twisted Protocol class, used Twisted's
-  ``namingSchemeForMethodsAndAttributes.``
-
-- All IPython's official interfaces should use our conventions.  In some cases
-  this will mean that you need to provide shadow names (first implement
-  ``fooBar`` and then ``foo_bar = fooBar``).  We want to avoid this at all
-  costs, but it will probably be necessary at times.  But, please use this
-  sparingly!
-
-Implementation-specific *private* methods will use
-``_single_underscore_prefix``.  Names with a leading double underscore will
-*only* be used in special cases, as they makes subclassing difficult (such
-names are not easily seen by child classes).
-
-Occasionally some run-in lowercase names are used, but mostly for very short
-names or where we are implementing methods very similar to existing ones in a
-base class (like ``runlines()`` where ``runsource()`` and ``runcode()`` had
-established precedent).
-
-The old IPython codebase has a big mix of classes and modules prefixed with an
-explicit ``IP``. In Python this is mostly unnecessary, redundant and frowned
-upon, as namespaces offer cleaner prefixing. The only case where this approach
-is justified is for classes which are expected to be imported into external
-namespaces and a very generic name (like Shell) is too likely to clash with
-something else.  We'll need to revisit this issue as we clean up and refactor
-the code, but in general we should remove as many unnecessary ``IP``/``ip``
-prefixes as possible. However, if a prefix seems absolutely necessary the more
-specific ``IPY`` or ``ipy`` are preferred.
-
-Older material
-==============
-
-General
--------
-
-In general, we'll try to follow the standard Python style conventions as
-described here:
-
-* `Style Guide for Python Code  <http://www.python.org/peps/pep-0008.html>`_  
-
-
-Other comments:
-
-* In a large file, top level classes and functions should be 
-  separated by 2-3 lines to make it easier to separate them visually.
-* Use 4 spaces for indentation.
-* Keep the ordering of methods the same in classes that have the same 
-  methods.  This is particularly true for classes that implement an interface.
-
-Naming conventions
-------------------
-
-In terms of naming conventions, we'll follow the guidelines from the `Style
-Guide for Python Code`_.
-
-For all new IPython code (and much existing code is being refactored), we'll
-use:
-
-* All ``lowercase`` module names.
-
-* ``CamelCase`` for class names.
-
-* ``lowercase_with_underscores`` for methods, functions, variables and
-  attributes.
-
-There are, however, some important exceptions to these rules. In some cases,
-IPython code will interface with packages (Twisted, Wx, Qt) that use other
-conventions. At some level this makes it impossible to adhere to our own
-standards at all times. In particular, when subclassing classes that use other
-naming conventions, you must follow their naming conventions. To deal with
-cases like this, we propose the following policy:
-
 * If you are subclassing a class that uses different conventions, use its
   naming conventions throughout your subclass.  Thus, if you are creating a
   Twisted Protocol class, used Twisted's
   ``namingSchemeForMethodsAndAttributes.``
 
 * All IPython's official interfaces should use our conventions.  In some cases
   this will mean that you need to provide shadow names (first implement
   ``fooBar`` and then ``foo_bar = fooBar``).  We want to avoid this at all
   costs, but it will probably be necessary at times.  But, please use this
   sparingly!
 
 Implementation-specific *private* methods will use
 ``_single_underscore_prefix``. Names with a leading double underscore will
 *only* be used in special cases, as they makes subclassing difficult (such
 names are not easily seen by child classes).
 
 Occasionally some run-in lowercase names are used, but mostly for very short
 names or where we are implementing methods very similar to existing ones in a
 base class (like ``runlines()`` where ``runsource()`` and ``runcode()`` had
 established precedent).
 
 The old IPython codebase has a big mix of classes and modules prefixed with an
 explicit ``IP``. In Python this is mostly unnecessary, redundant and frowned
 upon, as namespaces offer cleaner prefixing. The only case where this approach
 is justified is for classes which are expected to be imported into external
 namespaces and a very generic name (like Shell) is too likely to clash with
-something else. We'll need to revisit this issue as we clean up and refactor
-the code, but in general we should remove as many unnecessary ``IP``/``ip``
-prefixes as possible. However, if a prefix seems absolutely necessary the more
+something else.  However, if a prefix seems absolutely necessary the more
 specific ``IPY`` or ``ipy`` are preferred.
 
+.. [PEP8] Python Enhancement Proposal 8.  http://www.python.org/peps/pep-0008.html
 

docs/source/development/contributing.txt

@@ -1,191 +1,209 b''
+.. _contributing:
+
+============================
 How to contribute to IPython
 ============================
 
+Overview
+========
+
 IPython development is done using Bazaar [Bazaar]_ and Launchpad [Launchpad]_.
 This makes it easy for people to contribute to the development of IPython.
 There are several ways in which you can join in.
 
-If you have a small change that you want to send to the team, you can edit your
-bazaar checkout of IPython (see below) in-place, and ask bazaar for the
-differences::
+If you have a small change that you want to contribute, you can edit your
+Bazaar checkout of IPython (see below) in-place, and ask Bazaar for the
+differences:
+
+.. code-block:: bash
 
     $ cd /path/to/your/copy/of/ipython
     $ bzr diff > my_fixes.diff
 
 This produces a patch file with your fixes, which we can apply to the source
 tree.  This file should then be attached to a ticket in our `bug tracker
 <https://bugs.launchpad.net/ipython>`_, indicating what it does.
 
 This model of creating small, self-contained patches works very well and there
 are open source projects that do their entire development this way.  However,
 in IPython we have found that for tracking larger changes, making use of
-bazaar's full capabilities in conjunction with Launchpad's code hosting
+Bazaar's full capabilities in conjunction with Launchpad's code hosting
 services makes for a much better experience.
 
 Making your own branch of IPython allows you to refine your changes over time,
 track the development of the main team, and propose your own full version of
 the code for others to use and review, with a minimum amount of fuss.  The next
 parts of this document will explain how to do this.
 
 Install Bazaar and create a Launchpad account
 ---------------------------------------------
 
 First make sure you have installed Bazaar (see their `website
 <http://bazaar-vcs.org/>`_). To see that Bazaar is installed and knows about
-you, try the following::
+you, try the following:
+
+.. code-block:: bash
 
     $ bzr whoami
     Joe Coder <[e-mail]>
 
 This should display your name and email. Next, you will want to create an
 account on the `Launchpad website <http://www.launchpad.net>`_ and setup your
 ssh keys. For more information of setting up your ssh keys, see `this link
 <https://help.launchpad.net/YourAccount/CreatingAnSSHKeyPair>`_.
 
 Get the main IPython branch from Launchpad
 ------------------------------------------
 
 Now, you can get a copy of the main IPython development branch (we call this
-the "trunk")::
+the "trunk"):
+
+.. code-block:: bash
 
     $ bzr branch lp:ipython
 
 Create a working branch
 -----------------------
 
 When working on IPython, you won't actually make edits directly to the
 :file:`lp:ipython` branch. Instead, you will create a separate branch for your
 changes. For now, let's assume you want to do your work in a branch named
-"ipython-mybranch". Create this branch by doing::
+"ipython-mybranch". Create this branch by doing:
+
+.. code-block:: bash
 
     $ bzr branch ipython ipython-mybranch
 
 When you actually create a branch, you will want to give it a name that
 reflects the nature of the work that you will be doing in it, like
 "install-docs-update".
 
 Make edits in your working branch
 ---------------------------------
 
 Now you are ready to actually make edits in your :file:`ipython-mybranch`
 branch. Before doing this, it is helpful to install this branch so you can
 test your changes as you work. This is easiest if you have setuptools
-installed. Then, just do::
+installed. Then, just do:
+
+.. code-block:: bash
 
     $ cd ipython-mybranch
     $ python setupegg.py develop
 
 Now, make some changes. After a while, you will want to commit your changes.
 This let's Bazaar know that you like the changes you have made and gives you
 an opportunity to keep a nice record of what you have done. This looks like
-this::
+this:
+
+.. code-block:: bash
 
     $ ...do work in ipython-mybranch...
     $ bzr commit -m "the commit message goes here"
 
 Please note that since we now don't use an old-style linear ChangeLog (that
 tends to cause problems with distributed version control systems), you should
 ensure that your log messages are reasonably detailed.  Use a docstring-like
 approach in the commit messages (including the second line being left
 *blank*)::
 
   Single line summary of  changes being committed.
 
   * more details when warranted ...
   * including crediting outside contributors if they sent the
     code/bug/idea!
 
 As you work, you will repeat this edit/commit cycle many times. If you work on
 your branch for a long time, you will also want to get the latest changes from
 the :file:`lp:ipython` branch. This can be done with the following sequence of
-commands::
+commands:
+
+.. code-block:: bash
 
     $ ls
     ipython
     ipython-mybranch
     
     $ cd ipython
     $ bzr pull
     $ cd ../ipython-mybranch
     $ bzr merge ../ipython
     $ bzr commit -m "Merging changes from trunk"
 
-Along the way, you should also run the IPython test suite.  You can do this
-using the :command:`iptest` command (which is basically a customized version of
-:command:`nosetests`)::
-
-    $ cd
-    $ iptest
-
-The :command:`iptest` command will also pick up and run any tests you have
-written. See :ref:`testing documentation <devel_testing>` for further details
-on the testing system.
-
-
 Post your branch and request a code review
 ------------------------------------------
 
 Once you are done with your edits, you should post your branch on Launchpad so
 that other IPython developers can review the changes and help you merge your
 changes into the main development branch. To post your branch on Launchpad,
-do::
+do:
+
+.. code-block:: bash
 
     $ cd ipython-mybranch
     $ bzr push lp:~yourusername/ipython/ipython-mybranch
 
-Then, go to the `IPython Launchpad site <www.launchpad.net/ipython>`_, and you
-should see your branch under the "Code" tab. If you click on your branch, you
-can provide a short description of the branch as well as mark its status. Most
-importantly, you should click the link that reads "Propose for merging into
-another branch". What does this do?
+Then, go to the `IPython Launchpad site <http://www.launchpad.net/ipython>`_,
+and you should see your branch under the "Code" tab. If you click on your
+branch, you can provide a short description of the branch as well as mark its
+status. Most importantly, you should click the link that reads "Propose for
+merging into another branch". What does this do?
 
 This let's the other IPython developers know that your branch is ready to be
 reviewed and merged into the main development branch. During this review
 process, other developers will give you feedback and help you get your code
 ready to be merged. What types of things will we be looking for:
 
-* All code is documented.
-* All code has tests.
+* All code is documented.  How to document your code is described in 
+  :ref:`this section <documenting-ipython>`.
+* All code has tests.  How to write and run tests is described in 
+  :ref:`this section <testing>`.
 * The entire IPython test suite passes.
 
+You should also provide us with a list of changes that your branch contains.
+See the :ref:`What's new <whatsnew_index>` section of our documentation
+(:file:`docs/source/whatsnew`) for details on the format and content of this.
+
 Once your changes have been reviewed and approved, someone will merge them
 into the main development branch.
 
 
-Some notes for core developers when merging third-party contributions
-=====================================================================
+Merging a branch into trunk
+===========================
 
 Core developers, who ultimately merge any approved branch (from themselves,
 another developer, or any third-party contribution) will typically use
 :command:`bzr merge` to merge the branch into the trunk and push it to the
-main Launcphad site. This is a short list of things to keep in mind when doing
-this process, so that the project history is easy to understand in the long
+main Launchpad site. There are a number of things to keep in mind when doing
+this, so that the project history is easy to understand in the long
 run, and that generating release notes is as painless and accurate as
 possible.
 
-- When you merge any non-trivial functionality (from one small bug fix to a
-  big feature branch), please remember to always edit the :file:`changes.txt`
-  file accordingly. This file has one main section for each release, and if
-  you edit it as you go, noting what new features, bug fixes or API changes
-  you have made, the release notes will be almost finished when they are
-  needed later. This is much easier if done when you merge the work, rather
-  than weeks or months later by re-reading a massive Bazaar log.
-
-- When big merges are done, the practice of putting a summary commit message
-  in the merge is *extremely* useful. It makes this kind of job much nicer,
-  because that summary log message can be almost copy/pasted without changes,
-  if it was well written, rather than dissecting the next-level messages from
-  the individual commits.
-
-- It's important that we remember to always credit who gave us something if
+* When you merge any non-trivial functionality (from one small bug fix to a
+  big feature branch), please remember to always edit the appropriate file in
+  the :ref:`What's new <whatsnew_index>` section of our documentation.
+  Ideally, the author of the branch should provide this content when they
+  submit the branch for review. But if they don't it is the responsibility of
+  the developer doing the merge to add this information.
+
+* When merges are done, the practice of putting a summary commit message in
+  the merge is *extremely* useful. It is probably easiest if you simply use
+  the same list of changes that were added to the :ref:`What's new
+  <whatsnew_index>` section of the documentation.
+
+* It's important that we remember to always credit who gave us something if
   it's not the committer. In general, we have been fairly good on this front,
   this is just a reminder to keep things up. As a note, if you are ever
   committing something that is completely (or almost so) a third-party
   contribution, do the commit as::
 
     $ bzr commit --author="Someone Else"
 
   This way it will show that name separately in the log, which makes it even
   easier to spot. Obviously we often rework third party contributions
   extensively, but this is still good to keep in mind for cases when we don't
   touch the code too much.
+
+
+.. [Bazaar] Bazaar. http://bazaar-vcs.org/
+.. [Launchpad] Launchpad. http://www.launchpad.net/ipython

docs/source/development/doc_guide.txt

@@ -1,141 +1,104 b''
 .. _documenting-ipython:
 
 =====================
  Documenting IPython
 =====================
 
 Standalone documentation
 ========================
 
 All standalone documentation should be written in plain text (``.txt``) files
-using `reStructuredText`_ for markup and formatting. All such documentation
-should be placed in the top level directory ``docs`` of the IPython source
-tree. Or, when appropriate, a suitably named subdirectory should be used. The
-documentation in this location will serve as the main source for IPython
-documentation and all existing documentation should be converted to this
-format.
-
-The actual HTML and PDF docs are built using the Sphinx_ documentation
-generation tool.  Sphinx has been adopted as the default documentation tool for
-Python itself as of version 2.6, as well as by a number of projects that
-IPython is related with, such as numpy, scipy, matplotlib, sage and nipy.
+using reStructuredText [reStructuredText]_ for markup and formatting. All such
+documentation should be placed in the directory :file:`docs/source` of the
+IPython source tree. Or, when appropriate, a suitably named subdirectory
+should be used. The documentation in this location will serve as the main
+source for IPython documentation.
 
-.. _reStructuredText: http://docutils.sourceforge.net/rst.html
-.. _Sphinx: http://sphinx.pocoo.org/
+The actual HTML and PDF docs are built using the Sphinx [Sphinx]_
+documentation generation tool. Once you have Sphinx installed, you can build
+the html docs yourself by doing:
 
+.. code-block:: bash
 
-The rest of this document is mostly taken from the `matploblib
-documentation`__; we are using a number of Sphinx tools and extensions written
-by the matplotlib team and will mostly follow their conventions, which are
-nicely spelled out in their guide.  What follows is thus a lightly adapted
-version of the matplotlib documentation guide, taken with permission from the
-MPL team.
+    $ cd ipython-mybranch/docs
+    $ make html
 
-.. __: http://matplotlib.sourceforge.net/devel/documenting_mpl.html
+Our usage of Sphinx follows that of matplotlib [Matplotlib]_ closely. We are
+using a number of Sphinx tools and extensions written by the matplotlib team
+and will mostly follow their conventions, which are nicely spelled out in
+their documentation guide [MatplotlibDocGuide]_. What follows is thus a
+abridged version of the matplotlib documentation guide, taken with permission
+from the matplotlib team.
 
+If you are reading this in a web browser, you can click on the "Show Source"
+link to see the original reStricturedText for the following examples.
 
 A bit of Python code::
 
     for i in range(10):
         print i,
     print "A big number:",2**34
 
 An interactive Python session::
 
     >>> from IPython import genutils
     >>> genutils.get_ipython_dir()
     '/home/fperez/.ipython'
 
-
 An IPython session:
 
 .. code-block:: ipython
 
   In [7]: import IPython
 
   In [8]: print "This IPython is version:",IPython.__version__
   This IPython is version: 0.9.1
 
   In [9]: 2+4
   Out[9]: 6
 
 
 A bit of shell code:
 
 .. code-block:: bash
 
     cd /tmp
     echo "My home directory is: $HOME"
     ls
 
-  
 Docstring format
 ================
 
 Good docstrings are very important.  Unfortunately, Python itself only provides
-a rather loose standard for docstrings (`PEP 257`_), and there is no universally
+a rather loose standard for docstrings [PEP257]_, and there is no universally
 accepted convention for all the different parts of a complete docstring.
 However, the NumPy project has established a very reasonable standard, and has
 developed some tools to support the smooth inclusion of such docstrings in
 Sphinx-generated manuals.  Rather than inventing yet another pseudo-standard,
 IPython will be henceforth documented using the NumPy conventions; we carry
 copies of some of the NumPy support tools to remain self-contained, but share
 back upstream with NumPy any improvements or fixes we may make to the tools.
 
-The `NumPy documentation guidelines`_ contain detailed information on this
-standard, and for a quick overview, the NumPy `example docstring`_ is a useful
-read. 
+The NumPy documentation guidelines [NumPyDocGuide]_ contain detailed
+information on this standard, and for a quick overview, the NumPy example
+docstring [NumPyExampleDocstring]_ is a useful read.
 
-As in the past IPython used epydoc, currently many docstrings still use epydoc
-conventions.  We will update them as we go, but all new code should be fully
+In the past IPython used epydoc so currently many docstrings still use epydoc
+conventions.  We will update them as we go, but all new code should be
 documented using the NumPy standard.
 
-.. _PEP 257: http://www.python.org/peps/pep-0257.html
-.. _NumPy documentation guidelines: http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines
-
-.. _example docstring: http://projects.scipy.org/numpy/browser/trunk/doc/EXAMPLE_DOCSTRING.txt
-
-Additional PEPs of interest regarding documentation of code.  While both of
-these were rejected, the ideas therein form much of the basis of docutils (the
-machinery to process reStructuredText):
-
-- `Docstring Processing System Framework <http://www.python.org/peps/pep-0256.html>`_
-- `Docutils Design Specification <http://www.python.org/peps/pep-0258.html>`_
-
-Older material
-==============
-
-Documentation
-=============
-
-Standalone documentation
-------------------------
+Here are two additional PEPs of interest regarding documentation of code.
+While both of these were rejected, the ideas therein form much of the basis of
+docutils (the machinery to process reStructuredText):
 
-All standalone documentation should be written in plain text (``.txt``) files
-using reStructuredText [reStructuredText]_ for markup and formatting. All such
-documentation should be placed in directory :file:`docs/source` of the IPython
-source tree. The documentation in this location will serve as the main source
-for IPython documentation and all existing documentation should be converted
-to this format.
-
-To build the final documentation, we use Sphinx [Sphinx]_.  Once you have
-Sphinx installed, you can build the html docs yourself by doing::
-
-    $ cd ipython-mybranch/docs
-    $ make html
-
-Docstring format
-----------------
-
-Good docstrings are very important. All new code should have docstrings that
-are formatted using reStructuredText for markup and formatting, since it is
-understood by a wide variety of tools. Details about using reStructuredText
-for docstrings can be found `here
-<http://epydoc.sourceforge.net/manual-othermarkup.html>`_.
-
-Additional PEPs of interest regarding documentation of code:
-
-* `Docstring Conventions <http://www.python.org/peps/pep-0257.html>`_
 * `Docstring Processing System Framework <http://www.python.org/peps/pep-0256.html>`_
 * `Docutils Design Specification <http://www.python.org/peps/pep-0258.html>`_
 
+
+.. [reStructuredText] reStructuredText.  http://docutils.sourceforge.net/rst.html
+.. [Sphinx] Sphinx. http://sphinx.pocoo.org/
+.. [MatplotlibDocGuide] http://matplotlib.sourceforge.net/devel/documenting_mpl.html
+.. [PEP257] PEP 257.  http://www.python.org/peps/pep-0257.html
+.. [NumPyDocGuide] NumPy documentation guide.  http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines
+.. [NumPyExampleDocstring] NumPy example docstring. http://projects.scipy.org/numpy/browser/trunk/doc/EXAMPLE_DOCSTRING.txt
+

docs/source/development/index.txt

@@ -1,25 +1,18 b''
 .. _developer_guide:
 
 =========================
 IPython developer's guide
 =========================
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    contributing.txt
    coding_guide.txt
    doc_guide.txt
    testing.txt
    release.txt
    roadmap.txt
-
-   notification_blueprint.txt
    reorg.txt
+   notification_blueprint.txt
 
-
-.. [Bazaar] Bazaar. http://bazaar-vcs.org/
-.. [Launchpad] Launchpad. http://www.launchpad.net/ipython
-.. [reStructuredText] reStructuredText.  http://docutils.sourceforge.net/rst.html
-.. [Sphinx] Sphinx. http://sphinx.pocoo.org/
-.. [Nose] Nose: a discovery based unittest extension. http://code.google.com/p/python-nose/
  No newline at end of file

docs/source/development/release.txt

@@ -1,25 +1,32 b''
-Release checklist
+.. _releasing_ipython:
+
+=================
+Releasing IPython
 =================
 
+This section contains notes about the process that is used to release IPython.
+Our release process is currently not very formal and could be improved.
+
 Most of the release process is automated by the :file:`release` script in the
 :file:`tools` directory.  This is just a handy reminder for the release manager.
 
 #. First, run :file:`build_release`, which does all the file checking and
    building that the real release script will do.  This will let you do test
    installations, check that the build procedure runs OK, etc.  You may want to
    disable a few things like multi-version RPM building while testing, because
    otherwise the build takes really long.
 
 #. Run the release script, which makes the tar.gz, eggs and Win32 .exe
    installer.  It posts them to the site and registers the release with PyPI.
 
-#. Updating the website with announcements and links to the updated
-   changes.txt in html form. Remember to put a short note both on the news
-   page of the site and on Launcphad.
+#. Update the website with announcements and links to the updated changes.txt
+   in html form. Remember to put a short note both on the news page of the 
+   site and on Launcphad.
 
 #. Drafting a short release announcement with i) highlights and ii) a link to
-   the html changes.txt.
+   the html version of the :ref:`Whats new <whatsnew_index>` section of the 
+   documentation.
 
 #. Make sure that the released version of the docs is live on the site.
 
 #. Celebrate!
  No newline at end of file

docs/source/development/reorg.txt

@@ -1,85 +1,64 b''
-=============================
-IPython module reorganization
-=============================
+.. _module_reorg:
 
-Currently, IPython has many top-level modules that serve many different
-purposes. The lack of organization make it very difficult for developers to
-work on IPython and understand its design. This document contains notes about
-how we will reorganize the modules into sub-packages.
+===========================
+IPython module organization
+===========================
 
-.. warning::
+As of the 0.11 release of IPython, the top-level packages and modules have
+been completely reorganized.  This section describes the purpose of the
+top-level IPython subpackages.
 
-    This effort will possibly break third party packages that use IPython as
-    a library or hack on the IPython internals.
+Subpackage descriptions
+=======================
 
-.. warning::
+* :mod:`IPython.config`.  This package contains the configuration system of 
+  IPython, as well as default configuration files for the different IPython
+  applications.
 
-    This effort will result in the removal from IPython of certain modules
-    that are not used anymore, don't currently work, are unmaintained, etc.
-
-
-Current subpackges
-==================
-
-IPython currently has the following sub-packages:
-
-* :mod:`IPython.config`
-
-* :mod:`IPython.Extensions`
-
-* :mod:`IPython.external`
-
-* :mod:`IPython.frontend`
-
-* :mod:`IPython.gui`
-
-* :mod:`IPython.kernel`
+* :mod:`IPython.core`. This sub-package contains the core of the IPython
+  interpreter, but none of its extended capabilities.
 
-* :mod:`IPython.testing`
+* :mod:`IPython.deathrow`. This is for code that is outdated, untested,
+  rotting, or that belongs in a separate third party project. Eventually all
+  this code will either i) be revived by someone willing to maintain it with
+  tests and docs and re-included into IPython or 2) be removed from IPython
+  proper, but put into a separate third-party Python package. No new code will
+  be allowed here.
 
-* :mod:`IPython.tests`
+* :mod:`IPython.extensions`.  This package contains fully supported IPython
+  extensions.  These extensions adhere to the official IPython extension API
+  and can be enabled by adding them to a field in the configuration file.
 
-* :mod:`IPython.tools`
+* :mod:`IPython.external`. This package contains third party packages and
+  modules that IPython ships internally to reduce the number of dependencies.
+  Usually, these are short, single file modules.
 
-* :mod:`IPython.UserConfig`
+* :mod:`IPython.frontend`. This package contains the various IPython
+  frontends. Currently, the code in this subpackage is very experimental and
+  may be broken.
 
-New Subpackages to be created
-=============================
+* :mod:`IPython.gui`.  Another semi-experimental wxPython based IPython GUI.
 
-We propose to create the following new sub-packages:
-
-* :mod:`IPython.core`. This sub-package will contain the core of the IPython
-  interpreter, but none of its extended capabilities.
+* :mod:`IPython.kernel`.  This contains IPython's parallel computing system.
 
 * :mod:`IPython.lib`. IPython has many extended capabilities that are not part
-  of the IPython core. These things will go here.
-
-* :mod:`IPython.utils`. This sub-package will contain anything that might
-  eventually be found in the Python standard library, like things in
-  :mod:`genutils`. Each sub-module in this sub-package should contain
-  functions and classes that serve a single purpose.
-
-* :mod:`IPython.deathrow`. This is for code that is untested and/or rotting
-  and needs to be removed from IPython. Eventually all this code will either
-  i) be revived by someone willing to maintain it with tests and docs and
-  re-included into IPython or 2) be removed from IPython proper, but put into
-  a separate top-level (not IPython) package that we keep around. No new code
-  will be allowed here.
+  of the IPython core. These things will go here and in.  Modules in this
+  package are similar to extensions, but don't adhere to the official
+  IPython extension API.
 
 * :mod:`IPython.quarantine`.  This is for code that doesn't meet IPython's 
   standards, but that we plan on keeping.  To be moved out of this sub-package
-  a module needs to have a maintainer, tests and documentation.
-
-Procedure
-=========
+  a module needs to have approval of the core IPython developers, tests and
+  documentation.
 
-1. Move the file to its new location with its new name.
-2. Rename all import statements to reflect the change.
-3. Run PyFlakes on each changes module.
-4. Add tests/test_imports.py to test it.
+* :mod:`IPython.scripts`.  This package contains a variety of top-level
+  command line scripts.  Eventually, these should be moved to the 
+  :file:`scripts` subdirectory of the appropriate IPython subpackage.
 
-Status
-======
+* :mod:`IPython.utils`. This sub-package will contain anything that might
+  eventually be found in the Python standard library, like things in
+  :mod:`genutils`. Each sub-module in this sub-package should contain
+  functions and classes that serve a single purpose and that don't
+  depend on things in the rest of IPython.
 
-This branch was merged into trunk in early August of 2009.
 

docs/source/development/roadmap.txt

@@ -1,114 +1,121 b''
 .. _roadmap:
 
 ===================
 Development roadmap
 ===================
 
 IPython is an ambitious project that is still under heavy development.
 However, we want IPython to become useful to as many people as possible, as
 quickly as possible. To help us accomplish this, we are laying out a roadmap
 of where we are headed and what needs to happen to get there. Hopefully, this
 will help the IPython developers figure out the best things to work on for
 each upcoming release.
 
 Work targeted to particular releases
 ====================================
 
 Release 0.11
 ------------
 
 * Full module and package reorganization (done).
 
 * Removal of the threaded shells and new implementation of GUI support
   based on ``PyOSInputHook`` (done).
 
 * Refactor the configuration system (done).
 
 * Prepare to refactor IPython's core by creating a new component and 
   application system (done).
 
 * Start to refactor IPython's core by turning everything into components
   (started).
 
 Release 0.12
 ------------
 
 * Continue to refactor IPython's core by turning everything into components.
 
 
 Major areas of work
 ===================
 
 Refactoring the main IPython core
 ---------------------------------
 
 During the summer of 2009, we began refactoring IPython's core.  The main
-thrust in this work was make the IPython core into a set of loosely coupled
+thrust in this work was to make the IPython core into a set of loosely coupled
 components.  The base component class for this is 
-:class:`IPython.core.component.Component`.  This section outlines the status
-of this work.
+:class:`IPython.core.component.Component`. This section outlines the status of
+this work.
 
 Parts of the IPython core that have been turned into components:
 
-* The main :class:`InteractiveShell` class.
-* The aliases (:mod:`IPython.core.aliases`).
+* The main :class:`~IPython.core.iplib.InteractiveShell` class.
+* The aliases (:mod:`IPython.core.alias`).
 * The display and builtin traps (:mod:`IPython.core.display_trap` and
   :mod:`IPython.core.builtin_trap`).
 * The prefilter machinery (:mod:`IPython.core.prefilter`).
 
-Parts of the IPythoncore that need to be turned into components:
+Parts of the IPythoncore that still need to be turned into components:
 
 * Magics.
 * Input and output history management.
 * Prompts.
-* Completers.
+* Tab completers.
 * Logging.
 * Exception handling.
 * Anything else.
 
 Process management for :mod:`IPython.kernel`
 --------------------------------------------
 
+A number of things need to be done to improve how processes are started
+up and managed for the parallel computing side of IPython:
+
+* All of the processes need to use the new configuration system, components
+  and application.
+* We need to add support for other batch systems.
+
 Performance problems
 --------------------
 
 Currently, we have a number of performance issues in :mod:`IPython.kernel`:
 
 * The controller stores a large amount of state in Python dictionaries. Under
   heavy usage, these dicts with get very large, causing memory usage problems.
   We need to develop more scalable solutions to this problem. This will also
   help the controller to be more fault tolerant.
 
 * We currently don't have a good way of handling large objects in the
   controller. The biggest problem is that because we don't have any way of
   streaming objects, we get lots of temporary copies in the low-level buffers.
   We need to implement a better serialization approach and true streaming
   support.
 
 * The controller currently unpickles and repickles objects. We need to use the
   [push|pull]_serialized methods instead.
 
 * Currently the controller is a bottleneck. The best approach for this is to
   separate the controller itself into multiple processes, one for the core
   controller and one each for the controller interfaces.
 
 Porting to 3.0
 ==============
 
 There are no definite plans for porting of IPython to Python 3. The major 
 issue is the dependency on Twisted framework for the networking/threading 
 stuff. It is possible that it the traditional IPython interactive console 
 could be ported more easily since it has no such dependency. Here are a few
 things that will need to be considered when doing such a port especially
 if we want to have a codebase that works directly on both 2.x and 3.x.
 
 1. The syntax for exceptions changed (PEP 3110). The old `except exc, var`
 changed to `except exc as var`. At last count there was 78 occurrences of this
 usage in the code base. This is a particularly problematic issue, because it's
 not easy to implement it in a 2.5-compatible way.
 
 Because it is quite difficult to support simultaneously Python 2.5 and 3.x, we
 will likely at some point put out a release that requires strictly 2.6 and
 abandons 2.5 compatibility.  This will then allow us to port the code to using
 :func:`print` as a function, `except exc as var` syntax, etc.  But as of
 version 0.11 at least, we will retain Python 2.5 compatibility.

docs/source/development/testing.txt

@@ -1,194 +1,54 b''
 .. _testing:
 
 =========================
 Writing and running tests
 =========================
 
 Overview
 ========
 
 It is extremely important that all code contributed to IPython has tests.
 Tests should be written as unittests, doctests or other entities that the
 IPython test system can detect.  See below for more details on this.
 
 Each subpackage in IPython should have its own :file:`tests` directory that
 contains all of the tests for that subpackage. All of the files in the
 :file:`tests` directory should have the word "tests" in them to enable
 the testing framework to find them.  
 
 If a subpackage has any dependencies beyond the Python standard library, the
 tests for that subpackage should be skipped if the dependencies are not found.
 This is very important so users don't get tests failing simply because they
 don't have dependencies.  We are still figuring out the best way for this
 to be handled.
 
 Status
 ======
 
 Currently IPython's testing system is being reworked.  In the meantime,
 we recommend the following testing practices:
 
-* To run regular tests, use the :cmd:`nosetests` command on a per file basis:
+* To run regular tests, use the :command:`nosetests` command that Nose [Nose]_ 
+  provides on a per file basis:
 
 .. code-block:: bash
 
     nosetests -vvs IPython.core.tests.test_component
 
-* To run Twisted-using tests, use the :cmd:`trial` command on a per file 
+* To run Twisted-using tests, use the :command:`trial` command on a per file 
   basis:
 
 .. code-block:: bash
 
     trial IPython.kernel
 
 * For now, regular tests (of non-Twisted using code) should be written as
   unit tests.  They should be subclasses of :class:`unittest.TestCase`.
 
 * Tests of Twisted [Twisted]_ using code should be written by subclassing the
-  ``TestCase`` class that comes with ``twisted.trial.unittest``.
-
-.. _devel_testing:
-
-Older material
-==============
-
-It is extremely important that all code contributed to IPython has tests.
-Tests should be written as unittests, doctests or as entities that the Nose
-[Nose]_ testing package will find. Regardless of how the tests are written, we
-will use Nose for discovering and running the tests. Nose will be required to
-run the IPython test suite, but will not be required to simply use IPython.
-
-Tests of Twisted using code need to follow two additional guidelines:
-
-1. Twisted using tests should be written by subclassing the :class:`TestCase`
-   class that comes with :mod:`twisted.trial.unittest`. 
-
-2. All :class:`Deferred` instances that are created in the test must be
+  ``TestCase`` class that comes with ``twisted.trial.unittest``. Furthermore,
+  all :class:`Deferred` instances that are created in the test must be
    properly chained and the final one *must* be the return value of the test
    method.
 
-When these two things are done, Nose will be able to run the tests and the
-twisted reactor will be handled correctly.
-
-Each subpackage in IPython should have its own :file:`tests` directory that
-contains all of the tests for that subpackage. This allows each subpackage to
-be self-contained.  A good convention to follow is to have a file named
-:file:`test_foo.py` for each module :file:`foo.py` in the package.  This makes
-it easy to organize the tests, though like most conventions, it's OK to break
-it if logic and common sense dictate otherwise.
-
-If a subpackage has any dependencies beyond the Python standard library, the
-tests for that subpackage should be skipped if the dependencies are not
-found. This is very important so users don't get tests failing simply because
-they don't have dependencies.  We ship a set of decorators in the
-:mod:`IPython.testing` package to tag tests that may be platform-specific or
-otherwise may have restrictions; if the existing ones don't fit your needs, add
-a new decorator in that location so other tests can reuse it.
-
-To run the IPython test suite, use the :command:`iptest` command that is
-installed with IPython (if you are using IPython in-place, without installing
-it, you can find this script in the :file:`scripts` directory)::
-
-    $ iptest
-
-This command colects all IPython tests into separate groups, and then calls
-either Nose with the proper options and extensions, or Twisted's
-:command:`trial`.  This ensures that tests that need the Twisted reactor
-management facilities execute separate of Nose.  If any individual test group
-fails, :command:`iptest` will print what you need to type so you can rerun that
-particular test group alone for debugging.
-
-By default, :command:`iptest` runs the entire IPython test
-suite (skipping tests that may be platform-specific or which depend on tools
-you may not have).  But you can also use it to run only one specific test file,
-or a specific test function.  For example, this will run only the
-:file:`test_magic` file from the test suite::
-
-    $ iptest IPython.tests.test_magic
-    ----------------------------------------------------------------------
-    Ran 10 tests in 0.348s
-
-    OK (SKIP=3)
-    Deleting object: second_pass
-
-while the ``path:function`` syntax allows you to select a specific function in
-that file to run::
-
-    $ iptest IPython.tests.test_magic:test_obj_del
-    ----------------------------------------------------------------------
-    Ran 1 test in 0.204s
-
-    OK
-
-Since :command:`iptest` is based on nosetests, you can pass it any regular
-nosetests option.  For example, you can use ``--pdb`` or ``--pdb-failures`` to
-automatically activate the interactive Pdb debugger on errors or failures.  See
-the nosetests documentation for further details.
-
-
-A few tips for writing tests
-----------------------------
-
-You can write tests either as normal test files, using all the conventions that
-Nose recognizes, or as doctests.  Note that *all* IPython functions should have
-at least one example that serves as a doctest, whenever technically feasible.
-However, example doctests should only be in the main docstring if they are *a
-good example*, i.e. if they convey useful information about the function.  If
-you simply would like to write a test as a doctest, put it in a separate test
-file and write a no-op function whose only purpose is its docstring.
-
-Note, however, that in a file named :file:`test_X`, functions whose only test
-is their docstring (as a doctest) and which have no test functionality of their
-own, should be called *doctest_foo* instead of *test_foo*, otherwise they get
-double-counted (the empty function call is counted as a test, which just
-inflates tests numbers artificially).  This restriction does not apply to
-functions in files with other names, due to how Nose discovers tests.
-
-You can use IPython examples in your docstrings.  Those can make full use of
-IPython functionality (magics, variable substitution, etc), but be careful to
-keep them generic enough that they run identically on all Operating Systems.
-
-The prompts in your doctests can be either of the plain Python ``>>>`` variety
-or ``In [1]:`` IPython style.  Since this is the IPython system, after all, we
-encourage you to use IPython prompts throughout, unless you are illustrating a
-specific aspect of the normal prompts (such as the ``%doctest_mode`` magic).
-
-If a test isn't safe to run inside the main nose process (e.g. because it loads
-a GUI toolkit), consider running it in a subprocess and capturing its output
-for evaluation and test decision later.  Here is an example of how to do it, by
-relying on the builtin ``_ip`` object that contains the public IPython api as
-defined in :mod:`IPython.ipapi`::
-
-   def test_obj_del():
-       """Test that object's __del__ methods are called on exit."""
-       test_dir = os.path.dirname(__file__)
-       del_file = os.path.join(test_dir,'obj_del.py')
-       out = _ip.IP.getoutput('ipython %s' % del_file)
-       nt.assert_equals(out,'object A deleted')
-
-
-
-If a doctest contains input whose output you don't want to verify identically
-via doctest (random output, an object id, etc), you can mark a docstring with
-``#random``.  All of these test will have their code executed but no output
-checking will be done::
-
-       >>> 1+3
-       junk goes here...  # random
-
-       >>> 1+2
-       again,  anything goes #random
-       if multiline, the random mark is only needed once.
-
-       >>> 1+2
-       You can also put the random marker at the end:
-       # random
-
-       >>> 1+2
-       # random
-       .. or at the beginning.
-
-In a case where you want an *entire* docstring to be executed but not verified
-(this only serves to check that the code runs without crashing, so it should be
-used very sparingly), you can put ``# all-random`` in the docstring.
-
+.. [Nose] Nose: a discovery based unittest extension. http://code.google.com/p/python-nose/

docs/source/install/install.txt

@@ -1,328 +1,341 b''
 Overview
 ========
 
 This document describes the steps required to install IPython.  IPython is
 organized into a number of subpackages, each of which has its own dependencies.
 All of the subpackages come with IPython, so you don't need to download and
 install them separately.  However, to use a given subpackage, you will need to
 install all of its dependencies.
 
 
 Please let us know if you have problems installing IPython or any of its
 dependencies. Officially, IPython requires Python version 2.5 or 2.6.  We
 have *not* yet started to port IPython to Python 3.0.
 
 .. warning::
 
     Officially, IPython supports Python versions 2.5 and 2.6.
 
     IPython 0.10 has only been well tested with Python 2.5 and 2.6.  Parts of
     it may work with Python 2.4, but we do not officially support Python 2.4
     anymore.  If you need to use 2.4, you can still run IPython 0.9.
 
 Some of the installation approaches use the :mod:`setuptools` package and its
 :command:`easy_install` command line program.  In many scenarios, this provides
 the most simple method of installing IPython and its dependencies.  It is not
 required though.  More information about :mod:`setuptools` can be found on its
 website.
 
 More general information about installing Python packages can be found in
 Python's documentation at http://www.python.org/doc/.
 
 Quickstart
 ==========
 
 If you have :mod:`setuptools` installed and you are on OS X or Linux (not
 Windows), the following will download and install IPython *and* the main
 optional dependencies:
 
 .. code-block:: bash
 
-    easy_install ipython[kernel,security,test]
+    $ easy_install ipython[kernel,security,test]
 
 This will get Twisted, zope.interface and Foolscap, which are needed for
 IPython's parallel computing features as well as the nose package, which will
-enable you to run IPython's test suite.  To run IPython's test suite, use the
-:command:`iptest` command:
+enable you to run IPython's test suite.  
+
+.. warning::
+
+    IPython's test system is being refactored and currently the
+    :command:`iptest` shown below does not work. More details about the
+    testing situation can be found :ref:`here <testing>`
+
+To run IPython's test suite, use the :command:`iptest` command:
 
 .. code-block:: bash
 
-    iptest
+    $ iptest
 
 Read on for more specific details and instructions for Windows.
 
 Installing IPython itself
 =========================
 
 Given a properly built Python, the basic interactive IPython shell will work
 with no external dependencies.  However, some Python distributions
 (particularly on Windows and OS X), don't come with a working :mod:`readline`
 module.  The IPython shell will work without :mod:`readline`, but will lack
 many features that users depend on, such as tab completion and command line
 editing.  See below for details of how to make sure you have a working
 :mod:`readline`.
 
 Installation using easy_install
 -------------------------------
 
 If you have :mod:`setuptools` installed, the easiest way of getting IPython is
 to simple use :command:`easy_install`:
 
 .. code-block:: bash
 
-    easy_install ipython
+    $ easy_install ipython
 
 That's it.
 
 Installation from source
 ------------------------
 
 If you don't want to use :command:`easy_install`, or don't have it installed,
 just grab the latest stable build of IPython from `here
 <http://ipython.scipy.org/dist/>`_.  Then do the following:
 
 .. code-block:: bash
 
-    tar -xzf ipython.tar.gz
-    cd ipython
-    python setup.py install
+    $ tar -xzf ipython.tar.gz
+    $ cd ipython
+    $ python setup.py install
 
 If you are installing to a location (like ``/usr/local``) that requires higher
 permissions, you may need to run the last command with :command:`sudo`.
 
 Windows
 -------
 
 There are a few caveats for Windows users.  The main issue is that a basic
 ``python setup.py install`` approach won't create ``.bat`` file or Start Menu
 shortcuts, which most users want.  To get an installation with these, you can
 use any of the following alternatives:
 
 1. Install using :command:`easy_install`.
 
 2. Install using our binary ``.exe`` Windows installer, which can be found at
    `here <http://ipython.scipy.org/dist/>`_
 
 3. Install from source, but using :mod:`setuptools` (``python setupegg.py
    install``).
 
 IPython by default runs in a terminal window, but the normal terminal
 application supplied by Microsoft Windows is very primitive.  You may want to
 download the excellent and free Console_ application instead, which is a far
 superior tool.  You can even configure Console to give you by default an
 IPython tab, which is very convenient to create new IPython sessions directly
 from the working terminal.
 
 .. _Console:  http://sourceforge.net/projects/console
 
    
 Installing the development version
 ----------------------------------
 
 It is also possible to install the development version of IPython from our
 `Bazaar <http://bazaar-vcs.org/>`_ source code repository.  To do this you will
 need to have Bazaar installed on your system.  Then just do:
 
 .. code-block:: bash
 
-    bzr branch lp:ipython
-    cd ipython
-    python setup.py install
+    $ bzr branch lp:ipython
+    $ cd ipython
+    $ python setup.py install
 
 Again, this last step on Windows won't create ``.bat`` files or Start Menu
 shortcuts, so you will have to use one of the other approaches listed above.
 
 Some users want to be able to follow the development branch as it changes.  If
 you have :mod:`setuptools` installed, this is easy. Simply replace the last
 step by:
 
 .. code-block:: bash
 
-    python setupegg.py develop
+    $ python setupegg.py develop
 
 This creates links in the right places and installs the command line script to
 the appropriate places.  Then, if you want to update your IPython at any time,
 just do:
 
 .. code-block:: bash
 
-    bzr pull
+    $ bzr pull
 
 Basic optional dependencies
 ===========================
 
 There are a number of basic optional dependencies that most users will want to
 get.  These are:
 
 * readline (for command line editing, tab completion, etc.)
 * nose (to run the IPython test suite)
 * pexpect (to use things like irunner)
 
 If you are comfortable installing these things yourself, have at it, otherwise
 read on for more details.
 
 readline
 --------
 
 In principle, all Python distributions should come with a working
 :mod:`readline` module.  But, reality is not quite that simple.  There are two
 common situations where you won't have a working :mod:`readline` module:
 
 * If you are using the built-in Python on Mac OS X.
 
 * If you are running Windows, which doesn't have a :mod:`readline` module.
 
 On OS X, the built-in Python doesn't not have :mod:`readline` because of
 license issues.  Starting with OS X 10.5 (Leopard), Apple's built-in Python has
 a BSD-licensed not-quite-compatible readline replacement. As of IPython 0.9,
 many of the issues related to the differences between readline and libedit have
 been resolved.  For many users, libedit may be sufficient.
 
 Most users on OS X will want to get the full :mod:`readline` module.  To get a
 working :mod:`readline` module, just do (with :mod:`setuptools` installed):
 
 .. code-block:: bash
 
-    easy_install readline
+    $ easy_install readline
 
-.. note:
+.. note::
 
     Other Python distributions on OS X (such as fink, MacPorts and the 
     official python.org binaries) already have readline installed so 
     you don't have to do this step.
 
 If needed, the readline egg can be build and installed from source (see the
 wiki page at http://ipython.scipy.org/moin/InstallationOSXLeopard).
 
 On Windows, you will need the PyReadline module. PyReadline is a separate,
 Windows only implementation of readline that uses native Windows calls through
 :mod:`ctypes`. The easiest way of installing PyReadline is you use the binary
 installer available `here <http://ipython.scipy.org/dist/>`_. The :mod:`ctypes`
 module, which comes with Python 2.5 and greater, is required by PyReadline. It
 is available for Python 2.4 at http://python.net/crew/theller/ctypes.
 
 nose
 ----
 
 To run the IPython test suite you will need the :mod:`nose` package.  Nose
 provides a great way of sniffing out and running all of the IPython tests.  The
 simplest way of getting nose, is to use :command:`easy_install`:
 
 .. code-block:: bash
 
-    easy_install nose
+    $ easy_install nose
 
 Another way of getting this is to do:
 
 .. code-block:: bash
 
-    easy_install ipython[test]
+    $ easy_install ipython[test]
 
 For more installation options, see the `nose website
-<http://somethingaboutorange.com/mrl/projects/nose/>`_.  Once you have nose
-installed, you can run IPython's test suite using the iptest command:
+<http://somethingaboutorange.com/mrl/projects/nose/>`_.  
+
+.. warning::
+
+    As described above, the :command:`iptest` command currently doesn't work.
+
+Once you have nose installed, you can run IPython's test suite using the
+iptest command:
 
 .. code-block:: bash
 
-    iptest
+    $ iptest
 
 pexpect
 -------
 
 The `pexpect <http://www.noah.org/wiki/Pexpect>`_ package is used in IPython's
 :command:`irunner` script.  On Unix platforms (including OS X), just do:
 
 .. code-block:: bash
 
-    easy_install pexpect
+    $ easy_install pexpect
 
 Windows users are out of luck as pexpect does not run there.
 
 Dependencies for IPython.kernel (parallel computing)
 ====================================================
 
 The IPython kernel provides a nice architecture for parallel computing.  The
 main focus of this architecture is on interactive parallel computing.  These
 features require a number of additional packages:
 
 * zope.interface (yep, we use interfaces)
 * Twisted (asynchronous networking framework)
 * Foolscap (a nice, secure network protocol)
 * pyOpenSSL (security for network connections)
 
 On a Unix style platform (including OS X), if you want to use
 :mod:`setuptools`, you can just do:
 
 .. code-block:: bash
 
-    easy_install ipython[kernel]    # the first three
-    easy_install ipython[security]  # pyOpenSSL
+    $ easy_install ipython[kernel]    # the first three
+    $ easy_install ipython[security]  # pyOpenSSL
 
 zope.interface and Twisted
 --------------------------
 
 Twisted [Twisted]_ and zope.interface [ZopeInterface]_ are used for networking
 related things. On Unix style platforms (including OS X), the simplest way of
 getting the these is to use :command:`easy_install`:
 
 .. code-block:: bash
 
-    easy_install zope.interface
-    easy_install Twisted
+    $ easy_install zope.interface
+    $ easy_install Twisted
 
 Of course, you can also download the source tarballs from the Twisted website
-[Twisted]_ and the `zope.interface page at PyPI
-<http://pypi.python.org/pypi/zope.interface>`_ and do the usual ``python
-setup.py install`` if you prefer.
+[Twisted]_ and the 
+`zope.interface page at PyPI <http://pypi.python.org/pypi/zope.interface>`_ 
+and do the usual ``python setup.py install`` if you prefer.
 
 Windows is a bit different. For zope.interface and Twisted, simply get the
 latest binary ``.exe`` installer from the Twisted website. This installer
 includes both zope.interface and Twisted and should just work.
 
 Foolscap
 --------
 
 Foolscap [Foolscap]_ uses Twisted to provide a very nice secure RPC protocol that we use to implement our parallel computing features.
 
 On all platforms a simple:
 
 .. code-block:: bash
 
-    easy_install foolscap
+    $ easy_install foolscap
 
 should work.  You can also download the source tarballs from the `Foolscap
 website <http://foolscap.lothar.com/trac>`_ and do ``python setup.py install``
 if you prefer.
 
 pyOpenSSL
 ---------
 
 IPython does not work with version 0.7 of pyOpenSSL [pyOpenSSL]_. It is known
 to work with version 0.6 and will likely work with the more recent 0.8 and 0.9
 versions. There are a couple of options for getting this:
 
 1. Most Linux distributions have packages for pyOpenSSL.
 2. The built-in Python 2.5 on OS X 10.5 already has it installed.
 3. There are source tarballs on the pyOpenSSL website.  On Unix-like
    platforms, these can be built using ``python seutp.py install``.
 4. There is also a binary ``.exe`` Windows installer on the 
    `pyOpenSSL website <http://pyopenssl.sourceforge.net/>`_.
 
 Dependencies for IPython.frontend (the IPython GUI)
 ===================================================
 
 wxPython
 --------
 
 Starting with IPython 0.9, IPython has a new :mod:`IPython.frontend` package
 that has a nice wxPython based IPython GUI. As you would expect, this GUI
 requires wxPython. Most Linux distributions have wxPython packages available
 and the built-in Python on OS X comes with wxPython preinstalled. For Windows,
 a binary installer is available on the `wxPython website
 <http://www.wxpython.org/>`_.
 
 .. [Twisted] Twisted matrix.  http://twistedmatrix.org
 .. [ZopeInterface] http://pypi.python.org/pypi/zope.interface
 .. [Foolscap] Foolscap network protocol.  http://foolscap.lothar.com/trac
 .. [pyOpenSSL] pyOpenSSL.  http://pyopenssl.sourceforge.net