Merge pull request #8476 from takluyver/docs-cleanup-config

Remove documentation of machinery that is now in traitlets

docs/source/conf.py

@@ -191,7 +191,9 b' html_additional_pages = {'
 htmlhelp_basename = 'ipythondoc'
 
 intersphinx_mapping = {'python': ('http://docs.python.org/2/', None),
-                       'rpy2': ('http://rpy.sourceforge.net/rpy2/doc-2.4/html/', None)}
+                       'rpy2': ('http://rpy.sourceforge.net/rpy2/doc-2.4/html/', None),
+                       'traitlets': ('http://traitlets.readthedocs.org/en/latest/', None),
+                      }
 
 # Options for LaTeX output
 # ------------------------

docs/source/config/intro.rst

@@ -37,7 +37,7 b' no error.'
 
 To add to a collection which may have already been defined elsewhere,
 you can use methods like those found on lists, dicts and sets: append,
-extend, :meth:`~traitlets.config.loader.LazyConfigValue.prepend` (like
+extend, :meth:`~traitlets.config.LazyConfigValue.prepend` (like
 extend, but at the front), add and update (which works both for dicts
 and sets)::
 
@@ -154,3 +154,6 b' the directory :file:`~/.ipython/` by default.'
 
    This command line option can also be used to override the default
    IPython directory.
+
+To see where IPython is looking for the IPython directory, use the command
+``ipython locate``, or the Python function :func:`IPython.paths.get_ipython_dir`.

docs/source/development/config.rst

@@ -4,296 +4,9 b''
 Overview of the IPython configuration system
 ============================================
 
-This section describes the IPython configuration system. 
-
-The main concepts
-=================
-
-There are a number of abstractions that the IPython configuration system uses.
-Each of these abstractions is represented by a Python class.
-
-Configuration object: :class:`~traitlets.config.loader.Config`
-    A configuration object is a simple dictionary-like class that holds
-    configuration attributes and sub-configuration objects. These classes
-    support dotted attribute style access (``cfg.Foo.bar``) in addition to the
-    regular dictionary style access (``cfg['Foo']['bar']``).
-    The Config object is a wrapper around a simple dictionary with some convenience methods,
-    such as merging and automatic section creation.
-
-Application: :class:`~traitlets.config.application.Application`
-    An application is a process that does a specific job. The most obvious
-    application is the :command:`ipython` command line program. Each
-    application reads *one or more* configuration files and a single set of
-    command line options
-    and then produces a master configuration object for the application. This
-    configuration object is then passed to the configurable objects that the
-    application creates. These configurable objects implement the actual logic
-    of the application and know how to configure themselves given the
-    configuration object.
-    
-    Applications always have a `log` attribute that is a configured Logger.
-    This allows centralized logging configuration per-application.
-
-Configurable: :class:`~traitlets.config.configurable.Configurable`
-    A configurable is a regular Python class that serves as a base class for
-    all main classes in an application. The
-    :class:`~traitlets.config.configurable.Configurable` base class is
-    lightweight and only does one things.
-
-    This :class:`~traitlets.config.configurable.Configurable` is a subclass
-    of :class:`~traitlets.HasTraits` that knows how to configure
-    itself. Class level traits with the metadata ``config=True`` become
-    values that can be configured from the command line and configuration
-    files.
-    
-    Developers create :class:`~traitlets.config.configurable.Configurable`
-    subclasses that implement all of the logic in the application. Each of
-    these subclasses has its own configuration information that controls how
-    instances are created.
-
-Singletons: :class:`~traitlets.config.configurable.SingletonConfigurable`
-    Any object for which there is a single canonical instance. These are
-    just like Configurables, except they have a class method 
-    :meth:`~traitlets.config.configurable.SingletonConfigurable.instance`,
-    that returns the current active instance (or creates one if it
-    does not exist).  Examples of singletons include
-    :class:`~traitlets.config.application.Application`s and
-    :class:`~IPython.core.interactiveshell.InteractiveShell`.  This lets
-    objects easily connect to the current running Application without passing
-    objects around everywhere.  For instance, to get the current running 
-    Application instance, simply do: ``app = Application.instance()``.
-
-
-.. note::
-
-    Singletons are not strictly enforced - you can have many instances
-    of a given singleton class, but the :meth:`instance` method will always
-    return the same one.
-
-Having described these main concepts, we can now state the main idea in our
-configuration system: *"configuration" allows the default values of class
-attributes to be controlled on a class by class basis*. Thus all instances of
-a given class are configured in the same way. Furthermore, if two instances
-need to be configured differently, they need to be instances of two different
-classes. While this model may seem a bit restrictive, we have found that it
-expresses most things that need to be configured extremely well. However, it
-is possible to create two instances of the same class that have different
-trait values. This is done by overriding the configuration.
-
-Now, we show what our configuration objects and files look like.
-
-Configuration objects and files
-===============================
-
-A configuration object is little more than a wrapper around a dictionary.
-A configuration *file* is simply a mechanism for producing that object.
-The main IPython configuration file is a plain Python script,
-which can perform extensive logic to populate the config object.
-IPython 2.0 introduces a JSON configuration file,
-which is just a direct JSON serialization of the config dictionary,
-which is easily processed by external software.
-
-When both Python and JSON configuration file are present, both will be loaded,
-with JSON configuration having higher priority.
-
-Python configuration Files
---------------------------
-
-A Python configuration file is a pure Python file that populates a configuration object.
-This configuration object is a :class:`~traitlets.config.loader.Config` instance.
-While in a configuration file, to get a reference to this object, simply call the :func:`get_config`
-function, which is available in the global namespace of the script.
-
-Here is an example of a super simple configuration file that does nothing::
-
-    c = get_config()
-
-Once you get a reference to the configuration object, you simply set
-attributes on it.  All you have to know is:
-
-* The name of the class to configure.
-* The name of the attribute.
-* The type of each attribute.
-
-The answers to these questions are provided by the various
-:class:`~traitlets.config.configurable.Configurable` subclasses that an
-application uses. Let's look at how this would work for a simple configurable
-subclass::
-
-    # Sample configurable:
-    from traitlets.config.configurable import Configurable
-    from traitlets import Int, Float, Unicode, Bool
-    
-    class MyClass(Configurable):
-        name = Unicode(u'defaultname', config=True)
-        ranking = Int(0, config=True)
-        value = Float(99.0)
-        # The rest of the class implementation would go here..
-
-In this example, we see that :class:`MyClass` has three attributes, two
-of which (``name``, ``ranking``) can be configured.  All of the attributes
-are given types and default values.  If a :class:`MyClass` is instantiated,
-but not configured, these default values will be used.  But let's see how
-to configure this class in a configuration file::
-
-    # Sample config file
-    c = get_config()
-    
-    c.MyClass.name = 'coolname'
-    c.MyClass.ranking = 10
-
-After this configuration file is loaded, the values set in it will override
-the class defaults anytime a :class:`MyClass` is created.  Furthermore,
-these attributes will be type checked and validated anytime they are set.
-This type checking is handled by the :mod:`traitlets` module,
-which provides the :class:`Unicode`, :class:`Int` and :class:`Float` types.
-In addition to these traitlets, the :mod:`traitlets` provides
-traitlets for a number of other types.
-
-.. note::
-
-    Underneath the hood, the :class:`Configurable` base class is a subclass of
-    :class:`traitlets.HasTraits`. The
-    :mod:`traitlets` module is a lightweight version of
-    :mod:`enthought.traits`. Our implementation is a pure Python subset
-    (mostly API compatible) of :mod:`enthought.traits` that does not have any
-    of the automatic GUI generation capabilities. Our plan is to achieve 100%
-    API compatibility to enable the actual :mod:`enthought.traits` to
-    eventually be used instead. Currently, we cannot use
-    :mod:`enthought.traits` as we are committed to the core of IPython being
-    pure Python.
-
-It should be very clear at this point what the naming convention is for 
-configuration attributes::
-
-    c.ClassName.attribute_name = attribute_value
-
-Here, ``ClassName`` is the name of the class whose configuration attribute you
-want to set, ``attribute_name`` is the name of the attribute you want to set
-and ``attribute_value`` the the value you want it to have. The ``ClassName``
-attribute of ``c`` is not the actual class, but instead is another
-:class:`~traitlets.config.loader.Config` instance.
-
-.. note::
-
-    The careful reader may wonder how the ``ClassName`` (``MyClass`` in
-    the above example) attribute of the configuration object ``c`` gets
-    created. These attributes are created on the fly by the
-    :class:`~traitlets.config.loader.Config` instance, using a simple naming
-    convention. Any attribute of a :class:`~traitlets.config.loader.Config`
-    instance whose name begins with an uppercase character is assumed to be a
-    sub-configuration and a new empty :class:`~traitlets.config.loader.Config`
-    instance is dynamically created for that attribute. This allows deeply
-    hierarchical information created easily (``c.Foo.Bar.value``) on the fly.
-
-JSON configuration Files
-------------------------
-
-A JSON configuration file is simply a file that contains a
-:class:`~traitlets.config.loader.Config` dictionary serialized to JSON.
-A JSON configuration file has the same base name as a Python configuration file,
-but with a .json extension.
-
-Configuration described in previous section could be written as follows in a
-JSON configuration file:
-
-.. sourcecode:: json
-
-    {
-      "version": "1.0",
-      "MyClass": {
-        "name": "coolname",
-        "ranking": 10
-      }
-    }
-
-JSON configuration files can be more easily generated or processed by programs
-or other languages.
-
-
-Configuration files inheritance
-===============================
-
-.. note::
-
-    This section only apply to Python configuration files.
-
-Let's say you want to have different configuration files for various purposes.
-Our configuration system makes it easy for one configuration file to inherit
-the information in another configuration file. The :func:`load_subconfig`
-command can be used in a configuration file for this purpose. Here is a simple
-example that loads all of the values from the file :file:`base_config.py`::
-
-    # base_config.py
-    c = get_config()
-    c.MyClass.name = 'coolname'
-    c.MyClass.ranking = 100
-
-into the configuration file :file:`main_config.py`::
-
-    # main_config.py
-    c = get_config()
-    
-    # Load everything from base_config.py
-    load_subconfig('base_config.py')
-    
-    # Now override one of the values
-    c.MyClass.name = 'bettername'
-
-In a situation like this the :func:`load_subconfig` makes sure that the
-search path for sub-configuration files is inherited from that of the parent.
-Thus, you can typically put the two in the same directory and everything will
-just work.
-
-You can also load configuration files by profile, for instance:
-
-.. sourcecode:: python
-
-    load_subconfig('ipython_config.py', profile='default')
-
-to inherit your default configuration as a starting point.
-
-
-Class based configuration inheritance
-=====================================
-
-There is another aspect of configuration where inheritance comes into play.
-Sometimes, your classes will have an inheritance hierarchy that you want
-to be reflected in the configuration system.  Here is a simple example::
-
-    from traitlets.config.configurable import Configurable
-    from traitlets import Int, Float, Unicode, Bool
-    
-    class Foo(Configurable):
-        name = Unicode(u'fooname', config=True)
-        value = Float(100.0, config=True)
-    
-    class Bar(Foo):
-        name = Unicode(u'barname', config=True)
-        othervalue = Int(0, config=True)
-
-Now, we can create a configuration file to configure instances of :class:`Foo`
-and :class:`Bar`::
-
-    # config file
-    c = get_config()
-    
-    c.Foo.name = u'bestname'
-    c.Bar.othervalue = 10
-
-This class hierarchy and configuration file accomplishes the following:
-
-* The default value for :attr:`Foo.name` and :attr:`Bar.name` will be
-  'bestname'.  Because :class:`Bar` is a :class:`Foo` subclass it also
-  picks up the configuration information for :class:`Foo`.
-* The default value for :attr:`Foo.value` and :attr:`Bar.value` will be
-  ``100.0``, which is the value specified as the class default.
-* The default value for :attr:`Bar.othervalue` will be 10 as set in the
-  configuration file.  Because :class:`Foo` is the parent of :class:`Bar`
-  it doesn't know anything about the :attr:`othervalue` attribute.
-
-
-.. _ipython_dir:
+This section describes the IPython configuration system. This is based on
+:mod:`traitlets.config`; see that documentation for more information
+about the overall architecture.
 
 Configuration file location
 ===========================
@@ -305,7 +18,7 b' following algorithm:'
 
 * If the ``ipython-dir`` command line flag is given, its value is used.
 
-* If not, the value returned by :func:`IPython.utils.path.get_ipython_dir`
+* If not, the value returned by :func:`IPython.paths.get_ipython_dir`
   is used. This function will first look at the :envvar:`IPYTHONDIR`
   environment variable and then default to :file:`~/.ipython`.
   Historical support for the :envvar:`IPYTHON_DIR` environment variable will
@@ -400,167 +113,10 b' you create profiles with the name of one of our shipped profiles, these config'
 files will be copied over instead of starting with the automatically generated
 config files.
 
-Security Files
---------------
-
-If you are using the notebook, qtconsole, or parallel code, IPython stores
-connection information in small JSON files in the active profile's security
-directory. This directory is made private, so only you can see the files inside. If
-you need to move connection files around to other computers, this is where they will
-be. If you want your code to be able to open security files by name, we have a
-convenience function :func:`IPython.utils.path.get_security_file`, which will return
-the absolute path to a security file from its filename and [optionally] profile
-name.
-
-.. _startup_files:
-
-Startup Files
--------------
-
-If you want some code to be run at the beginning of every IPython session with
-a particular profile, the easiest way is to add Python (``.py``) or 
-IPython (``.ipy``) scripts to your :file:`<profile>/startup` directory. Files
-in this directory will always be executed as soon as the IPython shell is 
-constructed, and before any other code or scripts you have specified. If you 
-have multiple files in the startup directory, they will be run in 
-lexicographical order, so you can control the ordering by adding a '00-' 
-prefix.
-
-
-.. _commandline:
-
-Command-line arguments
-======================
-
-IPython exposes *all* configurable options on the command-line. The command-line
-arguments are generated from the Configurable traits of the classes associated
-with a given Application.  Configuring IPython from the command-line may look
-very similar to an IPython config file
-
-IPython applications use a parser called
-:class:`~traitlets.config.loader.KeyValueLoader` to load values into a Config
-object.  Values are assigned in much the same way as in a config file:
-
-.. code-block:: bash
-
-    $ ipython --InteractiveShell.use_readline=False --BaseIPythonApplication.profile='myprofile'
-
-Is the same as adding:
+IPython extends the config loader for Python files so that you can inherit
+config from another profile. To do this, use a line like this in your Python
+config file:
 
 .. sourcecode:: python
 
-    c.InteractiveShell.use_readline=False
-    c.BaseIPythonApplication.profile='myprofile'
-
-to your config file. Key/Value arguments *always* take a value, separated by '='
-and no spaces.
-
-Common Arguments
-----------------
-
-Since the strictness and verbosity of the KVLoader above are not ideal for everyday
-use, common arguments can be specified as flags_ or aliases_.
-
-Flags and Aliases are handled by :mod:`argparse` instead, allowing for more flexible
-parsing. In general, flags and aliases are prefixed by ``--``, except for those
-that are single characters, in which case they can be specified with a single ``-``, e.g.:
-
-.. code-block:: bash
-
-    $ ipython -i -c "import numpy; x=numpy.linspace(0,1)" --profile testing --colors=lightbg
-
-Aliases
-*******
-
-For convenience, applications have a mapping of commonly used traits, so you don't have
-to specify the whole class name:
-
-.. code-block:: bash
-
-    $ ipython --profile myprofile
-    # and
-    $ ipython --profile='myprofile'
-    # are equivalent to
-    $ ipython --BaseIPythonApplication.profile='myprofile'
-
-Flags
-*****
-
-Applications can also be passed **flags**. Flags are options that take no
-arguments. They are simply wrappers for
-setting one or more configurables with predefined values, often True/False.
-
-For instance:
-
-.. code-block:: bash
-
-    $ ipcontroller --debug
-    # is equivalent to
-    $ ipcontroller --Application.log_level=DEBUG
-    # and
-    $ ipython --matplotlib
-    # is equivalent to
-    $ ipython --matplotlib auto
-    # or
-    $ ipython --no-banner
-    # is equivalent to
-    $ ipython --TerminalIPythonApp.display_banner=False
-
-Subcommands
------------
-
-
-Some IPython applications have **subcommands**. Subcommands are modeled after
-:command:`git`, and are called with the form :command:`command subcommand
-[...args]`.  Currently, the QtConsole is a subcommand of terminal IPython:
-
-.. code-block:: bash
-
-    $ ipython qtconsole --profile myprofile
-
-and :command:`ipcluster` is simply a wrapper for its various subcommands (start,
-stop, engines).
-
-.. code-block:: bash
-
-    $ ipcluster start --profile=myprofile -n 4
-
-
-To see a list of the available aliases, flags, and subcommands for an IPython application, simply pass ``-h`` or ``--help``.  And to see the full list of configurable options (*very* long), pass ``--help-all``.
-
-
-Design requirements
-===================
-
-Here are the main requirements we wanted our configuration system to have:
-
-* Support for hierarchical configuration information.
-
-* Full integration with command line option parsers.  Often, you want to read
-  a configuration file, but then override some of the values with command line
-  options.  Our configuration system automates this process and allows each
-  command line option to be linked to a particular attribute in the 
-  configuration hierarchy that it will override.
-
-* Configuration files that are themselves valid Python code. This accomplishes
-  many things. First, it becomes possible to put logic in your configuration
-  files that sets attributes based on your operating system, network setup,
-  Python version, etc. Second, Python has a super simple syntax for accessing
-  hierarchical data structures, namely regular attribute access
-  (``Foo.Bar.Bam.name``). Third, using Python makes it easy for users to
-  import configuration attributes from one configuration file to another.
-  Fourth, even though Python is dynamically typed, it does have types that can
-  be checked at runtime. Thus, a ``1`` in a config file is the integer '1',
-  while a ``'1'`` is a string.
-
-* A fully automated method for getting the configuration information to the
-  classes that need it at runtime. Writing code that walks a configuration
-  hierarchy to extract a particular attribute is painful. When you have
-  complex configuration information with hundreds of attributes, this makes
-  you want to cry.
-
-* Type checking and validation that doesn't require the entire configuration
-  hierarchy to be specified statically before runtime. Python is a very
-  dynamic language and you don't always know everything that needs to be
-  configured when a program starts.
-
+    load_subconfig('ipython_config.py', profile='default')