The main concepts

There are a number of abstractions that the IPython configuration system uses. Each of these abstractions is represented by a Python class.

Configuration object: :class:`~traitlets.config.loader.Config`

A configuration object is a simple dictionary-like class that holds configuration attributes and sub-configuration objects. These classes support dotted attribute style access (cfg.Foo.bar) in addition to the regular dictionary style access (cfg['Foo']['bar']). The Config object is a wrapper around a simple dictionary with some convenience methods, such as merging and automatic section creation.

Application: :class:`~traitlets.config.application.Application`

An application is a process that does a specific job. The most obvious application is the :command:`ipython` command line program. Each application reads one or more configuration files and a single set of command line options and then produces a master configuration object for the application. This configuration object is then passed to the configurable objects that the application creates. These configurable objects implement the actual logic of the application and know how to configure themselves given the configuration object.

Applications always have a log attribute that is a configured Logger. This allows centralized logging configuration per-application.

Configurable: :class:`~traitlets.config.configurable.Configurable`

A configurable is a regular Python class that serves as a base class for all main classes in an application. The :class:`~traitlets.config.configurable.Configurable` base class is lightweight and only does one things.

This :class:`~traitlets.config.configurable.Configurable` is a subclass of :class:`~traitlets.HasTraits` that knows how to configure itself. Class level traits with the metadata config=True become values that can be configured from the command line and configuration files.

Developers create :class:`~traitlets.config.configurable.Configurable` subclasses that implement all of the logic in the application. Each of these subclasses has its own configuration information that controls how instances are created.

Singletons: :class:`~traitlets.config.configurable.SingletonConfigurable`

Any object for which there is a single canonical instance. These are just like Configurables, except they have a class method :meth:`~traitlets.config.configurable.SingletonConfigurable.instance`, that returns the current active instance (or creates one if it does not exist). Examples of singletons include :class:`~traitlets.config.application.Application`s and :class:`~IPython.core.interactiveshell.InteractiveShell`. This lets objects easily connect to the current running Application without passing objects around everywhere. For instance, to get the current running Application instance, simply do: app = Application.instance().

Having described these main concepts, we can now state the main idea in our configuration system: "configuration" allows the default values of class attributes to be controlled on a class by class basis. Thus all instances of a given class are configured in the same way. Furthermore, if two instances need to be configured differently, they need to be instances of two different classes. While this model may seem a bit restrictive, we have found that it expresses most things that need to be configured extremely well. However, it is possible to create two instances of the same class that have different trait values. This is done by overriding the configuration.

Now, we show what our configuration objects and files look like.

Configuration objects and files

A configuration object is little more than a wrapper around a dictionary. A configuration file is simply a mechanism for producing that object. The main IPython configuration file is a plain Python script, which can perform extensive logic to populate the config object. IPython 2.0 introduces a JSON configuration file, which is just a direct JSON serialization of the config dictionary, which is easily processed by external software.

When both Python and JSON configuration file are present, both will be loaded, with JSON configuration having higher priority.

c = get_config()

# Sample configurable:
from traitlets.config.configurable import Configurable
from traitlets import Int, Float, Unicode, Bool

class MyClass(Configurable):
    name = Unicode(u'defaultname', config=True)
    ranking = Int(0, config=True)
    value = Float(99.0)
    # The rest of the class implementation would go here..

# Sample config file
c = get_config()

c.MyClass.name = 'coolname'
c.MyClass.ranking = 10

c.ClassName.attribute_name = attribute_value

{
  "version": "1.0",
  "MyClass": {
    "name": "coolname",
    "ranking": 10
  }
}

Configuration files inheritance

Let's say you want to have different configuration files for various purposes. Our configuration system makes it easy for one configuration file to inherit the information in another configuration file. The :func:`load_subconfig` command can be used in a configuration file for this purpose. Here is a simple example that loads all of the values from the file :file:`base_config.py`:

# base_config.py
c = get_config()
c.MyClass.name = 'coolname'
c.MyClass.ranking = 100

into the configuration file :file:`main_config.py`:

# main_config.py
c = get_config()

# Load everything from base_config.py
load_subconfig('base_config.py')

# Now override one of the values
c.MyClass.name = 'bettername'

In a situation like this the :func:`load_subconfig` makes sure that the search path for sub-configuration files is inherited from that of the parent. Thus, you can typically put the two in the same directory and everything will just work.

You can also load configuration files by profile, for instance:

load_subconfig('ipython_config.py', profile='default')

to inherit your default configuration as a starting point.

Class based configuration inheritance

There is another aspect of configuration where inheritance comes into play. Sometimes, your classes will have an inheritance hierarchy that you want to be reflected in the configuration system. Here is a simple example:

from traitlets.config.configurable import Configurable
from traitlets import Int, Float, Unicode, Bool

class Foo(Configurable):
    name = Unicode(u'fooname', config=True)
    value = Float(100.0, config=True)

class Bar(Foo):
    name = Unicode(u'barname', config=True)
    othervalue = Int(0, config=True)

Now, we can create a configuration file to configure instances of :class:`Foo` and :class:`Bar`:

# config file
c = get_config()

c.Foo.name = u'bestname'
c.Bar.othervalue = 10

This class hierarchy and configuration file accomplishes the following:

• The default value for :attr:`Foo.name` and :attr:`Bar.name` will be 'bestname'. Because :class:`Bar` is a :class:`Foo` subclass it also picks up the configuration information for :class:`Foo`.
• The default value for :attr:`Foo.value` and :attr:`Bar.value` will be 100.0, which is the value specified as the class default.
• The default value for :attr:`Bar.othervalue` will be 10 as set in the configuration file. Because :class:`Foo` is the parent of :class:`Bar` it doesn't know anything about the :attr:`othervalue` attribute.

Configuration file location

So where should you put your configuration files? IPython uses "profiles" for configuration, and by default, all profiles will be stored in the so called "IPython directory". The location of this directory is determined by the following algorithm:

• If the ipython-dir command line flag is given, its value is used.
• If not, the value returned by :func:`IPython.utils.path.get_ipython_dir` is used. This function will first look at the :envvar:`IPYTHONDIR` environment variable and then default to :file:`~/.ipython`. Historical support for the :envvar:`IPYTHON_DIR` environment variable will be removed in a future release.

For most users, the configuration directory will be :file:`~/.ipython`.

Previous versions of IPython on Linux would use the XDG config directory, creating :file:`~/.config/ipython` by default. We have decided to go back to :file:`~/.ipython` for consistency among systems. IPython will issue a warning if it finds the XDG location, and will move it to the new location if there isn't already a directory there.

Once the location of the IPython directory has been determined, you need to know which profile you are using. For users with a single configuration, this will simply be 'default', and will be located in :file:`<IPYTHONDIR>/profile_default`.

The next thing you need to know is what to call your configuration file. The basic idea is that each application has its own default configuration filename. The default named used by the :command:`ipython` command line program is :file:`ipython_config.py`, and all IPython applications will use this file. Other applications, such as the parallel :command:`ipcluster` scripts or the QtConsole will load their own config files after :file:`ipython_config.py`. To load a particular configuration file instead of the default, the name can be overridden by the config_file command line flag.

To generate the default configuration files, do:

$ ipython profile create

and you will have a default :file:`ipython_config.py` in your IPython directory under :file:`profile_default`. If you want the default config files for the :mod:`IPython.parallel` applications, add --parallel to the end of the command-line args.

$ ipython locate
/home/you/.ipython

$ ipython locate profile foo
/home/you/.ipython/profile_foo

Profiles

A profile is a directory containing configuration and runtime files, such as logs, connection info for the parallel apps, and your IPython command history.

The idea is that users often want to maintain a set of configuration files for different purposes: one for doing numerical computing with NumPy and SciPy and another for doing symbolic computing with SymPy. Profiles make it easy to keep a separate configuration files, logs, and histories for each of these purposes.

Let's start by showing how a profile is used:

$ ipython --profile=sympy

This tells the :command:`ipython` command line program to get its configuration from the "sympy" profile. The file names for various profiles do not change. The only difference is that profiles are named in a special way. In the case above, the "sympy" profile means looking for :file:`ipython_config.py` in :file:`<IPYTHONDIR>/profile_sympy`.

The general pattern is this: simply create a new profile with:

$ ipython profile create <name>

which adds a directory called profile_<name> to your IPython directory. Then you can load this profile by adding --profile=<name> to your command line options. Profiles are supported by all IPython applications.

IPython ships with some sample profiles in :file:`IPython/config/profile`. If you create profiles with the name of one of our shipped profiles, these config files will be copied over instead of starting with the automatically generated config files.

Command-line arguments

IPython exposes all configurable options on the command-line. The command-line arguments are generated from the Configurable traits of the classes associated with a given Application. Configuring IPython from the command-line may look very similar to an IPython config file

IPython applications use a parser called :class:`~traitlets.config.loader.KeyValueLoader` to load values into a Config object. Values are assigned in much the same way as in a config file:

$ ipython --InteractiveShell.use_readline=False --BaseIPythonApplication.profile='myprofile'

Is the same as adding:

c.InteractiveShell.use_readline=False
c.BaseIPythonApplication.profile='myprofile'

to your config file. Key/Value arguments always take a value, separated by '=' and no spaces.

$ ipython -i -c "import numpy; x=numpy.linspace(0,1)" --profile testing --colors=lightbg

$ ipython --profile myprofile
# and
$ ipython --profile='myprofile'
# are equivalent to
$ ipython --BaseIPythonApplication.profile='myprofile'

$ ipcontroller --debug
# is equivalent to
$ ipcontroller --Application.log_level=DEBUG
# and
$ ipython --matplotlib
# is equivalent to
$ ipython --matplotlib auto
# or
$ ipython --no-banner
# is equivalent to
$ ipython --TerminalIPythonApp.display_banner=False

$ ipython qtconsole --profile myprofile

$ ipcluster start --profile=myprofile -n 4

Design requirements

Here are the main requirements we wanted our configuration system to have:

• Support for hierarchical configuration information.
• Full integration with command line option parsers. Often, you want to read a configuration file, but then override some of the values with command line options. Our configuration system automates this process and allows each command line option to be linked to a particular attribute in the configuration hierarchy that it will override.
• Configuration files that are themselves valid Python code. This accomplishes many things. First, it becomes possible to put logic in your configuration files that sets attributes based on your operating system, network setup, Python version, etc. Second, Python has a super simple syntax for accessing hierarchical data structures, namely regular attribute access (Foo.Bar.Bam.name). Third, using Python makes it easy for users to import configuration attributes from one configuration file to another. Fourth, even though Python is dynamically typed, it does have types that can be checked at runtime. Thus, a 1 in a config file is the integer '1', while a '1' is a string.
• A fully automated method for getting the configuration information to the classes that need it at runtime. Writing code that walks a configuration hierarchy to extract a particular attribute is painful. When you have complex configuration information with hundreds of attributes, this makes you want to cry.
• Type checking and validation that doesn't require the entire configuration hierarchy to be specified statically before runtime. Python is a very dynamic language and you don't always know everything that needs to be configured when a program starts.