##// END OF EJS Templates
Fix for looking up print function in Python 3.
Fix for looking up print function in Python 3.

File last commit:

r4608:c8faecb2
r4741:a7e8c702
Show More
overview.txt
499 lines | 20.4 KiB | text/plain | TextLexer
Brian Granger
Major work on the documentation....
r2277 .. _config_overview:
============================================
Overview of the IPython configuration system
============================================
This section describes the IPython configuration system. Starting with version
0.11, IPython has a completely new configuration system that is quite
different from the older :file:`ipythonrc` or :file:`ipy_user_conf.py`
approaches. The new configuration system was designed from scratch to address
the particular configuration needs of IPython. While there are many
other excellent configuration systems out there, we found that none of them
met our requirements.
.. warning::
If you are upgrading to version 0.11 of IPython, you will need to migrate
your old :file:`ipythonrc` or :file:`ipy_user_conf.py` configuration files
to the new system. Read on for information on how to do this.
MinRK
update config docs with recent changes...
r4040 The discussion that follows is focused on teaching users how to configure
IPython to their liking. Developers who want to know more about how they
Brian Granger
Major work on the documentation....
r2277 can enable their objects to take advantage of the configuration system
should consult our :ref:`developer guide <developer_guide>`
The main concepts
=================
There are a number of abstractions that the IPython configuration system uses.
Each of these abstractions is represented by a Python class.
Configuration object: :class:`~IPython.config.loader.Config`
A configuration object is a simple dictionary-like class that holds
configuration attributes and sub-configuration objects. These classes
support dotted attribute style access (``Foo.bar``) in addition to the
regular dictionary style access (``Foo['bar']``). Configuration objects
are smart. They know how to merge themselves with other configuration
objects and they automatically create sub-configuration objects.
MinRK
update config docs with recent changes...
r4040 Application: :class:`~IPython.config.application.Application`
Brian Granger
Major work on the documentation....
r2277 An application is a process that does a specific job. The most obvious
application is the :command:`ipython` command line program. Each
MinRK
update config docs with recent changes...
r4040 application reads *one or more* configuration files and a single set of
command line options
Brian Granger
Major work on the documentation....
r2277 and then produces a master configuration object for the application. This
Brian Granger
Updating the docs.
r2748 configuration object is then passed to the configurable objects that the
application creates. These configurable objects implement the actual logic
of the application and know how to configure themselves given the
configuration object.
MinRK
update config docs with recent changes...
r4040
Applications always have a `log` attribute that is a configured Logger.
This allows centralized logging configuration per-application.
Brian Granger
Updating the docs.
r2748
MinRK
doc: updated roadmap, which had become stale....
r4101 Configurable: :class:`~IPython.config.configurable.Configurable`
Brian Granger
Updating the docs.
r2748 A configurable is a regular Python class that serves as a base class for
all main classes in an application. The
:class:`~IPython.config.configurable.Configurable` base class is
lightweight and only does one things.
This :class:`~IPython.config.configurable.Configurable` is a subclass
of :class:`~IPython.utils.traitlets.HasTraits` that knows how to configure
itself. Class level traits with the metadata ``config=True`` become
values that can be configured from the command line and configuration
files.
Brian Granger
Major work on the documentation....
r2277
Brian Granger
Updating the docs.
r2748 Developers create :class:`~IPython.config.configurable.Configurable`
subclasses that implement all of the logic in the application. Each of
these subclasses has its own configuration information that controls how
Brian Granger
Major work on the documentation....
r2277 instances are created.
MinRK
update config docs with recent changes...
r4040 Singletons: :class:`~IPython.config.configurable.SingletonConfigurable`
Any object for which there is a single canonical instance. These are
just like Configurables, except they have a class method
:meth:`~IPython.config.configurable.SingletonConfigurable.instance`,
that returns the current active instance (or creates one if it
does not exist). Examples of singletons include
:class:`~IPython.config.application.Application`s and
:class:`~IPython.core.interactiveshell.InteractiveShell`. This lets
objects easily connect to the current running Application without passing
objects around everywhere. For instance, to get the current running
Application instance, simply do: ``app = Application.instance()``.
.. note::
Singletons are not strictly enforced - you can have many instances
of a given singleton class, but the :meth:`instance` method will always
return the same one.
Brian Granger
Major work on the documentation....
r2277 Having described these main concepts, we can now state the main idea in our
configuration system: *"configuration" allows the default values of class
attributes to be controlled on a class by class basis*. Thus all instances of
a given class are configured in the same way. Furthermore, if two instances
need to be configured differently, they need to be instances of two different
classes. While this model may seem a bit restrictive, we have found that it
Brian Granger
Updating the docs.
r2748 expresses most things that need to be configured extremely well. However, it
is possible to create two instances of the same class that have different
trait values. This is done by overriding the configuration.
Brian Granger
Major work on the documentation....
r2277
Now, we show what our configuration objects and files look like.
Configuration objects and files
===============================
A configuration file is simply a pure Python file that sets the attributes
of a global, pre-created configuration object. This configuration object is a
:class:`~IPython.config.loader.Config` instance. While in a configuration
file, to get a reference to this object, simply call the :func:`get_config`
function. We inject this function into the global namespace that the
configuration file is executed in.
Here is an example of a super simple configuration file that does nothing::
c = get_config()
Once you get a reference to the configuration object, you simply set
attributes on it. All you have to know is:
* The name of each attribute.
* The type of each attribute.
The answers to these two questions are provided by the various
Brian Granger
Updating the docs.
r2748 :class:`~IPython.config.configurable.Configurable` subclasses that an
MinRK
doc: updated roadmap, which had become stale....
r4101 application uses. Let's look at how this would work for a simple configurable
Brian Granger
Updating the docs.
r2748 subclass::
Brian Granger
Major work on the documentation....
r2277
MinRK
doc: updated roadmap, which had become stale....
r4101 # Sample configurable:
Brian Granger
Updating the docs.
r2748 from IPython.config.configurable import Configurable
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 from IPython.utils.traitlets import Int, Float, Unicode, Bool
Brian Granger
Major work on the documentation....
r2277
Brian Granger
Updating the docs.
r2748 class MyClass(Configurable):
MinRK
update config docs with recent changes...
r4040 name = Unicode(u'defaultname', config=True)
Brian Granger
Major work on the documentation....
r2277 ranking = Int(0, config=True)
value = Float(99.0)
# The rest of the class implementation would go here..
Brian Granger
Updating the docs.
r2748 In this example, we see that :class:`MyClass` has three attributes, two
Brian Granger
Major work on the documentation....
r2277 of whom (``name``, ``ranking``) can be configured. All of the attributes
Brian Granger
Updating the docs.
r2748 are given types and default values. If a :class:`MyClass` is instantiated,
Brian Granger
Major work on the documentation....
r2277 but not configured, these default values will be used. But let's see how
to configure this class in a configuration file::
# Sample config file
c = get_config()
Brian Granger
Updating the docs.
r2748 c.MyClass.name = 'coolname'
c.MyClass.ranking = 10
Brian Granger
Major work on the documentation....
r2277
After this configuration file is loaded, the values set in it will override
Brian Granger
Updating the docs.
r2748 the class defaults anytime a :class:`MyClass` is created. Furthermore,
Brian Granger
Major work on the documentation....
r2277 these attributes will be type checked and validated anytime they are set.
This type checking is handled by the :mod:`IPython.utils.traitlets` module,
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 which provides the :class:`Unicode`, :class:`Int` and :class:`Float` types.
In addition to these traitlets, the :mod:`IPython.utils.traitlets` provides
Brian Granger
Major work on the documentation....
r2277 traitlets for a number of other types.
.. note::
Brian Granger
Updating the docs.
r2748 Underneath the hood, the :class:`Configurable` base class is a subclass of
Dav Clark
missed something in a doc in converting to traits-compatible API
r2470 :class:`IPython.utils.traitlets.HasTraits`. The
Brian Granger
Major work on the documentation....
r2277 :mod:`IPython.utils.traitlets` module is a lightweight version of
:mod:`enthought.traits`. Our implementation is a pure Python subset
(mostly API compatible) of :mod:`enthought.traits` that does not have any
of the automatic GUI generation capabilities. Our plan is to achieve 100%
API compatibility to enable the actual :mod:`enthought.traits` to
eventually be used instead. Currently, we cannot use
:mod:`enthought.traits` as we are committed to the core of IPython being
pure Python.
It should be very clear at this point what the naming convention is for
configuration attributes::
c.ClassName.attribute_name = attribute_value
Here, ``ClassName`` is the name of the class whose configuration attribute you
want to set, ``attribute_name`` is the name of the attribute you want to set
and ``attribute_value`` the the value you want it to have. The ``ClassName``
attribute of ``c`` is not the actual class, but instead is another
:class:`~IPython.config.loader.Config` instance.
.. note::
Brian Granger
Updating the docs.
r2748 The careful reader may wonder how the ``ClassName`` (``MyClass`` in
Brian Granger
Major work on the documentation....
r2277 the above example) attribute of the configuration object ``c`` gets
created. These attributes are created on the fly by the
:class:`~IPython.config.loader.Config` instance, using a simple naming
convention. Any attribute of a :class:`~IPython.config.loader.Config`
instance whose name begins with an uppercase character is assumed to be a
sub-configuration and a new empty :class:`~IPython.config.loader.Config`
instance is dynamically created for that attribute. This allows deeply
Brian Granger
Updating the docs.
r2748 hierarchical information created easily (``c.Foo.Bar.value``) on the fly.
Brian Granger
Major work on the documentation....
r2277
Configuration files inheritance
===============================
Let's say you want to have different configuration files for various purposes.
Our configuration system makes it easy for one configuration file to inherit
the information in another configuration file. The :func:`load_subconfig`
command can be used in a configuration file for this purpose. Here is a simple
example that loads all of the values from the file :file:`base_config.py`::
# base_config.py
c = get_config()
Brian Granger
Updating the docs.
r2748 c.MyClass.name = 'coolname'
c.MyClass.ranking = 100
Brian Granger
Major work on the documentation....
r2277
into the configuration file :file:`main_config.py`::
# main_config.py
c = get_config()
# Load everything from base_config.py
load_subconfig('base_config.py')
# Now override one of the values
Brian Granger
Updating the docs.
r2748 c.MyClass.name = 'bettername'
Brian Granger
Major work on the documentation....
r2277
In a situation like this the :func:`load_subconfig` makes sure that the
search path for sub-configuration files is inherited from that of the parent.
Thus, you can typically put the two in the same directory and everything will
just work.
MinRK
update config docs with recent changes...
r4040 You can also load configuration files by profile, for instance:
.. sourcecode:: python
load_subconfig('ipython_config.py', profile='default')
to inherit your default configuration as a starting point.
Brian Granger
Major work on the documentation....
r2277 Class based configuration inheritance
=====================================
There is another aspect of configuration where inheritance comes into play.
Sometimes, your classes will have an inheritance hierarchy that you want
to be reflected in the configuration system. Here is a simple example::
Brian Granger
Updating the docs.
r2748 from IPython.config.configurable import Configurable
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 from IPython.utils.traitlets import Int, Float, Unicode, Bool
Brian Granger
Major work on the documentation....
r2277
Brian Granger
Updating the docs.
r2748 class Foo(Configurable):
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 name = Unicode(u'fooname', config=True)
Brian Granger
Major work on the documentation....
r2277 value = Float(100.0, config=True)
class Bar(Foo):
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 name = Unicode(u'barname', config=True)
Brian Granger
Major work on the documentation....
r2277 othervalue = Int(0, config=True)
Now, we can create a configuration file to configure instances of :class:`Foo`
and :class:`Bar`::
# config file
c = get_config()
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 c.Foo.name = u'bestname'
Brian Granger
Major work on the documentation....
r2277 c.Bar.othervalue = 10
This class hierarchy and configuration file accomplishes the following:
* The default value for :attr:`Foo.name` and :attr:`Bar.name` will be
'bestname'. Because :class:`Bar` is a :class:`Foo` subclass it also
picks up the configuration information for :class:`Foo`.
* The default value for :attr:`Foo.value` and :attr:`Bar.value` will be
``100.0``, which is the value specified as the class default.
* The default value for :attr:`Bar.othervalue` will be 10 as set in the
configuration file. Because :class:`Foo` is the parent of :class:`Bar`
it doesn't know anything about the :attr:`othervalue` attribute.
MinRK
update docs to reflect XDG support, new ~/.config/ipython default
r3351
.. _ipython_dir:
Brian Granger
Major work on the documentation....
r2277 Configuration file location
===========================
MinRK
update config docs with recent changes...
r4040 So where should you put your configuration files? IPython uses "profiles" for
configuration, and by default, all profiles will be stored in the so called
"IPython directory". The location of this directory is determined by the
following algorithm:
Brian Granger
Major work on the documentation....
r2277
MinRK
update docs with new cl-arguments
r3966 * If the ``ipython_dir`` command line flag is given, its value is used.
Brian Granger
Major work on the documentation....
r2277
Brian Granger
Work to address the review comments on Fernando's branch....
r2498 * If not, the value returned by :func:`IPython.utils.path.get_ipython_dir`
Brian Granger
Lots of work on command line options and env vars....
r2322 is used. This function will first look at the :envvar:`IPYTHON_DIR`
MinRK
update docs to reflect XDG support, new ~/.config/ipython default
r3351 environment variable and then default to a platform-specific default.
On posix systems (Linux, Unix, etc.), IPython respects the ``$XDG_CONFIG_HOME``
part of the `XDG Base Directory`_ specification. If ``$XDG_CONFIG_HOME`` is
defined and exists ( ``XDG_CONFIG_HOME`` has a default interpretation of
:file:`$HOME/.config`), then IPython's config directory will be located in
:file:`$XDG_CONFIG_HOME/ipython`. If users still have an IPython directory
in :file:`$HOME/.ipython`, then that will be used. in preference to the
system default.
Brian Granger
Major work on the documentation....
r2277
For most users, the default value will simply be something like
MinRK
update docs to reflect XDG support, new ~/.config/ipython default
r3351 :file:`$HOME/.config/ipython` on Linux, or :file:`$HOME/.ipython`
elsewhere.
Brian Granger
Major work on the documentation....
r2277
MinRK
update config docs with recent changes...
r4040 Once the location of the IPython directory has been determined, you need to know
which profile you are using. For users with a single configuration, this will
simply be 'default', and will be located in
:file:`<IPYTHON_DIR>/profile_default`.
The next thing you need to know is what to call your configuration file. The
basic idea is that each application has its own default configuration filename.
The default named used by the :command:`ipython` command line program is
:file:`ipython_config.py`, and *all* IPython applications will use this file.
Other applications, such as the parallel :command:`ipcluster` scripts or the
QtConsole will load their own config files *after* :file:`ipython_config.py`. To
load a particular configuration file instead of the default, the name can be
overridden by the ``config_file`` command line flag.
To generate the default configuration files, do::
$> ipython profile create
and you will have a default :file:`ipython_config.py` in your IPython directory
under :file:`profile_default`. If you want the default config files for the
:mod:`IPython.parallel` applications, add ``--parallel`` to the end of the
command-line args.
Brian Granger
Major work on the documentation....
r2277
.. _Profiles:
Profiles
========
MinRK
update config docs with recent changes...
r4040 A profile is a directory containing configuration and runtime files, such as
logs, connection info for the parallel apps, and your IPython command history.
The idea is that users often want to maintain a set of configuration files for
different purposes: one for doing numerical computing with NumPy and SciPy and
another for doing symbolic computing with SymPy. Profiles make it easy to keep a
separate configuration files, logs, and histories for each of these purposes.
Brian Granger
Major work on the documentation....
r2277
Let's start by showing how a profile is used:
.. code-block:: bash
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $ ipython --profile=sympy
Brian Granger
Major work on the documentation....
r2277
MinRK
update config docs with recent changes...
r4040 This tells the :command:`ipython` command line program to get its configuration
from the "sympy" profile. The file names for various profiles do not change. The
only difference is that profiles are named in a special way. In the case above,
the "sympy" profile means looking for :file:`ipython_config.py` in :file:`<IPYTHON_DIR>/profile_sympy`.
The general pattern is this: simply create a new profile with:
.. code-block:: bash
ipython profile create <name>
which adds a directory called ``profile_<name>`` to your IPython directory. Then
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 you can load this profile by adding ``--profile=<name>`` to your command line
MinRK
update config docs with recent changes...
r4040 options. Profiles are supported by all IPython applications.
IPython ships with some sample profiles in :file:`IPython/config/profile`. If
you create profiles with the name of one of our shipped profiles, these config
files will be copied over instead of starting with the automatically generated
config files.
.. _commandline:
Command-line arguments
======================
IPython exposes *all* configurable options on the command-line. The command-line
arguments are generated from the Configurable traits of the classes associated
with a given Application. Configuring IPython from the command-line may look
very similar to an IPython config file
IPython applications use a parser called
:class:`~IPython.config.loader.KeyValueLoader` to load values into a Config
object. Values are assigned in much the same way as in a config file:
.. code-block:: bash
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipython --InteractiveShell.use_readline=False --BaseIPythonApplication.profile='myprofile'
MinRK
update config docs with recent changes...
r4040
Is the same as adding:
.. sourcecode:: python
c.InteractiveShell.use_readline=False
c.BaseIPythonApplication.profile='myprofile'
to your config file. Key/Value arguments *always* take a value, separated by '='
and no spaces.
MinRK
update docs to reflect relaxed syntax of argparse
r4608 Common Arguments
****************
Since the strictness and verbosity of the KVLoader above are not ideal for everyday
use, common arguments can be specified as flags_ or aliases_.
Flags and Aliases are handled by :mod:`argparse` instead, allowing for more flexible
parsing. In general, flags and aliases are prefixed by ``--``, except for those
that are single characters, in which case they can be specified with a single ``-``, e.g.:
.. code-block:: bash
$> ipython -i -c "import numpy; x=numpy.linspace(0,1)" --profile testing --colors=lightbg
MinRK
update config docs with recent changes...
r4040 Aliases
-------
MinRK
update docs to reflect relaxed syntax of argparse
r4608 For convenience, applications have a mapping of commonly used traits, so you don't have
to specify the whole class name:
MinRK
update config docs with recent changes...
r4040
.. code-block:: bash
MinRK
update docs to reflect relaxed syntax of argparse
r4608 $> ipython --profile myprofile
# and
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipython --profile='myprofile'
MinRK
update docs to reflect relaxed syntax of argparse
r4608 # are equivalent to
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipython --BaseIPythonApplication.profile='myprofile'
MinRK
update config docs with recent changes...
r4040
Flags
-----
Applications can also be passed **flags**. Flags are options that take no
MinRK
update docs to reflect relaxed syntax of argparse
r4608 arguments. They are simply wrappers for
MinRK
update config docs with recent changes...
r4040 setting one or more configurables with predefined values, often True/False.
For instance:
.. code-block:: bash
$> ipcontroller --debug
# is equivalent to
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipcontroller --Application.log_level=DEBUG
MinRK
update docs to reflect relaxed syntax of argparse
r4608 # and
MinRK
update config docs with recent changes...
r4040 $> ipython --pylab
# is equivalent to
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipython --pylab=auto
MinRK
update docs to reflect relaxed syntax of argparse
r4608 # or
$> ipython --no-banner
# is equivalent to
$> ipython --TerminalIPythonApp.display_banner=False
MinRK
update config docs with recent changes...
r4040
Subcommands
MinRK
update docs to reflect relaxed syntax of argparse
r4608 ***********
MinRK
update config docs with recent changes...
r4040
Some IPython applications have **subcommands**. Subcommands are modeled after
:command:`git`, and are called with the form :command:`command subcommand
[...args]`. Currently, the QtConsole is a subcommand of terminal IPython:
.. code-block:: bash
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipython qtconsole --profile=myprofile
MinRK
update config docs with recent changes...
r4040
and :command:`ipcluster` is simply a wrapper for its various subcommands (start,
stop, engines).
.. code-block:: bash
Thomas Kluyver
Update more examples of command line args in the docs, and make a few minor formatting corrections.
r4198 $> ipcluster start --profile=myprofile --n=4
MinRK
update config docs with recent changes...
r4040
Brian Granger
Major work on the documentation....
r2277
MinRK
update config docs with recent changes...
r4040 To see a list of the available aliases, flags, and subcommands for an IPython application, simply pass ``-h`` or ``--help``. And to see the full list of configurable options (*very* long), pass ``--help-all``.
Brian Granger
Major work on the documentation....
r2277
Design requirements
===================
Here are the main requirements we wanted our configuration system to have:
* Support for hierarchical configuration information.
* Full integration with command line option parsers. Often, you want to read
a configuration file, but then override some of the values with command line
options. Our configuration system automates this process and allows each
command line option to be linked to a particular attribute in the
configuration hierarchy that it will override.
* Configuration files that are themselves valid Python code. This accomplishes
many things. First, it becomes possible to put logic in your configuration
files that sets attributes based on your operating system, network setup,
Python version, etc. Second, Python has a super simple syntax for accessing
hierarchical data structures, namely regular attribute access
(``Foo.Bar.Bam.name``). Third, using Python makes it easy for users to
import configuration attributes from one configuration file to another.
Thomas Kluyver
Update config docs so they don't refer to Str traitlet type.
r4066 Fourth, even though Python is dynamically typed, it does have types that can
Brian Granger
Major work on the documentation....
r2277 be checked at runtime. Thus, a ``1`` in a config file is the integer '1',
while a ``'1'`` is a string.
* A fully automated method for getting the configuration information to the
classes that need it at runtime. Writing code that walks a configuration
hierarchy to extract a particular attribute is painful. When you have
complex configuration information with hundreds of attributes, this makes
you want to cry.
* Type checking and validation that doesn't require the entire configuration
hierarchy to be specified statically before runtime. Python is a very
dynamic language and you don't always know everything that needs to be
configured when a program starts.
MinRK
update docs to reflect XDG support, new ~/.config/ipython default
r3351
.. _`XDG Base Directory`: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html