allow to load config from json file

json takes highest precedence and is loaded last

docs/source/whatsnew/pr/jsonconfig.rst

@@ -0,0 +1,3 b''
+* IPython config objects can be loaded from and serialized to JSON.
+  JSON config file have the same base name as their ``.py`` counterpart,
+  and will be loaded with higher priority if found.

IPython/config/application.py

@@ -1,582 +1,600 b''
 # encoding: utf-8
 """
 A base class for a configurable application.
 
 Authors:
 
 * Brian Granger
 * Min RK
 """
 from __future__ import print_function
 
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2008-2011  The IPython Development Team
 #
 #  Distributed under the terms of the BSD License.  The full license is in
 #  the file COPYING, distributed as part of this software.
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Imports
 #-----------------------------------------------------------------------------
 
 import logging
 import os
 import re
 import sys
 from copy import deepcopy
 from collections import defaultdict
 
 from IPython.external.decorator import decorator
 
 from IPython.config.configurable import SingletonConfigurable
 from IPython.config.loader import (
-    KVArgParseConfigLoader, PyFileConfigLoader, Config, ArgumentError, ConfigFileNotFound,
+    KVArgParseConfigLoader, PyFileConfigLoader, Config, ArgumentError, ConfigFileNotFound, JSONFileConfigLoader
 )
 
 from IPython.utils.traitlets import (
     Unicode, List, Enum, Dict, Instance, TraitError
 )
 from IPython.utils.importstring import import_item
 from IPython.utils.text import indent, wrap_paragraphs, dedent
 from IPython.utils import py3compat
 from IPython.utils.py3compat import string_types, iteritems
 
 #-----------------------------------------------------------------------------
 # function for re-wrapping a helpstring
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Descriptions for the various sections
 #-----------------------------------------------------------------------------
 
 # merge flags&aliases into options
 option_description = """
 Arguments that take values are actually convenience aliases to full
 Configurables, whose aliases are listed on the help line. For more information
 on full configurables, see '--help-all'.
 """.strip() # trim newlines of front and back
 
 keyvalue_description = """
 Parameters are set from command-line arguments of the form:
 `--Class.trait=value`.
 This line is evaluated in Python, so simple expressions are allowed, e.g.::
 `--C.a='range(3)'` For setting C.a=[0,1,2].
 """.strip() # trim newlines of front and back
 
 # sys.argv can be missing, for example when python is embedded. See the docs
 # for details: http://docs.python.org/2/c-api/intro.html#embedding-python
 if not hasattr(sys, "argv"):
     sys.argv = [""]
 
 subcommand_description = """
 Subcommands are launched as `{app} cmd [args]`. For information on using
 subcommand 'cmd', do: `{app} cmd -h`.
 """.strip().format(app=os.path.basename(sys.argv[0]))
 # get running program name
 
 #-----------------------------------------------------------------------------
 # Application class
 #-----------------------------------------------------------------------------
 
 @decorator
 def catch_config_error(method, app, *args, **kwargs):
     """Method decorator for catching invalid config (Trait/ArgumentErrors) during init.
 
     On a TraitError (generally caused by bad config), this will print the trait's
     message, and exit the app.
     
     For use on init methods, to prevent invoking excepthook on invalid input.
     """
     try:
         return method(app, *args, **kwargs)
     except (TraitError, ArgumentError) as e:
         app.print_help()
         app.log.fatal("Bad config encountered during initialization:")
         app.log.fatal(str(e))
         app.log.debug("Config at the time: %s", app.config)
         app.exit(1)
 
 
 class ApplicationError(Exception):
     pass
 
 class LevelFormatter(logging.Formatter):
     """Formatter with additional `highlevel` record
     
     This field is empty if log level is less than highlevel_limit,
     otherwise it is formatted with self.highlevel_format.
     
     Useful for adding 'WARNING' to warning messages,
     without adding 'INFO' to info, etc.
     """
     highlevel_limit = logging.WARN
     highlevel_format = " %(levelname)s |"
     
     def format(self, record):
         if record.levelno >= self.highlevel_limit:
             record.highlevel = self.highlevel_format % record.__dict__
         else:
             record.highlevel = ""
         return super(LevelFormatter, self).format(record)
             
 
 class Application(SingletonConfigurable):
     """A singleton application with full configuration support."""
 
     # The name of the application, will usually match the name of the command
     # line application
     name = Unicode(u'application')
 
     # The description of the application that is printed at the beginning
     # of the help.
     description = Unicode(u'This is an application.')
     # default section descriptions
     option_description = Unicode(option_description)
     keyvalue_description = Unicode(keyvalue_description)
     subcommand_description = Unicode(subcommand_description)
 
     # The usage and example string that goes at the end of the help string.
     examples = Unicode()
 
     # A sequence of Configurable subclasses whose config=True attributes will
     # be exposed at the command line.
     classes = List([])
 
     # The version string of this application.
     version = Unicode(u'0.0')
     
     # the argv used to initialize the application
     argv = List()
 
     # The log level for the application
     log_level = Enum((0,10,20,30,40,50,'DEBUG','INFO','WARN','ERROR','CRITICAL'),
                     default_value=logging.WARN,
                     config=True,
                     help="Set the log level by value or name.")
     def _log_level_changed(self, name, old, new):
         """Adjust the log level when log_level is set."""
         if isinstance(new, string_types):
             new = getattr(logging, new)
             self.log_level = new
         self.log.setLevel(new)
     
     log_datefmt = Unicode("%Y-%m-%d %H:%M:%S", config=True,
         help="The date format used by logging formatters for %(asctime)s"
     )
     def _log_datefmt_changed(self, name, old, new):
         self._log_format_changed()
     
     log_format = Unicode("[%(name)s]%(highlevel)s %(message)s", config=True,
         help="The Logging format template",
     )
     def _log_format_changed(self, name, old, new):
         """Change the log formatter when log_format is set."""
         _log_handler = self.log.handlers[0]
         _log_formatter = LevelFormatter(new, datefmt=self.log_datefmt)
         _log_handler.setFormatter(_log_formatter)
 
     log = Instance(logging.Logger)
     def _log_default(self):
         """Start logging for this application.
 
         The default is to log to stderr using a StreamHandler, if no default
         handler already exists.  The log level starts at logging.WARN, but this
         can be adjusted by setting the ``log_level`` attribute.
         """
         log = logging.getLogger(self.__class__.__name__)
         log.setLevel(self.log_level)
         log.propagate = False
         _log = log # copied from Logger.hasHandlers() (new in Python 3.2)
         while _log:
             if _log.handlers:
                 return log
             if not _log.propagate:
                 break
             else:
                 _log = _log.parent
         if sys.executable.endswith('pythonw.exe'):
             # this should really go to a file, but file-logging is only
             # hooked up in parallel applications
             _log_handler = logging.StreamHandler(open(os.devnull, 'w'))
         else:
             _log_handler = logging.StreamHandler()
         _log_formatter = LevelFormatter(self.log_format, datefmt=self.log_datefmt)
         _log_handler.setFormatter(_log_formatter)
         log.addHandler(_log_handler)
         return log
 
     # the alias map for configurables
     aliases = Dict({'log-level' : 'Application.log_level'})
 
     # flags for loading Configurables or store_const style flags
     # flags are loaded from this dict by '--key' flags
     # this must be a dict of two-tuples, the first element being the Config/dict
     # and the second being the help string for the flag
     flags = Dict()
     def _flags_changed(self, name, old, new):
         """ensure flags dict is valid"""
         for key,value in iteritems(new):
             assert len(value) == 2, "Bad flag: %r:%s"%(key,value)
             assert isinstance(value[0], (dict, Config)), "Bad flag: %r:%s"%(key,value)
             assert isinstance(value[1], string_types), "Bad flag: %r:%s"%(key,value)
 
 
     # subcommands for launching other applications
     # if this is not empty, this will be a parent Application
     # this must be a dict of two-tuples,
     # the first element being the application class/import string
     # and the second being the help string for the subcommand
     subcommands = Dict()
     # parse_command_line will initialize a subapp, if requested
     subapp = Instance('IPython.config.application.Application', allow_none=True)
 
     # extra command-line arguments that don't set config values
     extra_args = List(Unicode)
 
 
     def __init__(self, **kwargs):
         SingletonConfigurable.__init__(self, **kwargs)
         # Ensure my class is in self.classes, so my attributes appear in command line
         # options and config files.
         if self.__class__ not in self.classes:
             self.classes.insert(0, self.__class__)
 
     def _config_changed(self, name, old, new):
         SingletonConfigurable._config_changed(self, name, old, new)
         self.log.debug('Config changed:')
         self.log.debug(repr(new))
 
     @catch_config_error
     def initialize(self, argv=None):
         """Do the basic steps to configure me.
 
         Override in subclasses.
         """
         self.parse_command_line(argv)
 
 
     def start(self):
         """Start the app mainloop.
 
         Override in subclasses.
         """
         if self.subapp is not None:
             return self.subapp.start()
 
     def print_alias_help(self):
         """Print the alias part of the help."""
         if not self.aliases:
             return
 
         lines = []
         classdict = {}
         for cls in self.classes:
             # include all parents (up to, but excluding Configurable) in available names
             for c in cls.mro()[:-3]:
                 classdict[c.__name__] = c
 
         for alias, longname in iteritems(self.aliases):
             classname, traitname = longname.split('.',1)
             cls = classdict[classname]
 
             trait = cls.class_traits(config=True)[traitname]
             help = cls.class_get_trait_help(trait).splitlines()
             # reformat first line
             help[0] = help[0].replace(longname, alias) + ' (%s)'%longname
             if len(alias) == 1:
                 help[0] = help[0].replace('--%s='%alias, '-%s '%alias)
             lines.extend(help)
         # lines.append('')
         print(os.linesep.join(lines))
 
     def print_flag_help(self):
         """Print the flag part of the help."""
         if not self.flags:
             return
 
         lines = []
         for m, (cfg,help) in iteritems(self.flags):
             prefix = '--' if len(m) > 1 else '-'
             lines.append(prefix+m)
             lines.append(indent(dedent(help.strip())))
         # lines.append('')
         print(os.linesep.join(lines))
 
     def print_options(self):
         if not self.flags and not self.aliases:
             return
         lines = ['Options']
         lines.append('-'*len(lines[0]))
         lines.append('')
         for p in wrap_paragraphs(self.option_description):
             lines.append(p)
             lines.append('')
         print(os.linesep.join(lines))
         self.print_flag_help()
         self.print_alias_help()
         print()
 
     def print_subcommands(self):
         """Print the subcommand part of the help."""
         if not self.subcommands:
             return
 
         lines = ["Subcommands"]
         lines.append('-'*len(lines[0]))
         lines.append('')
         for p in wrap_paragraphs(self.subcommand_description):
             lines.append(p)
             lines.append('')
         for subc, (cls, help) in iteritems(self.subcommands):
             lines.append(subc)
             if help:
                 lines.append(indent(dedent(help.strip())))
         lines.append('')
         print(os.linesep.join(lines))
 
     def print_help(self, classes=False):
         """Print the help for each Configurable class in self.classes.
 
         If classes=False (the default), only flags and aliases are printed.
         """
         self.print_description()
         self.print_subcommands()
         self.print_options()
 
         if classes:
             if self.classes:
                 print("Class parameters")
                 print("----------------")
                 print()
                 for p in wrap_paragraphs(self.keyvalue_description):
                     print(p)
                     print()
 
             for cls in self.classes:
                 cls.class_print_help()
                 print()
         else:
             print("To see all available configurables, use `--help-all`")
             print()
 
         self.print_examples()
 
 
     def print_description(self):
         """Print the application description."""
         for p in wrap_paragraphs(self.description):
             print(p)
             print()
 
     def print_examples(self):
         """Print usage and examples.
 
         This usage string goes at the end of the command line help string
         and should contain examples of the application's usage.
         """
         if self.examples:
             print("Examples")
             print("--------")
             print()
             print(indent(dedent(self.examples.strip())))
             print()
 
     def print_version(self):
         """Print the version string."""
         print(self.version)
 
     def update_config(self, config):
         """Fire the traits events when the config is updated."""
         # Save a copy of the current config.
         newconfig = deepcopy(self.config)
         # Merge the new config into the current one.
         newconfig.merge(config)
         # Save the combined config as self.config, which triggers the traits
         # events.
         self.config = newconfig
 
     @catch_config_error
     def initialize_subcommand(self, subc, argv=None):
         """Initialize a subcommand with argv."""
         subapp,help = self.subcommands.get(subc)
 
         if isinstance(subapp, string_types):
             subapp = import_item(subapp)
 
         # clear existing instances
         self.__class__.clear_instance()
         # instantiate
         self.subapp = subapp.instance(config=self.config)
         # and initialize subapp
         self.subapp.initialize(argv)
     
     def flatten_flags(self):
         """flatten flags and aliases, so cl-args override as expected.
         
         This prevents issues such as an alias pointing to InteractiveShell,
         but a config file setting the same trait in TerminalInteraciveShell
         getting inappropriate priority over the command-line arg.
 
         Only aliases with exactly one descendent in the class list
         will be promoted.
         
         """
         # build a tree of classes in our list that inherit from a particular
         # it will be a dict by parent classname of classes in our list
         # that are descendents
         mro_tree = defaultdict(list)
         for cls in self.classes:
             clsname = cls.__name__
             for parent in cls.mro()[1:-3]:
                 # exclude cls itself and Configurable,HasTraits,object
                 mro_tree[parent.__name__].append(clsname)
         # flatten aliases, which have the form:
         # { 'alias' : 'Class.trait' }
         aliases = {}
         for alias, cls_trait in iteritems(self.aliases):
             cls,trait = cls_trait.split('.',1)
             children = mro_tree[cls]
             if len(children) == 1:
                 # exactly one descendent, promote alias
                 cls = children[0]
             aliases[alias] = '.'.join([cls,trait])
         
         # flatten flags, which are of the form:
         # { 'key' : ({'Cls' : {'trait' : value}}, 'help')}
         flags = {}
         for key, (flagdict, help) in iteritems(self.flags):
             newflag = {}
             for cls, subdict in iteritems(flagdict):
                 children = mro_tree[cls]
                 # exactly one descendent, promote flag section
                 if len(children) == 1:
                     cls = children[0]
                 newflag[cls] = subdict
             flags[key] = (newflag, help)
         return flags, aliases
 
     @catch_config_error
     def parse_command_line(self, argv=None):
         """Parse the command line arguments."""
         argv = sys.argv[1:] if argv is None else argv
         self.argv = [ py3compat.cast_unicode(arg) for arg in argv ]
         
         if argv and argv[0] == 'help':
             # turn `ipython help notebook` into `ipython notebook -h`
             argv = argv[1:] + ['-h']
 
         if self.subcommands and len(argv) > 0:
             # we have subcommands, and one may have been specified
             subc, subargv = argv[0], argv[1:]
             if re.match(r'^\w(\-?\w)*$', subc) and subc in self.subcommands:
                 # it's a subcommand, and *not* a flag or class parameter
                 return self.initialize_subcommand(subc, subargv)
 
         # Arguments after a '--' argument are for the script IPython may be
         # about to run, not IPython iteslf. For arguments parsed here (help and
         # version), we want to only search the arguments up to the first
         # occurrence of '--', which we're calling interpreted_argv.
         try:
             interpreted_argv = argv[:argv.index('--')]
         except ValueError:
             interpreted_argv = argv
 
         if any(x in interpreted_argv for x in ('-h', '--help-all', '--help')):
             self.print_help('--help-all' in interpreted_argv)
             self.exit(0)
 
         if '--version' in interpreted_argv or '-V' in interpreted_argv:
             self.print_version()
             self.exit(0)
         
         # flatten flags&aliases, so cl-args get appropriate priority:
         flags,aliases = self.flatten_flags()
-
         loader = KVArgParseConfigLoader(argv=argv, aliases=aliases,
-                                        flags=flags)
+                                        flags=flags, log=self.log)
         config = loader.load_config()
         self.update_config(config)
         # store unparsed args in extra_args
         self.extra_args = loader.extra_args
 
+    @classmethod
+    def _load_config_file(cls, basefilename, path=None, log=None):
+        """Load config files (json/py) by filename and path."""
+
+        pyloader = PyFileConfigLoader(basefilename+'.py', path=path, log=log)
+        jsonloader = JSONFileConfigLoader(basefilename+'.json', path=path, log=log)
+        config_found = False
+        config = None
+        for loader in [pyloader, jsonloader]:
+            try:
+                config = loader.load_config()
+                config_found = True
+            except ConfigFileNotFound:
+                pass
+            except Exception:
+                # try to get the full filename, but it will be empty in the
+                # unlikely event that the error raised before filefind finished
+                filename = loader.full_filename or filename
+                # problem while running the file
+                log.error("Exception while loading config file %s",
+                                filename, exc_info=True)
+            else:
+                log.debug("Loaded config file: %s", loader.full_filename)
+            if config :
+                 yield config
+
+        if not config_found :
+            raise ConfigFileNotFound('Neither .json, not .py file found.')
+        raise StopIteration
+
+
     @catch_config_error
     def load_config_file(self, filename, path=None):
-        """Load a .py based config file by filename and path."""
-        loader = PyFileConfigLoader(filename, path=path)
-        try:
-            config = loader.load_config()
-        except ConfigFileNotFound:
-            # problem finding the file, raise
-            raise
-        except Exception:
-            # try to get the full filename, but it will be empty in the
-            # unlikely event that the error raised before filefind finished
-            filename = loader.full_filename or filename
-            # problem while running the file
-            self.log.error("Exception while loading config file %s",
-                            filename, exc_info=True)
-        else:
-            self.log.debug("Loaded config file: %s", loader.full_filename)
+        """Load config files (json/py) by filename and path."""
+        filename, ext = os.path.splitext(filename)
+        for config in self._load_config_file(filename, path=path , log=self.log):
             self.update_config(config)
 
+
     def generate_config_file(self):
         """generate default config file from Configurables"""
         lines = ["# Configuration file for %s."%self.name]
         lines.append('')
         lines.append('c = get_config()')
         lines.append('')
         for cls in self.classes:
             lines.append(cls.class_config_section())
         return '\n'.join(lines)
 
     def exit(self, exit_status=0):
         self.log.debug("Exiting application: %s" % self.name)
         sys.exit(exit_status)
 
     @classmethod
     def launch_instance(cls, argv=None, **kwargs):
         """Launch a global instance of this Application
         
         If a global instance already exists, this reinitializes and starts it
         """
         app = cls.instance(**kwargs)
         app.initialize(argv)
         app.start()
 
 #-----------------------------------------------------------------------------
 # utility functions, for convenience
 #-----------------------------------------------------------------------------
 
 def boolean_flag(name, configurable, set_help='', unset_help=''):
     """Helper for building basic --trait, --no-trait flags.
 
     Parameters
     ----------
 
     name : str
         The name of the flag.
     configurable : str
         The 'Class.trait' string of the trait to be set/unset with the flag
     set_help : unicode
         help string for --name flag
     unset_help : unicode
         help string for --no-name flag
 
     Returns
     -------
 
     cfg : dict
         A dict with two keys: 'name', and 'no-name', for setting and unsetting
         the trait, respectively.
     """
     # default helpstrings
     set_help = set_help or "set %s=True"%configurable
     unset_help = unset_help or "set %s=False"%configurable
 
     cls,trait = configurable.split('.')
 
     setter = {cls : {trait : True}}
     unsetter = {cls : {trait : False}}
     return {name : (setter, set_help), 'no-'+name : (unsetter, unset_help)}
 

IPython/config/loader.py

@@ -1,802 +1,841 b''
 """A simple configuration system.
 
 Inheritance diagram:
 
 .. inheritance-diagram:: IPython.config.loader
    :parts: 3
 
 Authors
 -------
 * Brian Granger
 * Fernando Perez
 * Min RK
 """
 
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2008-2011  The IPython Development Team
 #
 #  Distributed under the terms of the BSD License.  The full license is in
 #  the file COPYING, distributed as part of this software.
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Imports
 #-----------------------------------------------------------------------------
 
 import argparse
 import copy
 import os
 import re
 import sys
+import json
 
 from IPython.utils.path import filefind, get_ipython_dir
-from IPython.utils import py3compat, warn
+from IPython.utils import py3compat
 from IPython.utils.encoding import DEFAULT_ENCODING
 from IPython.utils.py3compat import unicode_type, iteritems
 from IPython.utils.traitlets import HasTraits, List, Any, TraitError
 
 #-----------------------------------------------------------------------------
 # Exceptions
 #-----------------------------------------------------------------------------
 
 
 class ConfigError(Exception):
     pass
 
 class ConfigLoaderError(ConfigError):
     pass
 
 class ConfigFileNotFound(ConfigError):
     pass
 
 class ArgumentError(ConfigLoaderError):
     pass
 
 #-----------------------------------------------------------------------------
 # Argparse fix
 #-----------------------------------------------------------------------------
 
 # Unfortunately argparse by default prints help messages to stderr instead of
 # stdout.  This makes it annoying to capture long help screens at the command
 # line, since one must know how to pipe stderr, which many users don't know how
 # to do.  So we override the print_help method with one that defaults to
 # stdout and use our class instead.
 
 class ArgumentParser(argparse.ArgumentParser):
     """Simple argparse subclass that prints help to stdout by default."""
 
     def print_help(self, file=None):
         if file is None:
             file = sys.stdout
         return super(ArgumentParser, self).print_help(file)
 
     print_help.__doc__ = argparse.ArgumentParser.print_help.__doc__
 
 #-----------------------------------------------------------------------------
 # Config class for holding config information
 #-----------------------------------------------------------------------------
 
 class LazyConfigValue(HasTraits):
     """Proxy object for exposing methods on configurable containers
     
     Exposes:
     
     - append, extend, insert on lists
     - update on dicts
     - update, add on sets
     """
     
     _value = None
     
     # list methods
     _extend = List()
     _prepend = List()
     
     def append(self, obj):
         self._extend.append(obj)
     
     def extend(self, other):
         self._extend.extend(other)
     
     def prepend(self, other):
         """like list.extend, but for the front"""
         self._prepend[:0] = other
     
     _inserts = List()
     def insert(self, index, other):
         if not isinstance(index, int):
             raise TypeError("An integer is required")
         self._inserts.append((index, other))
     
     # dict methods
     # update is used for both dict and set
     _update = Any()
     def update(self, other):
         if self._update is None:
             if isinstance(other, dict):
                 self._update = {}
             else:
                 self._update = set()
         self._update.update(other)
     
     # set methods
     def add(self, obj):
         self.update({obj})
     
     def get_value(self, initial):
         """construct the value from the initial one
         
         after applying any insert / extend / update changes
         """
         if self._value is not None:
             return self._value
         value = copy.deepcopy(initial)
         if isinstance(value, list):
             for idx, obj in self._inserts:
                 value.insert(idx, obj)
             value[:0] = self._prepend
             value.extend(self._extend)
         
         elif isinstance(value, dict):
             if self._update:
                 value.update(self._update)
         elif isinstance(value, set):
             if self._update:
                 value.update(self._update)
         self._value = value
         return value
     
     def to_dict(self):
         """return JSONable dict form of my data
         
         Currently update as dict or set, extend, prepend as lists, and inserts as list of tuples.
         """
         d = {}
         if self._update:
             d['update'] = self._update
         if self._extend:
             d['extend'] = self._extend
         if self._prepend:
             d['prepend'] = self._prepend
         elif self._inserts:
             d['inserts'] = self._inserts
         return d
 
 
 def _is_section_key(key):
     """Is a Config key a section name (does it start with a capital)?"""
     if key and key[0].upper()==key[0] and not key.startswith('_'):
         return True
     else:
         return False
 
 
 class Config(dict):
     """An attribute based dict that can do smart merges."""
 
     def __init__(self, *args, **kwds):
         dict.__init__(self, *args, **kwds)
         self._ensure_subconfig()
     
     def _ensure_subconfig(self):
         """ensure that sub-dicts that should be Config objects are
         
         casts dicts that are under section keys to Config objects,
         which is necessary for constructing Config objects from dict literals.
         """
         for key in self:
             obj = self[key]
             if _is_section_key(key) \
                     and isinstance(obj, dict) \
                     and not isinstance(obj, Config):
                 setattr(self, key, Config(obj))
     
     def _merge(self, other):
         """deprecated alias, use Config.merge()"""
         self.merge(other)
     
     def merge(self, other):
         """merge another config object into this one"""
         to_update = {}
         for k, v in iteritems(other):
             if k not in self:
                 to_update[k] = copy.deepcopy(v)
             else: # I have this key
                 if isinstance(v, Config) and isinstance(self[k], Config):
                     # Recursively merge common sub Configs
                     self[k].merge(v)
                 else:
                     # Plain updates for non-Configs
                     to_update[k] = copy.deepcopy(v)
 
         self.update(to_update)
 
     def __contains__(self, key):
         # allow nested contains of the form `"Section.key" in config`
         if '.' in key:
             first, remainder = key.split('.', 1)
             if first not in self:
                 return False
             return remainder in self[first]
         
         return super(Config, self).__contains__(key)
     
     # .has_key is deprecated for dictionaries.
     has_key = __contains__
     
     def _has_section(self, key):
         return _is_section_key(key) and key in self
     
     def copy(self):
         return type(self)(dict.copy(self))
 
     def __copy__(self):
         return self.copy()
 
     def __deepcopy__(self, memo):
         import copy
         return type(self)(copy.deepcopy(list(self.items())))
     
     def __getitem__(self, key):
         try:
             return dict.__getitem__(self, key)
         except KeyError:
             if _is_section_key(key):
                 c = Config()
                 dict.__setitem__(self, key, c)
                 return c
             else:
                 # undefined, create lazy value, used for container methods
                 v = LazyConfigValue()
                 dict.__setitem__(self, key, v)
                 return v
 
     def __setitem__(self, key, value):
         if _is_section_key(key):
             if not isinstance(value, Config):
                 raise ValueError('values whose keys begin with an uppercase '
                                  'char must be Config instances: %r, %r' % (key, value))
         dict.__setitem__(self, key, value)
 
     def __getattr__(self, key):
         if key.startswith('__'):
             return dict.__getattr__(self, key)
         try:
             return self.__getitem__(key)
         except KeyError as e:
             raise AttributeError(e)
 
     def __setattr__(self, key, value):
         if key.startswith('__'):
             return dict.__setattr__(self, key, value)
         try:
             self.__setitem__(key, value)
         except KeyError as e:
             raise AttributeError(e)
 
     def __delattr__(self, key):
         if key.startswith('__'):
             return dict.__delattr__(self, key)
         try:
             dict.__delitem__(self, key)
         except KeyError as e:
             raise AttributeError(e)
 
 
 #-----------------------------------------------------------------------------
 # Config loading classes
 #-----------------------------------------------------------------------------
 
 
 class ConfigLoader(object):
     """A object for loading configurations from just about anywhere.
 
     The resulting configuration is packaged as a :class:`Struct`.
 
     Notes
     -----
     A :class:`ConfigLoader` does one thing: load a config from a source
     (file, command line arguments) and returns the data as a :class:`Struct`.
     There are lots of things that :class:`ConfigLoader` does not do.  It does
     not implement complex logic for finding config files.  It does not handle
     default values or merge multiple configs.  These things need to be
     handled elsewhere.
     """
 
-    def __init__(self):
+    def _log_default(self):
+        from IPython.config.application import Application
+        return Application.instance().log
+
+    def __init__(self, log=None):
         """A base class for config loaders.
 
+        log : instance of :class:`logging.Logger` to use.
+              By default loger of :meth:`IPython.config.application.Application.instance()`
+              will be used
+
         Examples
         --------
 
         >>> cl = ConfigLoader()
         >>> config = cl.load_config()
         >>> config
         {}
         """
         self.clear()
+        if log is None :
+            self.log = self._log_default()
+            self.log.debug('Using default logger')
+        else :
+            self.log = log
 
     def clear(self):
         self.config = Config()
 
     def load_config(self):
         """Load a config from somewhere, return a :class:`Config` instance.
 
         Usually, this will cause self.config to be set and then returned.
         However, in most cases, :meth:`ConfigLoader.clear` should be called
         to erase any previous state.
         """
         self.clear()
         return self.config
 
 
 class FileConfigLoader(ConfigLoader):
     """A base class for file based configurations.
 
     As we add more file based config loaders, the common logic should go
     here.
     """
-    pass
-
-
-class PyFileConfigLoader(FileConfigLoader):
-    """A config loader for pure python files.
-
-    This calls execfile on a plain python file and looks for attributes
-    that are all caps.  These attribute are added to the config Struct.
-    """
 
-    def __init__(self, filename, path=None):
+    def __init__(self, filename, path=None, **kw):
         """Build a config loader for a filename and path.
 
         Parameters
         ----------
         filename : str
             The file name of the config file.
         path : str, list, tuple
             The path to search for the config file on, or a sequence of
             paths to try in order.
         """
-        super(PyFileConfigLoader, self).__init__()
+        super(FileConfigLoader, self).__init__(**kw)
         self.filename = filename
         self.path = path
         self.full_filename = ''
-        self.data = None
+
+    def _find_file(self):
+        """Try to find the file by searching the paths."""
+        self.full_filename = filefind(self.filename, self.path)
+
+class JSONFileConfigLoader(FileConfigLoader):
+    """A Json file loader for config"""
+
+    def load_config(self):
+        """Load the config from a file and return it as a Struct."""
+        self.clear()
+        try:
+            self._find_file()
+        except IOError as e:
+            raise ConfigFileNotFound(str(e))
+        dct = self._read_file_as_dict()
+        self.config = self._convert_to_config(dct)
+        return self.config
+
+    def _read_file_as_dict(self):
+        with open(self.full_filename) as f :
+            return json.load(f)
+
+    def _convert_to_config(self, dictionary):
+        if 'version' in dictionary:
+            version = dictionary.pop('version')
+        else :
+            version = 1
+            self.log.warn("Unrecognized JSON config file version, assuming version : {}".format(version))
+
+        if version == 1:
+            return Config(dictionary)
+        else :
+            raise ValueError('Unknown version of JSON config file : version number {version}'.format(version=version))
+
+
+class PyFileConfigLoader(FileConfigLoader):
+    """A config loader for pure python files.
+
+    This is responsible for locating a Python config file by filename and
+    profile name, then executing it in a namespace where it could have access
+    to subconfigs.
+    """
 
     def load_config(self):
         """Load the config from a file and return it as a Struct."""
         self.clear()
         try:
             self._find_file()
         except IOError as e:
             raise ConfigFileNotFound(str(e))
         self._read_file_as_dict()
-        self._convert_to_config()
         return self.config
 
-    def _find_file(self):
-        """Try to find the file by searching the paths."""
-        self.full_filename = filefind(self.filename, self.path)
 
     def _read_file_as_dict(self):
         """Load the config file into self.config, with recursive loading."""
         # This closure is made available in the namespace that is used
         # to exec the config file.  It allows users to call
         # load_subconfig('myconfig.py') to load config files recursively.
         # It needs to be a closure because it has references to self.path
         # and self.config.  The sub-config is loaded with the same path
         # as the parent, but it uses an empty config which is then merged
         # with the parents.
 
         # If a profile is specified, the config file will be loaded
         # from that profile
 
         def load_subconfig(fname, profile=None):
             # import here to prevent circular imports
             from IPython.core.profiledir import ProfileDir, ProfileDirError
             if profile is not None:
                 try:
                     profile_dir = ProfileDir.find_profile_dir_by_name(
                             get_ipython_dir(),
                             profile,
                     )
                 except ProfileDirError:
                     return
                 path = profile_dir.location
             else:
                 path = self.path
             loader = PyFileConfigLoader(fname, path)
             try:
                 sub_config = loader.load_config()
             except ConfigFileNotFound:
                 # Pass silently if the sub config is not there. This happens
                 # when a user s using a profile, but not the default config.
                 pass
             else:
                 self.config.merge(sub_config)
 
         # Again, this needs to be a closure and should be used in config
         # files to get the config being loaded.
         def get_config():
             return self.config
 
         namespace = dict(
             load_subconfig=load_subconfig,
             get_config=get_config,
             __file__=self.full_filename,
         )
         fs_encoding = sys.getfilesystemencoding() or 'ascii'
         conf_filename = self.full_filename.encode(fs_encoding)
         py3compat.execfile(conf_filename, namespace)
 
-    def _convert_to_config(self):
-        if self.data is None:
-            ConfigLoaderError('self.data does not exist')
-
 
 class CommandLineConfigLoader(ConfigLoader):
     """A config loader for command line arguments.
 
     As we add more command line based loaders, the common logic should go
     here.
     """
 
     def _exec_config_str(self, lhs, rhs):
         """execute self.config.<lhs> = <rhs>
         
         * expands ~ with expanduser
         * tries to assign with raw eval, otherwise assigns with just the string,
           allowing `--C.a=foobar` and `--C.a="foobar"` to be equivalent.  *Not*
           equivalent are `--C.a=4` and `--C.a='4'`.
         """
         rhs = os.path.expanduser(rhs)
         try:
             # Try to see if regular Python syntax will work. This
             # won't handle strings as the quote marks are removed
             # by the system shell.
             value = eval(rhs)
         except (NameError, SyntaxError):
             # This case happens if the rhs is a string.
             value = rhs
 
         exec(u'self.config.%s = value' % lhs)
 
     def _load_flag(self, cfg):
         """update self.config from a flag, which can be a dict or Config"""
         if isinstance(cfg, (dict, Config)):
             # don't clobber whole config sections, update
             # each section from config:
             for sec,c in iteritems(cfg):
                 self.config[sec].update(c)
         else:
             raise TypeError("Invalid flag: %r" % cfg)
 
 # raw --identifier=value pattern
 # but *also* accept '-' as wordsep, for aliases
 # accepts:  --foo=a
 #           --Class.trait=value
 #           --alias-name=value
 # rejects:  -foo=value
 #           --foo
 #           --Class.trait
 kv_pattern = re.compile(r'\-\-[A-Za-z][\w\-]*(\.[\w\-]+)*\=.*')
 
 # just flags, no assignments, with two *or one* leading '-'
 # accepts:  --foo
 #           -foo-bar-again
 # rejects:  --anything=anything
 #           --two.word
 
 flag_pattern = re.compile(r'\-\-?\w+[\-\w]*$')
 
 class KeyValueConfigLoader(CommandLineConfigLoader):
     """A config loader that loads key value pairs from the command line.
 
     This allows command line options to be gives in the following form::
 
         ipython --profile="foo" --InteractiveShell.autocall=False
     """
 
-    def __init__(self, argv=None, aliases=None, flags=None):
+    def __init__(self, argv=None, aliases=None, flags=None, **kw):
         """Create a key value pair config loader.
 
         Parameters
         ----------
         argv : list
             A list that has the form of sys.argv[1:] which has unicode
             elements of the form u"key=value". If this is None (default),
             then sys.argv[1:] will be used.
         aliases : dict
             A dict of aliases for configurable traits.
             Keys are the short aliases, Values are the resolved trait.
             Of the form: `{'alias' : 'Configurable.trait'}`
         flags : dict
             A dict of flags, keyed by str name. Vaues can be Config objects,
             dicts, or "key=value" strings.  If Config or dict, when the flag
             is triggered, The flag is loaded as `self.config.update(m)`.
 
         Returns
         -------
         config : Config
             The resulting Config object.
 
         Examples
         --------
 
             >>> from IPython.config.loader import KeyValueConfigLoader
             >>> cl = KeyValueConfigLoader()
             >>> d = cl.load_config(["--A.name='brian'","--B.number=0"])
             >>> sorted(d.items())
             [('A', {'name': 'brian'}), ('B', {'number': 0})]
         """
-        self.clear()
+        super(KeyValueConfigLoader, self).__init__(**kw)
         if argv is None:
             argv = sys.argv[1:]
         self.argv = argv
         self.aliases = aliases or {}
         self.flags = flags or {}
 
 
     def clear(self):
         super(KeyValueConfigLoader, self).clear()
         self.extra_args = []
 
 
     def _decode_argv(self, argv, enc=None):
         """decode argv if bytes, using stin.encoding, falling back on default enc"""
         uargv = []
         if enc is None:
             enc = DEFAULT_ENCODING
         for arg in argv:
             if not isinstance(arg, unicode_type):
                 # only decode if not already decoded
                 arg = arg.decode(enc)
             uargv.append(arg)
         return uargv
 
 
     def load_config(self, argv=None, aliases=None, flags=None):
         """Parse the configuration and generate the Config object.
 
         After loading, any arguments that are not key-value or
         flags will be stored in self.extra_args - a list of
         unparsed command-line arguments.  This is used for
         arguments such as input files or subcommands.
 
         Parameters
         ----------
         argv : list, optional
             A list that has the form of sys.argv[1:] which has unicode
             elements of the form u"key=value". If this is None (default),
             then self.argv will be used.
         aliases : dict
             A dict of aliases for configurable traits.
             Keys are the short aliases, Values are the resolved trait.
             Of the form: `{'alias' : 'Configurable.trait'}`
         flags : dict
             A dict of flags, keyed by str name. Values can be Config objects
             or dicts.  When the flag is triggered, The config is loaded as
             `self.config.update(cfg)`.
         """
         self.clear()
         if argv is None:
             argv = self.argv
         if aliases is None:
             aliases = self.aliases
         if flags is None:
             flags = self.flags
 
         # ensure argv is a list of unicode strings:
         uargv = self._decode_argv(argv)
         for idx,raw in enumerate(uargv):
             # strip leading '-'
             item = raw.lstrip('-')
 
             if raw == '--':
                 # don't parse arguments after '--'
                 # this is useful for relaying arguments to scripts, e.g.
                 # ipython -i foo.py --matplotlib=qt -- args after '--' go-to-foo.py
                 self.extra_args.extend(uargv[idx+1:])
                 break
 
             if kv_pattern.match(raw):
                 lhs,rhs = item.split('=',1)
                 # Substitute longnames for aliases.
                 if lhs in aliases:
                     lhs = aliases[lhs]
                 if '.' not in lhs:
                     # probably a mistyped alias, but not technically illegal
-                    warn.warn("Unrecognized alias: '%s', it will probably have no effect."%lhs)
+                    self.log.warn("Unrecognized alias: '%s', it will probably have no effect. %s,-- %s"%(lhs,raw, aliases))
                 try:
                     self._exec_config_str(lhs, rhs)
                 except Exception:
                     raise ArgumentError("Invalid argument: '%s'" % raw)
 
             elif flag_pattern.match(raw):
                 if item in flags:
                     cfg,help = flags[item]
                     self._load_flag(cfg)
                 else:
                     raise ArgumentError("Unrecognized flag: '%s'"%raw)
             elif raw.startswith('-'):
                 kv = '--'+item
                 if kv_pattern.match(kv):
                     raise ArgumentError("Invalid argument: '%s', did you mean '%s'?"%(raw, kv))
                 else:
                     raise ArgumentError("Invalid argument: '%s'"%raw)
             else:
                 # keep all args that aren't valid in a list,
                 # in case our parent knows what to do with them.
                 self.extra_args.append(item)
         return self.config
 
 class ArgParseConfigLoader(CommandLineConfigLoader):
     """A loader that uses the argparse module to load from the command line."""
 
-    def __init__(self, argv=None, aliases=None, flags=None, *parser_args, **parser_kw):
+    def __init__(self, argv=None, aliases=None, flags=None, log=None,  *parser_args, **parser_kw):
         """Create a config loader for use with argparse.
 
         Parameters
         ----------
 
         argv : optional, list
           If given, used to read command-line arguments from, otherwise
           sys.argv[1:] is used.
 
         parser_args : tuple
           A tuple of positional arguments that will be passed to the
           constructor of :class:`argparse.ArgumentParser`.
 
         parser_kw : dict
           A tuple of keyword arguments that will be passed to the
           constructor of :class:`argparse.ArgumentParser`.
 
         Returns
         -------
         config : Config
             The resulting Config object.
         """
-        super(CommandLineConfigLoader, self).__init__()
+        super(CommandLineConfigLoader, self).__init__(log=log)
         self.clear()
         if argv is None:
             argv = sys.argv[1:]
         self.argv = argv
         self.aliases = aliases or {}
         self.flags = flags or {}
 
         self.parser_args = parser_args
         self.version = parser_kw.pop("version", None)
         kwargs = dict(argument_default=argparse.SUPPRESS)
         kwargs.update(parser_kw)
         self.parser_kw = kwargs
 
     def load_config(self, argv=None, aliases=None, flags=None):
         """Parse command line arguments and return as a Config object.
 
         Parameters
         ----------
 
         args : optional, list
           If given, a list with the structure of sys.argv[1:] to parse
           arguments from. If not given, the instance's self.argv attribute
           (given at construction time) is used."""
         self.clear()
         if argv is None:
             argv = self.argv
         if aliases is None:
             aliases = self.aliases
         if flags is None:
             flags = self.flags
         self._create_parser(aliases, flags)
         self._parse_args(argv)
         self._convert_to_config()
         return self.config
 
     def get_extra_args(self):
         if hasattr(self, 'extra_args'):
             return self.extra_args
         else:
             return []
 
     def _create_parser(self, aliases=None, flags=None):
         self.parser = ArgumentParser(*self.parser_args, **self.parser_kw)
         self._add_arguments(aliases, flags)
 
     def _add_arguments(self, aliases=None, flags=None):
         raise NotImplementedError("subclasses must implement _add_arguments")
 
     def _parse_args(self, args):
         """self.parser->self.parsed_data"""
         # decode sys.argv to support unicode command-line options
         enc = DEFAULT_ENCODING
         uargs = [py3compat.cast_unicode(a, enc) for a in args]
         self.parsed_data, self.extra_args = self.parser.parse_known_args(uargs)
 
     def _convert_to_config(self):
         """self.parsed_data->self.config"""
         for k, v in iteritems(vars(self.parsed_data)):
             exec("self.config.%s = v"%k, locals(), globals())
 
 class KVArgParseConfigLoader(ArgParseConfigLoader):
     """A config loader that loads aliases and flags with argparse,
     but will use KVLoader for the rest.  This allows better parsing
     of common args, such as `ipython -c 'print 5'`, but still gets
     arbitrary config with `ipython --InteractiveShell.use_readline=False`"""
 
     def _add_arguments(self, aliases=None, flags=None):
         self.alias_flags = {}
         # print aliases, flags
         if aliases is None:
             aliases = self.aliases
         if flags is None:
             flags = self.flags
         paa = self.parser.add_argument
         for key,value in iteritems(aliases):
             if key in flags:
                 # flags
                 nargs = '?'
             else:
                 nargs = None
             if len(key) is 1:
                 paa('-'+key, '--'+key, type=unicode_type, dest=value, nargs=nargs)
             else:
                 paa('--'+key, type=unicode_type, dest=value, nargs=nargs)
         for key, (value, help) in iteritems(flags):
             if key in self.aliases:
                 #
                 self.alias_flags[self.aliases[key]] = value
                 continue
             if len(key) is 1:
                 paa('-'+key, '--'+key, action='append_const', dest='_flags', const=value)
             else:
                 paa('--'+key, action='append_const', dest='_flags', const=value)
 
     def _convert_to_config(self):
         """self.parsed_data->self.config, parse unrecognized extra args via KVLoader."""
         # remove subconfigs list from namespace before transforming the Namespace
         if '_flags' in self.parsed_data:
             subcs = self.parsed_data._flags
             del self.parsed_data._flags
         else:
             subcs = []
 
         for k, v in iteritems(vars(self.parsed_data)):
             if v is None:
                 # it was a flag that shares the name of an alias
                 subcs.append(self.alias_flags[k])
             else:
                 # eval the KV assignment
                 self._exec_config_str(k, v)
 
         for subc in subcs:
             self._load_flag(subc)
 
         if self.extra_args:
-            sub_parser = KeyValueConfigLoader()
+            sub_parser = KeyValueConfigLoader(log=self.log)
             sub_parser.load_config(self.extra_args)
             self.config.merge(sub_parser.config)
             self.extra_args = sub_parser.extra_args
 
 
 def load_pyconfig_files(config_files, path):
     """Load multiple Python config files, merging each of them in turn.
 
     Parameters
     ==========
     config_files : list of str
         List of config files names to load and merge into the config.
     path : unicode
         The full path to the location of the config files.
     """
     config = Config()
     for cf in config_files:
         loader = PyFileConfigLoader(cf, path=path)
         try:
             next_config = loader.load_config()
         except ConfigFileNotFound:
             pass
         except:
             raise
         else:
             config.merge(next_config)
     return config

IPython/config/tests/test_loader.py

@@ -1,337 +1,388 b''
 # encoding: utf-8
 """
 Tests for IPython.config.loader
 
 Authors:
 
 * Brian Granger
 * Fernando Perez (design help)
 """
 
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2008 The IPython Development Team
 #
 #  Distributed under the terms of the BSD License.  The full license is in
 #  the file COPYING, distributed as part of this software.
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Imports
 #-----------------------------------------------------------------------------
 
 import os
 import pickle
 import sys
+import json
+
 from tempfile import mkstemp
 from unittest import TestCase
 
 from nose import SkipTest
+import nose.tools as nt
+
 
-from IPython.testing.tools import mute_warn
 
 from IPython.config.loader import (
     Config,
     LazyConfigValue,
     PyFileConfigLoader,
+    JSONFileConfigLoader,
     KeyValueConfigLoader,
     ArgParseConfigLoader,
     KVArgParseConfigLoader,
     ConfigError,
 )
 
 #-----------------------------------------------------------------------------
 # Actual tests
 #-----------------------------------------------------------------------------
 
 
 pyfile = """
 c = get_config()
 c.a=10
 c.b=20
 c.Foo.Bar.value=10
 c.Foo.Bam.value=list(range(10))  # list() is just so it's the same on Python 3
 c.D.C.value='hi there'
 """
 
-class TestPyFileCL(TestCase):
+json1file = """
+{
+  "version": 1,
+  "a": 10,
+  "b": 20,
+  "Foo": {
+    "Bam": {
+      "value": [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
+    },
+    "Bar": {
+      "value": 10
+    }
+  },
+  "D": {
+    "C": {
+      "value": "hi there"
+    }
+  }
+}
+"""
 
-    def test_basic(self):
+# should not load
+json2file = """
+{
+  "version": 2
+}
+"""
+
+import logging
+log = logging.getLogger('devnull')
+log.setLevel(0)
+
+class TestFileCL(TestCase):
+
+    def _check_conf(self, config):
+        self.assertEqual(config.a, 10)
+        self.assertEqual(config.b, 20)
+        self.assertEqual(config.Foo.Bar.value, 10)
+        self.assertEqual(config.Foo.Bam.value, list(range(10)))
+        self.assertEqual(config.D.C.value, 'hi there')
+
+    def test_python(self):
         fd, fname = mkstemp('.py')
         f = os.fdopen(fd, 'w')
         f.write(pyfile)
         f.close()
         # Unlink the file
-        cl = PyFileConfigLoader(fname)
+        cl = PyFileConfigLoader(fname, log=log)
         config = cl.load_config()
-        self.assertEqual(config.a, 10)
-        self.assertEqual(config.b, 20)
-        self.assertEqual(config.Foo.Bar.value, 10)
-        self.assertEqual(config.Foo.Bam.value, list(range(10)))
-        self.assertEqual(config.D.C.value, 'hi there')
+        self._check_conf(config)
+
+    def test_json(self):
+        fd, fname = mkstemp('.json')
+        f = os.fdopen(fd, 'w')
+        f.write(json1file)
+        f.close()
+        # Unlink the file
+        cl = JSONFileConfigLoader(fname, log=log)
+        config = cl.load_config()
+        self._check_conf(config)
+
+    def test_v2raise(self):
+        fd, fname = mkstemp('.json')
+        f = os.fdopen(fd, 'w')
+        f.write(json2file)
+        f.close()
+        # Unlink the file
+        cl = JSONFileConfigLoader(fname, log=log)
+        with nt.assert_raises(ValueError):
+            cl.load_config()
+
 
 class MyLoader1(ArgParseConfigLoader):
     def _add_arguments(self, aliases=None, flags=None):
         p = self.parser
         p.add_argument('-f', '--foo', dest='Global.foo', type=str)
         p.add_argument('-b', dest='MyClass.bar', type=int)
         p.add_argument('-n', dest='n', action='store_true')
         p.add_argument('Global.bam', type=str)
 
 class MyLoader2(ArgParseConfigLoader):
     def _add_arguments(self, aliases=None, flags=None):
         subparsers = self.parser.add_subparsers(dest='subparser_name')
         subparser1 = subparsers.add_parser('1')
         subparser1.add_argument('-x',dest='Global.x')
         subparser2 = subparsers.add_parser('2')
         subparser2.add_argument('y')
 
 class TestArgParseCL(TestCase):
 
     def test_basic(self):
         cl = MyLoader1()
         config = cl.load_config('-f hi -b 10 -n wow'.split())
         self.assertEqual(config.Global.foo, 'hi')
         self.assertEqual(config.MyClass.bar, 10)
         self.assertEqual(config.n, True)
         self.assertEqual(config.Global.bam, 'wow')
         config = cl.load_config(['wow'])
         self.assertEqual(list(config.keys()), ['Global'])
         self.assertEqual(list(config.Global.keys()), ['bam'])
         self.assertEqual(config.Global.bam, 'wow')
 
     def test_add_arguments(self):
         cl = MyLoader2()
         config = cl.load_config('2 frobble'.split())
         self.assertEqual(config.subparser_name, '2')
         self.assertEqual(config.y, 'frobble')
         config = cl.load_config('1 -x frobble'.split())
         self.assertEqual(config.subparser_name, '1')
         self.assertEqual(config.Global.x, 'frobble')
 
     def test_argv(self):
         cl = MyLoader1(argv='-f hi -b 10 -n wow'.split())
         config = cl.load_config()
         self.assertEqual(config.Global.foo, 'hi')
         self.assertEqual(config.MyClass.bar, 10)
         self.assertEqual(config.n, True)
         self.assertEqual(config.Global.bam, 'wow')
 
 
 class TestKeyValueCL(TestCase):
     klass = KeyValueConfigLoader
 
     def test_basic(self):
-        cl = self.klass()
+        cl = self.klass(log=log)
         argv = ['--'+s.strip('c.') for s in pyfile.split('\n')[2:-1]]
-        with mute_warn():
-            config = cl.load_config(argv)
+        config = cl.load_config(argv)
         self.assertEqual(config.a, 10)
         self.assertEqual(config.b, 20)
         self.assertEqual(config.Foo.Bar.value, 10)
         self.assertEqual(config.Foo.Bam.value, list(range(10)))
         self.assertEqual(config.D.C.value, 'hi there')
     
     def test_expanduser(self):
-        cl = self.klass()
+        cl = self.klass(log=log)
         argv = ['--a=~/1/2/3', '--b=~', '--c=~/', '--d="~/"']
-        with mute_warn():
-            config = cl.load_config(argv)
+        config = cl.load_config(argv)
         self.assertEqual(config.a, os.path.expanduser('~/1/2/3'))
         self.assertEqual(config.b, os.path.expanduser('~'))
         self.assertEqual(config.c, os.path.expanduser('~/'))
         self.assertEqual(config.d, '~/')
     
     def test_extra_args(self):
-        cl = self.klass()
-        with mute_warn():
-            config = cl.load_config(['--a=5', 'b', '--c=10', 'd'])
+        cl = self.klass(log=log)
+        config = cl.load_config(['--a=5', 'b', '--c=10', 'd'])
         self.assertEqual(cl.extra_args, ['b', 'd'])
         self.assertEqual(config.a, 5)
         self.assertEqual(config.c, 10)
-        with mute_warn():
-            config = cl.load_config(['--', '--a=5', '--c=10'])
+        config = cl.load_config(['--', '--a=5', '--c=10'])
         self.assertEqual(cl.extra_args, ['--a=5', '--c=10'])
     
     def test_unicode_args(self):
-        cl = self.klass()
+        cl = self.klass(log=log)
         argv = [u'--a=épsîlön']
-        with mute_warn():
-            config = cl.load_config(argv)
+        config = cl.load_config(argv)
         self.assertEqual(config.a, u'épsîlön')
     
     def test_unicode_bytes_args(self):
         uarg = u'--a=é'
         try:
             barg = uarg.encode(sys.stdin.encoding)
         except (TypeError, UnicodeEncodeError):
             raise SkipTest("sys.stdin.encoding can't handle 'é'")
         
-        cl = self.klass()
-        with mute_warn():
-            config = cl.load_config([barg])
+        cl = self.klass(log=log)
+        config = cl.load_config([barg])
         self.assertEqual(config.a, u'é')
     
     def test_unicode_alias(self):
-        cl = self.klass()
+        cl = self.klass(log=log)
         argv = [u'--a=épsîlön']
-        with mute_warn():
-            config = cl.load_config(argv, aliases=dict(a='A.a'))
+        config = cl.load_config(argv, aliases=dict(a='A.a'))
         self.assertEqual(config.A.a, u'épsîlön')
 
 
 class TestArgParseKVCL(TestKeyValueCL):
     klass = KVArgParseConfigLoader
 
     def test_expanduser2(self):
-        cl = self.klass()
+        cl = self.klass(log=log)
         argv = ['-a', '~/1/2/3', '--b', "'~/1/2/3'"]
-        with mute_warn():
-            config = cl.load_config(argv, aliases=dict(a='A.a', b='A.b'))
+        config = cl.load_config(argv, aliases=dict(a='A.a', b='A.b'))
         self.assertEqual(config.A.a, os.path.expanduser('~/1/2/3'))
         self.assertEqual(config.A.b, '~/1/2/3')
     
     def test_eval(self):
-        cl = self.klass()
+        cl = self.klass(log=log)
         argv = ['-c', 'a=5']
-        with mute_warn():
-            config = cl.load_config(argv, aliases=dict(c='A.c'))
+        config = cl.load_config(argv, aliases=dict(c='A.c'))
         self.assertEqual(config.A.c, u"a=5")
     
 
 class TestConfig(TestCase):
 
     def test_setget(self):
         c = Config()
         c.a = 10
         self.assertEqual(c.a, 10)
         self.assertEqual('b' in c, False)
 
     def test_auto_section(self):
         c = Config()
         self.assertNotIn('A', c)
         assert not c._has_section('A')
         A = c.A
         A.foo = 'hi there'
         self.assertIn('A', c)
         assert c._has_section('A')
         self.assertEqual(c.A.foo, 'hi there')
         del c.A
         self.assertEqual(c.A, Config())
 
     def test_merge_doesnt_exist(self):
         c1 = Config()
         c2 = Config()
         c2.bar = 10
         c2.Foo.bar = 10
         c1.merge(c2)
         self.assertEqual(c1.Foo.bar, 10)
         self.assertEqual(c1.bar, 10)
         c2.Bar.bar = 10
         c1.merge(c2)
         self.assertEqual(c1.Bar.bar, 10)
 
     def test_merge_exists(self):
         c1 = Config()
         c2 = Config()
         c1.Foo.bar = 10
         c1.Foo.bam = 30
         c2.Foo.bar = 20
         c2.Foo.wow = 40
         c1.merge(c2)
         self.assertEqual(c1.Foo.bam, 30)
         self.assertEqual(c1.Foo.bar, 20)
         self.assertEqual(c1.Foo.wow, 40)
         c2.Foo.Bam.bam = 10
         c1.merge(c2)
         self.assertEqual(c1.Foo.Bam.bam, 10)
 
     def test_deepcopy(self):
         c1 = Config()
         c1.Foo.bar = 10
         c1.Foo.bam = 30
         c1.a = 'asdf'
         c1.b = range(10)
         import copy
         c2 = copy.deepcopy(c1)
         self.assertEqual(c1, c2)
         self.assertTrue(c1 is not c2)
         self.assertTrue(c1.Foo is not c2.Foo)
 
     def test_builtin(self):
         c1 = Config()
         c1.format = "json"
     
     def test_fromdict(self):
         c1 = Config({'Foo' : {'bar' : 1}})
         self.assertEqual(c1.Foo.__class__, Config)
         self.assertEqual(c1.Foo.bar, 1)
     
     def test_fromdictmerge(self):
         c1 = Config()
         c2 = Config({'Foo' : {'bar' : 1}})
         c1.merge(c2)
         self.assertEqual(c1.Foo.__class__, Config)
         self.assertEqual(c1.Foo.bar, 1)
 
     def test_fromdictmerge2(self):
         c1 = Config({'Foo' : {'baz' : 2}})
         c2 = Config({'Foo' : {'bar' : 1}})
         c1.merge(c2)
         self.assertEqual(c1.Foo.__class__, Config)
         self.assertEqual(c1.Foo.bar, 1)
         self.assertEqual(c1.Foo.baz, 2)
         self.assertNotIn('baz', c2.Foo)
     
     def test_contains(self):
         c1 = Config({'Foo' : {'baz' : 2}})
         c2 = Config({'Foo' : {'bar' : 1}})
         self.assertIn('Foo', c1)
         self.assertIn('Foo.baz', c1)
         self.assertIn('Foo.bar', c2)
         self.assertNotIn('Foo.bar', c1)
     
     def test_pickle_config(self):
         cfg = Config()
         cfg.Foo.bar = 1
         pcfg = pickle.dumps(cfg)
         cfg2 = pickle.loads(pcfg)
         self.assertEqual(cfg2, cfg)
     
     def test_getattr_section(self):
         cfg = Config()
         self.assertNotIn('Foo', cfg)
         Foo = cfg.Foo
         assert isinstance(Foo, Config)
         self.assertIn('Foo', cfg)
 
     def test_getitem_section(self):
         cfg = Config()
         self.assertNotIn('Foo', cfg)
         Foo = cfg['Foo']
         assert isinstance(Foo, Config)
         self.assertIn('Foo', cfg)
 
     def test_getattr_not_section(self):
         cfg = Config()
         self.assertNotIn('foo', cfg)
         foo = cfg.foo
         assert isinstance(foo, LazyConfigValue)
         self.assertIn('foo', cfg)
 
     def test_getitem_not_section(self):
         cfg = Config()
         self.assertNotIn('foo', cfg)
         foo = cfg['foo']
         assert isinstance(foo, LazyConfigValue)
         self.assertIn('foo', cfg)
     
     def test_merge_copies(self):
         c = Config()
         c2 = Config()
         c2.Foo.trait = []
         c.merge(c2)
         c2.Foo.trait.append(1)
         self.assertIsNot(c.Foo, c2.Foo)
         self.assertEqual(c.Foo.trait, [])
         self.assertEqual(c2.Foo.trait, [1])
 

IPython/terminal/ipapp.py

@@ -1,389 +1,387 b''
 #!/usr/bin/env python
 # encoding: utf-8
 """
 The :class:`~IPython.core.application.Application` object for the command
 line :command:`ipython` program.
 
 Authors
 -------
 
 * Brian Granger
 * Fernando Perez
 * Min Ragan-Kelley
 """
 
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2008-2011  The IPython Development Team
 #
 #  Distributed under the terms of the BSD License.  The full license is in
 #  the file COPYING, distributed as part of this software.
 #-----------------------------------------------------------------------------
 
 #-----------------------------------------------------------------------------
 # Imports
 #-----------------------------------------------------------------------------
 
 from __future__ import absolute_import
 from __future__ import print_function
 
 import logging
 import os
 import sys
 
 from IPython.config.loader import (
     Config, PyFileConfigLoader, ConfigFileNotFound
 )
-from IPython.config.application import boolean_flag, catch_config_error
+from IPython.config.application import boolean_flag, catch_config_error, Application
 from IPython.core import release
 from IPython.core import usage
 from IPython.core.completer import IPCompleter
 from IPython.core.crashhandler import CrashHandler
 from IPython.core.formatters import PlainTextFormatter
 from IPython.core.history import HistoryManager
 from IPython.core.prompts import PromptManager
 from IPython.core.application import (
     ProfileDir, BaseIPythonApplication, base_flags, base_aliases
 )
 from IPython.core.magics import ScriptMagics
 from IPython.core.shellapp import (
     InteractiveShellApp, shell_flags, shell_aliases
 )
 from IPython.extensions.storemagic import StoreMagics
 from IPython.terminal.interactiveshell import TerminalInteractiveShell
 from IPython.utils import warn
 from IPython.utils.path import get_ipython_dir, check_for_old_config
 from IPython.utils.traitlets import (
     Bool, List, Dict,
 )
 
 #-----------------------------------------------------------------------------
 # Globals, utilities and helpers
 #-----------------------------------------------------------------------------
 
 _examples = """
 ipython --matplotlib       # enable matplotlib integration
 ipython --matplotlib=qt    # enable matplotlib integration with qt4 backend
 
 ipython --log-level=DEBUG  # set logging to DEBUG
 ipython --profile=foo      # start with profile foo
 
 ipython qtconsole          # start the qtconsole GUI application
 ipython help qtconsole     # show the help for the qtconsole subcmd
 
 ipython console            # start the terminal-based console application
 ipython help console       # show the help for the console subcmd
 
 ipython notebook           # start the IPython notebook
 ipython help notebook      # show the help for the notebook subcmd
 
 ipython profile create foo # create profile foo w/ default config files
 ipython help profile       # show the help for the profile subcmd
 
 ipython locate             # print the path to the IPython directory
 ipython locate profile foo # print the path to the directory for profile `foo`
 
 ipython nbconvert           # convert notebooks to/from other formats
 """
 
 #-----------------------------------------------------------------------------
 # Crash handler for this application
 #-----------------------------------------------------------------------------
 
 class IPAppCrashHandler(CrashHandler):
     """sys.excepthook for IPython itself, leaves a detailed report on disk."""
 
     def __init__(self, app):
         contact_name = release.author
         contact_email = release.author_email
         bug_tracker = 'https://github.com/ipython/ipython/issues'
         super(IPAppCrashHandler,self).__init__(
             app, contact_name, contact_email, bug_tracker
         )
 
     def make_report(self,traceback):
         """Return a string containing a crash report."""
 
         sec_sep = self.section_sep
         # Start with parent report
         report = [super(IPAppCrashHandler, self).make_report(traceback)]
         # Add interactive-specific info we may have
         rpt_add = report.append
         try:
             rpt_add(sec_sep+"History of session input:")
             for line in self.app.shell.user_ns['_ih']:
                 rpt_add(line)
             rpt_add('\n*** Last line of input (may not be in above history):\n')
             rpt_add(self.app.shell._last_input_line+'\n')
         except:
             pass
 
         return ''.join(report)
 
 #-----------------------------------------------------------------------------
 # Aliases and Flags
 #-----------------------------------------------------------------------------
 flags = dict(base_flags)
 flags.update(shell_flags)
 frontend_flags = {}
 addflag = lambda *args: frontend_flags.update(boolean_flag(*args))
 addflag('autoedit-syntax', 'TerminalInteractiveShell.autoedit_syntax',
         'Turn on auto editing of files with syntax errors.',
         'Turn off auto editing of files with syntax errors.'
 )
 addflag('banner', 'TerminalIPythonApp.display_banner',
         "Display a banner upon starting IPython.",
         "Don't display a banner upon starting IPython."
 )
 addflag('confirm-exit', 'TerminalInteractiveShell.confirm_exit',
     """Set to confirm when you try to exit IPython with an EOF (Control-D
     in Unix, Control-Z/Enter in Windows). By typing 'exit' or 'quit',
     you can force a direct exit without any confirmation.""",
     "Don't prompt the user when exiting."
 )
 addflag('term-title', 'TerminalInteractiveShell.term_title',
     "Enable auto setting the terminal title.",
     "Disable auto setting the terminal title."
 )
 classic_config = Config()
 classic_config.InteractiveShell.cache_size = 0
 classic_config.PlainTextFormatter.pprint = False
 classic_config.PromptManager.in_template = '>>> '
 classic_config.PromptManager.in2_template = '... '
 classic_config.PromptManager.out_template = ''
 classic_config.InteractiveShell.separate_in = ''
 classic_config.InteractiveShell.separate_out = ''
 classic_config.InteractiveShell.separate_out2 = ''
 classic_config.InteractiveShell.colors = 'NoColor'
 classic_config.InteractiveShell.xmode = 'Plain'
 
 frontend_flags['classic']=(
     classic_config,
     "Gives IPython a similar feel to the classic Python prompt."
 )
 # # log doesn't make so much sense this way anymore
 # paa('--log','-l',
 #     action='store_true', dest='InteractiveShell.logstart',
 #     help="Start logging to the default log file (./ipython_log.py).")
 #
 # # quick is harder to implement
 frontend_flags['quick']=(
     {'TerminalIPythonApp' : {'quick' : True}},
     "Enable quick startup with no config files."
 )
 
 frontend_flags['i'] = (
     {'TerminalIPythonApp' : {'force_interact' : True}},
     """If running code from the command line, become interactive afterwards.
     Note: can also be given simply as '-i.'"""
 )
 flags.update(frontend_flags)
 
 aliases = dict(base_aliases)
 aliases.update(shell_aliases)
 
 #-----------------------------------------------------------------------------
 # Main classes and functions
 #-----------------------------------------------------------------------------
 
 
 class LocateIPythonApp(BaseIPythonApplication):
     description = """print the path to the IPython dir"""
     subcommands = Dict(dict(
         profile=('IPython.core.profileapp.ProfileLocate',
             "print the path to an IPython profile directory",
         ),
     ))
     def start(self):
         if self.subapp is not None:
             return self.subapp.start()
         else:
             print(self.ipython_dir)
 
 
 class TerminalIPythonApp(BaseIPythonApplication, InteractiveShellApp):
     name = u'ipython'
     description = usage.cl_usage
     crash_handler_class = IPAppCrashHandler
     examples = _examples
 
     flags = Dict(flags)
     aliases = Dict(aliases)
     classes = List()
     def _classes_default(self):
         """This has to be in a method, for TerminalIPythonApp to be available."""
         return [
             InteractiveShellApp, # ShellApp comes before TerminalApp, because
             self.__class__,      # it will also affect subclasses (e.g. QtConsole)
             TerminalInteractiveShell,
             PromptManager,
             HistoryManager,
             ProfileDir,
             PlainTextFormatter,
             IPCompleter,
             ScriptMagics,
             StoreMagics,
         ]
 
     subcommands = Dict(dict(
         qtconsole=('IPython.qt.console.qtconsoleapp.IPythonQtConsoleApp',
             """Launch the IPython Qt Console."""
         ),
         notebook=('IPython.html.notebookapp.NotebookApp',
             """Launch the IPython HTML Notebook Server."""
         ),
         profile = ("IPython.core.profileapp.ProfileApp",
             "Create and manage IPython profiles."
         ),
         kernel = ("IPython.kernel.zmq.kernelapp.IPKernelApp",
             "Start a kernel without an attached frontend."
         ),
         console=('IPython.terminal.console.app.ZMQTerminalIPythonApp',
             """Launch the IPython terminal-based Console."""
         ),
         locate=('IPython.terminal.ipapp.LocateIPythonApp',
             LocateIPythonApp.description
         ),
         history=('IPython.core.historyapp.HistoryApp',
             "Manage the IPython history database."
         ),
         nbconvert=('IPython.nbconvert.nbconvertapp.NbConvertApp',
             "Convert notebooks to/from other formats."
         ),
     ))
 
     # *do* autocreate requested profile, but don't create the config file.
     auto_create=Bool(True)
     # configurables
     ignore_old_config=Bool(False, config=True,
         help="Suppress warning messages about legacy config files"
     )
     quick = Bool(False, config=True,
         help="""Start IPython quickly by skipping the loading of config files."""
     )
     def _quick_changed(self, name, old, new):
         if new:
             self.load_config_file = lambda *a, **kw: None
             self.ignore_old_config=True
 
     display_banner = Bool(True, config=True,
         help="Whether to display a banner upon starting IPython."
     )
 
     # if there is code of files to run from the cmd line, don't interact
     # unless the --i flag (App.force_interact) is true.
     force_interact = Bool(False, config=True,
         help="""If a command or file is given via the command-line,
         e.g. 'ipython foo.py"""
     )
     def _force_interact_changed(self, name, old, new):
         if new:
             self.interact = True
 
     def _file_to_run_changed(self, name, old, new):
         if new:
             self.something_to_run = True
         if new and not self.force_interact:
                 self.interact = False
     _code_to_run_changed = _file_to_run_changed
     _module_to_run_changed = _file_to_run_changed
 
     # internal, not-configurable
     interact=Bool(True)
     something_to_run=Bool(False)
 
     def parse_command_line(self, argv=None):
         """override to allow old '-pylab' flag with deprecation warning"""
 
         argv = sys.argv[1:] if argv is None else argv
 
         if '-pylab' in argv:
             # deprecated `-pylab` given,
             # warn and transform into current syntax
             argv = argv[:] # copy, don't clobber
             idx = argv.index('-pylab')
             warn.warn("`-pylab` flag has been deprecated.\n"
             "    Use `--matplotlib <backend>` and import pylab manually.")
             argv[idx] = '--pylab'
 
         return super(TerminalIPythonApp, self).parse_command_line(argv)
     
     @catch_config_error
     def initialize(self, argv=None):
         """Do actions after construct, but before starting the app."""
         super(TerminalIPythonApp, self).initialize(argv)
         if self.subapp is not None:
             # don't bother initializing further, starting subapp
             return
         if not self.ignore_old_config:
             check_for_old_config(self.ipython_dir)
         # print self.extra_args
         if self.extra_args and not self.something_to_run:
             self.file_to_run = self.extra_args[0]
         self.init_path()
         # create the shell
         self.init_shell()
         # and draw the banner
         self.init_banner()
         # Now a variety of things that happen after the banner is printed.
         self.init_gui_pylab()
         self.init_extensions()
         self.init_code()
 
     def init_shell(self):
         """initialize the InteractiveShell instance"""
         # Create an InteractiveShell instance.
         # shell.display_banner should always be False for the terminal
         # based app, because we call shell.show_banner() by hand below
         # so the banner shows *before* all extension loading stuff.
         self.shell = TerminalInteractiveShell.instance(parent=self,
                         display_banner=False, profile_dir=self.profile_dir,
                         ipython_dir=self.ipython_dir, user_ns=self.user_ns)
         self.shell.configurables.append(self)
 
     def init_banner(self):
         """optionally display the banner"""
         if self.display_banner and self.interact:
             self.shell.show_banner()
         # Make sure there is a space below the banner.
         if self.log_level <= logging.INFO: print()
 
     def _pylab_changed(self, name, old, new):
         """Replace --pylab='inline' with --pylab='auto'"""
         if new == 'inline':
             warn.warn("'inline' not available as pylab backend, "
                       "using 'auto' instead.")
             self.pylab = 'auto'
 
     def start(self):
         if self.subapp is not None:
             return self.subapp.start()
         # perform any prexec steps:
         if self.interact:
             self.log.debug("Starting IPython's mainloop...")
             self.shell.mainloop()
         else:
             self.log.debug("IPython not interactive...")
 
-
 def load_default_config(ipython_dir=None):
     """Load the default config file from the default ipython_dir.
 
     This is useful for embedded shells.
     """
     if ipython_dir is None:
         ipython_dir = get_ipython_dir()
+
     profile_dir = os.path.join(ipython_dir, 'profile_default')
-    cl = PyFileConfigLoader("ipython_config.py", profile_dir)
-    try:
-        config = cl.load_config()
-    except ConfigFileNotFound:
-        # no config found
-        config = Config()
-    return config
 
+    config = Config()
+    for cf in Application._load_config_file(filename[:-3], path=profile_dir, log=None):
+        config.update(cf)
+
+    return config
 
 launch_new_instance = TerminalIPythonApp.launch_instance
 
 
 if __name__ == '__main__':
     launch_new_instance()

docs/source/development/config.rst

@@ -1,524 +1,566 b''
 .. _config_overview:
 
 ============================================
 Overview of the IPython configuration system
 ============================================
 
 This section describes the IPython configuration system. 
 
 The main concepts
 =================
 
 There are a number of abstractions that the IPython configuration system uses.
 Each of these abstractions is represented by a Python class.
 
 Configuration object: :class:`~IPython.config.loader.Config`
     A configuration object is a simple dictionary-like class that holds
     configuration attributes and sub-configuration objects. These classes
-    support dotted attribute style access (``Foo.bar``) in addition to the
-    regular dictionary style access (``Foo['bar']``). Configuration objects
-    are smart. They know how to merge themselves with other configuration
-    objects and they automatically create sub-configuration objects.
+    support dotted attribute style access (``cfg.Foo.bar``) in addition to the
+    regular dictionary style access (``cfg['Foo']['bar']``).
+    The Config object is a wrapper around a simple dictionary with some convenience methods,
+    such as merging and automatic section creation.
 
 Application: :class:`~IPython.config.application.Application`
     An application is a process that does a specific job. The most obvious
     application is the :command:`ipython` command line program. Each
     application reads *one or more* configuration files and a single set of
     command line options
     and then produces a master configuration object for the application. This
     configuration object is then passed to the configurable objects that the
     application creates. These configurable objects implement the actual logic
     of the application and know how to configure themselves given the
     configuration object.
     
     Applications always have a `log` attribute that is a configured Logger.
     This allows centralized logging configuration per-application.
 
 Configurable: :class:`~IPython.config.configurable.Configurable`
     A configurable is a regular Python class that serves as a base class for
     all main classes in an application. The
     :class:`~IPython.config.configurable.Configurable` base class is
     lightweight and only does one things.
 
     This :class:`~IPython.config.configurable.Configurable` is a subclass
     of :class:`~IPython.utils.traitlets.HasTraits` that knows how to configure
     itself. Class level traits with the metadata ``config=True`` become
     values that can be configured from the command line and configuration
     files.
     
     Developers create :class:`~IPython.config.configurable.Configurable`
     subclasses that implement all of the logic in the application. Each of
     these subclasses has its own configuration information that controls how
     instances are created.
 
 Singletons: :class:`~IPython.config.configurable.SingletonConfigurable`
     Any object for which there is a single canonical instance. These are
     just like Configurables, except they have a class method 
     :meth:`~IPython.config.configurable.SingletonConfigurable.instance`,
     that returns the current active instance (or creates one if it
     does not exist).  Examples of singletons include
     :class:`~IPython.config.application.Application`s and
     :class:`~IPython.core.interactiveshell.InteractiveShell`.  This lets
     objects easily connect to the current running Application without passing
     objects around everywhere.  For instance, to get the current running 
     Application instance, simply do: ``app = Application.instance()``.
 
 
 .. note::
 
     Singletons are not strictly enforced - you can have many instances
     of a given singleton class, but the :meth:`instance` method will always
     return the same one.
 
 Having described these main concepts, we can now state the main idea in our
 configuration system: *"configuration" allows the default values of class
 attributes to be controlled on a class by class basis*. Thus all instances of
 a given class are configured in the same way. Furthermore, if two instances
 need to be configured differently, they need to be instances of two different
 classes. While this model may seem a bit restrictive, we have found that it
 expresses most things that need to be configured extremely well. However, it
 is possible to create two instances of the same class that have different
 trait values. This is done by overriding the configuration.
 
 Now, we show what our configuration objects and files look like.
 
 Configuration objects and files
 ===============================
 
-A configuration file is simply a pure Python file that sets the attributes
-of a global, pre-created configuration object.  This configuration object is a 
-:class:`~IPython.config.loader.Config` instance.  While in a configuration
-file, to get a reference to this object, simply call the :func:`get_config`
-function.  We inject this function into the global namespace that the 
-configuration file is executed in.
+A configuration object is little more than a wrapper around a dictionary.
+A configuration *file* is simply a mechanism for producing that object.
+The main IPython configuration file is a plain Python script,
+which can perform extensive logic to populate the config object.
+IPython 2.0 introduces a JSON configuration file,
+which is just a direct JSON serialization of the config dictionary.
+The JSON format is easily processed by external software.
+
+When both Python and JSON configuration file are present, both will be loaded,
+with JSON configuration having higher priority.
+
+Python configuration Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A Python configuration file is a pure Python file that populates a configuration object.
+This configuration object is a :class:`~IPython.config.loader.Config` instance.
+While in a configuration file, to get a reference to this object, simply call the :func:`get_config`
+function, which is available in the global namespace of the script.
 
 Here is an example of a super simple configuration file that does nothing::
 
     c = get_config()
 
 Once you get a reference to the configuration object, you simply set
 attributes on it.  All you have to know is:
 
-* The name of each attribute.
+* The name of the class to configure.
+* The name of the attribute.
 * The type of each attribute.
 
-The answers to these two questions are provided by the various
+The answers to these questions are provided by the various
 :class:`~IPython.config.configurable.Configurable` subclasses that an
 application uses. Let's look at how this would work for a simple configurable
 subclass::
 
     # Sample configurable:
     from IPython.config.configurable import Configurable
     from IPython.utils.traitlets import Int, Float, Unicode, Bool
     
     class MyClass(Configurable):
         name = Unicode(u'defaultname', config=True)
         ranking = Int(0, config=True)
         value = Float(99.0)
         # The rest of the class implementation would go here..
 
 In this example, we see that :class:`MyClass` has three attributes, two
-of whom (``name``, ``ranking``) can be configured.  All of the attributes
+of  (``name``, ``ranking``) can be configured.  All of the attributes
 are given types and default values.  If a :class:`MyClass` is instantiated,
 but not configured, these default values will be used.  But let's see how
 to configure this class in a configuration file::
 
     # Sample config file
     c = get_config()
     
     c.MyClass.name = 'coolname'
     c.MyClass.ranking = 10
 
 After this configuration file is loaded, the values set in it will override
 the class defaults anytime a :class:`MyClass` is created.  Furthermore,
 these attributes will be type checked and validated anytime they are set.
 This type checking is handled by the :mod:`IPython.utils.traitlets` module,
 which provides the :class:`Unicode`, :class:`Int` and :class:`Float` types.
 In addition to these traitlets, the :mod:`IPython.utils.traitlets` provides
 traitlets for a number of other types.
 
 .. note::
 
     Underneath the hood, the :class:`Configurable` base class is a subclass of
     :class:`IPython.utils.traitlets.HasTraits`. The
     :mod:`IPython.utils.traitlets` module is a lightweight version of
     :mod:`enthought.traits`. Our implementation is a pure Python subset
     (mostly API compatible) of :mod:`enthought.traits` that does not have any
     of the automatic GUI generation capabilities. Our plan is to achieve 100%
     API compatibility to enable the actual :mod:`enthought.traits` to
     eventually be used instead. Currently, we cannot use
     :mod:`enthought.traits` as we are committed to the core of IPython being
     pure Python.
 
 It should be very clear at this point what the naming convention is for 
 configuration attributes::
 
     c.ClassName.attribute_name = attribute_value
 
 Here, ``ClassName`` is the name of the class whose configuration attribute you
 want to set, ``attribute_name`` is the name of the attribute you want to set
 and ``attribute_value`` the the value you want it to have. The ``ClassName``
 attribute of ``c`` is not the actual class, but instead is another
 :class:`~IPython.config.loader.Config` instance.
 
 .. note::
 
     The careful reader may wonder how the ``ClassName`` (``MyClass`` in
     the above example) attribute of the configuration object ``c`` gets
     created. These attributes are created on the fly by the
     :class:`~IPython.config.loader.Config` instance, using a simple naming
     convention. Any attribute of a :class:`~IPython.config.loader.Config`
     instance whose name begins with an uppercase character is assumed to be a
     sub-configuration and a new empty :class:`~IPython.config.loader.Config`
     instance is dynamically created for that attribute. This allows deeply
     hierarchical information created easily (``c.Foo.Bar.value``) on the fly.
 
+JSON configuration Files
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+A JSON configuration file is simply a file that contain a
+:class:`~IPython.config.loader.Config` dictionary serialized to JSON.
+A JSON configuration file has the same base name as a Python configuration file,
+just with a .json extension.
+
+Configuration described in previous section could be written as follow in a
+JSON configuration file:
+
+.. sourcecode:: json
+
+    {
+      "version": "1.0",
+      "MyClass": {
+        "name": "coolname",
+        "ranking": 10
+      }
+    }
+
+JSON configuration files can be more easily generated or processed by programs
+or other languages.
+
+
 Configuration files inheritance
 ===============================
 
+.. note::
+
+    This section only apply to Python configuration files.
+
 Let's say you want to have different configuration files for various purposes.
 Our configuration system makes it easy for one configuration file to inherit
 the information in another configuration file. The :func:`load_subconfig`
 command can be used in a configuration file for this purpose. Here is a simple
 example that loads all of the values from the file :file:`base_config.py`::
 
     # base_config.py
     c = get_config()
     c.MyClass.name = 'coolname'
     c.MyClass.ranking = 100
 
 into the configuration file :file:`main_config.py`::
 
     # main_config.py
     c = get_config()
     
     # Load everything from base_config.py
     load_subconfig('base_config.py')
     
     # Now override one of the values
     c.MyClass.name = 'bettername'
 
 In a situation like this the :func:`load_subconfig` makes sure that the
 search path for sub-configuration files is inherited from that of the parent.
 Thus, you can typically put the two in the same directory and everything will
 just work.
 
 You can also load configuration files by profile, for instance:
 
 .. sourcecode:: python
 
     load_subconfig('ipython_config.py', profile='default')
 
 to inherit your default configuration as a starting point.
 
 
 Class based configuration inheritance
 =====================================
 
 There is another aspect of configuration where inheritance comes into play.
 Sometimes, your classes will have an inheritance hierarchy that you want
 to be reflected in the configuration system.  Here is a simple example::
 
     from IPython.config.configurable import Configurable
     from IPython.utils.traitlets import Int, Float, Unicode, Bool
     
     class Foo(Configurable):
         name = Unicode(u'fooname', config=True)
         value = Float(100.0, config=True)
     
     class Bar(Foo):
         name = Unicode(u'barname', config=True)
         othervalue = Int(0, config=True)
 
 Now, we can create a configuration file to configure instances of :class:`Foo`
 and :class:`Bar`::
 
     # config file
     c = get_config()
     
     c.Foo.name = u'bestname'
     c.Bar.othervalue = 10
 
 This class hierarchy and configuration file accomplishes the following:
 
 * The default value for :attr:`Foo.name` and :attr:`Bar.name` will be
   'bestname'.  Because :class:`Bar` is a :class:`Foo` subclass it also
   picks up the configuration information for :class:`Foo`.
 * The default value for :attr:`Foo.value` and :attr:`Bar.value` will be
   ``100.0``, which is the value specified as the class default.
 * The default value for :attr:`Bar.othervalue` will be 10 as set in the
   configuration file.  Because :class:`Foo` is the parent of :class:`Bar`
   it doesn't know anything about the :attr:`othervalue` attribute.
 
 
 .. _ipython_dir:
 
 Configuration file location
 ===========================
 
 So where should you put your configuration files? IPython uses "profiles" for
 configuration, and by default, all profiles will be stored in the so called
 "IPython directory". The location of this directory is determined by the
 following algorithm:
 
 * If the ``ipython-dir`` command line flag is given, its value is used.
 
 * If not, the value returned by :func:`IPython.utils.path.get_ipython_dir`
   is used. This function will first look at the :envvar:`IPYTHONDIR`
   environment variable and then default to :file:`~/.ipython`.
   Historical support for the :envvar:`IPYTHON_DIR` environment variable will
   be removed in a future release.
 
 For most users, the configuration directory will be :file:`~/.ipython`.
 
 Previous versions of IPython on Linux would use the XDG config directory,
 creating :file:`~/.config/ipython` by default. We have decided to go
 back to :file:`~/.ipython` for consistency among systems. IPython will
 issue a warning if it finds the XDG location, and will move it to the new
 location if there isn't already a directory there.
 
 Once the location of the IPython directory has been determined, you need to know
 which profile you are using. For users with a single configuration, this will
 simply be 'default', and will be located in 
 :file:`<IPYTHONDIR>/profile_default`.
 
 The next thing you need to know is what to call your configuration file. The
 basic idea is that each application has its own default configuration filename.
 The default named used by the :command:`ipython` command line program is
 :file:`ipython_config.py`, and *all* IPython applications will use this file.
 Other applications, such as the parallel :command:`ipcluster` scripts or the
 QtConsole will load their own config files *after* :file:`ipython_config.py`. To
 load a particular configuration file instead of the default, the name can be
 overridden by the ``config_file`` command line flag.
 
 To generate the default configuration files, do::
 
     $ ipython profile create
 
 and you will have a default :file:`ipython_config.py` in your IPython directory
 under :file:`profile_default`. If you want the default config files for the
 :mod:`IPython.parallel` applications, add ``--parallel`` to the end of the
 command-line args.
 
 
 Locating these files
 --------------------
 
 From the command-line, you can quickly locate the IPYTHONDIR or a specific
 profile with:
 
 .. sourcecode:: bash
 
     $ ipython locate
     /home/you/.ipython
     
     $ ipython locate profile foo
     /home/you/.ipython/profile_foo
 
 These map to the utility functions: :func:`IPython.utils.path.get_ipython_dir`
 and :func:`IPython.utils.path.locate_profile` respectively.
 
 
 .. _profiles_dev:
 
 Profiles
 ========
 
 A profile is a directory containing configuration and runtime files, such as
 logs, connection info for the parallel apps, and your IPython command history.
 
 The idea is that users often want to maintain a set of configuration files for
 different purposes: one for doing numerical computing with NumPy and SciPy and
 another for doing symbolic computing with SymPy. Profiles make it easy to keep a
 separate configuration files, logs, and histories for each of these purposes.
 
 Let's start by showing how a profile is used:
 
 .. code-block:: bash
 
     $ ipython --profile=sympy
 
 This tells the :command:`ipython` command line program to get its configuration
 from the "sympy" profile. The file names for various profiles do not change. The
 only difference is that profiles are named in a special way. In the case above,
 the "sympy" profile means looking for :file:`ipython_config.py` in :file:`<IPYTHONDIR>/profile_sympy`.
 
 The general pattern is this: simply create a new profile with:
 
 .. code-block:: bash
 
     $ ipython profile create <name>
 
 which adds a directory called ``profile_<name>`` to your IPython directory. Then
 you can load this profile by adding ``--profile=<name>`` to your command line
 options. Profiles are supported by all IPython applications.
 
 IPython ships with some sample profiles in :file:`IPython/config/profile`. If
 you create profiles with the name of one of our shipped profiles, these config
 files will be copied over instead of starting with the automatically generated
 config files.
 
 Security Files
 --------------
 
 If you are using the notebook, qtconsole, or parallel code, IPython stores
 connection information in small JSON files in the active profile's security
 directory. This directory is made private, so only you can see the files inside. If
 you need to move connection files around to other computers, this is where they will
 be. If you want your code to be able to open security files by name, we have a
 convenience function :func:`IPython.utils.path.get_security_file`, which will return
 the absolute path to a security file from its filename and [optionally] profile
 name.
 
 .. _startup_files:
 
 Startup Files
 -------------
 
 If you want some code to be run at the beginning of every IPython session with
 a particular profile, the easiest way is to add Python (``.py``) or 
 IPython (``.ipy``) scripts to your :file:`<profile>/startup` directory. Files
 in this directory will always be executed as soon as the IPython shell is 
 constructed, and before any other code or scripts you have specified. If you 
 have multiple files in the startup directory, they will be run in 
 lexicographical order, so you can control the ordering by adding a '00-' 
 prefix.
 
 
 .. _commandline:
 
 Command-line arguments
 ======================
 
 IPython exposes *all* configurable options on the command-line. The command-line
 arguments are generated from the Configurable traits of the classes associated
 with a given Application.  Configuring IPython from the command-line may look
 very similar to an IPython config file
 
 IPython applications use a parser called
 :class:`~IPython.config.loader.KeyValueLoader` to load values into a Config
 object.  Values are assigned in much the same way as in a config file:
 
 .. code-block:: bash
 
     $ ipython --InteractiveShell.use_readline=False --BaseIPythonApplication.profile='myprofile'
 
 Is the same as adding:
 
 .. sourcecode:: python
 
     c.InteractiveShell.use_readline=False
     c.BaseIPythonApplication.profile='myprofile'
 
 to your config file. Key/Value arguments *always* take a value, separated by '='
 and no spaces.
 
 Common Arguments
 ----------------
 
 Since the strictness and verbosity of the KVLoader above are not ideal for everyday
 use, common arguments can be specified as flags_ or aliases_.
 
 Flags and Aliases are handled by :mod:`argparse` instead, allowing for more flexible
 parsing. In general, flags and aliases are prefixed by ``--``, except for those
 that are single characters, in which case they can be specified with a single ``-``, e.g.:
 
 .. code-block:: bash
 
     $ ipython -i -c "import numpy; x=numpy.linspace(0,1)" --profile testing --colors=lightbg
 
 Aliases
 *******
 
 For convenience, applications have a mapping of commonly used traits, so you don't have
 to specify the whole class name:
 
 .. code-block:: bash
 
     $ ipython --profile myprofile
     # and
     $ ipython --profile='myprofile'
     # are equivalent to
     $ ipython --BaseIPythonApplication.profile='myprofile'
 
 Flags
 *****
 
 Applications can also be passed **flags**. Flags are options that take no
 arguments. They are simply wrappers for
 setting one or more configurables with predefined values, often True/False.
 
 For instance:
 
 .. code-block:: bash
 
     $ ipcontroller --debug
     # is equivalent to
     $ ipcontroller --Application.log_level=DEBUG
     # and
     $ ipython --matploitlib
     # is equivalent to
     $ ipython --matplotlib auto
     # or
     $ ipython --no-banner
     # is equivalent to
     $ ipython --TerminalIPythonApp.display_banner=False
 
 Subcommands
 -----------
 
 
 Some IPython applications have **subcommands**. Subcommands are modeled after
 :command:`git`, and are called with the form :command:`command subcommand
 [...args]`.  Currently, the QtConsole is a subcommand of terminal IPython:
 
 .. code-block:: bash
 
     $ ipython qtconsole --profile myprofile
 
 and :command:`ipcluster` is simply a wrapper for its various subcommands (start,
 stop, engines).
 
 .. code-block:: bash
 
     $ ipcluster start --profile=myprofile -n 4
 
 
 To see a list of the available aliases, flags, and subcommands for an IPython application, simply pass ``-h`` or ``--help``.  And to see the full list of configurable options (*very* long), pass ``--help-all``.
 
 
 Design requirements
 ===================
 
 Here are the main requirements we wanted our configuration system to have:
 
 * Support for hierarchical configuration information.
 
 * Full integration with command line option parsers.  Often, you want to read
   a configuration file, but then override some of the values with command line
   options.  Our configuration system automates this process and allows each
   command line option to be linked to a particular attribute in the 
   configuration hierarchy that it will override.
 
 * Configuration files that are themselves valid Python code. This accomplishes
   many things. First, it becomes possible to put logic in your configuration
   files that sets attributes based on your operating system, network setup,
   Python version, etc. Second, Python has a super simple syntax for accessing
   hierarchical data structures, namely regular attribute access
   (``Foo.Bar.Bam.name``). Third, using Python makes it easy for users to
   import configuration attributes from one configuration file to another.
   Fourth, even though Python is dynamically typed, it does have types that can
   be checked at runtime. Thus, a ``1`` in a config file is the integer '1',
   while a ``'1'`` is a string.
 
 * A fully automated method for getting the configuration information to the
   classes that need it at runtime. Writing code that walks a configuration
   hierarchy to extract a particular attribute is painful. When you have
   complex configuration information with hundreds of attributes, this makes
   you want to cry.
 
 * Type checking and validation that doesn't require the entire configuration
   hierarchy to be specified statically before runtime. Python is a very
   dynamic language and you don't always know everything that needs to be
   configured when a program starts.