docs/source/development/reorg.txt

=============================
IPython module reorganization
=============================

Currently, IPython has many top-level modules that serve many different
purposes. The lack of organization make it very difficult for developers to
work on IPython and understand its design. This document contains notes about
how we will reorganize the modules into sub-packages.

.. warning::

    This effort will possibly break third party packages that use IPython as
    a library or hack on the IPython internals.

.. warning::

    This effort will result in the removal from IPython of certain modules
    that are not used anymore, don't currently work, are unmaintained, etc.


Current subpackges
==================

IPython currently has the following sub-packages:

* :mod:`IPython.config`

* :mod:`IPython.Extensions`

* :mod:`IPython.external`

* :mod:`IPython.frontend`

* :mod:`IPython.gui`

* :mod:`IPython.kernel`

* :mod:`IPython.testing`

* :mod:`IPython.tests`

* :mod:`IPython.tools`

* :mod:`IPython.UserConfig`

New Subpackages to be created
=============================

We propose to create the following new sub-packages:

* :mod:`IPython.core`. This sub-package will contain the core of the IPython
  interpreter, but none of its extended capabilities.

* :mod:`IPython.lib`. IPython has many extended capabilities that are not part
  of the IPython core. These things will go here.

* :mod:`IPython.utils`. This sub-package will contain anything that might
  eventually be found in the Python standard library, like things in
  :mod:`genutils`. Each sub-module in this sub-package should contain
  functions and classes that serve a single purpose.

* :mod:`IPython.deathrow`. This is for code that is untested and/or rotting
  and needs to be removed from IPython. Eventually all this code will either
  i) be revived by someone willing to maintain it with tests and docs and
  re-included into IPython or 2) be removed from IPython proper, but put into
  a separate top-level (not IPython) package that we keep around. No new code
  will be allowed here.

* :mod:`IPython.quarantine`.  This is for code that doesn't meet IPython's 
  standards, but that we plan on keeping.  To be moved out of this sub-package
  a module needs to have a maintainer, tests and documentation.

Procedure
=========

1. Move the file to its new location with its new name.
2. Rename all import statements to reflect the change.
3. Run PyFlakes on each changes module.
3. Add tests/test_imports.py to test it.

Status
======

The new subpackages have been created and the top-level modules have been
moved and renamed. Import tests have been created for all of the moved and
renamed modules. The build infrastructure (setup.py and friends) have been
updated and tested on Mac and Windows. Finally, a compatibility layer has been
added for iplib, ipapi and Shell. The follow things still need to be done::

* I need to modify iptests to properly skip modules that are no longer top
  level modules.

* When running python setup.py sdist, the Sphinx API docs fail to build
  because of something going on with IPython.core.fakemodule