Document single codebase Python 3 compatibility

docs/source/development/pycompat.rst

@@ -0,0 +1,122 b''
+Writing code for Python 2 and 3
+===============================
+
+Iterators
+---------
+
+Many built in functions and methods in Python 2 come in pairs, one
+returning a list, and one returning an iterator (e.g. :func:`range` and
+:func:`xrange`). In Python 3, there is usually only the iterator form,
+but it has the name which gives a list in Python 2 (e.g. :func:`range`).
+
+The way to write compatible code depends on what you need:
+
+* A list, e.g. for serialisation, or to test if something is in it.
+* Iteration, but it will never be used for very many items, so efficiency
+  isn't especially important.
+* Iteration over many items, where efficiency is important.
+
+================  =================  =======================
+list              iteration (small)  iteration(large)
+================  =================  =======================
+list(range(n))    range(n)           py3compat.xrange(n)
+list(map(f, it))  map(f, it)         --
+list(zip(a, b))   zip(a, b)          --
+list(d.items())   d.items()          py3compat.iteritems(d)
+list(d.values())  d.values()         py3compat.itervalues(d)
+================  =================  =======================
+
+Iterating over a dictionary yields its keys, so there is rarely a need
+to use :meth:`dict.keys` or :meth:`dict.iterkeys`.
+
+Avoid using :func:`map` to cause function side effects. This is more
+clearly written with a simple for loop.
+
+Changed standard library locations
+----------------------------------
+
+Several parts of the standard library have been renamed and moved. This
+is a short list of things that we're using. A couple of them have names
+in :mod:`IPython.utils.py3compat`, so you don't need both
+imports in each module that uses them.
+
+==================  ============  ===========
+Python 2            Python 3      py3compat
+==================  ============  ===========
+:func:`raw_input`   input         input
+:mod:`__builtin__`  builtins      builtin_mod
+:mod:`StringIO`     io
+:mod:`Queue`        queue
+:mod:`cPickle`      pickle
+:mod:`thread`       _thread
+:mod:`copy_reg`     copyreg
+:mod:`urlparse`     urllib.parse
+:mod:`repr`         reprlib
+:mod:`Tkinter`      tkinter
+:mod:`Cookie`       http.cookie
+:mod:`_winreg`      winreg
+==================  ============  ===========
+
+Be careful with StringIO: :class:`io.StringIO` is available in Python 2.7,
+but it behaves differently from :class:`StringIO.StringIO`, and much of
+our code assumes the use of the latter on Python 2. So a try/except on
+the import may cause problems.
+
+Unicode
+-------
+
+Always be explicit about what is text (unicode) and what is bytes.
+*Encoding* goes from unicode to bytes, and *decoding* goes from bytes
+to unicode.
+
+To open files for reading or writing text, use :func:`io.open`, which is
+the Python 3 builtin ``open`` function, available on Python 2 as well.
+We almost always need to specify the encoding parameter, because the
+default is platform dependent.
+
+Relative imports
+----------------
+
+::
+
+    # This makes Python 2 behave like Python 3:
+    from __future__ import absolute_import
+    
+    import io  # Imports the standard library io module
+    from . import io  # Import the io module from the package
+                      # containing the current module
+    from .io import foo  # foo from the io module next to this module
+    from IPython.utils import io  # This still works
+
+Print function
+--------------
+
+::
+
+    # Support the print function on Python 2:
+    from __future__ import print_function
+    
+    print(a, b)
+    print(foo, file=sys.stderr)
+    print(bar, baz, sep='\t', end='')
+
+Metaclasses
+-----------
+
+The syntax for declaring a class with a metaclass is different in
+Python 2 and 3. In most cases, the helper function
+:func:`~IPython.utils.py3compat.with_metaclass` (copied from the six
+library) can be used like this::
+
+    class FormatterABC(with_metaclass(abc.ABCMeta, object)):
+        ...
+
+Combining inheritance between Qt and the traitlets system, however, does
+not work with this. Instead, we do this::
+
+    class QtKernelClientMixin(MetaQObjectHasTraits('NewBase', (HasTraits, SuperQObject), {})): 
+        ...
+
+This gives the new class a metaclass of :class:`~IPython.qt.util.MetaQObjectHasTraits`,
+and the parent classes :class:`~IPython.utils.traitlets.HasTraits` and
+:class:`~IPython.qt.util.SuperQObject`.

docs/source/whatsnew/pr/single-codebase.rst

@@ -0,0 +1,8 b''
+Single codebase Python 3 support
+--------------------------------
+
+IPython previously supported Python 3 by running 2to3 during setup. We
+have now switched to a single codebase which runs natively on Python 2.7
+and 3.3.
+
+For notes on how to maintain this, see :doc:`/development/pycompat`.

docs/source/conf.py

@@ -1,237 +1,239 b''
 # -*- coding: utf-8 -*-
 #
 # IPython documentation build configuration file.
 
 # NOTE: This file has been edited manually from the auto-generated one from
 # sphinx.  Do NOT delete and re-generate.  If any changes from sphinx are
 # needed, generate a scratch one and merge by hand any new fields needed.
 
 #
 # This file is execfile()d with the current directory set to its containing dir.
 #
 # The contents of this file are pickled, so don't put values in the namespace
 # that aren't pickleable (module imports are okay, they're removed automatically).
 #
 # All configuration values have a default value; values that are commented out
 # serve to show the default value.
 
 import sys, os
 
 ON_RTD = os.environ.get('READTHEDOCS', None) == 'True'
 
 if ON_RTD:
     # Mock the presence of matplotlib, which we don't have on RTD
     # see
     # http://read-the-docs.readthedocs.org/en/latest/faq.html
     tags.add('rtd')
     
 # If your extensions are in another directory, add it here. If the directory
 # is relative to the documentation root, use os.path.abspath to make it
 # absolute, like shown here.
 sys.path.insert(0, os.path.abspath('../sphinxext'))
 
 # We load the ipython release info into a dict by explicit execution
 iprelease = {}
 execfile('../../IPython/core/release.py',iprelease)
 
 # General configuration
 # ---------------------
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 extensions = [
     'matplotlib.sphinxext.mathmpl',
     'matplotlib.sphinxext.only_directives',
     'matplotlib.sphinxext.plot_directive',
     'sphinx.ext.autodoc',
     'sphinx.ext.doctest',
     'sphinx.ext.inheritance_diagram',
+    'sphinx.ext.intersphinx',
     'IPython.sphinxext.ipython_console_highlighting',
     'IPython.sphinxext.ipython_directive',
     'numpydoc',  # to preprocess docstrings
     'github',  # for easy GitHub links
 ]
 
 if ON_RTD:
     # Remove extensions not currently supported on RTD
     extensions.remove('matplotlib.sphinxext.only_directives')
     extensions.remove('matplotlib.sphinxext.mathmpl')
     extensions.remove('matplotlib.sphinxext.plot_directive')
     extensions.remove('IPython.sphinxext.ipython_directive')
     extensions.remove('IPython.sphinxext.ipython_console_highlighting')
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
 # The suffix of source filenames.
 source_suffix = '.rst'
 
 if iprelease['_version_extra']:
     rst_prolog = """
     .. note::
 
         This documentation is for a development version of IPython. There may be
         significant differences from the latest stable release (1.1.0).
 
     """
 
 # The master toctree document.
 master_doc = 'index'
 
 # General substitutions.
 project = 'IPython'
 copyright = '2008, The IPython Development Team'
 
 # ghissue config
 github_project_url = "https://github.com/ipython/ipython"
 
 # The default replacements for |version| and |release|, also used in various
 # other places throughout the built documents.
 #
 # The full version, including alpha/beta/rc tags.
 codename = iprelease['codename']
 release = "%s: %s" % (iprelease['version'], codename)
 # Just the X.Y.Z part, no '-dev'
 version = iprelease['version'].split('-', 1)[0]
 
 
 # There are two options for replacing |today|: either, you set today to some
 # non-false value, then it is used:
 #today = ''
 # Else, today_fmt is used as the format for a strftime call.
 today_fmt = '%B %d, %Y'
 
 # List of documents that shouldn't be included in the build.
 #unused_docs = []
 
 # List of directories, relative to source directories, that shouldn't be searched
 # for source files.
 exclude_dirs = ['attic']
 
 # If true, '()' will be appended to :func: etc. cross-reference text.
 #add_function_parentheses = True
 
 # If true, the current module name will be prepended to all description
 # unit titles (such as .. function::).
 #add_module_names = True
 
 # If true, sectionauthor and moduleauthor directives will be shown in the
 # output. They are ignored by default.
 #show_authors = False
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
 
 
 # Options for HTML output
 # -----------------------
 
 # The style sheet to use for HTML and HTML Help pages. A file of that name
 # must exist either in Sphinx' static/ path, or in one of the custom paths
 # given in html_static_path.
 html_style = 'default.css'
 
 # The name for this set of Sphinx documents.  If None, it defaults to
 # "<project> v<release> documentation".
 #html_title = None
 
 # The name of an image file (within the static path) to place at the top of
 # the sidebar.
 #html_logo = None
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
 # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 # using the given strftime format.
 html_last_updated_fmt = '%b %d, %Y'
 
 # If true, SmartyPants will be used to convert quotes and dashes to
 # typographically correct entities.
 #html_use_smartypants = True
 
 # Custom sidebar templates, maps document names to template names.
 #html_sidebars = {}
 
 # Additional templates that should be rendered to pages, maps page names to
 # template names.
 html_additional_pages = {
                          'interactive/htmlnotebook': 'htmlnotebook.html',
 }
 
 # If false, no module index is generated.
 #html_use_modindex = True
 
 # If true, the reST sources are included in the HTML build as _sources/<name>.
 #html_copy_source = True
 
 # If true, an OpenSearch description file will be output, and all pages will
 # contain a <link> tag referring to it.  The value of this option must be the
 # base URL from which the finished HTML is served.
 #html_use_opensearch = ''
 
 # If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
 #html_file_suffix = ''
 
 # Output file base name for HTML help builder.
 htmlhelp_basename = 'ipythondoc'
 
+intersphinx_mapping = {'http://docs.python.org/2/': None}
 
 # Options for LaTeX output
 # ------------------------
 
 # The paper size ('letter' or 'a4').
 latex_paper_size = 'letter'
 
 # The font size ('10pt', '11pt' or '12pt').
 latex_font_size = '11pt'
 
 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title, author, document class [howto/manual]).
 
 latex_documents = [
     ('index', 'ipython.tex', 'IPython Documentation',
      ur"""The IPython Development Team""", 'manual', True),
     ('parallel/winhpc_index', 'winhpc_whitepaper.tex',
      'Using IPython on Windows HPC Server 2008',
      ur"Brian E. Granger", 'manual', True)
 ]
 
 # The name of an image file (relative to this directory) to place at the top of
 # the title page.
 #latex_logo = None
 
 # For "manual" documents, if this is true, then toplevel headings are parts,
 # not chapters.
 #latex_use_parts = False
 
 # Additional stuff for the LaTeX preamble.
 #latex_preamble = ''
 
 # Documents to append as an appendix to all manuals.
 #latex_appendices = []
 
 # If false, no module index is generated.
 latex_use_modindex = True
 
 
 # Options for texinfo output
 # --------------------------
 
 texinfo_documents = [
   (master_doc, 'ipython', 'IPython Documentation',
    'The IPython Development Team',
    'IPython',
    'IPython Documentation',
    'Programming',
    1),
 ]
 
 
 # Cleanup
 # -------
 # delete release info to avoid pickling errors from sphinx
 
 del iprelease

docs/source/development/index.rst

@@ -1,26 +1,27 b''
 .. _developer_guide:
 
 =========================
 IPython developer's guide
 =========================
 
 This are two categories of developer focused documentation:
 
 1. Documentation for developers of *IPython itself*.
 2. Documentation for developers of third party tools and libraries
    that use IPython.
 
 This part of our documentation only contains information in the second category.
 
 Developers interested in working on IPython itself should consult
 our `developer information <https://github.com/ipython/ipython/wiki/Dev:-Index>`_ 
 on the IPython GitHub wiki.
 
 .. toctree::
    :maxdepth: 1
 
  
    gitwash/index
    messaging
    parallel_messages
    parallel_connections
+   pycompat