Merge pull request #4712 from takluyver/docs-shotgun-3

Various docs fixes

* Tweak exclude patterns, which were accidentally excluding qt.console.frontend_widget
* Added example of adding input transformers (#4709)
* Tidied up and added some module docstrings, now that we're using the first line in the API index
* Update the inputsplitter module docstring, which was rather out of date.
* Fix some header levels introduced in  #4265
* pyreadline is apparently no longer mandatory on Windows (#4544)

IPython/core/inputsplitter.py

@@ -1,58 +1,24
-"""Analysis of text input into executable blocks.
+"""Input handling and transformation machinery.
 
-The main class in this module, :class:`InputSplitter`, is designed to break
-input from either interactive, line-by-line environments or block-based ones,
-into standalone blocks that can be executed by Python as 'single' statements
-(thus triggering sys.displayhook).
+The first class in this module, :class:`InputSplitter`, is designed to tell when
+input from a line-oriented frontend is complete and should be executed, and when
+the user should be prompted for another line of code instead. The name 'input
+splitter' is largely for historical reasons.
 
 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
 with full support for the extended IPython syntax (magics, system calls, etc).
+The code to actually do these transformations is in :mod:`IPython.core.inputtransformer`.
+:class:`IPythonInputSplitter` feeds the raw code to the transformers in order
+and stores the results.
 
-For more details, see the class docstring below.
-
-Syntax Transformations
-----------------------
-
-One of the main jobs of the code in this file is to apply all syntax
-transformations that make up 'the IPython language', i.e. magics, shell
-escapes, etc.  All transformations should be implemented as *fully stateless*
-entities, that simply take one line as their input and return a line.
-Internally for implementation purposes they may be a normal function or a
-callable object, but the only input they receive will be a single line and they
-should only return a line, without holding any data-dependent state between
-calls.
-
-As an example, the EscapedTransformer is a class so we can more clearly group
-together the functionality of dispatching to individual functions based on the
-starting escape character, but the only method for public use is its call
-method.
-
-
-ToDo
-----
-
-- Should we make push() actually raise an exception once push_accepts_more()
-  returns False?
-
-- Naming cleanups.  The tr_* names aren't the most elegant, though now they are
-  at least just attributes of a class so not really very exposed.
-
-- Think about the best way to support dynamic things: automagic, autocall,
-  macros, etc.
-
-- Think of a better heuristic for the application of the transforms in
-  IPythonInputSplitter.push() than looking at the buffer ending in ':'.  Idea:
-  track indentation change events (indent, dedent, nothing) and apply them only
-  if the indentation went up, but not otherwise.
-
-- Think of the cleanest way for supporting user-specified transformations (the
-  user prefilters we had before).
+For more details, see the class docstrings below.
 
 Authors
 -------
 
 * Fernando Perez
 * Brian Granger
+* Thomas Kluyver
 """
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2010  The IPython Development Team
@@ -598,9 +564,9 class IPythonInputSplitter(InputSplitter):
         -------
         is_complete : boolean
           True if the current input source (the result of the current input
-        plus prior inputs) forms a complete Python execution block.  Note that
-        this value is also stored as a private attribute (_is_complete), so it
-        can be queried at any time.
+          plus prior inputs) forms a complete Python execution block.  Note that
+          this value is also stored as a private attribute (_is_complete), so it
+          can be queried at any time.
         """
 
         # We must ensure all input is pure unicode

IPython/core/inputtransformer.py

@@ -1,3 +1,8
+"""Input transformer classes to support IPython special syntax.
+
+This includes the machinery to recognise and transform ``%magic`` commands,
+``!system`` commands, ``help?`` querying, prompt stripping, and so forth.
+"""
 import abc
 import functools
 import re

IPython/core/ultratb.py

@@ -1,6 +1,6
 # -*- coding: utf-8 -*-
 """
-ultratb.py -- Spice up your tracebacks!
+Verbose and colourful traceback formatting.
 
 **ColorTB**
 

IPython/kernel/zmq/datapub.py

@@ -1,4 +1,4
-"""Publishing
+"""Publishing native (typically pickled) objects.
 """
 
 #-----------------------------------------------------------------------------

IPython/kernel/zmq/displayhook.py

@@ -1,3 +1,5
+"""Replacements for sys.displayhook that publish over ZMQ.
+"""
 import sys
 
 from IPython.core.displayhook import DisplayHook

IPython/parallel/util.py

@@ -1,4 +1,4
-"""some generic utilities for dealing with classes, urls, and serialization
+"""Some generic utilities for dealing with classes, urls, and serialization.
 
 Authors:
 

IPython/qt/console/completion_html.py

@@ -1,4 +1,4
-"""a navigable completer for the qtconsole"""
+"""A navigable completer for the qtconsole"""
 # coding : utf-8
 #-----------------------------------------------------------------------------
 # Copyright (c) 2012, IPython Development Team.$

IPython/qt/console/completion_plain.py

@@ -1,4 +1,4
-"""a simple completer for the qtconsole"""
+"""A simple completer for the qtconsole"""
 #-----------------------------------------------------------------------------
 # Copyright (c) 2012, IPython Development Team.$
 #

IPython/qt/console/completion_widget.py

@@ -1,3 +1,4
+"""A dropdown completer widget for the qtconsole."""
 # System library imports
 from IPython.external.qt import QtCore, QtGui
 

IPython/qt/console/ipython_widget.py

@@ -1,5 +1,6
-""" A FrontendWidget that emulates the interface of the console IPython and
-    supports the additional functionality provided by the IPython kernel.
+"""A FrontendWidget that emulates the interface of the console IPython.
+
+This supports the additional functionality provided by the IPython kernel.
 """
 
 #-----------------------------------------------------------------------------

IPython/terminal/console/completer.py

@@ -1,3 +1,5
+"""Adapt readline completer interface to make ZMQ request.
+"""
 # -*- coding: utf-8 -*-
 import readline
 try:

IPython/utils/text.py

@@ -310,7 +310,8 def list_strings(arg):
     """Always return a list of strings, given a string or list of strings
     as input.
 
-    :Examples:
+    Examples
+    --------
     ::
 
         In [7]: list_strings('A single string')
@@ -330,7 +331,8 def list_strings(arg):
 def marquee(txt='',width=78,mark='*'):
     """Return the input string centered in a 'marquee'.
 
-    :Examples:
+    Examples
+    --------
     ::
 
         In [16]: marquee('A test',40)

IPython/utils/zmqrelated.py

@@ -1,4 +1,4
-"""utilities for checking zmq versions"""
+"""Utilities for checking zmq versions."""
 #-----------------------------------------------------------------------------
 #  Copyright (C) 2013  The IPython Development Team
 #

docs/autogen_api.py

@@ -30,7 +30,7 if __name__ == '__main__':
                                         r'\.ipdoctest',
                                         r'\.testing\.plugin',
                                         # This just prints a deprecation msg:
-                                        r'\.frontend',
+                                        r'\.frontend$',
                                         ]
     
     # We're rarely working on machines with the Azure SDK installed, so we

docs/source/config/inputtransforms.rst

@@ -41,7 +41,10 attributes :attr:`~IPython.core.interactiveshell.InteractiveShell.input_splitter
 to tell when a block of input is complete, and
 :attr:`~IPython.core.interactiveshell.InteractiveShell.input_transformer_manager`,
 to transform complete cells. If you add a transformer, you should make sure that
-it gets added to both.
+it gets added to both, e.g.::
+
+    ip.input_splitter.logical_line_transforms.append(my_transformer())
+    ip.input_transformer_manager.logical_line_transforms.append(my_transformer())
 
 Stateless transformations
 -------------------------

docs/source/development/config.rst

@@ -97,7 +97,7 When both Python and JSON configuration file are present, both will be loaded,
 with JSON configuration having higher priority.
 
 Python configuration Files
-~~~~~~~~~~~~~~~~~~~~~~~~~~
+--------------------------
 
 A Python configuration file is a pure Python file that populates a configuration object.
 This configuration object is a :class:`~IPython.config.loader.Config` instance.
@@ -187,7 +187,7 attribute of ``c`` is not the actual class, but instead is another
     hierarchical information created easily (``c.Foo.Bar.value``) on the fly.
 
 JSON configuration Files
-~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------
 
 A JSON configuration file is simply a file that contains a
 :class:`~IPython.config.loader.Config` dictionary serialized to JSON.

docs/source/install/install.rst

@@ -198,7 +198,8 read on for more details.
 readline
 --------
 
-As indicated above, on Windows, PyReadline is a *mandatory* dependency.
+As indicated above, on Windows, to get full functionality in the console
+version of IPython, PyReadline is needed.
 PyReadline is a separate, Windows only implementation of readline that uses
 native Windows calls through :mod:`ctypes`. The easiest way of installing
 PyReadline is you use the binary installer available `here