Miscellaneous doc fixes

docs/source/config/extensions/storemagic.rst

@@ -1,8 +1,9 b''
 .. _extensions_storemagic:
 
 ==========
 storemagic
 ==========
 
 .. automodule:: IPython.extensions.storemagic
-   :members: store
+
+.. automethod:: StoreMagics.store

docs/source/interactive/notebook.rst

@@ -1,495 +1,495 b''
 .. _htmlnotebook:
 
 The IPython Notebook
 ====================
 
 Introduction
 ------------
 
 The notebook extends the console-based approach to interactive computing in
 a qualitatively new direction, providing a web-based application suitable for
 capturing the whole computation process: developing, documenting, and
 executing code, as well as communicating the results.  The IPython notebook
 combines two components:
 
 **A web application**: a browser-based tool for interactive authoring of
 documents which combine explanatory text, mathematics, computations and their
 rich media output.
 
 **Notebook documents**: a representation of all content visible in the web
 application, including inputs and outputs of the computations, explanatory
 text, mathematics, images, and rich media representations of objects.
 
 .. seealso::
 
     See the :ref:`installation documentation <installnotebook>` for directions
     on how to install the notebook and its dependencies.
 
 
 Main features of the web application
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 * In-browser editing for code, with automatic syntax highlighting,
   indentation, and tab completion/introspection.
 
 * The ability to execute code from the browser, with the results of
   computations attached to the code which generated them.
 
 * Displaying the result of computation using rich media representations, such
   as HTML, LaTeX, PNG, SVG, etc. For example, publication-quality figures
   rendered by the matplotlib_ library, can be included inline.
 
 * In-browser editing for rich text using the Markdown_ markup language, which
   can provide commentary for the code, is not limited to plain text.
 
 * The ability to easily include mathematical notation within markdown cells
   using LaTeX, and rendered natively by MathJax_.
 
 
 
 .. _MathJax: http://www.mathjax.org/
 
 
 Notebook documents
 ~~~~~~~~~~~~~~~~~~
 Notebook documents contains the inputs and outputs of a interactive session as
 well as additional text that accompanies the code but is not meant for
 execution.  In this way, notebook files can serve as a complete computational
 record of a session, interleaving executable code with explanatory text,
 mathematics, and rich representations of resulting objects. These documents
 are internally JSON_ files and are saved with the ``.ipynb`` extension. Since
 JSON is a plain text format, they can be version-controlled and shared with
 colleagues.  
    
 .. _JSON: http://en.wikipedia.org/wiki/JSON
 
 Notebooks may be exported to a range of static formats, including HTML (for
 example, for blog posts), reStructeredText, LaTeX, PDF, and slide shows, via
 the new :ref:`nbconvert <nbconvert>` command.
 
 Furthermore, any  ``.ipynb`` notebook document available from a public 
 URL can be shared via the `IPython Notebook Viewer <nbviewer>`_ (nbviewer_).
 This service loads the notebook document from the URL and renders it as a
 static web page.  The results may thus be shared with a colleague, or as a
 public blog post, without other users needing to install IPython themselves.
 In effect, nbviewer_ is simply :ref:`nbconvert <nbconvert>` as a web service,
 so you can do your own static conversions with nbconvert, without relying on
 nbviewer.
 
 
 
 .. seealso::
 
     :ref:`Details on the notebook JSON file format <notebook_format>` 
    
 
 Starting the notebook server
 ----------------------------
 
 You can start running a notebook server from the command line using the
 following command::
 
     ipython notebook
 
 This will print some information about the notebook server in your console,
 and open a web browser to the URL of the web application (by default,
 ``http://127.0.0.1:8888``).
 
 The landing page of the IPython notebook web application, the **dashboard**,
 shows the notebooks currently available in the notebook directory (by default,
 the directory from which the notebook server was started).
 
 You can create new notebooks from the dashboard with the ``New Notebook``
 button, or open existing ones by clicking on their name.  You can also drag
 and drop ``.ipynb`` notebooks and standard ``.py`` Python source code files
 into the notebook list area.
 
 When starting a notebook server from the command line, you can also open a
 particular notebook directly, bypassing the dashboard, with ``ipython notebook
 my_notebook.ipynb``. The ``.ipynb`` extension is assumed if no extension is
 given.
 
 When you are inside an open notebook, the `File | Open...` menu option will
 open the dashboard in a new browser tab, to allow you to open another notebook
 from the notebook directory or to create a new notebook.
 
 
 .. note::
 
    You can start more than one notebook server at the same time, if you want 
    to work on notebooks in different directories.  By default the first 
    notebook server starts on port 8888, and later notebook servers search for  
    ports near that one.  You can also manually specify the port with the 
    ``--port`` option.
 
 Creating a new notebook document
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 A new notebook may be created at any time, either from the dashboard, or using
 the `File | New` menu option from within an active notebook. The new notebook
 is created within the same directory and will open in a new browser tab. It
 will also be reflected as a new entry in the notebook list on the dashboard.
 
 
 Opening notebooks
 ~~~~~~~~~~~~~~~~~
 An open notebook has **exactly one** interactive session connected to an
 :ref:`IPython kernel <ipythonzmq>`, which will execute code sent by the user
 and communicate back results.  This kernel remains active if the web browser
 window is closed, and reopening the same notebook from the dashboard will
 reconnect the web application to the same kernel. In the dashboard, notebooks
 with an active kernel have a ``Shutdown`` button next to them, whereas
 notebooks without an active kernel have a ``Delete`` button in its place.
 
 Other clients may connect to the same underlying IPython kernel.
 The notebook server always prints to the terminal the full details of 
 how to connect to each kernel, with messages such as the following::
 
     [NotebookApp] Kernel started: 87f7d2c0-13e3-43df-8bb8-1bd37aaf3373
 
 This long string is the kernel's ID which is sufficient for getting the
 information necessary to connect to the kernel.  You can also request this
 connection data by running the ``%connect_info`` :ref:`magic
 <magics_explained>`. This will print the same ID information as well as the
 content of the JSON data structure it contains.
 
 You can then, for example, manually start a Qt console connected to the *same*
 kernel from the command line, by passing a portion of the ID::
 
     $ ipython qtconsole --existing 87f7d2c0
 
 Without an ID, ``--existing`` will  connect to the most recently 
 started kernel. This can also be done by running the ``%qtconsole``
 :ref:`magic <magics_explained>` in the notebook.
 
 .. seealso::
 
     :ref:`ipythonzmq`
 
 Notebook user interface
 -----------------------
 
 When you create a new notebook document, you will be presented with the
 **notebook name**, a **menu bar**, a **toolbar** and an empty **code
 cell**.
 
 **notebook name**: The name of the notebook document is displayed at the top
 of the page, next to the ``IP[y]: Notebook`` logo. This name reflects the name
 of the ``.ipynb`` notebook document file.  Clicking on the notebook name
 brings up a dialog which allows you to rename it. Thus, renaming a notebook
 from "Untitled0" to "My first notebook" in the browser, renames the
 ``Untitled0.ipynb`` file to ``My first notebook.ipynb``. 
 
 **menu bar**: The menu bar presents different options that may be used to
 manipulate the way the notebook functions.
 
 **toolbar**: The tool bar gives a quick way of performing the most-used
 operations within the notebook, by clicking on an icon.
 
 **code cell**: the default type of cell, read on for an explanation of cells
 
 
 Structure of a notebook document
 --------------------------------
 
 The notebook consists of a sequence of cells.  A cell is a multi-line
 text input field, and its contents can be executed by using
 :kbd:`Shift-Enter`, or by clicking either the "Play" button the toolbar, or
 `Cell | Run` in the menu bar.  The execution behavior of a cell is determined
 the cell's type.  There are four types of cells: **code cells**, **markdown
 cells**, **raw cells** and **heading cells**.  Every cell starts off
 being a **code cell**, but its type can be changed by using a dropdown on the
 toolbar (which will be "Code", initially), or via :ref:`keyboard shortcuts
 <keyboard-shortcuts>`.  
 
 For more information on the different things you can do in a notebook,
 see the `collection of examples
 <https://github.com/ipython/ipython/tree/master/examples/notebooks#readme>`_.
 
 Code cells
 ~~~~~~~~~~
 A *code cell* allows you to edit and write new code, with full syntax
 highlighting and tab completion. By default, the language associated to a code
 cell is Python, but other languages, such as ``Julia`` and ``R``, can be
 handled using :ref:`cell magic commands <magics_explained>`.
 
 When a code cell is executed, code that it contains is sent to the kernel
 associated with the notebook.  The results that are returned from this 
 computation  are then displayed in the notebook as the cell's *output*. The
 output is not limited to text, with many other possible forms of output are
 also possible, including ``matplotlib`` figures and HTML tables (as used, for 
 example, in the ``pandas`` data analysis package). This is known as IPython's 
 *rich display* capability.
 
 .. seealso::
 
     `Basic Output`_  example notebook
 
     `Rich Display System`_  example notebook
 
 Markdown cells
 ~~~~~~~~~~~~~~
 You can document the computational process in a literate way, alternating 
 descriptive text with code, using *rich text*. In IPython this is accomplished 
 by marking up text with the Markdown language. The corresponding cells are 
 called *Markdown cells*. The Markdown language provides a simple way to 
 perform this text markup, that is, to specify which parts of the text should 
 be emphasized (italics), bold, form lists, etc. 
 
 
 When a Markdown cell is executed, the Markdown code is converted into 
 the corresponding formatted rich text. Markdown allows arbitrary HTML code for
 formatting.
 
 Within Markdown cells, you can also include *mathematics* in a straightforward 
 way, using standard LaTeX notation: ``$...$`` for inline mathematics and 
 ``$$...$$`` for displayed mathematics. When the Markdown cell is executed, 
 the LaTeX portions are automatically rendered in the HTML output as equations 
 with high quality typography. This is made possible by MathJax_, which 
 supports a `large subset <mathjax_tex>`_ of LaTeX functionality 
 
 .. _mathjax_tex: http://docs.mathjax.org/en/latest/tex.html
 
 Standard mathematics environments defined by LaTeX and AMS-LaTeX (the 
 `amsmath` package) also work, such as 
 ``\begin{equation}...\end{equation}``, and ``\begin{align}...\end{align}``.
 New LaTeX macros may be defined using standard methods, 
 such as ``\newcommand``, by placing them anywhere *between math delimiters* in 
 a Markdown cell. These definitions are then available throughout the rest of 
 the IPython session. 
 
 .. seealso::
 
     `Markdown Cells`_ example notebook
 
 Raw cells
 ~~~~~~~~~
 
 *Raw* cells provide a place in which you can write *output* directly.
 Raw cells are not evaluated by the notebook.
 When passed through :ref:`nbconvert <nbconvert>`, raw cells arrive in the
 destination format unmodified. For example, this allows you to type full LaTeX
 into a raw cell, which will only be rendered by LaTeX after conversion by
 nbconvert.
 
 Heading cells
 ~~~~~~~~~~~~~
 
 You can provide a conceptual structure for your computational document as a 
 whole using different levels of headings; there are 6 levels available, from 
 level 1 (top level) down to level 6 (paragraph). These can be used later for 
 constructing tables of contents, etc.  As with Markdown cells, a heading 
 cell is replaced by a rich text rendering of the heading when the cell is
 executed.
 
 
 Basic workflow
 --------------
 
 The normal workflow in a notebook is, then, quite similar to a standard 
 IPython session, with the difference that you can edit cells in-place multiple 
 times until you obtain the desired results, rather than having to 
 rerun separate scripts with the ``%run`` magic command. 
 
 
 Typically, you will work on a computational problem in pieces, organizing 
 related ideas into cells and moving forward once previous parts work 
 correctly. This is much more convenient for interactive exploration than 
 breaking up a computation into scripts that must be executed together, as was 
 previously necessary, especially if parts of them take a long time to run.
 
 At certain moments, it may be necessary to interrupt a calculation which is 
 taking too long to complete. This may be done with the `Kernel | Interrupt` 
 menu option, or the :kbd:`Ctrl-m i` keyboard shortcut.
 Similarly, it may be necessary or desirable to restart the whole computational 
 process, with the `Kernel | Restart` menu option or :kbd:`Ctrl-m .` 
 shortcut. 
 
 A notebook may be downloaded in either a ``.ipynb`` or ``.py`` file from the
 menu option `File | Download as`. Choosing the ``.py`` option downloads a
 Python ``.py`` script, in which all rich output has been removed and the
 content of markdown cells have been inserted as comments.  
 
 .. seealso::
 
     `Running Code in the IPython Notebook`_ example notebook
 
     `Basic Output`_ example notebook
 
     :ref:`a warning about doing "roundtrip" conversions <note_about_roundtrip>`.
 
 .. _keyboard-shortcuts:
 
 Keyboard shortcuts
 ~~~~~~~~~~~~~~~~~~
 All actions in the notebook can be performed with the mouse, but keyboard 
 shortcuts are also available for the most common ones. The essential shortcuts
 to remember are the following:
 
 * :kbd:`Shift-Enter`:  run cell
     Execute the current cell, show output (if any), and jump to the next cell 
     below. If :kbd:`Shift-Enter` is invoked on the last cell, a new code 
     cell will also be created. Note that in the notebook, typing :kbd:`Enter` 
     on its own *never* forces execution, but rather just inserts a new line in 
     the current cell. :kbd:`Shift-Enter` is equivalent to clicking the 
     ``Cell | Run`` menu item.
 
 * :kbd:`Ctrl-Enter`: run cell in-place
     Execute the current cell as if it were in "terminal mode", where any 
     output is shown, but the cursor *remains* in the current cell. The cell's
     entire contents are selected after execution, so you can just start typing
     and only the new input will be in the cell. This is convenient for doing
     quick experiments in place, or for querying things like filesystem
     content, without needing to create additional cells that you may not want
     to be saved in the notebook.
 
 * :kbd:`Alt-Enter`: run cell, insert below
     Executes the current cell, shows the output, and inserts a *new* 
     cell between the current cell and the cell below (if one exists). This  
     is thus a shortcut for the sequence :kbd:`Shift-Enter`, :kbd:`Ctrl-m a`.
     (:kbd:`Ctrl-m a` adds a new cell above the current one.)
   
 * :kbd:`Ctrl-m`: 
   This is the prefix for *all* other shortcuts, which consist of :kbd:`Ctrl-m` 
   followed by a single letter or character. For example, if you type 
   :kbd:`Ctrl-m h` (that is, the sole letter :kbd:`h` after :kbd:`Ctrl-m`), 
   IPython will show you all the available keyboard shortcuts.
 
 
 ..
     TODO: these live in IPython/html/static/notebook/js/quickhelp.js
     They were last updated for IPython 1.0 release, so update them again for
     future releases.
 
 Here is the complete set of keyboard shortcuts available:
 
 ============  ==========================
 **Shortcut**        **Action**
 ------------  --------------------------
 Shift-Enter    run cell
 Ctrl-Enter     run cell in-place
 Alt-Enter      run cell, insert below
 Ctrl-m x       cut cell
 Ctrl-m c       copy cell
 Ctrl-m v       paste cell
 Ctrl-m d       delete cell
 Ctrl-m z       undo last cell deletion
 Ctrl-m -       split cell
 Ctrl-m a       insert cell above
 Ctrl-m b       insert cell below
 Ctrl-m o       toggle output
 Ctrl-m O       toggle output scroll
 Ctrl-m l       toggle line numbers
 Ctrl-m s       save notebook
 Ctrl-m j       move cell down
 Ctrl-m k       move cell up
 Ctrl-m y       code cell
 Ctrl-m m       markdown cell
 Ctrl-m t       raw cell
 Ctrl-m 1-6     heading 1-6 cell
 Ctrl-m p       select previous
 Ctrl-m n       select next
 Ctrl-m i       interrupt kernel
 Ctrl-m .       restart kernel
 Ctrl-m h       show keyboard shortcuts
 ============  ==========================
 
    
 
 
 
 Plotting
 --------
 One major feature of the notebook is the ability to display plots that are the
 output of running code cells. IPython is designed to work seamlessly with the
 matplotlib_ plotting library to provide this functionality. 
 
 To set this up, before any plotting is performed you must execute the
 ``%matplotlib``  :ref:`magic command <magics_explained>`. This performs the
 necessary behind-the-scenes setup for IPython to work correctly hand in hand
 with ``matplotlib``; it does *not*, however, actually execute any Python
 ``import`` commands, that is, no names are added to the namespace.
 
 If the ``%matplotlib`` magic is called without an argument, the 
 output of a plotting command is displayed using the default ``matplotlib`` 
 backend in a separate window. Alternatively, the backend can be explicitly 
 requested using, for example::
 
   %matplotlib gtk
 
 A particularly interesting backend, provided by IPython, is the ``inline``
 backend.  This is available only for the IPython Notebook and the
 :ref:`IPython QtConsole <qtconsole>`.  It can be invoked as follows::
 
   %matplotlib inline
 
 With this backend, the output of plotting commands is displayed *inline*
 within the notebook, directly below the code cell that produced it. The
 resulting plots will then also be stored in the notebook document. 
 
 .. seealso::
 
     `Plotting with Matplotlib`_  example notebook
 
 
 Configuring the IPython Notebook
 --------------------------------
 The notebook server can be run with a variety of command line arguments. 
 To see a list of available options enter::
 
   $ ipython notebook --help 
 
 Defaults for these options can also be set by creating a file named 
 ``ipython_notebook_config.py`` in your IPython *profile folder*. The profile 
 folder is a subfolder of your IPython directory; to find out where it is 
 located, run::
 
   $ ipython locate
 
 To create a new set of default configuration files, with lots of information 
 on available options, use::
 
   $ ipython profile create
 
 .. seealso::
 
     :ref:`config_overview`, in particular :ref:`Profiles`.
     
     :ref:`notebook_security`
 
     :ref:`notebook_public_server`
 
 
 Importing ``.py`` files
 -----------------------
 
 ``.py`` files will be imported as a notebook with 
 the same basename, but an ``.ipynb`` extension, located in the notebook 
 directory. The notebook created will have just one cell, which will contain 
 all the code in the ``.py`` file. You can later manually partition this into 
 individual cells using the ``Edit | Split Cell`` menu option, or the 
 :kbd:`Ctrl-m -` keyboard shortcut.
 
-Note that ``.py`` scripts obtained from a notebook document using nbconvert_ 
+Note that ``.py`` scripts obtained from a notebook document using :doc:`nbconvert <nbconvert>`
 maintain the structure of the notebook in comments. Reimporting such a
 script back into a notebook will preserve this structure.
 
 .. _note_about_roundtrip:
 
 .. warning::
 
    While in simple cases you can "roundtrip" a notebook to Python, edit the
    Python file, and then import it back without loss of main content, this is 
    in general *not guaranteed to work*.  First, there is extra metadata
    saved in the notebook that may not be saved to the ``.py`` format.  And as
    the notebook format evolves in complexity, there will be attributes of the
    notebook that will not survive a roundtrip through the Python form.  You
    should think of the Python format as a way to output a script version of a
    notebook and the import capabilities as a way to load existing code to get 
    a notebook started.  But the Python version is *not* an alternate notebook
    format.
 
 .. seealso::
     :ref:`notebook_format`
 
 .. include:: ../links.txt

docs/source/interactive/public_server.rst

@@ -1,163 +1,163 b''
-.. _working_remotely.txt
+.. _working_remotely:
 
 Running a notebook server
 =========================
 
 
 The  :ref:`IPython notebook <htmlnotebook>` web-application is based on a
 server-client structure.  This server uses a :ref:`two-process kernel
 architecture <ipythonzmq>` based on ZeroMQ_, as well as Tornado_ for serving
 HTTP requests. By default, a notebook server runs on http://127.0.0.1:8888/
 and is accessible only from `localhost`. This document describes how you can
 :ref:`secure a notebook server <notebook_security>` and how to :ref:`run it on
 a public interface <notebook_public_server>`.
 
 .. _ZeroMQ: http://zeromq.org
 
 .. _Tornado: http://www.tornadoweb.org
 
 
 .. _notebook_security:
 
 Notebook security
 -----------------
 
 You can protect your notebook server with a simple single password by
 setting the :attr:`NotebookApp.password` configurable. You can prepare a
 hashed password using the function :func:`IPython.lib.security.passwd`:
 
 .. sourcecode:: ipython
 
     In [1]: from IPython.lib import passwd
     In [2]: passwd()
     Enter password: 
     Verify password: 
     Out[2]: 'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed'
     
 .. note::
 
   :func:`~IPython.lib.security.passwd` can also take the password as a string
   argument. **Do not** pass it as an argument inside an IPython session, as it
   will be saved in your input history.
 
 You can then add this to your :file:`ipython_notebook_config.py`, e.g.::
 
     # Password to use for web authentication
     c = get_config()
     c.NotebookApp.password = 
     u'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed'
 
 When using a password, it is a good idea to also use SSL, so that your 
 password is not sent unencrypted by your browser. You can start the notebook 
 to communicate via a secure protocol mode using a self-signed certificate with 
 the command::
 
     $ ipython notebook --certfile=mycert.pem
 
 .. note::
 
     A self-signed certificate can be generated with ``openssl``.  For example, 
     the following command will create a certificate valid for 365 days with 
     both the key and certificate data written to the same file::
 
         $ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem
 
 Your browser will warn you of a dangerous certificate because it is
 self-signed.  If you want to have a fully compliant certificate that will not
 raise warnings, it is possible (but rather involved) to obtain one,
 as explained in detail in `this tutorial`__.
 
 .. __: http://arstechnica.com/security/news/2009/12/how-to-get-set-with-a-secure-sertificate-for-free.ars
 	
 Keep in mind that when you enable SSL support, you will need to access the
 notebook server over ``https://``, not over plain ``http://``.  The startup
 message from the server prints this, but it is easy to overlook and think the
 server is for some reason non-responsive.
 
 
 .. _notebook_public_server:
 
 Running a public notebook server
 --------------------------------
 
 If you want to access your notebook server remotely via a web browser,
 you can do the following.  
 
 Start by creating a certificate file and a hashed password, as explained 
 above.  Then create a custom profile for the notebook, with the following 
 command line, type::
 
   $ ipython profile create nbserver
 
 In the profile directory just created, edit the file 
 ``ipython_notebook_config.py``.  By default, the file has all fields 
 commented; the minimum set you need to uncomment and edit is the following::
 
      c = get_config()
 
      # Kernel config
      c.IPKernelApp.pylab = 'inline'  # if you want plotting support always
 
      # Notebook config
      c.NotebookApp.certfile = u'/absolute/path/to/your/certificate/mycert.pem'
      c.NotebookApp.ip = '*'
      c.NotebookApp.open_browser = False
      c.NotebookApp.password = u'sha1:bcd259ccf...[your hashed password here]'
      # It is a good idea to put it on a known, fixed port
      c.NotebookApp.port = 9999
 
 You can then start the notebook and access it later by pointing your browser 
 to ``https://your.host.com:9999`` with ``ipython notebook 
 --profile=nbserver``.
 
 Running with a different URL prefix
 -----------------------------------
 
 The notebook dashboard (the landing page with an overview
 of the notebooks in your working directory) typically lives at the URL
 ``http://localhost:8888/``. If you prefer that it lives, together with the 
 rest of the notebook, under a sub-directory,
 e.g. ``http://localhost:8888/ipython/``, you can do so with
 configuration options like the following (see above for instructions about
 modifying ``ipython_notebook_config.py``)::
 
     c.NotebookApp.base_project_url = '/ipython/'
     c.NotebookApp.base_kernel_url = '/ipython/'
     c.NotebookApp.webapp_settings = {'static_url_prefix':'/ipython/static/'}
 
 Using a different notebook store
 --------------------------------
 
 By default, the notebook server stores the notebook documents that it saves as 
 files in the working directory of the notebook server, also known as the
 ``notebook_dir``. This  logic is implemented in the 
 :class:`FileNotebookManager` class. However, the server can be configured to 
 use a different notebook manager class, which can 
 store the notebooks in a different format. 
 
 Currently, we ship a :class:`AzureNotebookManager` class that stores notebooks 
 in Azure blob storage. This can be used by adding the following lines to your 
 ``ipython_notebook_config.py`` file::
 
     c.NotebookApp.notebook_manager_class = 
     'IPython.html.services.notebooks.azurenbmanager.AzureNotebookManager'
     c.AzureNotebookManager.account_name = u'paste_your_account_name_here'
     c.AzureNotebookManager.account_key = u'paste_your_account_key_here'
     c.AzureNotebookManager.container = u'notebooks'
 
 In addition to providing your Azure Blob Storage account name and key, you 
 will have to provide a container name; you can use multiple containers to 
 organize your notebooks.
 
 
 Known issues
 ------------
 
 When behind a proxy, especially if your system or browser is set to autodetect
 the proxy, the notebook web application might fail to connect to the server's
 websockets, and present you with a warning at startup. In this case, you need
 to configure your system not to use the proxy for the server's address.
 
 For example, in Firefox, go to the Preferences panel, Advanced section,
 Network tab, click 'Settings...', and add the address of the notebook server
 to the 'No proxy for' field.

docs/source/interactive/qtconsole.rst

@@ -1,610 +1,610 b''
 .. _qtconsole:
 
 =========================
 A Qt Console for IPython
 =========================
 
 We now have a version of IPython, using the new two-process :ref:`ZeroMQ Kernel
 <ipythonzmq>`, running in a PyQt_ GUI.  This is a very lightweight widget that
 largely feels like a terminal, but provides a number of enhancements only
 possible in a GUI, such as inline figures, proper multiline editing with syntax
 highlighting, graphical calltips, and much more.
 
-.. figure:: ../../_images/qtconsole.png
+.. figure:: ../_images/qtconsole.png
     :width: 400px
     :alt: IPython Qt console with embedded plots
     :align: center
     :target: ../_images/qtconsole.png
 
     The Qt console for IPython, using inline matplotlib plots.
 
 To get acquainted with the Qt console, type `%guiref` to see a quick
 introduction of its main features.
 
 The Qt frontend has hand-coded emacs-style bindings for text navigation. This
 is not yet configurable.
 
 .. tip::
 
    Since the Qt console tries hard to behave like a terminal, by default it
    immediately executes single lines of input that are complete.  If you want
    to force multiline input, hit :kbd:`Ctrl-Enter` at the end of the first line
    instead of :kbd:`Enter`, and it will open a new line for input.  At any
    point in a multiline block, you can force its execution (without having to
    go to the bottom) with :kbd:`Shift-Enter`.
 
 ``%load``
 =========
 
 The new ``%load`` magic (previously ``%loadpy``) takes any script, and pastes
 its contents as your next input, so you can edit it before executing. The
 script may be on your machine, but you can also specify an history range, or a
 url, and it will download the script from the web. This is particularly useful
 for playing with examples from documentation, such as matplotlib.
 
 .. sourcecode:: ipython
 
     In [6]: %load http://matplotlib.org/plot_directive/mpl_examples/mplot3d/contour3d_demo.py
 
     In [7]: from mpl_toolkits.mplot3d import axes3d
        ...: import matplotlib.pyplot as plt
        ...:
        ...: fig = plt.figure()
        ...: ax = fig.add_subplot(111, projection='3d')
        ...: X, Y, Z = axes3d.get_test_data(0.05)
        ...: cset = ax.contour(X, Y, Z)
        ...: ax.clabel(cset, fontsize=9, inline=1)
        ...:
        ...: plt.show()
 
 Inline Matplotlib
 =================
 
 One of the most exciting features of the QtConsole is embedded matplotlib
 figures. You can use any standard matplotlib GUI backend
 to draw the figures, and since there is now a two-process model, there is no
 longer a conflict between user input and the drawing eventloop.
 
 .. image:: figs/besselj.png
     :width: 519px
 
 .. _display:
 
 :func:`display`
 ***************
 
 IPython provides a function :func:`display` for displaying rich representations
 of objects if they are available. The IPython display
 system provides a mechanism for specifying PNG or SVG (and more)
 representations of objects for GUI frontends.
 When you enable matplotlib integration via the ``%matplotlib`` magic, IPython registers
 convenient PNG and SVG renderers for matplotlib figures, so you can embed them
 in your document by calling :func:`display` on one or more of them. This is
 especially useful for saving_ your work.
 
 .. sourcecode:: ipython
 
     In [4]: from IPython.display import display
     
     In [5]: plt.plot(range(5)) # plots in the matplotlib window
     
     In [6]: display(plt.gcf()) # embeds the current figure in the qtconsole
     
     In [7]: display(*getfigs()) # embeds all active figures in the qtconsole
 
 If you have a reference to a matplotlib figure object, you can always display
 that specific figure:
 
 .. sourcecode:: ipython
 
    In [1]: f = plt.figure()
 
    In [2]: plt.plot(np.rand(100))
    Out[2]: [<matplotlib.lines.Line2D at 0x7fc6ac03dd90>]
 
    In [3]: display(f)
 
    # Plot is shown here
 
    In [4]: plt.title('A title')
    Out[4]: <matplotlib.text.Text at 0x7fc6ac023450>
 
    In [5]: display(f)
 
    # Updated plot with title is shown here.
 
 .. _inline:
 
 ``--matplotlib inline``
 ***********************
 
 If you want to have all of your figures embedded in your session, instead of
 calling :func:`display`, you can specify ``--matplotlib inline`` when you start the
 console, and each time you make a plot, it will show up in your document, as if
 you had called :func:`display(fig)`.
 
 The inline backend can use either SVG or PNG figures (PNG being the default).
 It also supports the special key ``'retina'``, which is 2x PNG for high-DPI displays.
 To switch between them, set the ``InlineBackend.figure_format`` configurable
 in a config file, or via the ``%config`` magic:
 
 .. sourcecode:: ipython
 
     In [10]: %config InlineBackend.figure_format = 'svg'
 
 .. note::
 
     Changing the inline figure format also affects calls to :func:`display` above,
     even if you are not using the inline backend for all figures.
 
 By default, IPython closes all figures at the completion of each execution. This means you
 don't have to manually close figures, which is less convenient when figures aren't attached
 to windows with an obvious close button.  It also means that the first matplotlib call in
 each cell will always create a new figure:
 
 .. sourcecode:: ipython
 
     In [11]: plt.plot(range(100))
     <single-line plot>
     
     In [12]: plt.plot([1,3,2])
     <another single-line plot>
 
 
 However, it does prevent the list of active figures surviving from one input cell to the
 next, so if you want to continue working with a figure, you must hold on to a reference to
 it:
 
 .. sourcecode:: ipython
 
     In [11]: fig = gcf()
        ....: fig.plot(rand(100))
     <plot>
     In [12]: fig.title('Random Title')
     <redraw plot with title>
 
 This behavior is controlled by the :attr:`InlineBackend.close_figures` configurable, and
 if you set it to False, via %config or config file, then IPython will *not* close figures,
 and tools like :func:`gcf`, :func:`gca`, :func:`getfigs` will behave the same as they
 do with other backends.  You will, however, have to manually close figures:
 
 .. sourcecode:: ipython
 
     # close all active figures:
     In [13]: [ fig.close() for fig in getfigs() ]
 
 
 
 .. _saving:
 
 Saving and Printing
 ===================
 
 IPythonQt has the ability to save your current session, as either HTML or
 XHTML. If you have been using :func:`display` or inline_ matplotlib, your figures
 will be PNG in HTML, or inlined as SVG in XHTML. PNG images have the option to
 be either in an external folder, as in many browsers' "Webpage, Complete"
 option, or inlined as well, for a larger, but more portable file.
 
 .. note::
 
     Export to SVG+XHTML requires that you are using SVG figures, which is *not*
     the default.  To switch the inline figure format to use SVG during an active
     session, do:
     
     .. sourcecode:: ipython
     
         In [10]: %config InlineBackend.figure_format = 'svg'
     
     Or, you can add the same line (c.Inline... instead of %config Inline...) to
     your config files.
     
     This will only affect figures plotted after making this call
 
 
 The widget also exposes the ability to print directly, via the default print
 shortcut or context menu.
 
 
 .. Note:: 
 
     Saving is only available to richtext Qt widgets, which are used by default,
     but if you pass the ``--plain`` flag, saving will not be available to you.
 
 
 See these examples of :download:`png/html<figs/jn.html>` and
 :download:`svg/xhtml <figs/jn.xhtml>` output. Note that syntax highlighting
 does not survive export. This is a known issue, and is being investigated.
 
 
 Colors and Highlighting
 =======================
 
 Terminal IPython has always had some coloring, but never syntax
 highlighting. There are a few simple color choices, specified by the ``colors``
 flag or ``%colors`` magic:
 
 * LightBG for light backgrounds
 * Linux for dark backgrounds
 * NoColor for a simple colorless terminal
 
 The Qt widget has full support for the ``colors`` flag used in the terminal shell.
 
 The Qt widget, however, has full syntax highlighting as you type, handled by
 the `pygments`_ library. The ``style`` argument exposes access to any style by
 name that can be found by pygments, and there are several already
 installed. The ``colors`` argument, if unspecified, will be guessed based on
 the chosen style. Similarly, there are default styles associated with each
 ``colors`` option.
 
 
 Screenshot of ``ipython qtconsole --colors=linux``, which uses the 'monokai'
 theme by default:
 
 .. image:: figs/colors_dark.png
     :width: 627px
     
 .. Note::
 
     Calling ``ipython qtconsole -h`` will show all the style names that
     pygments can find on your system.
 
 You can also pass the filename of a custom CSS stylesheet, if you want to do
 your own coloring, via the ``stylesheet`` argument.  The default LightBG
 stylesheet:
 
 .. sourcecode:: css
 
     QPlainTextEdit, QTextEdit { background-color: white;
             color: black ;
             selection-background-color: #ccc}
     .error { color: red; }
     .in-prompt { color: navy; }
     .in-prompt-number { font-weight: bold; }
     .out-prompt { color: darkred; }
     .out-prompt-number { font-weight: bold; }
     /* .inverted is used to highlight selected completion */
     .inverted { background-color: black ; color: white; }
 
 Fonts
 =====
 
 The QtConsole has configurable via the ConsoleWidget. To change these, set the
 ``font_family`` or ``font_size`` traits of the ConsoleWidget. For instance, to
 use 9pt Anonymous Pro::
 
     $> ipython qtconsole --ConsoleWidget.font_family="Anonymous Pro" --ConsoleWidget.font_size=9
 
 Process Management
 ==================
 
 With the two-process ZMQ model, the frontend does not block input during
 execution. This means that actions can be taken by the frontend while the
 Kernel is executing, or even after it crashes. The most basic such command is
 via 'Ctrl-.', which restarts the kernel.  This can be done in the middle of a
 blocking execution. The frontend can also know, via a heartbeat mechanism, that
 the kernel has died. This means that the frontend can safely restart the
 kernel.
 
 .. _multiple_consoles:
 
 Multiple Consoles
 *****************
 
 Since the Kernel listens on the network, multiple frontends can connect to it.
 These do not have to all be qt frontends - any IPython frontend can connect and
 run code.  When you start ipython qtconsole, there will be an output line,
 like::
 
     [IPKernelApp] To connect another client to this kernel, use:
     [IPKernelApp] --existing kernel-12345.json
 
 Other frontends can connect to your kernel, and share in the execution. This is
 great for collaboration.  The ``--existing`` flag means connect to a kernel
 that already exists.  Starting other consoles
 with that flag will not try to start their own kernel, but rather connect to
 yours.  :file:`kernel-12345.json` is a small JSON file with the ip, port, and
 authentication information necessary to connect to your kernel. By default, this file
 will be in your default profile's security directory.  If it is somewhere else,
 the output line will print the full path of the connection file, rather than
 just its filename.
 
 If you need to find the connection info to send, and don't know where your connection file
 lives, there are a couple of ways to get it. If you are already running an IPython console
 connected to the kernel, you can use the ``%connect_info`` magic to display the information
 necessary to connect another frontend to the kernel.
 
 .. sourcecode:: ipython
 
     In [2]: %connect_info
     {
       "stdin_port":50255,
       "ip":"127.0.0.1",
       "hb_port":50256,
       "key":"70be6f0f-1564-4218-8cda-31be40a4d6aa",
       "shell_port":50253,
       "iopub_port":50254
     }
 
     Paste the above JSON into a file, and connect with:
         $> ipython <app> --existing <file>
     or, if you are local, you can connect with just:
         $> ipython <app> --existing kernel-12345.json
     or even just:
         $> ipython <app> --existing
     if this is the most recent IPython session you have started.
 
 Otherwise, you can find a connection file by name (and optionally profile) with
 :func:`IPython.lib.kernel.find_connection_file`:
 
 .. sourcecode:: bash
 
     $> python -c "from IPython.lib.kernel import find_connection_file;\
     print find_connection_file('kernel-12345.json')"
     /home/you/.ipython/profile_default/security/kernel-12345.json
 
 And if you are using a particular IPython profile:
 
 .. sourcecode:: bash
 
     $> python -c "from IPython.lib.kernel import find_connection_file;\
     print find_connection_file('kernel-12345.json', profile='foo')"
     /home/you/.ipython/profile_foo/security/kernel-12345.json
 
 You can even launch a standalone kernel, and connect and disconnect Qt Consoles
 from various machines.  This lets you keep the same running IPython session
 on your work machine (with matplotlib plots and everything), logging in from home,
 cafés, etc.::
 
     $> ipython kernel
     [IPKernelApp] To connect another client to this kernel, use:
     [IPKernelApp] --existing kernel-12345.json
 
 This is actually exactly the same as the subprocess launched by the qtconsole, so
 all the information about connecting to a standalone kernel is identical to that
 of connecting to the kernel attached to a running console.
 
 .. _kernel_security:
 
 Security
 --------
 
 .. warning::
 
     Since the ZMQ code currently has no encryption, listening on an
     external-facing IP is dangerous.  You are giving any computer that can see
     you on the network the ability to connect to your kernel, and view your traffic.
     Read the rest of this section before listening on external ports
     or running an IPython kernel on a shared machine.
 
 By default (for security reasons), the kernel only listens on localhost, so you
 can only connect multiple frontends to the kernel from your local machine. You
 can specify to listen on an external interface by specifying the ``ip``
 argument::
 
     $> ipython qtconsole --ip=192.168.1.123
 
 If you specify the ip as 0.0.0.0 or '*', that means all interfaces, so any
 computer that can see yours on the network can connect to the kernel.
 
 Messages are not encrypted, so users with access to the ports your kernel is using will be
 able to see any output of the kernel. They will **NOT** be able to issue shell commands as
 you due to message signatures, which are enabled by default as of IPython 0.12.
 
 .. warning::
 
     If you disable message signatures, then any user with access to the ports your
     kernel is listening on can issue arbitrary code as you. **DO NOT** disable message
     signatures unless you have a lot of trust in your environment.
 
 The one security feature IPython does provide is protection from unauthorized execution.
 IPython's messaging system will sign messages with HMAC digests using a shared-key. The key
 is never sent over the network, it is only used to generate a unique hash for each message,
 based on its content. When IPython receives a message, it will check that the digest
 matches, and discard the message. You can use any file that only you have access to to
 generate this key, but the default is just to generate a new UUID. You can generate a random
 private key with::
 
     # generate 1024b of random data, and store in a file only you can read:
     # (assumes IPYTHONDIR is defined, otherwise use your IPython directory)
     $> python -c "import os; print os.urandom(128).encode('base64')" > $IPYTHONDIR/sessionkey
     $> chmod 600 $IPYTHONDIR/sessionkey
 
 The *contents* of this file will be stored in the JSON connection file, so that file
 contains everything you need to connect to and use a kernel.
 
 To use this generated key, simply specify the ``Session.keyfile`` configurable
 in :file:`ipython_config.py` or at the command-line, as in::
 
     # instruct IPython to sign messages with that key, instead of a new UUID
     $> ipython qtconsole --Session.keyfile=$IPYTHONDIR/sessionkey
 
 .. _ssh_tunnels:
 
 SSH Tunnels
 -----------
 
 Sometimes you want to connect to machines across the internet, or just across
 a LAN that either doesn't permit open ports or you don't trust the other
 machines on the network.  To do this, you can use SSH tunnels.  SSH tunnels
 are a way to securely forward ports on your local machine to ports on another
 machine, to which you have SSH access.
 
 In simple cases, IPython's tools can forward ports over ssh by simply adding the
 ``--ssh=remote`` argument to the usual ``--existing...`` set of flags for connecting
 to a running kernel, after copying the JSON connection file (or its contents) to
 the second computer.
 
 .. warning::
 
     Using SSH tunnels does *not* increase localhost security.  In fact, when
     tunneling from one machine to another *both* machines have open
     ports on localhost available for connections to the kernel.
 
 There are two primary models for using SSH tunnels with IPython.  The first
 is to have the Kernel listen only on localhost, and connect to it from
 another machine on the same LAN.
 
 First, let's start a kernel on machine **worker**, listening only
 on loopback::
 
     user@worker $> ipython kernel
     [IPKernelApp] To connect another client to this kernel, use:
     [IPKernelApp] --existing kernel-12345.json
 
 In this case, the IP that you would connect
 to would still be 127.0.0.1, but you want to specify the additional ``--ssh`` argument
 with the hostname of the kernel (in this example, it's 'worker')::
 
     user@client $> ipython qtconsole  --ssh=worker --existing /path/to/kernel-12345.json
 
 Which will write a new connection file with the forwarded ports, so you can reuse them::
 
     [IPythonQtConsoleApp] To connect another client via this tunnel, use:
     [IPythonQtConsoleApp] --existing kernel-12345-ssh.json
 
 Note again that this opens ports on the *client* machine that point to your kernel.
 
 .. note::
 
     the ssh argument is simply passed to openssh, so it can be fully specified ``user@host:port``
     but it will also respect your aliases, etc. in :file:`.ssh/config` if you have any.
 
 The second pattern is for connecting to a machine behind a firewall across the internet
 (or otherwise wide network). This time, we have a machine **login** that you have ssh access
 to, which can see **kernel**, but **client** is on another network. The important difference
 now is that **client** can see **login**, but *not* **worker**. So we need to forward ports from
 client to worker *via* login. This means that the kernel must be started listening
 on external interfaces, so that its ports are visible to `login`::
 
     user@worker $> ipython kernel --ip=0.0.0.0
     [IPKernelApp] To connect another client to this kernel, use:
     [IPKernelApp] --existing kernel-12345.json
 
 Which we can connect to from the client with::
 
     user@client $> ipython qtconsole --ssh=login --ip=192.168.1.123 --existing /path/to/kernel-12345.json
 
 .. note::
 
     The IP here is the address of worker as seen from *login*, and need only be specified if
     the kernel used the ambiguous 0.0.0.0 (all interfaces) address. If it had used
     192.168.1.123 to start with, it would not be needed.
 
 
 Manual SSH tunnels
 ------------------
 
 It's possible that IPython's ssh helper functions won't work for you, for various
 reasons.  You can still connect to remote machines, as long as you set up the tunnels
 yourself.  The basic format of forwarding a local port to a remote one is::
 
     [client] $> ssh <server> <localport>:<remoteip>:<remoteport> -f -N
 
 This will forward local connections to **localport** on client to **remoteip:remoteport**
 *via* **server**. Note that remoteip is interpreted relative to *server*, not the client.
 So if you have direct ssh access to the machine to which you want to forward connections,
 then the server *is* the remote machine, and remoteip should be server's IP as seen from the
 server itself, i.e. 127.0.0.1.  Thus, to forward local port 12345 to remote port 54321 on
 a machine you can see, do::
 
     [client] $> ssh machine 12345:127.0.0.1:54321 -f -N
 
 But if your target is actually on a LAN at 192.168.1.123, behind another machine called **login**,
 then you would do::
 
     [client] $> ssh login 12345:192.168.1.16:54321 -f -N
 
 The ``-f -N`` on the end are flags that tell ssh to run in the background,
 and don't actually run any commands beyond creating the tunnel.
 
 .. seealso::
 
     A short discussion of ssh tunnels: http://www.revsys.com/writings/quicktips/ssh-tunnel.html
 
 
 
 Stopping Kernels and Consoles
 *****************************
 
 Since there can be many consoles per kernel, the shutdown mechanism and dialog
 are probably more complicated than you are used to. Since you don't always want
 to shutdown a kernel when you close a window, you are given the option to just
 close the console window or also close the Kernel and *all other windows*. Note
 that this only refers to all other *local* windows, as remote Consoles are not
 allowed to shutdown the kernel, and shutdowns do not close Remote consoles (to
 allow for saving, etc.).
 
 Rules:
 
     * Restarting the kernel automatically clears all *local* Consoles, and prompts remote
       Consoles about the reset.
     * Shutdown closes all *local* Consoles, and notifies remotes that
       the Kernel has been shutdown.
     * Remote Consoles may not restart or shutdown the kernel.
 
 Qt and the QtConsole
 ====================
 
 An important part of working with the QtConsole when you are writing your own
 Qt code is to remember that user code (in the kernel) is *not* in the same
 process as the frontend.  This means that there is not necessarily any Qt code
 running in the kernel, and under most normal circumstances there isn't. If,
 however, you specify ``--matplotlib qt`` at the command-line, then there *will* be a
 :class:`QCoreApplication` instance running in the kernel process along with
 user-code. To get a reference to this application, do:
 
 .. sourcecode:: python
 
     from PyQt4 import QtCore
     app = QtCore.QCoreApplication.instance()
     # app will be None if there is no such instance
 
 A common problem listed in the PyQt4 Gotchas_ is the fact that Python's garbage
 collection will destroy Qt objects (Windows, etc.) once there is no longer a
 Python reference to them, so you have to hold on to them.  For instance, in:
 
 .. sourcecode:: python
 
     def make_window():
         win = QtGui.QMainWindow()
     
     def make_and_return_window():
         win = QtGui.QMainWindow()
         return win
 
 :func:`make_window` will never draw a window, because garbage collection will
 destroy it before it is drawn, whereas :func:`make_and_return_window` lets the
 caller decide when the window object should be destroyed.  If, as a developer,
 you know that you always want your objects to last as long as the process, you
 can attach them to the QApplication instance itself:
 
 .. sourcecode:: python
     
     # do this just once:
     app = QtCore.QCoreApplication.instance()
     app.references = set()
     # then when you create Windows, add them to the set
     def make_window():
         win = QtGui.QMainWindow()
         app.references.add(win)
 
 Now the QApplication itself holds a reference to ``win``, so it will never be
 garbage collected until the application itself is destroyed.
 
 .. _Gotchas: http://www.riverbankcomputing.co.uk/static/Docs/PyQt4/html/gotchas.html#garbage-collection
 
 Regressions
 ===========
 
 There are some features, where the qt console lags behind the Terminal
 frontend:
 
 * !cmd input: Due to our use of pexpect, we cannot pass input to subprocesses
   launched using the '!' escape, so you should never call a command that
   requires interactive input.  For such cases, use the terminal IPython.  This
   will not be fixed, as abandoning pexpect would significantly degrade the
   console experience.
 
 .. _PyQt: http://www.riverbankcomputing.co.uk/software/pyqt/download
 .. _pygments: http://pygments.org/

docs/source/parallel/parallel_intro.rst

@@ -1,306 +1,307 b''
 .. _parallel_overview:
 
 ============================
 Overview and getting started
 ============================
 
 
 Examples
 ========
 
 We have various example scripts and notebooks for using IPython.parallel in our
 :file:`examples/parallel` directory, or they can be found `on GitHub`__.
 Some of these are covered in more detail in the :ref:`examples
 <parallel_examples>` section.
 
 .. __: https://github.com/ipython/ipython/tree/master/examples/parallel
 
 Introduction
 ============
 
 This section gives an overview of IPython's sophisticated and powerful
 architecture for parallel and distributed computing. This architecture
 abstracts out parallelism in a very general way, which enables IPython to
 support many different styles of parallelism including:
 
 * Single program, multiple data (SPMD) parallelism.
 * Multiple program, multiple data (MPMD) parallelism.
 * Message passing using MPI.
 * Task farming.
 * Data parallel.
 * Combinations of these approaches.
 * Custom user defined approaches.
 
 Most importantly, IPython enables all types of parallel applications to
 be developed, executed, debugged and monitored *interactively*. Hence,
 the ``I`` in IPython.  The following are some example usage cases for IPython:
 
 * Quickly parallelize algorithms that are embarrassingly parallel
   using a number of simple approaches.  Many simple things can be
   parallelized interactively in one or two lines of code.
 
 * Steer traditional MPI applications on a supercomputer from an
   IPython session on your laptop.
 
 * Analyze and visualize large datasets (that could be remote and/or
   distributed) interactively using IPython and tools like
   matplotlib/TVTK.
 
 * Develop, test and debug new parallel algorithms 
   (that may use MPI) interactively.
 
 * Tie together multiple MPI jobs running on different systems into
   one giant distributed and parallel system.
 
 * Start a parallel job on your cluster and then have a remote 	
   collaborator connect to it and pull back data into their 
   local IPython session for plotting and analysis.
 
 * Run a set of tasks on a set of CPUs using dynamic load balancing.
 
 .. tip::
 
    At the SciPy 2011 conference in Austin, Min Ragan-Kelley presented a
    complete 4-hour tutorial on the use of these features, and all the materials
    for the tutorial are now `available online`__.  That tutorial provides an
    excellent, hands-on oriented complement to the reference documentation
    presented here.
 
 .. __: http://minrk.github.com/scipy-tutorial-2011
 
 Architecture overview
 =====================
 
 .. figure:: figs/wideView.png
     :width: 300px
 
 
 The IPython architecture consists of four components:
 
 * The IPython engine.
 * The IPython hub.
 * The IPython schedulers.
 * The controller client.
 
 These components live in the :mod:`IPython.parallel` package and are
 installed with IPython.  They do, however, have additional dependencies
 that must be installed.  For more information, see our
 :ref:`installation documentation <install_index>`.
 
 .. TODO: include zmq in install_index
 
 IPython engine
 ---------------
 
 The IPython engine is a Python instance that takes Python commands over a
 network connection. Eventually, the IPython engine will be a full IPython
 interpreter, but for now, it is a regular Python interpreter. The engine
 can also handle incoming and outgoing Python objects sent over a network
 connection.  When multiple engines are started, parallel and distributed
 computing becomes possible. An important feature of an IPython engine is
 that it blocks while user code is being executed. Read on for how the
 IPython controller solves this problem to expose a clean asynchronous API
 to the user.
 
 IPython controller
 ------------------
 
 The IPython controller processes provide an interface for working with a set of engines.
 At a general level, the controller is a collection of processes to which IPython engines
 and clients can connect. The controller is composed of a :class:`Hub` and a collection of
 :class:`Schedulers`. These Schedulers are typically run in separate processes but on the
 same machine as the Hub, but can be run anywhere from local threads or on remote machines.
 
 The controller also provides a single point of contact for users who wish to
 utilize the engines connected to the controller. There are different ways of
 working with a controller. In IPython, all of these models are implemented via
 the :meth:`.View.apply` method, after
 constructing :class:`.View` objects to represent subsets of engines. The two
 primary models for interacting with engines are:
 
 * A **Direct** interface, where engines are addressed explicitly.
 * A **LoadBalanced** interface, where the Scheduler is trusted with assigning work to
   appropriate engines.
 
 Advanced users can readily extend the View models to enable other
 styles of parallelism. 
 
 .. note:: 
 
     A single controller and set of engines can be used with multiple models
     simultaneously. This opens the door for lots of interesting things.
 
 
 The Hub
 *******
 
 The center of an IPython cluster is the Hub. This is the process that keeps
 track of engine connections, schedulers, clients, as well as all task requests and
 results. The primary role of the Hub is to facilitate queries of the cluster state, and
 minimize the necessary information required to establish the many connections involved in
 connecting new clients and engines.
 
 
 Schedulers
 **********
 
 All actions that can be performed on the engine go through a Scheduler. While the engines
 themselves block when user code is run, the schedulers hide that from the user to provide
 a fully asynchronous interface to a set of engines.
 
 
 IPython client and views
 ------------------------
 
 There is one primary object, the :class:`~.parallel.Client`, for connecting to a cluster.
 For each execution model, there is a corresponding :class:`~.parallel.View`. These views
 allow users to interact with a set of engines through the interface. Here are the two default
 views:
 
 * The :class:`DirectView` class for explicit addressing.
 * The :class:`LoadBalancedView` class for destination-agnostic scheduling.
 
 Security
 --------
 
 IPython uses ZeroMQ for networking, which has provided many advantages, but
 one of the setbacks is its utter lack of security [ZeroMQ]_. By default, no IPython
 connections are encrypted, but open ports only listen on localhost. The only
 source of security for IPython is via ssh-tunnel. IPython supports both shell
 (`openssh`) and `paramiko` based tunnels for connections.  There is a key necessary
 to submit requests, but due to the lack of encryption, it does not provide
 significant security if loopback traffic is compromised.
 
 In our architecture, the controller is the only process that listens on
 network ports, and is thus the main point of vulnerability. The standard model
 for secure connections is to designate that the controller listen on
 localhost, and use ssh-tunnels to connect clients and/or
 engines.
 
 To connect and authenticate to the controller an engine or client needs
 some information that the controller has stored in a JSON file.
 Thus, the JSON files need to be copied to a location where
 the clients and engines can find them. Typically, this is the
 :file:`~/.ipython/profile_default/security` directory on the host where the 
 client/engine is running (which could be a different host than the controller). 
 Once the JSON files are copied over, everything should work fine.
 
 Currently, there are two JSON files that the controller creates:
 
 ipcontroller-engine.json
     This JSON file has the information necessary for an engine to connect
     to a controller.
 
 ipcontroller-client.json
     The client's connection information.  This may not differ from the engine's,
     but since the controller may listen on different ports for clients and
     engines, it is stored separately.
 
 ipcontroller-client.json will look something like this, under default localhost
 circumstances:
 
 .. sourcecode:: python
 
     {
       "url":"tcp:\/\/127.0.0.1:54424",
       "exec_key":"a361fe89-92fc-4762-9767-e2f0a05e3130",
       "ssh":"",
       "location":"10.19.1.135"
     }
 
 If, however, you are running the controller on a work node on a cluster, you will likely
 need to use ssh tunnels to connect clients from your laptop to it.  You will also
 probably need to instruct the controller to listen for engines coming from other work nodes
 on the cluster.  An example of ipcontroller-client.json, as created by::
 
     $> ipcontroller --ip=* --ssh=login.mycluster.com
 
 
 .. sourcecode:: python
 
     {
       "url":"tcp:\/\/*:54424",
       "exec_key":"a361fe89-92fc-4762-9767-e2f0a05e3130",
       "ssh":"login.mycluster.com",
       "location":"10.0.0.2"
     }
+
 More details of how these JSON files are used are given below.
 
 A detailed description of the security model and its implementation in IPython
 can be found :ref:`here <parallelsecurity>`.
 
 .. warning::
 
     Even at its most secure, the Controller listens on ports on localhost, and
     every time you make a tunnel, you open a localhost port on the connecting
     machine that points to the Controller. If localhost on the Controller's
     machine, or the machine of any client or engine, is untrusted, then your
     Controller is insecure. There is no way around this with ZeroMQ.
 
 
 
 Getting Started
 ===============
 
 To use IPython for parallel computing, you need to start one instance of the
 controller and one or more instances of the engine. Initially, it is best to
 simply start a controller and engines on a single host using the
 :command:`ipcluster` command. To start a controller and 4 engines on your
 localhost, just do::
 
     $ ipcluster start -n 4
 
 More details about starting the IPython controller and engines can be found
 :ref:`here <parallel_process>`
 
 Once you have started the IPython controller and one or more engines, you
 are ready to use the engines to do something useful. To make sure
 everything is working correctly, try the following commands:
 
 .. sourcecode:: ipython
 
 	In [1]: from IPython.parallel import Client
 	
 	In [2]: c = Client()
 	
 	In [4]: c.ids
 	Out[4]: set([0, 1, 2, 3])
 	
 	In [5]: c[:].apply_sync(lambda : "Hello, World")
 	Out[5]: [ 'Hello, World', 'Hello, World', 'Hello, World', 'Hello, World' ]
 
 
 When a client is created with no arguments, the client tries to find the corresponding JSON file
 in the local `~/.ipython/profile_default/security` directory. Or if you specified a profile,
 you can use that with the Client.  This should cover most cases:
 
 .. sourcecode:: ipython
 
     In [2]: c = Client(profile='myprofile')
 
 If you have put the JSON file in a different location or it has a different name, create the
 client like this:
 
 .. sourcecode:: ipython
 
     In [2]: c = Client('/path/to/my/ipcontroller-client.json')
 
 Remember, a client needs to be able to see the Hub's ports to connect. So if they are on a
 different machine, you may need to use an ssh server to tunnel access to that machine,
 then you would connect to it with:
 
 .. sourcecode:: ipython
 
     In [2]: c = Client('/path/to/my/ipcontroller-client.json', sshserver='[e-mail]')
 
 Where 'myhub.example.com' is the url or IP address of the machine on
 which the Hub process is running (or another machine that has direct access to the Hub's ports).
 
 The SSH server may already be specified in ipcontroller-client.json, if the controller was
 instructed at its launch time.
 
 You are now ready to learn more about the :ref:`Direct
 <parallel_multiengine>` and :ref:`LoadBalanced <parallel_task>` interfaces to the
 controller.
 
 .. [ZeroMQ] ZeroMQ.  http://www.zeromq.org