Merge pull request #9663 from Carreau/docs-overhall

Docs overhaul.

docs/requirements.txt

@@ -1,3 +1,3 b''
--e .
+-e ../.
 ipykernel
 setuptools>=18.5

docs/source/conf.py

@@ -34,6 +34,10 b' if ON_RTD:'
                 '__file__': fpath,
                 '__name__': '__main__',
             })
+else:
+    import sphinx_rtd_theme
+    html_theme = "sphinx_rtd_theme"
+    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
 
 # If your extensions are in another directory, add it here. If the directory
 # is relative to the documentation root, use os.path.abspath to make it
@@ -79,9 +83,18 b" templates_path = ['_templates']"
 # The suffix of source filenames.
 source_suffix = '.rst'
 
-if iprelease['_version_extra'] == 'dev':
+def is_stable(extra):
+    for ext in {'dev', 'b', 'rc'}:
+        if ext in extra:
+            return False
+    return True
+
+if is_stable(iprelease['_version_extra']):
+    tags.add('ipystable')
+else:
+    tags.add('ipydev')
     rst_prolog = """
-    .. note::
+    .. warning::
 
         This documentation is for a development version of IPython. There may be
         significant differences from the latest stable release.
@@ -148,8 +161,8 b" default_role = 'literal'"
 # The style sheet to use for HTML and HTML Help pages. A file of that name
 # must exist either in Sphinx' static/ path, or in one of the custom paths
 # given in html_static_path.
-html_style = 'default.css'
-html_favicon = 'favicon.ico'
+# html_style = 'default.css'
+
 
 # The name for this set of Sphinx documents.  If None, it defaults to
 # "<project> v<release> documentation".
@@ -164,6 +177,8 b" html_favicon = 'favicon.ico'"
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
+# Favicon needs the directory name
+html_favicon = '_static/favicon.ico'
 # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 # using the given strftime format.
 html_last_updated_fmt = '%b %d, %Y'

docs/source/coredev/release_process.rst

@@ -91,6 +91,9 b" release is: ``1.3rc1``. Notice that there is no separator between the '3' and"
 the 'r'. Check the environment variable `$VERSION` as well. 
 
 
+Comment remove the `development` entry in `whatsnew/index.rst`. TODO, figure
+out how to make that automatic. 
+
 5. Run the `tools/build_release` script
 ---------------------------------------
 
@@ -116,7 +119,8 b' Create and push the tag::'
     git tag -am "release $VERSION" "$VERSION"
     git push origin --tags
 
-Update release.py back to `x.y-dev` or `x.y-maint`, and push::
+Update release.py back to `x.y-dev` or `x.y-maint`, and re-add the
+`development` entry in `docs/source/whatsnew/index.rst` and push::
 
     git commit -am "back to development"
     git push origin $BRANCH

docs/source/index.rst

@@ -1,3 +1,5 b''
+.. _introduction:
+
 =====================
 IPython Documentation
 =====================
@@ -7,14 +9,82 b' IPython Documentation'
    :Release: |release|
    :Date: |today|
 
-Welcome to the official IPython documentation.
+Welcome to the official IPython documentation
+
+IPython provides a rich toolkit to help you make the most out of using Python
+interactively.  Its main components are:
+
+* A powerful interactive Python shell
+
+* A `Jupyter <http://jupyter.org/>`_ kernel to work with Python code in Jupyter
+  notebooks and other interactive frontends.
+
+The enhanced interactive Python shells and kernel have the following main
+features:
+
+* Comprehensive object introspection.
+
+* Input history, persistent across sessions.
+
+* Caching of output results during a session with automatically generated
+  references.
+
+* Extensible tab completion, with support by default for completion of python
+  variables and keywords, filenames and function keywords.
+
+* Extensible system of 'magic' commands for controlling the environment and
+  performing many tasks related either to IPython or the operating system.
+
+* A rich configuration system with easy switching between different setups
+  (simpler than changing $PYTHONSTARTUP environment variables every time).
+
+* Session logging and reloading.
+
+* Extensible syntax processing for special purpose situations.
+
+* Access to the system shell with user-extensible alias system.
+
+* Easily embeddable in other Python programs and GUIs.
+
+* Integrated access to the pdb debugger and the Python profiler.
+
+
+The Command line interface inherit all the above functionality and posses 
+ 
+* real multi-line editing.
+ 
+* syntax highlighting as you type
+
+* integration with command line editor for a better workflow.
+
+The kernel also have its share of feature, when used with a compatible frontend
+it allows for:
+
+* rich display system for object allowing to display Html, Images, Latex,Sounds
+  Video.
+
+* interactive widgets with the use of the ``ipywidgets`` package.
+
+
+This documentation will walk through most of the features of the IPython
+command line and kernel, as well as describe the internals mechanisms in order
+to improve your Python workflow.
+
+You can always find the table of content for this documentation in the left
+sidebar, allowing you to come back on previous section if needed, or skip ahead. 
+
+
+The latest development version is always available from IPython's `GitHub
+repository <http://github.com/ipython/ipython>`_.
+
+
 
-Contents
-========
 
 .. toctree::
    :maxdepth: 1
+   :hidden:
 
+   self
    overview
    whatsnew/index
    install/index

docs/source/install/index.rst

@@ -5,8 +5,54 b' Installation'
 ============
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 3
+   :hidden:
+
 
    install
    kernel_install
 
+
+
+This sections will guide you through `installing IPython itself <install>`_, and
+installing `kernels for Jupyter <kernel_install>`_ if you wish to work with
+multiple version of Python, or multiple environments.
+
+
+Quick install reminder
+~~~~~~~~~~~~~~~~~~~~~~
+
+Here is a quick reminder of the commands needed for installation if you are
+already familiar with IPython and are just searching to refresh your memory:
+
+Install IPython:
+
+.. code-block:: bash
+
+    $ pip install ipython
+
+
+Install and register an IPython kernel with Jupyter:
+
+
+.. code-block:: bash
+
+    $ python -m pip install ipykernel
+
+    $ python -m ipykernel install [--user] [--name <machine-readable-name>] [--display-name <"User Friendly Name">]
+
+for more help see 
+
+.. code-block:: bash
+    
+    $ python -m ipykernel install  --help
+    
+
+
+.. seealso::
+
+   `Installing Jupyter <http://jupyter.readthedocs.io/en/latest/install.html>`__
+     The Notebook, nbconvert, and many other former pieces of IPython are now
+     part of Project Jupyter.
+
+

docs/source/install/install.rst

@@ -1,34 +1,28 b''
-IPython requires Python 2.7 or ≥ 3.3.
+Installing IPython
+==================
 
-.. seealso::
 
-   `Installing Jupyter <http://jupyter.readthedocs.io/en/latest/install.html>`__
-     The Notebook, nbconvert, and many other former pieces of IPython are now
-     part of Project Jupyter.
+IPython requires Python 2.7 or ≥ 3.3.
 
 
-Quickstart
-==========
+Quick Install 
+-------------
 
-If you have :mod:`pip`,
-the quickest way to get up and running with IPython is:
+With ``pip`` already installed :
 
 .. code-block:: bash
 
     $ pip install ipython
 
-To use IPython with notebooks or the Qt console, you should also install
-``jupyter``.
+This installs IPython as well as its dependencies.
 
-To run IPython's test suite, use the :command:`iptest` command:
+If you want to use IPython with notebooks or the Qt console, you should also
+install Jupyter ``pip install jupyter``.
 
-.. code-block:: bash
-
-    $ iptest
 
 
 Overview
-========
+--------
 
 This document describes in detail the steps required to install IPython.
 For a few quick ways to get started with package managers or full Python distributions,
@@ -36,10 +30,10 b' see `the install page <http://ipython.org/install.html>`_ of the IPython website'
 
 Please let us know if you have problems installing IPython or any of its dependencies.
 
-IPython and most dependencies can be installed via :command:`pip`.
+IPython and most dependencies should be installed via :command:`pip`.
 In many scenarios, this is the simplest method of installing Python packages.
 More information about :mod:`pip` can be found on
-`its PyPI page <http://pypi.python.org/pypi/pip>`__.
+`its PyPI page <https://pip.pypa.io>`__.
 
 
 More general information about installing Python packages can be found in
@@ -47,21 +41,20 b' More general information about installing Python packages can be found in'
 
 
 Installing IPython itself
-=========================
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Given a properly built Python, the basic interactive IPython shell will work
-with no external dependencies.  However, some Python distributions
-(particularly on Windows and OS X), don't come with a working :mod:`readline`
-module.  The IPython shell will work without :mod:`readline`, but will lack
-many features that users depend on, such as tab completion and command line
-editing.  If you install IPython with :mod:`pip`,
-then the appropriate :mod:`readline` for your platform will be installed.
-See below for details of how to make sure you have a working :mod:`readline`.
+IPython requires several dependencies to work correctly, it is not recommended
+to install IPython and all its dependencies manually as this can be quite long and troublesome.
+You should use the python package manager ``pip``.
 
 Installation using pip
-----------------------
+~~~~~~~~~~~~~~~~~~~~~~
+
+Make sure you have the latest version of :mod:`pip` (the Python package
+manager) installed. If you do not, head to `Pip documentation
+<https://pip.pypa.io/en/stable/installing/>`_ and install :mod:`pip` first.
 
-If you have :mod:`pip`, the easiest way of getting IPython is:
+The quickest way to get up and running with IPython is to install it with pip:
 
 .. code-block:: bash
 
@@ -71,7 +64,7 b" That's it."
 
 
 Installation from source
-------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~
 
 If you don't want to use :command:`pip`, or don't have it installed,
 grab the latest stable tarball of IPython `from PyPI
@@ -83,12 +76,22 b' grab the latest stable tarball of IPython `from PyPI'
     $ cd ipython
     $ pip install .
 
+Do not invoke ``setup.py`` directly as this can have undesirable consequences for further upgrades. 
+Try to also avoid any usage of ``easy_install`` that can have similar undesirable consequences.
+
 If you are installing to a location (like ``/usr/local``) that requires higher
-permissions, you may need to run the last command with :command:`sudo`.
+permissions, you may need to run the last command with :command:`sudo`. You can
+also install in user specific location by using the ``--user`` flag in conjunction with pip.
+
+To run IPython's test suite, use the :command:`iptest` command from outside of the IPython source tree:
+
+.. code-block:: bash
+
+    $ iptest
 
 
 Installing the development version
-----------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 It is also possible to install the development version of IPython from our
 `Git <http://git-scm.com/>`_ source code repository.  To do this you will
@@ -100,15 +103,15 b' need to have Git installed on your system. Then do:'
     $ cd ipython
     $ pip install .
 
-Some users want to be able to follow the development branch as it changes.  If
-you have :mod:`pip`, you can replace the last step by:
+Some users want to be able to follow the development branch as it changes.
+With :mod:`pip` installed, you can replace the last step by:
 
 .. code-block:: bash
 
     $ pip install -e .
 
 This creates links in the right places and installs the command line script to
-the appropriate places. 
+the appropriate location. 
 
 Then, if you want to update your IPython at any time, do:
 
@@ -119,62 +122,11 b' Then, if you want to update your IPython at any time, do:'
 .. _dependencies:
 
 Dependencies
-============
+~~~~~~~~~~~~
 
 IPython relies on a number of other Python packages. Installing using a package
 manager like pip or conda will ensure the necessary packages are installed. If
 you install manually, it's up to you to make sure dependencies are installed.
-They're not listed here, because they may change from release to release, so a
-static list will inevitably get out of date.
-
-It also has one key non-Python dependency which you may need to install separately.
-
-readline
---------
-
-IPython's terminal interface relies on readline to provide features like tab
-completion and history navigation. If you only want to use IPython as a kernel
-for Jupyter notebooks and other frontends, you don't need readline.
-
-
-**On Windows**, to get full console functionality, *PyReadline* is required.
-PyReadline is a separate, Windows only implementation of readline that uses
-native Windows calls through :mod:`ctypes`. The easiest way of installing
-PyReadline is you use the binary installer available `here
-<http://pypi.python.org/pypi/pyreadline>`__.
-
-**On OS X**, if you are using the built-in Python shipped by Apple, you will be
-missing a proper readline implementation as Apple ships instead a library called
-``libedit`` that provides only some of readline's functionality.  While you may
-find libedit sufficient, we have occasional reports of bugs with it and several
-developers who use OS X as their main environment consider libedit unacceptable
-for productive, regular use with IPython.
-
-Therefore, IPython on OS X depends on the :mod:`gnureadline` module.
-We will *not* consider completion/history problems to be bugs for IPython if you
-are using libedit.
-
-To get a working :mod:`readline` module on OS X, do (with :mod:`pip`
-installed):
-
-.. code-block:: bash
-
-    $ pip install gnureadline
-
-.. note::
-
-    Other Python distributions on OS X (such as Anaconda, fink, MacPorts)
-    already have proper readline so you likely don't have to do this step.
-
-When IPython is installed with :mod:`pip`,
-the correct readline should be installed if you specify the `terminal`
-optional dependencies:
-
-.. code-block:: bash
-
-    $ pip install "ipython[terminal]"
-
-**On Linux**, readline is normally installed by default. If not, install it
-from your system package manager. If you are compiling your own Python, make
-sure you install the readline development headers first.
-
+They're not listed here since a static list would inevitably fall out of date as
+dependencies may change from release to release and also vary depending on
+the platform.

docs/source/overview.rst

@@ -1,9 +1,6 b''
 .. _overview:
 
-============
-Introduction
-============
-
+========
 Overview
 ========
 
@@ -18,10 +15,13 b' interactive and exploratory computing. To support this goal, IPython'
 has three main components:
 
 * An enhanced interactive Python shell.
+
 * A decoupled :ref:`two-process communication model <ipythonzmq>`, which
   allows for multiple clients to connect to a computation kernel, most notably
-  the web-based notebook.
-* An architecture for interactive parallel computing.
+  the web-based notebook provided with `Jupyter <https://jupyter.org>`_.
+
+* An architecture for interactive parallel computing now part of the
+  `ipyparallel` package.
 
 All of IPython is open source (released under the revised BSD license).
 
@@ -69,8 +69,7 b' Main features of the interactive shell'
 
 * Completion in the local namespace, by typing :kbd:`TAB` at the prompt.
   This works for keywords, modules, methods, variables and files in the
-  current directory. This is supported via the readline library, and
-  full access to configuring readline's behavior is provided. 
+  current directory. This is supported via the ``prompt_toolkit`` library.
   Custom completers can be implemented easily for different purposes
   (system commands, magic arguments etc.)
 
@@ -79,7 +78,7 b' Main features of the interactive shell'
   history and caching of all input and output.
 
 * User-extensible 'magic' commands. A set of commands prefixed with
-  :samp:`%` is available for controlling IPython itself and provides
+  :samp:`%`  or :samp:`%%` is available for controlling IPython itself and provides
   directory control, namespace information and many aliases to
   common system shell commands.
 
@@ -102,8 +101,8 b' Main features of the interactive shell'
   allows you to save arbitrary Python variables. These get restored
   when you run the :samp:`%store -r` command.
 
-* Automatic indentation (optional) of code as you type (through the
-  readline library).
+* Automatic indentation and highlighting of code as you type (through the
+  `prompt_toolkit` library).
 
 * Macro system for quickly re-executing multiple lines of previous
   input with a single name via the :samp:`%macro` command. Macros can be
@@ -204,10 +203,11 b' This decoupling allows us to have several clients connected to the same'
 kernel, and even allows clients and kernels to live on different machines.
 With the exclusion of the traditional single process terminal-based IPython
 (what you start if you run ``ipython`` without any subcommands), all
-other IPython machinery uses this two-process model. This includes ``ipython
-console``,  ``ipython qtconsole``, and ``ipython notebook``.
+other IPython machinery uses this two-process model. Most of this is now part
+of the `Jupyter` project, whis includes ``jupyter console``,  ``jupyter
+qtconsole``, and ``jupyter notebook``.
 
-As an example, this means that when you start ``ipython qtconsole``, you're
+As an example, this means that when you start ``jupyter qtconsole``, you're
 really starting two processes, a kernel and a Qt-based client can send
 commands to and receive results from that kernel. If there is already a kernel
 running that you want to connect to, you can pass the  ``--existing`` flag
@@ -217,9 +217,9 b' running, use the ``%connect_info`` magic to get the unique connection file,'
 which will be something like ``--existing kernel-19732.json`` but with
 different numbers which correspond to the Process ID of the kernel.
 
-You can read more about using `ipython qtconsole 
+You can read more about using `jupyter qtconsole 
 <http://jupyter.org/qtconsole/>`_, and
-`ipython notebook <http://jupyter-notebook.readthedocs.io/en/latest/>`_. There 
+`jupyter notebook <http://jupyter-notebook.readthedocs.io/en/latest/>`_. There
 is also a :ref:`message spec <messaging>` which documents the protocol for 
 communication between kernels
 and clients.
@@ -232,6 +232,11 b' and clients.'
 Interactive parallel computing
 ==============================
 
+.. note::
+    
+    This functionality is optional and now part of the `ipyparallel
+    <http://ipyparallel.readthedocs.io/>`_ project.
+
 Increasingly, parallel computer hardware, such as multicore CPUs, clusters and
 supercomputers, is becoming ubiquitous. Over the last several years, we have
 developed an architecture within IPython that allows such hardware to be used

docs/source/whatsnew/index.rst

@@ -20,8 +20,8 b' development work they do here in a user friendly format.'
 .. toctree::
    :maxdepth: 1
 
-   version5
    development
+   version5
    version4
    github-stats-4
    version3

docs/source/whatsnew/version5.rst

@@ -99,14 +99,16 b' snippet:'
     ip.sphinxify_docstring = True
     ip.enable_html_pager = True
 
+
 You can test the effect of various combinations of the above configuration in
 the Jupyter notebook, with things example like :
 
-.. code-block:: python
+.. code-block:: ipython
 
     import numpy as np
     np.histogram?
 
+
 This is part of an effort to make Documentation in Python richer and provide in
 the long term if possible dynamic examples that can contain math, images,
 widgets... As stated above this is nightly experimental feature with a lot of
@@ -117,47 +119,47 b' it.'
 Removed Feature
 ---------------
 
- - ``TerminalInteractiveShell.autoedit_syntax`` Has been broken for many years now
-apparently. It has been removed.
+- ``TerminalInteractiveShell.autoedit_syntax`` Has been broken for many years now
+  apparently. It has been removed.
 
 
 Deprecated Features
 -------------------
 
-Some deprecated feature, don't forget to enable `DeprecationWarning` as error
-of you are using IPython in Continuous Integration setup or in your testing in general:
+Some deprecated features are listed in this section. Don't forget to enable
+``DeprecationWarning`` as an error if you are using IPython in a Continuous
+Integration setup or in your testing in general:
 
-.. code::
-    :python:
+.. code-block:: python
 
     import warnings
     warnings.filterwarnings('error', '.*', DeprecationWarning, module='yourmodule.*')
 
 
- - `hooks.fix_error_editor` seem to be unused and is pending deprecation.
- - `IPython/core/excolors.py:ExceptionColors` is  deprecated.
- - `IPython.core.InteractiveShell:write()` is deprecated, use `sys.stdout` instead.
- - `IPython.core.InteractiveShell:write_err()` is deprecated, use `sys.stderr` instead.
- - The `formatter` keyword argument to `Inspector.info` in `IPython.core.oinspec` has now no effects.
- - The `global_ns` keyword argument of IPython Embed was deprecated, and  will now have no effect. Use `module` keyword argument instead.
+- ``hooks.fix_error_editor`` seems unused and is pending deprecation.
+- `IPython/core/excolors.py:ExceptionColors` is  deprecated.
+- `IPython.core.InteractiveShell:write()` is deprecated; use `sys.stdout` instead.
+- `IPython.core.InteractiveShell:write_err()` is deprecated; use `sys.stderr` instead.
+- The `formatter` keyword argument to `Inspector.info` in `IPython.core.oinspec` has no effect.
+- The `global_ns` keyword argument of IPython Embed was deprecated, and has no effect. Use `module` keyword argument instead.
 
 
 Known Issues:
 -------------
 
- - ``<Esc>`` Key does not dismiss the completer and does not clear the current
-   buffer. This is an on purpose modification due to current technical
-   limitation. Cf :ghpull:`9572`. Escape the control character which is used
-   for other shortcut, and there is no practical way to distinguish. Use Ctr-G
-   or Ctrl-C as an alternative. 
+- ``<Esc>`` Key does not dismiss the completer and does not clear the current
+  buffer. This is an on purpose modification due to current technical
+  limitation. Cf :ghpull:`9572`. Escape the control character which is used
+  for other shortcut, and there is no practical way to distinguish. Use Ctr-G
+  or Ctrl-C as an alternative.
 
- - Cannot use ``Shift-Enter`` and ``Ctrl-Enter`` to submit code in terminal. cf
-   :ghissue:`9587` and :ghissue:`9401`. In terminal there is no practical way to
-   distinguish these key sequences from a normal new line return. 
+- Cannot use ``Shift-Enter`` and ``Ctrl-Enter`` to submit code in terminal. cf
+  :ghissue:`9587` and :ghissue:`9401`. In terminal there is no practical way to
+  distinguish these key sequences from a normal new line return.
 
- - ``PageUp`` and ``pageDown`` do not move through completion menu.
+- ``PageUp`` and ``pageDown`` do not move through completion menu.
 
- - Color styles might not adapt to terminal emulator themes. This will need new
-   version of Pygments to be released, and can be mitigated with custom themes. 
+- Color styles might not adapt to terminal emulator themes. This will need new
+  version of Pygments to be released, and can be mitigated with custom themes.