diff --git a/docs/source/development/coding_guide.txt b/docs/source/development/coding_guide.txt deleted file mode 100644 index 9801147..0000000 --- a/docs/source/development/coding_guide.txt +++ /dev/null @@ -1,123 +0,0 @@ -============ -Coding guide -============ - -General coding conventions -========================== - -In general, we'll try to follow the standard Python style conventions as -described in Python's PEP 8 [PEP8]_, the official Python Style Guide. - -Other general comments: - -* In a large file, top level classes and functions should be separated by 2 - lines to make it easier to separate them visually. - -* Use 4 spaces for indentation, **never** use hard tabs. - -* Keep the ordering of methods the same in classes that have the same methods. - This is particularly true for classes that implement similar interfaces and - for interfaces that are similar. - -Naming conventions -================== - -In terms of naming conventions, we'll follow the guidelines of PEP 8 [PEP8]_. -Some of the existing code doesn't honor this perfectly, but for all new -IPython code (and much existing code is being refactored), we'll use: - -* All ``lowercase`` module names. - -* ``CamelCase`` for class names. - -* ``lowercase_with_underscores`` for methods, functions, variables and - attributes. - -This may be confusing as some of the existing codebase uses a different -convention (``lowerCamelCase`` for methods and attributes). Slowly, we will -move IPython over to the new convention, providing shadow names for backward -compatibility in public interfaces. - -There are, however, some important exceptions to these rules. In some cases, -IPython code will interface with packages (Twisted, Wx, Qt) that use other -conventions. At some level this makes it impossible to adhere to our own -standards at all times. In particular, when subclassing classes that use other -naming conventions, you must follow their naming conventions. To deal with -cases like this, we propose the following policy: - -* If you are subclassing a class that uses different conventions, use its - naming conventions throughout your subclass. Thus, if you are creating a - Twisted Protocol class, used Twisted's - ``namingSchemeForMethodsAndAttributes.`` - -* All IPython's official interfaces should use our conventions. In some cases - this will mean that you need to provide shadow names (first implement - ``fooBar`` and then ``foo_bar = fooBar``). We want to avoid this at all - costs, but it will probably be necessary at times. But, please use this - sparingly! - -Implementation-specific *private* methods will use -``_single_underscore_prefix``. Names with a leading double underscore will -*only* be used in special cases, as they makes subclassing difficult (such -names are not easily seen by child classes). - -Occasionally some run-in lowercase names are used, but mostly for very short -names or where we are implementing methods very similar to existing ones in a -base class (like ``runlines()`` where ``runsource()`` and ``runcode()`` had -established precedent). - -The old IPython codebase has a big mix of classes and modules prefixed with an -explicit ``IP``. In Python this is mostly unnecessary, redundant and frowned -upon, as namespaces offer cleaner prefixing. The only case where this approach -is justified is for classes which are expected to be imported into external -namespaces and a very generic name (like Shell) is too likely to clash with -something else. However, if a prefix seems absolutely necessary the more -specific ``IPY`` or ``ipy`` are preferred. - -.. [PEP8] Python Enhancement Proposal 8. http://www.python.org/peps/pep-0008.html - -Attribute declarations for objects -================================== - -In general, objects should declare in their *class* all attributes the object -is meant to hold throughout its life. While Python allows you to add an -attribute to an instance at any point in time, this makes the code harder to -read and requires methods to constantly use checks with hasattr() or try/except -calls. By declaring all attributes of the object in the class header, there is -a single place one can refer to for understanding the object's data interface, -where comments can explain the role of each variable and when possible, -sensible deafaults can be assigned. - -.. Warning:: - - If an attribute is meant to contain a mutable object, it should be set to - ``None`` in the class and its mutable value should be set in the object's - constructor. Since class attributes are shared by all instances, failure - to do this can lead to difficult to track bugs. But you should still set - it in the class declaration so the interface specification is complete and - documdented in one place. - -A simple example:: - - class foo: - # X does..., sensible default given: - x = 1 - # y does..., default will be set by constructor - y = None - # z starts as an empty list, must be set in constructor - z = None - - def __init__(self, y): - self.y = y - self.z = [] - - -New files -========= - -When starting a new file for IPython, you can use the following template as a -starting point that has a few common things pre-written for you. The template -is included in the documentation sources as -:file:`docs/sources/development/template.py`: - -.. literalinclude:: template.py diff --git a/docs/source/development/contributing.txt b/docs/source/development/contributing.txt deleted file mode 100644 index 8b3031b..0000000 --- a/docs/source/development/contributing.txt +++ /dev/null @@ -1,82 +0,0 @@ -.. _contributing: - -============================ -How to contribute to IPython -============================ - -Overview -======== - -IPython development is done using Git_ and Github_ (`Github Ipython Repository`_). -This makes it easy for people to contribute to the development of IPython. -There are several ways in which you can join in. - - -Merging a branch into trunk -=========================== - -Core developers, who ultimately merge any approved branch (from themselves, -another developer, or any third-party contribution) will typically use -:command:`git merge` to merge the branch into the trunk and push it to the main -Git repository. There are a number of things to keep in mind when doing this, -so that the project history is easy to understand in the long run, and that -generating release notes is as painless and accurate as possible. - -* When you merge any non-trivial functionality (from one small bug fix to a - big feature branch), please remember to always edit the appropriate file in - the :ref:`What's new ` section of our documentation. - Ideally, the author of the branch should provide this content when they - submit the branch for review. But if they don't it is the responsibility of - the developer doing the merge to add this information. - -* When merges are done, the practice of putting a summary commit message in - the merge is *extremely* useful. It is probably easiest if you simply use - the same list of changes that were added to the :ref:`What's new - ` section of the documentation. - -* It's important that we remember to always credit who gave us something if - it's not the committer. In general, we have been fairly good on this front, - this is just a reminder to keep things up. As a note, if you are ever - committing something that is completely (or almost so) a third-party - contribution, do the commit as:: - - $ git commit --author="Someone Else " - - This way it will show that name separately in the log, which makes it even - easier to spot. Obviously we often rework third party contributions - extensively, but this is still good to keep in mind for cases when we don't - touch the code too much. - - - - -Commit messages -=============== - -Good commit messages are very important; they provide a verbal account of what -happened that is often invaluable for anyone trying to undestand the intent of -a commit later on (including the original author!). And git's log command is a -very versatile and powerful tool, capable of extracting a lot of information -from the commit logs, so it's important that these logs actually have useful -information in them. - -In short, a commit message should have the form:: - - One line summary. - - More detailed description of what was done, using multiple lines and even - more than one paragraph if needed. For very simple commits this may not be - necessary, but non-trivial ones should always have it. - - Closes gh-NNN. # if the commit closes issue NNN on github. - -This format is understood by many git tools that expect a *single line* -summary, so please do respect it. - -An excellent reference on commits message is `this blog post`_, please take a -moment to read it (it's short but very informative). - -.. _this blog post: http://who-t.blogspot.com/2009/12/on-commit-messages.html -.. _Github IPython Repository: ipython_github_ - -.. include:: ../links.rst diff --git a/docs/source/development/doc_guide.txt b/docs/source/development/doc_guide.txt deleted file mode 100644 index 32ffac3..0000000 --- a/docs/source/development/doc_guide.txt +++ /dev/null @@ -1,167 +0,0 @@ -.. _documenting-ipython: - -===================== - Documenting IPython -===================== - -When contributing code to IPython, you should strive for clarity and -consistency, without falling prey to a style straitjacket. Basically, -'document everything, try to be consistent, do what makes sense.' - -By and large we follow existing Python practices in major projects like Python -itself or NumPy, this document provides some additional detail for IPython. - - -Standalone documentation -======================== - -All standalone documentation should be written in plain text (``.txt``) files -using reStructuredText [reStructuredText]_ for markup and formatting. All such -documentation should be placed in the directory :file:`docs/source` of the -IPython source tree. Or, when appropriate, a suitably named subdirectory -should be used. The documentation in this location will serve as the main -source for IPython documentation. - -The actual HTML and PDF docs are built using the Sphinx [Sphinx]_ -documentation generation tool. Once you have Sphinx installed, you can build -the html docs yourself by doing: - -.. code-block:: bash - - $ cd ipython-mybranch/docs - $ make html - -Our usage of Sphinx follows that of matplotlib [Matplotlib]_ closely. We are -using a number of Sphinx tools and extensions written by the matplotlib team -and will mostly follow their conventions, which are nicely spelled out in -their documentation guide [MatplotlibDocGuide]_. What follows is thus a -abridged version of the matplotlib documentation guide, taken with permission -from the matplotlib team. - -If you are reading this in a web browser, you can click on the "Show Source" -link to see the original reStricturedText for the following examples. - -A bit of Python code:: - - for i in range(10): - print i, - print "A big number:",2**34 - -An interactive Python session:: - - >>> from IPython.utils.path import get_ipython_dir - >>> get_ipython_dir() - '/home/fperez/.config/ipython' - -An IPython session: - -.. code-block:: ipython - - In [7]: import IPython - - In [8]: print "This IPython is version:",IPython.__version__ - This IPython is version: 0.9.1 - - In [9]: 2+4 - Out[9]: 6 - - -A bit of shell code: - -.. code-block:: bash - - cd /tmp - echo "My home directory is: $HOME" - ls - -Docstring format -================ - -Good docstrings are very important. Unfortunately, Python itself only provides -a rather loose standard for docstrings [PEP257]_, and there is no universally -accepted convention for all the different parts of a complete docstring. -However, the NumPy project has established a very reasonable standard, and has -developed some tools to support the smooth inclusion of such docstrings in -Sphinx-generated manuals. Rather than inventing yet another pseudo-standard, -IPython will be henceforth documented using the NumPy conventions; we carry -copies of some of the NumPy support tools to remain self-contained, but share -back upstream with NumPy any improvements or fixes we may make to the tools. - -The NumPy documentation guidelines [NumPyDocGuide]_ contain detailed -information on this standard, and for a quick overview, the NumPy example -docstring [NumPyExampleDocstring]_ is a useful read. - - -For user-facing APIs, we try to be fairly strict about following the above -standards (even though they mean more verbose and detailed docstrings). -Wherever you can reasonably expect people to do introspection with:: - - In [1]: some_function? - -the docstring should follow the NumPy style and be fairly detailed. - -For purely internal methods that are only likely to be read by others extending -IPython itself we are a bit more relaxed, especially for small/short methods -and functions whose intent is reasonably obvious. We still expect docstrings -to be written, but they can be simpler. For very short functions with a -single-line docstring you can use something like:: - - def add(a, b): - """The sum of two numbers. - """ - code - -and for longer multiline strings:: - - def add(a, b): - """The sum of two numbers. - - Here is the rest of the docs. - """ - code - - -Here are two additional PEPs of interest regarding documentation of code. -While both of these were rejected, the ideas therein form much of the basis of -docutils (the machinery to process reStructuredText): - -* `Docstring Processing System Framework `_ -* `Docutils Design Specification `_ - -.. note:: - - In the past IPython used epydoc so currently many docstrings still use - epydoc conventions. We will update them as we go, but all new code should - be documented using the NumPy standard. - -Building and uploading -====================== -The built docs are stored in a separate repository. Through some github magic, -they're automatically exposed as a website. It works like this: - -* You will need to have sphinx and latex installed. In Ubuntu, install - ``texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended``. - Install the latest version of sphinx from PyPI (``pip install sphinx``). -* Ensure that the development version of IPython is the first in your system - path. You can either use a virtualenv, or modify your PYTHONPATH. -* Switch into the docs directory, and run ``make gh-pages``. This will build - your updated docs as html and pdf, then automatically check out the latest - version of the docs repository, copy the built docs into it, and commit your - changes. -* Open the built docs in a web browser, and check that they're as expected. -* (When building the docs for a new tagged release, you will have to add its link to - index.rst, then run ``python build_index.py`` to update index.html. Commit the - change.) -* Upload the docs with ``git push``. This only works if you have write access to - the docs repository. -* If you are building a version that is not the current dev branch, nor a tagged release, - then you must run gh-pages.py directly with ``python gh-pages.py ``, and *not* - with ``make gh-pages``. - -.. [reStructuredText] reStructuredText. http://docutils.sourceforge.net/rst.html -.. [Sphinx] Sphinx. http://sphinx.pocoo.org/ -.. [MatplotlibDocGuide] http://matplotlib.sourceforge.net/devel/documenting_mpl.html -.. [PEP257] PEP 257. http://www.python.org/peps/pep-0257.html -.. [NumPyDocGuide] NumPy documentation guide. https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt -.. [NumPyExampleDocstring] NumPy example docstring. https://github.com/numpy/numpy/blob/master/doc/HOWTO_BUILD_DOCS.rst.txt - diff --git a/docs/source/development/index.txt b/docs/source/development/index.txt index d1de11b..a13a17b 100644 --- a/docs/source/development/index.txt +++ b/docs/source/development/index.txt @@ -7,14 +7,8 @@ IPython developer's guide .. toctree:: :maxdepth: 1 - contributing.txt + gitwash/index.txt - coding_guide.txt - doc_guide.txt - testing.txt - release.txt - reorg.txt messaging.txt parallel_messages.txt parallel_connections.txt - ipython_directive.txt diff --git a/docs/source/development/ipython_directive.txt b/docs/source/development/ipython_directive.txt deleted file mode 100644 index 8ebb554..0000000 --- a/docs/source/development/ipython_directive.txt +++ /dev/null @@ -1,419 +0,0 @@ -.. _ipython_directive: - -======================== -IPython Sphinx Directive -======================== - -The ipython directive is a stateful ipython shell for embedding in -sphinx documents. It knows about standard ipython prompts, and -extracts the input and output lines. These prompts will be renumbered -starting at ``1``. The inputs will be fed to an embedded ipython -interpreter and the outputs from that interpreter will be inserted as -well. For example, code blocks like the following:: - - .. ipython:: - - In [136]: x = 2 - - In [137]: x**3 - Out[137]: 8 - -will be rendered as - -.. ipython:: - - In [136]: x = 2 - - In [137]: x**3 - Out[137]: 8 - -.. note:: - - This tutorial should be read side-by-side with the Sphinx source - for this document because otherwise you will see only the rendered - output and not the code that generated it. Excepting the example - above, we will not in general be showing the literal ReST in this - document that generates the rendered output. - - -The state from previous sessions is stored, and standard error is -trapped. At doc build time, ipython's output and std err will be -inserted, and prompts will be renumbered. So the prompt below should -be renumbered in the rendered docs, and pick up where the block above -left off. - -.. ipython:: - - In [138]: z = x*3 # x is recalled from previous block - - In [139]: z - Out[139]: 6 - - In [140]: print z - --------> print(z) - 6 - - In [141]: q = z[) # this is a syntax error -- we trap ipy exceptions - ------------------------------------------------------------ - File "", line 1 - q = z[) # this is a syntax error -- we trap ipy exceptions - ^ - SyntaxError: invalid syntax - - -The embedded interpreter supports some limited markup. For example, -you can put comments in your ipython sessions, which are reported -verbatim. There are some handy "pseudo-decorators" that let you -doctest the output. The inputs are fed to an embedded ipython -session and the outputs from the ipython session are inserted into -your doc. If the output in your doc and in the ipython session don't -match on a doctest assertion, an error will be - - -.. ipython:: - - In [1]: x = 'hello world' - - # this will raise an error if the ipython output is different - @doctest - In [2]: x.upper() - Out[2]: 'HELLO WORLD' - - # some readline features cannot be supported, so we allow - # "verbatim" blocks, which are dumped in verbatim except prompts - # are continuously numbered - @verbatim - In [3]: x.st - x.startswith x.strip - - -Multi-line input is supported. - -.. ipython:: - - In [130]: url = 'http://ichart.finance.yahoo.com/table.csv?s=CROX\ - .....: &d=9&e=22&f=2009&g=d&a=1&br=8&c=2006&ignore=.csv' - - In [131]: print url.split('&') - --------> print(url.split('&')) - ['http://ichart.finance.yahoo.com/table.csv?s=CROX', 'd=9', 'e=22', - -You can do doctesting on multi-line output as well. Just be careful -when using non-deterministic inputs like random numbers in the ipython -directive, because your inputs are ruin through a live interpreter, so -if you are doctesting random output you will get an error. Here we -"seed" the random number generator for deterministic output, and we -suppress the seed line so it doesn't show up in the rendered output - -.. ipython:: - - In [133]: import numpy.random - - @suppress - In [134]: numpy.random.seed(2358) - - @doctest - In [135]: numpy.random.rand(10,2) - Out[135]: - array([[ 0.64524308, 0.59943846], - [ 0.47102322, 0.8715456 ], - [ 0.29370834, 0.74776844], - [ 0.99539577, 0.1313423 ], - [ 0.16250302, 0.21103583], - [ 0.81626524, 0.1312433 ], - [ 0.67338089, 0.72302393], - [ 0.7566368 , 0.07033696], - [ 0.22591016, 0.77731835], - [ 0.0072729 , 0.34273127]]) - - -Another demonstration of multi-line input and output - -.. ipython:: - - In [106]: print x - --------> print(x) - jdh - - In [109]: for i in range(10): - .....: print i - .....: - .....: - 0 - 1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - - -Most of the "pseudo-decorators" can be used an options to ipython -mode. For example, to setup matplotlib pylab but suppress the output, -you can do. When using the matplotlib ``use`` directive, it should -occur before any import of pylab. This will not show up in the -rendered docs, but the commands will be executed in the embedded -interpreter and subsequent line numbers will be incremented to reflect -the inputs:: - - - .. ipython:: - :suppress: - - In [144]: from pylab import * - - In [145]: ion() - -.. ipython:: - :suppress: - - In [144]: from pylab import * - - In [145]: ion() - -Likewise, you can set ``:doctest:`` or ``:verbatim:`` to apply these -settings to the entire block. For example, - -.. ipython:: - :verbatim: - - In [9]: cd mpl/examples/ - /home/jdhunter/mpl/examples - - In [10]: pwd - Out[10]: '/home/jdhunter/mpl/examples' - - - In [14]: cd mpl/examples/ - mpl/examples/animation/ mpl/examples/misc/ - mpl/examples/api/ mpl/examples/mplot3d/ - mpl/examples/axes_grid/ mpl/examples/pylab_examples/ - mpl/examples/event_handling/ mpl/examples/widgets - - In [14]: cd mpl/examples/widgets/ - /home/msierig/mpl/examples/widgets - - In [15]: !wc * - 2 12 77 README.txt - 40 97 884 buttons.py - 26 90 712 check_buttons.py - 19 52 416 cursor.py - 180 404 4882 menu.py - 16 45 337 multicursor.py - 36 106 916 radio_buttons.py - 48 226 2082 rectangle_selector.py - 43 118 1063 slider_demo.py - 40 124 1088 span_selector.py - 450 1274 12457 total - -You can create one or more pyplot plots and insert them with the -``@savefig`` decorator. - -.. ipython:: - - @savefig plot_simple.png width=4in - In [151]: plot([1,2,3]); - - # use a semicolon to suppress the output - @savefig hist_simple.png width=4in - In [151]: hist(np.random.randn(10000), 100); - -In a subsequent session, we can update the current figure with some -text, and then resave - -.. ipython:: - - - In [151]: ylabel('number') - - In [152]: title('normal distribution') - - @savefig hist_with_text.png width=4in - In [153]: grid(True) - -You can also have function definitions included in the source. - -.. ipython:: - - In [3]: def square(x): - ...: """ - ...: An overcomplicated square function as an example. - ...: """ - ...: if x < 0: - ...: x = abs(x) - ...: y = x * x - ...: return y - ...: - -Then call it from a subsequent section. - -.. ipython:: - - In [4]: square(3) - Out [4]: 9 - - In [5]: square(-2) - Out [5]: 4 - - -Writing Pure Python Code ------------------------- - -Pure python code is supported by the optional argument `python`. In this pure -python syntax you do not include the output from the python interpreter. The -following markup:: - - .. ipython:: python - - foo = 'bar' - print foo - foo = 2 - foo**2 - -Renders as - -.. ipython:: python - - foo = 'bar' - print foo - foo = 2 - foo**2 - -We can even plot from python, using the savefig decorator, as well as, suppress -output with a semicolon - -.. ipython:: python - - @savefig plot_simple_python.png width=4in - plot([1,2,3]); - -Similarly, std err is inserted - -.. ipython:: python - - foo = 'bar' - foo[) - -Comments are handled and state is preserved - -.. ipython:: python - - # comments are handled - print foo - -If you don't see the next code block then the options work. - -.. ipython:: python - :suppress: - - ioff() - ion() - -Multi-line input is handled. - -.. ipython:: python - - line = 'Multi\ - line &\ - support &\ - works' - print line.split('&') - -Functions definitions are correctly parsed - -.. ipython:: python - - def square(x): - """ - An overcomplicated square function as an example. - """ - if x < 0: - x = abs(x) - y = x * x - return y - -And persist across sessions - -.. ipython:: python - - print square(3) - print square(-2) - -Pretty much anything you can do with the ipython code, you can do with -with a simple python script. Obviously, though it doesn't make sense -to use the doctest option. - -Pseudo-Decorators -================= - -Here are the supported decorators, and any optional arguments they -take. Some of the decorators can be used as options to the entire -block (eg ``verbatim`` and ``suppress``), and some only apply to the -line just below them (eg ``savefig``). - -@suppress - - execute the ipython input block, but suppress the input and output - block from the rendered output. Also, can be applied to the entire - ``..ipython`` block as a directive option with ``:suppress:``. - -@verbatim - - insert the input and output block in verbatim, but auto-increment - the line numbers. Internally, the interpreter will be fed an empty - string, so it is a no-op that keeps line numbering consistent. - Also, can be applied to the entire ``..ipython`` block as a - directive option with ``:verbatim:``. - -@savefig OUTFILE [IMAGE_OPTIONS] - - save the figure to the static directory and insert it into the - document, possibly binding it into a minipage and/or putting - code/figure label/references to associate the code and the - figure. Takes args to pass to the image directive (*scale*, - *width*, etc can be kwargs); see `image options - `_ - for details. - -@doctest - - Compare the pasted in output in the ipython block with the output - generated at doc build time, and raise errors if they don’t - match. Also, can be applied to the entire ``..ipython`` block as a - directive option with ``:doctest:``. - -Configuration Options -===================== - -ipython_savefig_dir - - The directory in which to save the figures. This is relative to the - Sphinx source directory. The default is `html_static_path`. - -ipython_rgxin - - The compiled regular expression to denote the start of IPython input - lines. The default is re.compile('In \[(\d+)\]:\s?(.*)\s*'). You - shouldn't need to change this. - -ipython_rgxout - - The compiled regular expression to denote the start of IPython output - lines. The default is re.compile('Out\[(\d+)\]:\s?(.*)\s*'). You - shouldn't need to change this. - - -ipython_promptin - - The string to represent the IPython input prompt in the generated ReST. - The default is 'In [%d]:'. This expects that the line numbers are used - in the prompt. - -ipython_promptout - - The string to represent the IPython prompt in the generated ReST. The - default is 'Out [%d]:'. This expects that the line numbers are used - in the prompt. diff --git a/docs/source/development/release.txt b/docs/source/development/release.txt deleted file mode 100644 index 6d3df66..0000000 --- a/docs/source/development/release.txt +++ /dev/null @@ -1,53 +0,0 @@ -.. _releasing_ipython: - -================= -Releasing IPython -================= - -This section contains notes about the process that is used to release IPython. -Our release process is currently not very formal and could be improved. - -Most of the release process is automated by the :file:`release` script in the -:file:`tools` directory. This is just a handy reminder for the release manager. - -#. For writing release notes, this will cleanly show who contributed as author - of commits (get the previous release name from the tag list with ``git - tag``):: - - git log --format="* %aN" $PREV_RELEASE... | sort -u - - .. note:: - - use:: - - git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f - - To find duplicates and update :file:`.mailmap` - -#. Run :file:`build_release`, which does all the file checking and building - that the real release script will do. This will let you do test - installations, check that the build procedure runs OK, etc. You may want to - disable a few things like multi-version RPM building while testing, because - otherwise the build takes really long. - -#. Run the release script, which makes the tar.gz, eggs and Win32 .exe - installer. It posts them to the site and registers the release with PyPI. - -#. Update the website with announcements and links to the updated changes.txt - in html form. Remember to put a short note on the news page of the site. - -#. Drafting a short release announcement with i) highlights and ii) a link to - the html version of the :ref:`Whats new ` section of the - documentation. - -#. Make sure that the released version of the docs is live on the site. For - this we are now using the gh-pages system: - - - Make a static directory for the final copy of the release docs. - - Update the :file:`index.rst` file and run :file:`build_index.py` to update - the html version. - - Update the ``stable`` symlink to point to the released version. - - Run ``git add`` for all the new files and commit. - - Run ``git push`` to update the public version of the docs on gh-pages. - -#. Celebrate! diff --git a/docs/source/development/reorg.txt b/docs/source/development/reorg.txt deleted file mode 100644 index 10e7fed..0000000 --- a/docs/source/development/reorg.txt +++ /dev/null @@ -1,85 +0,0 @@ -.. _module_reorg: - -=========================== -IPython module organization -=========================== - -As of the 0.11 release of IPython, the top-level packages and modules have -been completely reorganized. This section describes the purpose of the -top-level IPython subpackages. - -Subpackage descriptions -======================= - -* :mod:`IPython.config`. This package contains the :ref:`configuration system - ` of IPython, as well as default configuration files for the - different IPython applications. - -* :mod:`IPython.core`. This sub-package contains the core of the IPython - interpreter, but none of its extended capabilities. - -* :mod:`IPython.deathrow`. This is for code that is outdated, untested, - rotting, or that belongs in a separate third party project. Eventually all - this code will either 1) be revived by someone willing to maintain it with - tests and docs and re-included into IPython or 2) be removed from IPython - proper, but put into a separate third-party Python package. No new code will - be allowed here. If your favorite extension has been moved here please - contact the IPython developer mailing list to help us determine the best - course of action. - -* :mod:`IPython.extensions`. This package contains fully supported IPython - extensions. These extensions adhere to the official IPython extension API - and can be enabled by adding them to a field in the configuration file. - If your extension is no longer in this location, please look in - :mod:`IPython.quarantine` and :mod:`IPython.deathrow` and contact the - IPython developer mailing list. - -* :mod:`IPython.external`. This package contains third party packages and - modules that IPython ships internally to reduce the number of dependencies. - Usually, these are short, single file modules. - -* :mod:`IPython.frontend`. This package contains the various IPython - frontends which communicate with the :mod:`IPython.kernel.zmq` kernels (see - :ref:`Messaging in IPython `). This includes the - :ref:`ipython notebook `, :ref:`ipython qtconsole - `, and :ref:`ipython console ` entry points. - -* :mod:`IPython.lib`. IPython has many extended capabilities that are not part - of the IPython core. These things will go here and in. Modules in this - package are similar to extensions, but don't adhere to the official - IPython extension API. - -* :mod:`IPython.nbformat`. This package contains code related to reading and - writing :ref:`IPython Notebook's ` file format (`.ipynb` - files). - -* :mod:`IPython.parallel`. This contains :ref:`IPython's parallel computing - system `. This previously lived under :mod:`IPython.kernel`, - but that module has been deprecated. - -* :mod:`IPython.quarantine`. This is for code that doesn't meet IPython's - standards, but that we plan on keeping. To be moved out of this sub-package - a module needs to have approval of the core IPython developers, tests and - documentation. If your favorite extension has been moved here please contact - the IPython developer mailing list to help us determine the best course of - action. - -* :mod:`IPython.scripts`. This package contains a variety of top-level - command line scripts. Eventually, these should be moved to the - :file:`scripts` subdirectory of the appropriate IPython subpackage. - -* :mod:`IPython.testing`. This package contains code related to the IPython - test suite, which locates and executes the `tests` submodules of all - IPython sub-packages. It also contains decorators and utilities relevant for - testing. - -* :mod:`IPython.utils`. This sub-package will contain anything that might - eventually be found in the Python standard library, like things in - :mod:`genutils`. Each sub-module in this sub-package should contain - functions and classes that serve a single purpose and that don't - depend on things in the rest of IPython. - -* :mod:`IPython.kernel.zmq`. This sub-package contains code related to starting - and managing IPython kernels, which :mod:`IPython.frontend` instances can then - communicate with (see :ref:`Messaging in IPython `). - diff --git a/docs/source/development/session_format.txt b/docs/source/development/session_format.txt deleted file mode 100644 index 26b24d6..0000000 --- a/docs/source/development/session_format.txt +++ /dev/null @@ -1,111 +0,0 @@ -=============================== - IPython session storage notes -=============================== - -This document serves as a sample/template for ideas on how to store session -data on disk. This stems from discussions we had on various mailing lists, and -should be considered a pure work in progress. We haven't settled these ideas -completely yet, and there's a lot to discuss; this document should just serve -as a reference of the distilled points from various conversations on multiple -mailing lists, and will congeal over time on a specific design we implement. - -The frontend would store, for now, 5 types of data: - -#. Input: this is python/ipython code to be executed. - -#. Output (python): result of executing Inputs. - -#. Standard output: from subprocesses. - -#. Standard error: from subprocesses. - -#. Text: arbitrary text. For now, we'll just store plain text and will defer - to the user on how to format it, though it should be valid reST if it is - later to be converted into html/pdf. - -The non-text cells would be stored on-disk as follows:: - - .. input-cell:: - :id: 1 - - 3+3 - - .. output-cell:: - :id: 1 - - 6 - - .. input-cell:: - :id: 2 - - ls - - .. stdout-cell:: - :id: 2 - - a.py b.py - - .. input-cell:: - :id: 3 - - !askdfj - - .. stderr-cell:: - :id: 3 - - sh: askdfj: command not found - -Brian made some interesting points on the mailing list inspired by the -Mathematica format, reproduced here for reference: - -The Mathematica notebook format is a plain text file that itself is *valid -Mathematica code*. This id documented here: - -http://reference.wolfram.com/mathematica/guide/LowLevelNotebookProgramming.html - -For examples a simple notebook with one text cell is just:: - - Notebook[{Cell['Here is my text', 'Text']}] - -Everything - input cells, output cells, static images and all are represented -in this way and embedded in the plain text notebook file. The Python -generalization of this would be the following: - -* A Python notebook is plain text, importable Python code. - -* That code is simply a tree of objects that declare the relevant parts of the - notebook. - -This has a number of advantages: - -* A notebook can be imported, manipulated and run by anyone who has the support - code (the notebook module that defines the relevant classes). - -* A notebook doesn't need to be parsed. It is valid Python and can be imported - or exec'd. Once that is done, you have the full notebook in memory. You can - immediately do anything you want with it. - -* The various Notebook, Cell, Image, etc. classes can know about how to output - to various formats, latex, html, reST, XML, etc:: - - import mynotebook - mynotebook.notebook.export('rest') - -* Each individual format (HTML, reST, latex) has weaknesses. If you pick any - one to be *the* notebook format, you are building those weaknesses into your - design. A pure python based notebook format won't suffer from that syndrome. - -* It is a clean separation of the model (Notebook, Cell, Image, etc.) and the - view (HTML, reST, etc.). Picking HTML or reST for the notebook format - confuses (at some level) the model and view... - -* Third party code can define new Notebook elements that specify how they can - be rendered in different contexts. For example, matplotlib could ship a - Figure element that knows how to render itself as a native PyQt GUI, a static - image, a web page, etc. - -* A notebook remains a single plain text file that anyone can edit - even if it - has embedded images. Neither HTML nor reST have the ability to inline - graphics in plain text files. While I love reST, it is a pain that I need an - entire directory of files to render a single Sphinx doc. - \ No newline at end of file diff --git a/docs/source/development/template.py b/docs/source/development/template.py deleted file mode 100644 index 385e58d..0000000 --- a/docs/source/development/template.py +++ /dev/null @@ -1,51 +0,0 @@ -"""A one-line description. - -A longer description that spans multiple lines. Explain the purpose of the -file and provide a short list of the key classes/functions it contains. This -is the docstring shown when some does 'import foo;foo?' in IPython, so it -should be reasonably useful and informative. -""" -#----------------------------------------------------------------------------- -# Copyright (c) 2011, the IPython Development Team. -# -# Distributed under the terms of the Modified BSD License. -# -# The full license is in the file COPYING.txt, distributed with this software. -#----------------------------------------------------------------------------- - -#----------------------------------------------------------------------------- -# Imports -#----------------------------------------------------------------------------- -from __future__ import print_function - -# [remove this comment in production] -# -# List all imports, sorted within each section (stdlib/third-party/ipython). -# For 'import foo', use one import per line. For 'from foo.bar import a, b, c' -# it's OK to import multiple items, use the parenthesized syntax 'from foo -# import (a, b, ...)' if the list needs multiple lines. - -# Stdlib imports - -# Third-party imports - -# Our own imports - - -# [remove this comment in production] -# -# Use broad section headers like this one that make it easier to navigate the -# file, with descriptive titles. For complex classes, simliar (but indented) -# headers are useful to organize the internal class structure. - -#----------------------------------------------------------------------------- -# Globals and constants -#----------------------------------------------------------------------------- - -#----------------------------------------------------------------------------- -# Local utilities -#----------------------------------------------------------------------------- - -#----------------------------------------------------------------------------- -# Classes and functions -#----------------------------------------------------------------------------- diff --git a/docs/source/development/testing.txt b/docs/source/development/testing.txt deleted file mode 100644 index 107dbaa..0000000 --- a/docs/source/development/testing.txt +++ /dev/null @@ -1,387 +0,0 @@ -.. _testing: - -========================================== - Testing IPython for users and developers -========================================== - -Overview -======== - -It is extremely important that all code contributed to IPython has tests. -Tests should be written as unittests, doctests or other entities that the -IPython test system can detect. See below for more details on this. - -Each subpackage in IPython should have its own :file:`tests` directory that -contains all of the tests for that subpackage. All of the files in the -:file:`tests` directory should have the word "tests" in them to enable -the testing framework to find them. - -In docstrings, examples (either using IPython prompts like ``In [1]:`` or -'classic' python ``>>>`` ones) can and should be included. The testing system -will detect them as doctests and will run them; it offers control to skip parts -or all of a specific doctest if the example is meant to be informative but -shows non-reproducible information (like filesystem data). - -If a subpackage has any dependencies beyond the Python standard library, the -tests for that subpackage should be skipped if the dependencies are not found. -This is very important so users don't get tests failing simply because they -don't have dependencies. - -The testing system we use is an extension of the nose_ test runner. -In particular we've -developed a nose plugin that allows us to paste verbatim IPython sessions and -test them as doctests, which is extremely important for us. - -.. _nose: http://code.google.com/p/python-nose - - -For the impatient: running the tests -==================================== - -You can run IPython from the source download directory without even installing -it system-wide or having configure anything, by typing at the terminal: - -.. code-block:: bash - - python ipython.py - -In order to run the test suite, you must at least be able to import IPython, -even if you haven't fully installed the user-facing scripts yet (common in a -development environment). You can then run the tests with: - -.. code-block:: bash - - python -c "import IPython; IPython.test()" - -Once you have installed IPython either via a full install or using: - -.. code-block:: bash - - python setup.py develop - -you will have available a system-wide script called :file:`iptest` that runs -the full test suite. You can then run the suite with: - -.. code-block:: bash - - iptest [args] - -By default, this excludes the relatively slow tests for ``IPython.parallel``. To -run these, use ``iptest --all``. - -Regardless of how you run things, you should eventually see something like: - -.. code-block:: bash - - ********************************************************************** - Test suite completed for system with the following information: - {'commit_hash': '144fdae', - 'commit_source': 'repository', - 'ipython_path': '/home/fperez/usr/lib/python2.6/site-packages/IPython', - 'ipython_version': '0.11.dev', - 'os_name': 'posix', - 'platform': 'Linux-2.6.35-22-generic-i686-with-Ubuntu-10.10-maverick', - 'sys_executable': '/usr/bin/python', - 'sys_platform': 'linux2', - 'sys_version': '2.6.6 (r266:84292, Sep 15 2010, 15:52:39) \n[GCC 4.4.5]'} - - Tools and libraries available at test time: - curses matplotlib pymongo qt sqlite3 tornado wx wx.aui zmq - - Ran 9 test groups in 67.213s - - Status: - OK - - -If not, there will be a message indicating which test group failed and how to -rerun that group individually. For example, this tests the -:mod:`IPython.utils` subpackage, the :option:`-v` option shows progress -indicators: - -.. code-block:: bash - - $ iptest -v IPython.utils - ..........................SS..SSS............................S.S... - ......................................................... - ---------------------------------------------------------------------- - Ran 125 tests in 0.119s - - OK (SKIP=7) - - -Because the IPython test machinery is based on nose, you can use all nose -options and syntax, typing ``iptest -h`` shows all available options. For -example, this lets you run the specific test :func:`test_rehashx` inside the -:mod:`test_magic` module: - -.. code-block:: bash - - $ iptest -vv IPython.core.tests.test_magic:test_rehashx - IPython.core.tests.test_magic.test_rehashx(True,) ... ok - IPython.core.tests.test_magic.test_rehashx(True,) ... ok - - ---------------------------------------------------------------------- - Ran 2 tests in 0.100s - - OK - -When developing, the :option:`--pdb` and :option:`--pdb-failures` of nose are -particularly useful, these drop you into an interactive pdb session at the -point of the error or failure respectively. - -.. note:: - - The system information summary printed above is accessible from the top - level package. If you encounter a problem with IPython, it's useful to - include this information when reporting on the mailing list; use:: - - from IPython import sys_info - print sys_info() - - and include the resulting information in your query. - -Testing pull requests ---------------------- - -We have a script that fetches a pull request from Github, merges it with master, -and runs the test suite on different versions of Python. This uses a separate -copy of the repository, so you can keep working on the code while it runs. To -run it:: - - python tools/test_pr.py -p 1234 - -The number is the pull request number from Github; the ``-p`` flag makes it post -the results to a comment on the pull request. Any further arguments are passed -to ``iptest``. - -This requires the `requests `_ and -`keyring `_ packages. - -For developers: writing tests -============================= - -By now IPython has a reasonable test suite, so the best way to see what's -available is to look at the :file:`tests` directory in most subpackages. But -here are a few pointers to make the process easier. - - -Main tools: :mod:`IPython.testing` ----------------------------------- - -The :mod:`IPython.testing` package is where all of the machinery to test -IPython (rather than the tests for its various parts) lives. In particular, -the :mod:`iptest` module in there has all the smarts to control the test -process. In there, the :func:`make_exclude` function is used to build a -blacklist of exclusions, these are modules that do not get even imported for -tests. This is important so that things that would fail to even import because -of missing dependencies don't give errors to end users, as we stated above. - -The :mod:`decorators` module contains a lot of useful decorators, especially -useful to mark individual tests that should be skipped under certain conditions -(rather than blacklisting the package altogether because of a missing major -dependency). - -Our nose plugin for doctests ----------------------------- - -The :mod:`plugin` subpackage in testing contains a nose plugin called -:mod:`ipdoctest` that teaches nose about IPython syntax, so you can write -doctests with IPython prompts. You can also mark doctest output with ``# -random`` for the output corresponding to a single input to be ignored (stronger -than using ellipsis and useful to keep it as an example). If you want the -entire docstring to be executed but none of the output from any input to be -checked, you can use the ``# all-random`` marker. The -:mod:`IPython.testing.plugin.dtexample` module contains examples of how to use -these; for reference here is how to use ``# random``:: - - def ranfunc(): - """A function with some random output. - - Normal examples are verified as usual: - >>> 1+3 - 4 - - But if you put '# random' in the output, it is ignored: - >>> 1+3 - junk goes here... # random - - >>> 1+2 - again, anything goes #random - if multiline, the random mark is only needed once. - - >>> 1+2 - You can also put the random marker at the end: - # random - - >>> 1+2 - # random - .. or at the beginning. - - More correct input is properly verified: - >>> ranfunc() - 'ranfunc' - """ - return 'ranfunc' - -and an example of ``# all-random``:: - - def random_all(): - """A function where we ignore the output of ALL examples. - - Examples: - - # all-random - - This mark tells the testing machinery that all subsequent examples - should be treated as random (ignoring their output). They are still - executed, so if a they raise an error, it will be detected as such, - but their output is completely ignored. - - >>> 1+3 - junk goes here... - - >>> 1+3 - klasdfj; - - In [8]: print 'hello' - world # random - - In [9]: iprand() - Out[9]: 'iprand' - """ - return 'iprand' - - -When writing docstrings, you can use the ``@skip_doctest`` decorator to -indicate that a docstring should *not* be treated as a doctest at all. The -difference between ``# all-random`` and ``@skip_doctest`` is that the former -executes the example but ignores output, while the latter doesn't execute any -code. ``@skip_doctest`` should be used for docstrings whose examples are -purely informational. - -If a given docstring fails under certain conditions but otherwise is a good -doctest, you can use code like the following, that relies on the 'null' -decorator to leave the docstring intact where it works as a test:: - - # The docstring for full_path doctests differently on win32 (different path - # separator) so just skip the doctest there, and use a null decorator - # elsewhere: - - doctest_deco = dec.skip_doctest if sys.platform == 'win32' else dec.null_deco - - @doctest_deco - def full_path(startPath,files): - """Make full paths for all the listed files, based on startPath...""" - - # function body follows... - -With our nose plugin that understands IPython syntax, an extremely effective -way to write tests is to simply copy and paste an interactive session into a -docstring. You can writing this type of test, where your docstring is meant -*only* as a test, by prefixing the function name with ``doctest_`` and leaving -its body *absolutely empty* other than the docstring. In -:mod:`IPython.core.tests.test_magic` you can find several examples of this, but -for completeness sake, your code should look like this (a simple case):: - - def doctest_time(): - """ - In [10]: %time None - CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s - Wall time: 0.00 s - """ - -This function is only analyzed for its docstring but it is not considered a -separate test, which is why its body should be empty. - - -Parametric tests done right ---------------------------- - -If you need to run multiple tests inside the same standalone function or method -of a :class:`unittest.TestCase` subclass, IPython provides the ``parametric`` -decorator for this purpose. This is superior to how test generators work in -nose, because IPython's keeps intact your stack, which makes debugging vastly -easier. For example, these are some parametric tests both in class form and as -a standalone function (choose in each situation the style that best fits the -problem at hand, since both work):: - - from IPython.testing import decorators as dec - - def is_smaller(i,j): - assert i>>`` and the input is pure Python. - - The prompts are of the form ``In [1]:`` and the input can contain extended - IPython expressions. - - In the first case, Nose will recognize the doctests as long as it is called - with the ``--with-doctest`` flag. But the second case will likely require - modifications or the writing of a new doctest plugin for Nose that is - IPython-aware. - -3. ReStructuredText files that contain code blocks. For this type of file, we - have three distinct possibilities for the code blocks: - - - They use ``>>>`` prompts. - - They use ``In [1]:`` prompts. - - They are standalone blocks of pure Python code without any prompts. - - The first two cases are similar to the situation #2 above, except that in - this case the doctests must be extracted from input code blocks using - docutils instead of from the Python docstrings. - - In the third case, we must have a convention for distinguishing code blocks - that are meant for execution from others that may be snippets of shell code - or other examples not meant to be run. One possibility is to assume that - all indented code blocks are meant for execution, but to have a special - docutils directive for input that should not be executed. - - For those code blocks that we will execute, the convention used will simply - be that they get called and are considered successful if they run to - completion without raising errors. This is similar to what Nose does for - standalone test functions, and by putting asserts or other forms of - exception-raising statements it becomes possible to have literate examples - that double as lightweight tests. - -4. Extension modules with doctests in function and method docstrings. - Currently Nose simply can't find these docstrings correctly, because the - underlying doctest DocTestFinder object fails there. Similarly to #2 above, - the docstrings could have either pure python or IPython prompts. - -Of these, only 3-c (reST with standalone code blocks) is not implemented at -this point.