.. _development: ============================== IPython development guidelines ============================== Overview ======== This document describes IPython from the perspective of developers. Most importantly, it gives information for people who want to contribute to the development of IPython. So if you want to help out, read on! How to contribute to IPython ============================ IPython development is done using Bazaar [Bazaar]_ and Launchpad [Launchpad]_. This makes it easy for people to contribute to the development of IPython. Here is a sketch of how to get going. Install Bazaar and create a Launchpad account --------------------------------------------- First make sure you have installed Bazaar (see their `website `_). To see that Bazaar is installed and knows about you, try the following:: $ bzr whoami Joe Coder This should display your name and email. Next, you will want to create an account on the `Launchpad website `_ and setup your ssh keys. For more information of setting up your ssh keys, see `this link `_. Get the main IPython branch from Launchpad ------------------------------------------ Now, you can get a copy of the main IPython development branch (we call this the "trunk"):: $ bzr branch lp:ipython Create a working branch ----------------------- When working on IPython, you won't actually make edits directly to the :file:`lp:ipython` branch. Instead, you will create a separate branch for your changes. For now, let's assume you want to do your work in a branch named "ipython-mybranch". Create this branch by doing:: $ bzr branch ipython ipython-mybranch When you actually create a branch, you will want to give it a name that reflects the nature of the work that you will be doing in it, like "install-docs-update". Make edits in your working branch --------------------------------- Now you are ready to actually make edits in your :file:`ipython-mybranch` branch. Before doing this, it is helpful to install this branch so you can test your changes as you work. This is easiest if you have setuptools installed. Then, just do:: $ cd ipython-mybranch $ python setupegg.py develop Now, make some changes. After a while, you will want to commit your changes. This let's Bazaar know that you like the changes you have made and gives you an opportunity to keep a nice record of what you have done. This looks like this:: $ ...do work in ipython-mybranch... $ bzr commit -m "the commit message goes here" Please note that since we now don't use an old-style linear ChangeLog (that tends to cause problems with distributed version control systems), you should ensure that your log messages are reasonably detailed. Use a docstring-like approach in the commit messages (including the second line being left *blank*):: Single line summary of changes being committed. * more details when warranted ... * including crediting outside contributors if they sent the code/bug/idea! As you work, you will repeat this edit/commit cycle many times. If you work on your branch for a long time, you will also want to get the latest changes from the :file:`lp:ipython` branch. This can be done with the following sequence of commands:: $ ls ipython ipython-mybranch $ cd ipython $ bzr pull $ cd ../ipython-mybranch $ bzr merge ../ipython $ bzr commit -m "Merging changes from trunk" Along the way, you should also run the IPython test suite. You can do this using the :command:`iptest` command:: $ cd $ iptest The :command:`iptest` command will also pick up and run any tests you have written. Post your branch and request a code review ------------------------------------------ Once you are done with your edits, you should post your branch on Launchpad so that other IPython developers can review the changes and help you merge your changes into the main development branch. To post your branch on Launchpad, do:: $ cd ipython-mybranch $ bzr push lp:~yourusername/ipython/ipython-mybranch Then, go to the `IPython Launchpad site `_, and you should see your branch under the "Code" tab. If you click on your branch, you can provide a short description of the branch as well as mark its status. Most importantly, you should click the link that reads "Propose for merging into another branch". What does this do? This let's the other IPython developers know that your branch is ready to be reviewed and merged into the main development branch. During this review process, other developers will give you feedback and help you get your code ready to be merged. What types of things will we be looking for: * All code is documented. * All code has tests. * The entire IPython test suite passes. Once your changes have been reviewed and approved, someone will merge them into the main development branch. Documentation ============= Standalone documentation ------------------------ All standalone documentation should be written in plain text (``.txt``) files using reStructuredText [reStructuredText]_ for markup and formatting. All such documentation should be placed in directory :file:`docs/source` of the IPython source tree. The documentation in this location will serve as the main source for IPython documentation and all existing documentation should be converted to this format. To build the final documentation, we use Sphinx [Sphinx]_. Once you have Sphinx installed, you can build the html docs yourself by doing:: $ cd ipython-mybranch/docs $ make html Docstring format ---------------- Good docstrings are very important. All new code should have docstrings that are formatted using reStructuredText for markup and formatting, since it is understood by a wide variety of tools. Details about using reStructuredText for docstrings can be found `here `_. Additional PEPs of interest regarding documentation of code: * `Docstring Conventions `_ * `Docstring Processing System Framework `_ * `Docutils Design Specification `_ Coding conventions ================== General ------- In general, we'll try to follow the standard Python style conventions as described here: * `Style Guide for Python Code `_ Other comments: * In a large file, top level classes and functions should be separated by 2-3 lines to make it easier to separate them visually. * Use 4 spaces for indentation. * Keep the ordering of methods the same in classes that have the same methods. This is particularly true for classes that implement an interface. Naming conventions ------------------ In terms of naming conventions, we'll follow the guidelines from the `Style Guide for Python Code`_. For all new IPython code (and much existing code is being refactored), we'll use: * All ``lowercase`` module names. * ``CamelCase`` for class names. * ``lowercase_with_underscores`` for methods, functions, variables and attributes. There are, however, some important exceptions to these rules. In some cases, IPython code will interface with packages (Twisted, Wx, Qt) that use other conventions. At some level this makes it impossible to adhere to our own standards at all times. In particular, when subclassing classes that use other naming conventions, you must follow their naming conventions. To deal with cases like this, we propose the following policy: * If you are subclassing a class that uses different conventions, use its naming conventions throughout your subclass. Thus, if you are creating a Twisted Protocol class, used Twisted's ``namingSchemeForMethodsAndAttributes.`` * All IPython's official interfaces should use our conventions. In some cases this will mean that you need to provide shadow names (first implement ``fooBar`` and then ``foo_bar = fooBar``). We want to avoid this at all costs, but it will probably be necessary at times. But, please use this sparingly! Implementation-specific *private* methods will use ``_single_underscore_prefix``. Names with a leading double underscore will *only* be used in special cases, as they makes subclassing difficult (such names are not easily seen by child classes). Occasionally some run-in lowercase names are used, but mostly for very short names or where we are implementing methods very similar to existing ones in a base class (like ``runlines()`` where ``runsource()`` and ``runcode()`` had established precedent). The old IPython codebase has a big mix of classes and modules prefixed with an explicit ``IP``. In Python this is mostly unnecessary, redundant and frowned upon, as namespaces offer cleaner prefixing. The only case where this approach is justified is for classes which are expected to be imported into external namespaces and a very generic name (like Shell) is too likely to clash with something else. We'll need to revisit this issue as we clean up and refactor the code, but in general we should remove as many unnecessary ``IP``/``ip`` prefixes as possible. However, if a prefix seems absolutely necessary the more specific ``IPY`` or ``ipy`` are preferred. .. _devel_testing: Testing system ============== It is extremely important that all code contributed to IPython has tests. Tests should be written as unittests, doctests or as entities that the Nose [Nose]_ testing package will find. Regardless of how the tests are written, we will use Nose for discovering and running the tests. Nose will be required to run the IPython test suite, but will not be required to simply use IPython. Tests of Twisted using code need to follow two additional guidelines: 1. Twisted using tests should be written by subclassing the :class:`TestCase` class that comes with :mod:`twisted.trial.unittest`. 2. All :class:`Deferred` instances that are created in the test must be properly chained and the final one *must* be the return value of the test method. When these two things are done, Nose will be able to run the tests and the twisted reactor will be handled correctly. Each subpackage in IPython should have its own :file:`tests` directory that contains all of the tests for that subpackage. This allows each subpackage to be self-contained. If a subpackage has any dependencies beyond the Python standard library, the tests for that subpackage should be skipped if the dependencies are not found. This is very important so users don't get tests failing simply because they don't have dependencies. To run the IPython test suite, use the :command:`iptest` command that is installed with IPython:: $ iptest This command runs Nose with the proper options and extensions. .. _devel_config: Release checklist ================= Most of the release process is automated by the :file:`release` script in the :file:`tools` directory. This is just a handy reminder for the release manager. #. Run the release script, which makes the tar.gz, eggs and Win32 .exe installer. It posts them to the site and registers the release with PyPI. #. Updating the website with announcements and links to the updated changes.txt in html form. Remember to put a short note both on the news page of the site and on Launcphad. #. Drafting a short release announcement with i) highlights and ii) a link to the html changes.txt. #. Make sure that the released version of the docs is live on the site. #. Celebrate! .. [Bazaar] Bazaar. http://bazaar-vcs.org/ .. [Launchpad] Launchpad. http://www.launchpad.net/ipython .. [reStructuredText] reStructuredText. http://docutils.sourceforge.net/rst.html .. [Sphinx] Sphinx. http://sphinx.pocoo.org/ .. [Nose] Nose: a discovery based unittest extension. http://code.google.com/p/python-nose/