Show More
@@ -1,325 +1,334 b'' | |||
|
1 | 1 | .. _changes: |
|
2 | 2 | |
|
3 | 3 | ========== |
|
4 | 4 | What's new |
|
5 | 5 | ========== |
|
6 | 6 | |
|
7 | 7 | .. contents:: |
|
8 | 8 | .. |
|
9 | 1 Release 0.9 | |
|
10 | 1.1 New features | |
|
11 |
|
|
|
12 | 1.3 Backwards incompatible changes | |
|
13 | 1.4 Changes merged in from IPython1 | |
|
14 | 1.4.1 New features | |
|
15 |
|
|
|
16 | 1.4.3 Backwards incompatible changes | |
|
17 | 2 Release 0.8.4 | |
|
18 |
3 Release 0.8. |
|
|
19 |
4 Release 0.8. |
|
|
20 |
5 |
|
|
9 | 1 Release 0.9.1 | |
|
10 | 2 Release 0.9 | |
|
11 | 2.1 New features | |
|
12 | 2.2 Bug fixes | |
|
13 | 2.3 Backwards incompatible changes | |
|
14 | 2.4 Changes merged in from IPython1 | |
|
15 | 2.4.1 New features | |
|
16 | 2.4.2 Bug fixes | |
|
17 | 2.4.3 Backwards incompatible changes | |
|
18 | 3 Release 0.8.4 | |
|
19 | 4 Release 0.8.3 | |
|
20 | 5 Release 0.8.2 | |
|
21 | 6 Older releases | |
|
21 | 22 | .. |
|
22 | 23 | |
|
23 | 24 | |
|
25 | Release 0.9.1 | |
|
26 | ============= | |
|
27 | ||
|
28 | This release was quickly made to restore compatibility with Python 2.4, which | |
|
29 | version 0.9 accidentally broke. No new features were introduced, other than | |
|
30 | some additional testing support for internal use. | |
|
31 | ||
|
32 | ||
|
24 | 33 | Release 0.9 |
|
25 | 34 | =========== |
|
26 | 35 | |
|
27 | 36 | New features |
|
28 | 37 | ------------ |
|
29 | 38 | |
|
30 | 39 | * All furl files and security certificates are now put in a read-only directory |
|
31 | 40 | named ~./ipython/security. |
|
32 | 41 | |
|
33 | 42 | * A single function :func:`get_ipython_dir`, in :mod:`IPython.genutils` that |
|
34 | 43 | determines the user's IPython directory in a robust manner. |
|
35 | 44 | |
|
36 | 45 | * Laurent's WX application has been given a top-level script called ipython-wx, |
|
37 | 46 | and it has received numerous fixes. We expect this code to be |
|
38 | 47 | architecturally better integrated with Gael's WX 'ipython widget' over the |
|
39 | 48 | next few releases. |
|
40 | 49 | |
|
41 | 50 | * The Editor synchronization work by Vivian De Smedt has been merged in. This |
|
42 | 51 | code adds a number of new editor hooks to synchronize with editors under |
|
43 | 52 | Windows. |
|
44 | 53 | |
|
45 | 54 | * A new, still experimental but highly functional, WX shell by Gael Varoquaux. |
|
46 | 55 | This work was sponsored by Enthought, and while it's still very new, it is |
|
47 | 56 | based on a more cleanly organized arhictecture of the various IPython |
|
48 | 57 | components. We will continue to develop this over the next few releases as a |
|
49 | 58 | model for GUI components that use IPython. |
|
50 | 59 | |
|
51 | 60 | * Another GUI frontend, Cocoa based (Cocoa is the OSX native GUI framework), |
|
52 | 61 | authored by Barry Wark. Currently the WX and the Cocoa ones have slightly |
|
53 | 62 | different internal organizations, but the whole team is working on finding |
|
54 | 63 | what the right abstraction points are for a unified codebase. |
|
55 | 64 | |
|
56 | 65 | * As part of the frontend work, Barry Wark also implemented an experimental |
|
57 | 66 | event notification system that various ipython components can use. In the |
|
58 | 67 | next release the implications and use patterns of this system regarding the |
|
59 | 68 | various GUI options will be worked out. |
|
60 | 69 | |
|
61 | 70 | * IPython finally has a full test system, that can test docstrings with |
|
62 | 71 | IPython-specific functionality. There are still a few pieces missing for it |
|
63 | 72 | to be widely accessible to all users (so they can run the test suite at any |
|
64 | 73 | time and report problems), but it now works for the developers. We are |
|
65 | 74 | working hard on continuing to improve it, as this was probably IPython's |
|
66 | 75 | major Achilles heel (the lack of proper test coverage made it effectively |
|
67 | 76 | impossible to do large-scale refactoring). The full test suite can now |
|
68 | 77 | be run using the :command:`iptest` command line program. |
|
69 | 78 | |
|
70 | 79 | * The notion of a task has been completely reworked. An `ITask` interface has |
|
71 | 80 | been created. This interface defines the methods that tasks need to |
|
72 | 81 | implement. These methods are now responsible for things like submitting |
|
73 | 82 | tasks and processing results. There are two basic task types: |
|
74 | 83 | :class:`IPython.kernel.task.StringTask` (this is the old `Task` object, but |
|
75 | 84 | renamed) and the new :class:`IPython.kernel.task.MapTask`, which is based on |
|
76 | 85 | a function. |
|
77 | 86 | |
|
78 | 87 | * A new interface, :class:`IPython.kernel.mapper.IMapper` has been defined to |
|
79 | 88 | standardize the idea of a `map` method. This interface has a single `map` |
|
80 | 89 | method that has the same syntax as the built-in `map`. We have also defined |
|
81 | 90 | a `mapper` factory interface that creates objects that implement |
|
82 | 91 | :class:`IPython.kernel.mapper.IMapper` for different controllers. Both the |
|
83 | 92 | multiengine and task controller now have mapping capabilties. |
|
84 | 93 | |
|
85 | 94 | * The parallel function capabilities have been reworks. The major changes are |
|
86 | 95 | that i) there is now an `@parallel` magic that creates parallel functions, |
|
87 | 96 | ii) the syntax for mulitple variable follows that of `map`, iii) both the |
|
88 | 97 | multiengine and task controller now have a parallel function implementation. |
|
89 | 98 | |
|
90 | 99 | * All of the parallel computing capabilities from `ipython1-dev` have been |
|
91 | 100 | merged into IPython proper. This resulted in the following new subpackages: |
|
92 | 101 | :mod:`IPython.kernel`, :mod:`IPython.kernel.core`, :mod:`IPython.config`, |
|
93 | 102 | :mod:`IPython.tools` and :mod:`IPython.testing`. |
|
94 | 103 | |
|
95 | 104 | * As part of merging in the `ipython1-dev` stuff, the `setup.py` script and |
|
96 | 105 | friends have been completely refactored. Now we are checking for |
|
97 | 106 | dependencies using the approach that matplotlib uses. |
|
98 | 107 | |
|
99 | 108 | * The documentation has been completely reorganized to accept the documentation |
|
100 | 109 | from `ipython1-dev`. |
|
101 | 110 | |
|
102 | 111 | * We have switched to using Foolscap for all of our network protocols in |
|
103 | 112 | :mod:`IPython.kernel`. This gives us secure connections that are both |
|
104 | 113 | encrypted and authenticated. |
|
105 | 114 | |
|
106 | 115 | * We have a brand new `COPYING.txt` files that describes the IPython license |
|
107 | 116 | and copyright. The biggest change is that we are putting "The IPython |
|
108 | 117 | Development Team" as the copyright holder. We give more details about |
|
109 | 118 | exactly what this means in this file. All developer should read this and use |
|
110 | 119 | the new banner in all IPython source code files. |
|
111 | 120 | |
|
112 | 121 | * sh profile: ./foo runs foo as system command, no need to do !./foo anymore |
|
113 | 122 | |
|
114 | 123 | * String lists now support ``sort(field, nums = True)`` method (to easily sort |
|
115 | 124 | system command output). Try it with ``a = !ls -l ; a.sort(1, nums=1)``. |
|
116 | 125 | |
|
117 | 126 | * '%cpaste foo' now assigns the pasted block as string list, instead of string |
|
118 | 127 | |
|
119 | 128 | * The ipcluster script now run by default with no security. This is done |
|
120 | 129 | because the main usage of the script is for starting things on localhost. |
|
121 | 130 | Eventually when ipcluster is able to start things on other hosts, we will put |
|
122 | 131 | security back. |
|
123 | 132 | |
|
124 | 133 | * 'cd --foo' searches directory history for string foo, and jumps to that dir. |
|
125 | 134 | Last part of dir name is checked first. If no matches for that are found, |
|
126 | 135 | look at the whole path. |
|
127 | 136 | |
|
128 | 137 | |
|
129 | 138 | Bug fixes |
|
130 | 139 | --------- |
|
131 | 140 | |
|
132 | 141 | * The Windows installer has been fixed. Now all IPython scripts have ``.bat`` |
|
133 | 142 | versions created. Also, the Start Menu shortcuts have been updated. |
|
134 | 143 | |
|
135 | 144 | * The colors escapes in the multiengine client are now turned off on win32 as |
|
136 | 145 | they don't print correctly. |
|
137 | 146 | |
|
138 | 147 | * The :mod:`IPython.kernel.scripts.ipengine` script was exec'ing |
|
139 | 148 | mpi_import_statement incorrectly, which was leading the engine to crash when |
|
140 | 149 | mpi was enabled. |
|
141 | 150 | |
|
142 | 151 | * A few subpackages had missing ``__init__.py`` files. |
|
143 | 152 | |
|
144 | 153 | * The documentation is only created if Sphinx is found. Previously, the |
|
145 | 154 | ``setup.py`` script would fail if it was missing. |
|
146 | 155 | |
|
147 | 156 | * Greedy ``cd`` completion has been disabled again (it was enabled in 0.8.4) as |
|
148 | 157 | it caused problems on certain platforms. |
|
149 | 158 | |
|
150 | 159 | |
|
151 | 160 | Backwards incompatible changes |
|
152 | 161 | ------------------------------ |
|
153 | 162 | |
|
154 | 163 | * The ``clusterfile`` options of the :command:`ipcluster` command has been |
|
155 | 164 | removed as it was not working and it will be replaced soon by something much |
|
156 | 165 | more robust. |
|
157 | 166 | |
|
158 | 167 | * The :mod:`IPython.kernel` configuration now properly find the user's |
|
159 | 168 | IPython directory. |
|
160 | 169 | |
|
161 | 170 | * In ipapi, the :func:`make_user_ns` function has been replaced with |
|
162 | 171 | :func:`make_user_namespaces`, to support dict subclasses in namespace |
|
163 | 172 | creation. |
|
164 | 173 | |
|
165 | 174 | * :class:`IPython.kernel.client.Task` has been renamed |
|
166 | 175 | :class:`IPython.kernel.client.StringTask` to make way for new task types. |
|
167 | 176 | |
|
168 | 177 | * The keyword argument `style` has been renamed `dist` in `scatter`, `gather` |
|
169 | 178 | and `map`. |
|
170 | 179 | |
|
171 | 180 | * Renamed the values that the rename `dist` keyword argument can have from |
|
172 | 181 | `'basic'` to `'b'`. |
|
173 | 182 | |
|
174 | 183 | * IPython has a larger set of dependencies if you want all of its capabilities. |
|
175 | 184 | See the `setup.py` script for details. |
|
176 | 185 | |
|
177 | 186 | * The constructors for :class:`IPython.kernel.client.MultiEngineClient` and |
|
178 | 187 | :class:`IPython.kernel.client.TaskClient` no longer take the (ip,port) tuple. |
|
179 | 188 | Instead they take the filename of a file that contains the FURL for that |
|
180 | 189 | client. If the FURL file is in your IPYTHONDIR, it will be found automatically |
|
181 | 190 | and the constructor can be left empty. |
|
182 | 191 | |
|
183 | 192 | * The asynchronous clients in :mod:`IPython.kernel.asyncclient` are now created |
|
184 | 193 | using the factory functions :func:`get_multiengine_client` and |
|
185 | 194 | :func:`get_task_client`. These return a `Deferred` to the actual client. |
|
186 | 195 | |
|
187 | 196 | * The command line options to `ipcontroller` and `ipengine` have changed to |
|
188 | 197 | reflect the new Foolscap network protocol and the FURL files. Please see the |
|
189 | 198 | help for these scripts for details. |
|
190 | 199 | |
|
191 | 200 | * The configuration files for the kernel have changed because of the Foolscap |
|
192 | 201 | stuff. If you were using custom config files before, you should delete them |
|
193 | 202 | and regenerate new ones. |
|
194 | 203 | |
|
195 | 204 | Changes merged in from IPython1 |
|
196 | 205 | ------------------------------- |
|
197 | 206 | |
|
198 | 207 | New features |
|
199 | 208 | ............ |
|
200 | 209 | |
|
201 | 210 | * Much improved ``setup.py`` and ``setupegg.py`` scripts. Because Twisted and |
|
202 | 211 | zope.interface are now easy installable, we can declare them as dependencies |
|
203 | 212 | in our setupegg.py script. |
|
204 | 213 | |
|
205 | 214 | * IPython is now compatible with Twisted 2.5.0 and 8.x. |
|
206 | 215 | |
|
207 | 216 | * Added a new example of how to use :mod:`ipython1.kernel.asynclient`. |
|
208 | 217 | |
|
209 | 218 | * Initial draft of a process daemon in :mod:`ipython1.daemon`. This has not |
|
210 | 219 | been merged into IPython and is still in `ipython1-dev`. |
|
211 | 220 | |
|
212 | 221 | * The ``TaskController`` now has methods for getting the queue status. |
|
213 | 222 | |
|
214 | 223 | * The ``TaskResult`` objects not have information about how long the task |
|
215 | 224 | took to run. |
|
216 | 225 | |
|
217 | 226 | * We are attaching additional attributes to exceptions ``(_ipython_*)`` that |
|
218 | 227 | we use to carry additional info around. |
|
219 | 228 | |
|
220 | 229 | * New top-level module :mod:`asyncclient` that has asynchronous versions (that |
|
221 | 230 | return deferreds) of the client classes. This is designed to users who want |
|
222 | 231 | to run their own Twisted reactor. |
|
223 | 232 | |
|
224 | 233 | * All the clients in :mod:`client` are now based on Twisted. This is done by |
|
225 | 234 | running the Twisted reactor in a separate thread and using the |
|
226 | 235 | :func:`blockingCallFromThread` function that is in recent versions of Twisted. |
|
227 | 236 | |
|
228 | 237 | * Functions can now be pushed/pulled to/from engines using |
|
229 | 238 | :meth:`MultiEngineClient.push_function` and |
|
230 | 239 | :meth:`MultiEngineClient.pull_function`. |
|
231 | 240 | |
|
232 | 241 | * Gather/scatter are now implemented in the client to reduce the work load |
|
233 | 242 | of the controller and improve performance. |
|
234 | 243 | |
|
235 | 244 | * Complete rewrite of the IPython docuementation. All of the documentation |
|
236 | 245 | from the IPython website has been moved into docs/source as restructured |
|
237 | 246 | text documents. PDF and HTML documentation are being generated using |
|
238 | 247 | Sphinx. |
|
239 | 248 | |
|
240 | 249 | * New developer oriented documentation: development guidelines and roadmap. |
|
241 | 250 | |
|
242 | 251 | * Traditional ``ChangeLog`` has been changed to a more useful ``changes.txt`` |
|
243 | 252 | file that is organized by release and is meant to provide something more |
|
244 | 253 | relevant for users. |
|
245 | 254 | |
|
246 | 255 | Bug fixes |
|
247 | 256 | ......... |
|
248 | 257 | |
|
249 | 258 | * Created a proper ``MANIFEST.in`` file to create source distributions. |
|
250 | 259 | |
|
251 | 260 | * Fixed a bug in the ``MultiEngine`` interface. Previously, multi-engine |
|
252 | 261 | actions were being collected with a :class:`DeferredList` with |
|
253 | 262 | ``fireononeerrback=1``. This meant that methods were returning |
|
254 | 263 | before all engines had given their results. This was causing extremely odd |
|
255 | 264 | bugs in certain cases. To fix this problem, we have 1) set |
|
256 | 265 | ``fireononeerrback=0`` to make sure all results (or exceptions) are in |
|
257 | 266 | before returning and 2) introduced a :exc:`CompositeError` exception |
|
258 | 267 | that wraps all of the engine exceptions. This is a huge change as it means |
|
259 | 268 | that users will have to catch :exc:`CompositeError` rather than the actual |
|
260 | 269 | exception. |
|
261 | 270 | |
|
262 | 271 | Backwards incompatible changes |
|
263 | 272 | .............................. |
|
264 | 273 | |
|
265 | 274 | * All names have been renamed to conform to the lowercase_with_underscore |
|
266 | 275 | convention. This will require users to change references to all names like |
|
267 | 276 | ``queueStatus`` to ``queue_status``. |
|
268 | 277 | |
|
269 | 278 | * Previously, methods like :meth:`MultiEngineClient.push` and |
|
270 | 279 | :meth:`MultiEngineClient.push` used ``*args`` and ``**kwargs``. This was |
|
271 | 280 | becoming a problem as we weren't able to introduce new keyword arguments into |
|
272 | 281 | the API. Now these methods simple take a dict or sequence. This has also |
|
273 | 282 | allowed us to get rid of the ``*All`` methods like :meth:`pushAll` and |
|
274 | 283 | :meth:`pullAll`. These things are now handled with the ``targets`` keyword |
|
275 | 284 | argument that defaults to ``'all'``. |
|
276 | 285 | |
|
277 | 286 | * The :attr:`MultiEngineClient.magicTargets` has been renamed to |
|
278 | 287 | :attr:`MultiEngineClient.targets`. |
|
279 | 288 | |
|
280 | 289 | * All methods in the MultiEngine interface now accept the optional keyword |
|
281 | 290 | argument ``block``. |
|
282 | 291 | |
|
283 | 292 | * Renamed :class:`RemoteController` to :class:`MultiEngineClient` and |
|
284 | 293 | :class:`TaskController` to :class:`TaskClient`. |
|
285 | 294 | |
|
286 | 295 | * Renamed the top-level module from :mod:`api` to :mod:`client`. |
|
287 | 296 | |
|
288 | 297 | * Most methods in the multiengine interface now raise a :exc:`CompositeError` |
|
289 | 298 | exception that wraps the user's exceptions, rather than just raising the raw |
|
290 | 299 | user's exception. |
|
291 | 300 | |
|
292 | 301 | * Changed the ``setupNS`` and ``resultNames`` in the ``Task`` class to ``push`` |
|
293 | 302 | and ``pull``. |
|
294 | 303 | |
|
295 | 304 | |
|
296 | 305 | Release 0.8.4 |
|
297 | 306 | ============= |
|
298 | 307 | |
|
299 | 308 | This was a quick release to fix an unfortunate bug that slipped into the 0.8.3 |
|
300 | 309 | release. The ``--twisted`` option was disabled, as it turned out to be broken |
|
301 | 310 | across several platforms. |
|
302 | 311 | |
|
303 | 312 | |
|
304 | 313 | Release 0.8.3 |
|
305 | 314 | ============= |
|
306 | 315 | |
|
307 | 316 | * pydb is now disabled by default (due to %run -d problems). You can enable |
|
308 | 317 | it by passing -pydb command line argument to IPython. Note that setting |
|
309 | 318 | it in config file won't work. |
|
310 | 319 | |
|
311 | 320 | |
|
312 | 321 | Release 0.8.2 |
|
313 | 322 | ============= |
|
314 | 323 | |
|
315 | 324 | * %pushd/%popd behave differently; now "pushd /foo" pushes CURRENT directory |
|
316 | 325 | and jumps to /foo. The current behaviour is closer to the documented |
|
317 | 326 | behaviour, and should not trip anyone. |
|
318 | 327 | |
|
319 | 328 | |
|
320 | 329 | Older releases |
|
321 | 330 | ============== |
|
322 | 331 | |
|
323 | 332 | Changes in earlier releases of IPython are described in the older file |
|
324 | 333 | ``ChangeLog``. Please refer to this document for details. |
|
325 | 334 |
@@ -1,426 +1,446 b'' | |||
|
1 | 1 | .. _development: |
|
2 | 2 | |
|
3 |
============================== |
|
|
3 | ============================== | |
|
4 | 4 | IPython development guidelines |
|
5 |
============================== |
|
|
6 | ||
|
7 | .. contents:: | |
|
5 | ============================== | |
|
8 | 6 | |
|
9 | 7 | |
|
10 | 8 | Overview |
|
11 | 9 | ======== |
|
12 | 10 | |
|
13 | 11 | IPython is the next generation of IPython. It is named such for two reasons: |
|
14 | 12 | |
|
15 | 13 | - Eventually, IPython will become IPython version 1.0. |
|
16 | 14 | - This new code base needs to be able to co-exist with the existing IPython until |
|
17 | 15 | it is a full replacement for it. Thus we needed a different name. We couldn't |
|
18 | 16 | use ``ipython`` (lowercase) as some files systems are case insensitive. |
|
19 | 17 | |
|
20 | 18 | There are two, no three, main goals of the IPython effort: |
|
21 | 19 | |
|
22 | 20 | 1. Clean up the existing codebase and write lots of tests. |
|
23 | 21 | 2. Separate the core functionality of IPython from the terminal to enable IPython |
|
24 | 22 | to be used from within a variety of GUI applications. |
|
25 | 23 | 3. Implement a system for interactive parallel computing. |
|
26 | 24 | |
|
27 | 25 | While the third goal may seem a bit unrelated to the main focus of IPython, it |
|
28 | 26 | turns out that the technologies required for this goal are nearly identical |
|
29 | 27 | with those required for goal two. This is the main reason the interactive |
|
30 | 28 | parallel computing capabilities are being put into IPython proper. Currently |
|
31 | 29 | the third of these goals is furthest along. |
|
32 | 30 | |
|
33 | 31 | This document describes IPython from the perspective of developers. |
|
34 | 32 | |
|
35 | 33 | |
|
36 | 34 | Project organization |
|
37 | 35 | ==================== |
|
38 | 36 | |
|
39 | 37 | Subpackages |
|
40 | 38 | ----------- |
|
41 | 39 | |
|
42 | 40 | IPython is organized into semi self-contained subpackages. Each of the |
|
43 | 41 | subpackages will have its own: |
|
44 | 42 | |
|
45 | 43 | - **Dependencies**. One of the most important things to keep in mind in |
|
46 | 44 | partitioning code amongst subpackages, is that they should be used to cleanly |
|
47 | 45 | encapsulate dependencies. |
|
48 | 46 | |
|
49 | 47 | - **Tests**. Each subpackage shoud have its own ``tests`` subdirectory that |
|
50 | 48 | contains all of the tests for that package. For information about writing |
|
51 | 49 | tests for IPython, see the `Testing System`_ section of this document. |
|
52 | 50 | |
|
53 | 51 | - **Configuration**. Each subpackage should have its own ``config`` |
|
54 | 52 | subdirectory that contains the configuration information for the components |
|
55 | 53 | of the subpackage. For information about how the IPython configuration |
|
56 | 54 | system works, see the `Configuration System`_ section of this document. |
|
57 | 55 | |
|
58 | 56 | - **Scripts**. Each subpackage should have its own ``scripts`` subdirectory |
|
59 | 57 | that contains all of the command line scripts associated with the subpackage. |
|
60 | 58 | |
|
61 | 59 | Installation and dependencies |
|
62 | 60 | ----------------------------- |
|
63 | 61 | |
|
64 | 62 | IPython will not use `setuptools`_ for installation. Instead, we will use |
|
65 | 63 | standard ``setup.py`` scripts that use `distutils`_. While there are a number a |
|
66 | 64 | extremely nice features that `setuptools`_ has (like namespace packages), the |
|
67 | 65 | current implementation of `setuptools`_ has performance problems, particularly |
|
68 | 66 | on shared file systems. In particular, when Python packages are installed on |
|
69 | 67 | NSF file systems, import times become much too long (up towards 10 seconds). |
|
70 | 68 | |
|
71 | 69 | Because IPython is being used extensively in the context of high performance |
|
72 | 70 | computing, where performance is critical but shared file systems are common, we |
|
73 | 71 | feel these performance hits are not acceptable. Thus, until the performance |
|
74 | 72 | problems associated with `setuptools`_ are addressed, we will stick with plain |
|
75 | 73 | `distutils`_. We are hopeful that these problems will be addressed and that we |
|
76 | 74 | will eventually begin using `setuptools`_. Because of this, we are trying to |
|
77 | 75 | organize IPython in a way that will make the eventual transition to |
|
78 | 76 | `setuptools`_ as painless as possible. |
|
79 | 77 | |
|
80 | 78 | Because we will be using `distutils`_, there will be no method for |
|
81 | 79 | automatically installing dependencies. Instead, we are following the approach |
|
82 | 80 | of `Matplotlib`_ which can be summarized as follows: |
|
83 | 81 | |
|
84 | 82 | - Distinguish between required and optional dependencies. However, the required |
|
85 | 83 | dependencies for IPython should be only the Python standard library. |
|
86 | 84 | |
|
87 | 85 | - Upon installation check to see which optional dependencies are present and |
|
88 | 86 | tell the user which parts of IPython need which optional dependencies. |
|
89 | 87 | |
|
90 | 88 | It is absolutely critical that each subpackage of IPython has a clearly |
|
91 | 89 | specified set of dependencies and that dependencies are not carelessly |
|
92 | 90 | inherited from other IPython subpackages. Furthermore, tests that have certain |
|
93 | 91 | dependencies should not fail if those dependencies are not present. Instead |
|
94 | 92 | they should be skipped and print a message. |
|
95 | 93 | |
|
96 | 94 | .. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools |
|
97 | 95 | .. _distutils: http://docs.python.org/lib/module-distutils.html |
|
98 | 96 | .. _Matplotlib: http://matplotlib.sourceforge.net/ |
|
99 | 97 | |
|
100 | 98 | Specific subpackages |
|
101 | 99 | -------------------- |
|
102 | 100 | |
|
103 | 101 | ``core`` |
|
104 | 102 | This is the core functionality of IPython that is independent of the |
|
105 | 103 | terminal, network and GUIs. Most of the code that is in the current |
|
106 | 104 | IPython trunk will be refactored, cleaned up and moved here. |
|
107 | 105 | |
|
108 | 106 | ``kernel`` |
|
109 | 107 | The enables the IPython core to be expose to a the network. This is |
|
110 | 108 | also where all of the parallel computing capabilities are to be found. |
|
111 | 109 | |
|
112 | 110 | ``config`` |
|
113 | 111 | The configuration package used by IPython. |
|
114 | 112 | |
|
115 | 113 | ``frontends`` |
|
116 | 114 | The various frontends for IPython. A frontend is the end-user application |
|
117 | 115 | that exposes the capabilities of IPython to the user. The most basic |
|
118 | 116 | frontend will simply be a terminal based application that looks just like |
|
119 | 117 | today 's IPython. Other frontends will likely be more powerful and based |
|
120 | 118 | on GUI toolkits. |
|
121 | 119 | |
|
122 | 120 | ``notebook`` |
|
123 | 121 | An application that allows users to work with IPython notebooks. |
|
124 | 122 | |
|
125 | 123 | ``tools`` |
|
126 | 124 | This is where general utilities go. |
|
127 | 125 | |
|
128 | 126 | |
|
129 | 127 | Version control |
|
130 | 128 | =============== |
|
131 | 129 | |
|
132 | 130 | In the past, IPython development has been done using `Subversion`__. Recently, |
|
133 | 131 | we made the transition to using `Bazaar`__ and `Launchpad`__. This makes it |
|
134 | 132 | much easier for people to contribute code to IPython. Here is a sketch of how |
|
135 | 133 | to use Bazaar for IPython development. First, you should install Bazaar. |
|
136 | 134 | After you have done that, make sure that it is working by getting the latest |
|
137 | 135 | main branch of IPython:: |
|
138 | 136 | |
|
139 | 137 | $ bzr branch lp:ipython |
|
140 | 138 | |
|
141 | 139 | Now you can create a new branch for you to do your work in:: |
|
142 | 140 | |
|
143 | 141 | $ bzr branch ipython ipython-mybranch |
|
144 | 142 | |
|
145 | 143 | The typical work cycle in this branch will be to make changes in |
|
146 | 144 | ``ipython-mybranch`` and then commit those changes using the commit command:: |
|
147 | 145 | |
|
148 | 146 | $ ...do work in ipython-mybranch... |
|
149 | 147 | $ bzr ci -m "the commit message goes here" |
|
150 | 148 | |
|
151 | 149 | Please note that since we now don't use an old-style linear ChangeLog (that |
|
152 | 150 | tends to cause problems with distributed version control systems), you should |
|
153 | 151 | ensure that your log messages are reasonably detailed. Use a docstring-like |
|
154 | 152 | approach in the commit messages (including the second line being left |
|
155 | 153 | *blank*):: |
|
156 | 154 | |
|
157 | 155 | Single line summary of changes being committed. |
|
158 | 156 | |
|
159 | 157 | - more details when warranted ... |
|
160 | 158 | - including crediting outside contributors if they sent the |
|
161 | 159 | code/bug/idea! |
|
162 | 160 | |
|
163 | 161 | If we couple this with a policy of making single commits for each reasonably |
|
164 | 162 | atomic change, the bzr log should give an excellent view of the project, and |
|
165 | 163 | the `--short` log option becomes a nice summary. |
|
166 | 164 | |
|
167 | 165 | While working with this branch, it is a good idea to merge in changes that have |
|
168 | 166 | been made upstream in the parent branch. This can be done by doing:: |
|
169 | 167 | |
|
170 | 168 | $ bzr pull |
|
171 | 169 | |
|
172 | 170 | If this command shows that the branches have diverged, then you should do a |
|
173 | 171 | merge instead:: |
|
174 | 172 | |
|
175 | 173 | $ bzr merge lp:ipython |
|
176 | 174 | |
|
177 | 175 | If you want others to be able to see your branch, you can create an account |
|
178 | 176 | with launchpad and push the branch to your own workspace:: |
|
179 | 177 | |
|
180 | 178 | $ bzr push bzr+ssh://<me>@bazaar.launchpad.net/~<me>/+junk/ipython-mybranch |
|
181 | 179 | |
|
182 | 180 | Finally, once the work in your branch is done, you can merge your changes back |
|
183 | 181 | into the `ipython` branch by using merge:: |
|
184 | 182 | |
|
185 | 183 | $ cd ipython |
|
186 | 184 | $ merge ../ipython-mybranch |
|
187 | 185 | [resolve any conflicts] |
|
188 | 186 | $ bzr ci -m "Fixing that bug" |
|
189 | 187 | $ bzr push |
|
190 | 188 | |
|
191 | 189 | But this will require you to have write permissions to the `ipython` branch. |
|
192 | 190 | It you don't you can tell one of the IPython devs about your branch and they |
|
193 | 191 | can do the merge for you. |
|
194 | 192 | |
|
195 | 193 | More information about Bazaar workflows can be found `here`__. |
|
196 | 194 | |
|
197 | 195 | .. __: http://subversion.tigris.org/ |
|
198 | 196 | .. __: http://bazaar-vcs.org/ |
|
199 | 197 | .. __: http://www.launchpad.net/ipython |
|
200 | 198 | .. __: http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html |
|
201 | 199 | |
|
202 | 200 | Documentation |
|
203 | 201 | ============= |
|
204 | 202 | |
|
205 | 203 | Standalone documentation |
|
206 | 204 | ------------------------ |
|
207 | 205 | |
|
208 | 206 | All standalone documentation should be written in plain text (``.txt``) files |
|
209 | 207 | using `reStructuredText`_ for markup and formatting. All such documentation |
|
210 | 208 | should be placed in the top level directory ``docs`` of the IPython source |
|
211 | 209 | tree. Or, when appropriate, a suitably named subdirectory should be used. The |
|
212 | 210 | documentation in this location will serve as the main source for IPython |
|
213 | 211 | documentation and all existing documentation should be converted to this |
|
214 | 212 | format. |
|
215 | 213 | |
|
216 | 214 | In the future, the text files in the ``docs`` directory will be used to |
|
217 | 215 | generate all forms of documentation for IPython. This include documentation on |
|
218 | 216 | the IPython website as well as *pdf* documentation. |
|
219 | 217 | |
|
220 | 218 | .. _reStructuredText: http://docutils.sourceforge.net/rst.html |
|
221 | 219 | |
|
222 | 220 | Docstring format |
|
223 | 221 | ---------------- |
|
224 | 222 | |
|
225 | 223 | Good docstrings are very important. All new code will use `Epydoc`_ for |
|
226 | 224 | generating API docs, so we will follow the `Epydoc`_ conventions. More |
|
227 | 225 | specifically, we will use `reStructuredText`_ for markup and formatting, since |
|
228 | 226 | it is understood by a wide variety of tools. This means that if in the future |
|
229 | 227 | we have any reason to change from `Epydoc`_ to something else, we'll have fewer |
|
230 | 228 | transition pains. |
|
231 | 229 | |
|
232 | 230 | Details about using `reStructuredText`_ for docstrings can be found `here |
|
233 | 231 | <http://epydoc.sourceforge.net/manual-othermarkup.html>`_. |
|
234 | 232 | |
|
235 | 233 | .. _Epydoc: http://epydoc.sourceforge.net/ |
|
236 | 234 | |
|
237 | 235 | Additional PEPs of interest regarding documentation of code: |
|
238 | 236 | |
|
239 | 237 | - `Docstring Conventions <http://www.python.org/peps/pep-0257.html>`_ |
|
240 | 238 | - `Docstring Processing System Framework <http://www.python.org/peps/pep-0256.html>`_ |
|
241 | 239 | - `Docutils Design Specification <http://www.python.org/peps/pep-0258.html>`_ |
|
242 | 240 | |
|
243 | 241 | |
|
244 | 242 | Coding conventions |
|
245 | 243 | ================== |
|
246 | 244 | |
|
247 | 245 | General |
|
248 | 246 | ------- |
|
249 | 247 | |
|
250 | 248 | In general, we'll try to follow the standard Python style conventions as |
|
251 | 249 | described here: |
|
252 | 250 | |
|
253 | 251 | - `Style Guide for Python Code <http://www.python.org/peps/pep-0008.html>`_ |
|
254 | 252 | |
|
255 | 253 | |
|
256 | 254 | Other comments: |
|
257 | 255 | |
|
258 | 256 | - In a large file, top level classes and functions should be |
|
259 | 257 | separated by 2-3 lines to make it easier to separate them visually. |
|
260 | 258 | - Use 4 spaces for indentation. |
|
261 | 259 | - Keep the ordering of methods the same in classes that have the same |
|
262 | 260 | methods. This is particularly true for classes that implement |
|
263 | 261 | similar interfaces and for interfaces that are similar. |
|
264 | 262 | |
|
265 | 263 | Naming conventions |
|
266 | 264 | ------------------ |
|
267 | 265 | |
|
268 | 266 | In terms of naming conventions, we'll follow the guidelines from the `Style |
|
269 | 267 | Guide for Python Code`_. |
|
270 | 268 | |
|
271 | 269 | For all new IPython code (and much existing code is being refactored), we'll use: |
|
272 | 270 | |
|
273 | 271 | - All ``lowercase`` module names. |
|
274 | 272 | |
|
275 | 273 | - ``CamelCase`` for class names. |
|
276 | 274 | |
|
277 | 275 | - ``lowercase_with_underscores`` for methods, functions, variables and |
|
278 | 276 | attributes. |
|
279 | 277 | |
|
280 | 278 | This may be confusing as most of the existing IPython codebase uses a different |
|
281 | 279 | convention (``lowerCamelCase`` for methods and attributes). Slowly, we will |
|
282 | 280 | move IPython over to the new convention, providing shadow names for backward |
|
283 | 281 | compatibility in public interfaces. |
|
284 | 282 | |
|
285 | 283 | There are, however, some important exceptions to these rules. In some cases, |
|
286 | 284 | IPython code will interface with packages (Twisted, Wx, Qt) that use other |
|
287 | 285 | conventions. At some level this makes it impossible to adhere to our own |
|
288 | 286 | standards at all times. In particular, when subclassing classes that use other |
|
289 | 287 | naming conventions, you must follow their naming conventions. To deal with |
|
290 | 288 | cases like this, we propose the following policy: |
|
291 | 289 | |
|
292 | 290 | - If you are subclassing a class that uses different conventions, use its |
|
293 | 291 | naming conventions throughout your subclass. Thus, if you are creating a |
|
294 | 292 | Twisted Protocol class, used Twisted's |
|
295 | 293 | ``namingSchemeForMethodsAndAttributes.`` |
|
296 | 294 | |
|
297 | 295 | - All IPython's official interfaces should use our conventions. In some cases |
|
298 | 296 | this will mean that you need to provide shadow names (first implement |
|
299 | 297 | ``fooBar`` and then ``foo_bar = fooBar``). We want to avoid this at all |
|
300 | 298 | costs, but it will probably be necessary at times. But, please use this |
|
301 | 299 | sparingly! |
|
302 | 300 | |
|
303 | 301 | Implementation-specific *private* methods will use |
|
304 | 302 | ``_single_underscore_prefix``. Names with a leading double underscore will |
|
305 | 303 | *only* be used in special cases, as they makes subclassing difficult (such |
|
306 | 304 | names are not easily seen by child classes). |
|
307 | 305 | |
|
308 | 306 | Occasionally some run-in lowercase names are used, but mostly for very short |
|
309 | 307 | names or where we are implementing methods very similar to existing ones in a |
|
310 | 308 | base class (like ``runlines()`` where ``runsource()`` and ``runcode()`` had |
|
311 | 309 | established precedent). |
|
312 | 310 | |
|
313 | 311 | The old IPython codebase has a big mix of classes and modules prefixed with an |
|
314 | 312 | explicit ``IP``. In Python this is mostly unnecessary, redundant and frowned |
|
315 | 313 | upon, as namespaces offer cleaner prefixing. The only case where this approach |
|
316 | 314 | is justified is for classes which are expected to be imported into external |
|
317 | 315 | namespaces and a very generic name (like Shell) is too likely to clash with |
|
318 | 316 | something else. We'll need to revisit this issue as we clean up and refactor |
|
319 | 317 | the code, but in general we should remove as many unnecessary ``IP``/``ip`` |
|
320 | 318 | prefixes as possible. However, if a prefix seems absolutely necessary the more |
|
321 | 319 | specific ``IPY`` or ``ipy`` are preferred. |
|
322 | 320 | |
|
323 | 321 | .. _devel_testing: |
|
324 | 322 | |
|
325 | 323 | Testing system |
|
326 | 324 | ============== |
|
327 | 325 | |
|
328 | 326 | It is extremely important that all code contributed to IPython has tests. Tests |
|
329 | 327 | should be written as unittests, doctests or as entities that the `Nose`_ |
|
330 | 328 | testing package will find. Regardless of how the tests are written, we will use |
|
331 | 329 | `Nose`_ for discovering and running the tests. `Nose`_ will be required to run |
|
332 | 330 | the IPython test suite, but will not be required to simply use IPython. |
|
333 | 331 | |
|
334 | 332 | .. _Nose: http://code.google.com/p/python-nose/ |
|
335 | 333 | |
|
336 | 334 | Tests of `Twisted`__ using code should be written by subclassing the |
|
337 | 335 | ``TestCase`` class that comes with ``twisted.trial.unittest``. When this is |
|
338 | 336 | done, `Nose`_ will be able to run the tests and the twisted reactor will be |
|
339 | 337 | handled correctly. |
|
340 | 338 | |
|
341 | 339 | .. __: http://www.twistedmatrix.com |
|
342 | 340 | |
|
343 | 341 | Each subpackage in IPython should have its own ``tests`` directory that |
|
344 | 342 | contains all of the tests for that subpackage. This allows each subpackage to |
|
345 | 343 | be self-contained. If a subpackage has any dependencies beyond the Python |
|
346 | 344 | standard library, the tests for that subpackage should be skipped if the |
|
347 | 345 | dependencies are not found. This is very important so users don't get tests |
|
348 | 346 | failing simply because they don't have dependencies. |
|
349 | 347 | |
|
350 | 348 | We also need to look into use Noses ability to tag tests to allow a more |
|
351 | 349 | modular approach of running tests. |
|
352 | 350 | |
|
353 | 351 | .. _devel_config: |
|
354 | 352 | |
|
355 | 353 | Configuration system |
|
356 | 354 | ==================== |
|
357 | 355 | |
|
358 | 356 | IPython uses `.ini`_ files for configuration purposes. This represents a huge |
|
359 | 357 | improvement over the configuration system used in IPython. IPython works with |
|
360 | 358 | these files using the `ConfigObj`_ package, which IPython includes as |
|
361 | 359 | ``ipython1/external/configobj.py``. |
|
362 | 360 | |
|
363 | 361 | Currently, we are using raw `ConfigObj`_ objects themselves. Each subpackage of |
|
364 | 362 | IPython should contain a ``config`` subdirectory that contains all of the |
|
365 | 363 | configuration information for the subpackage. To see how configuration |
|
366 | 364 | information is defined (along with defaults) see at the examples in |
|
367 | 365 | ``ipython1/kernel/config`` and ``ipython1/core/config``. Likewise, to see how |
|
368 | 366 | the configuration information is used, see examples in |
|
369 | 367 | ``ipython1/kernel/scripts/ipengine.py``. |
|
370 | 368 | |
|
371 | 369 | Eventually, we will add a new layer on top of the raw `ConfigObj`_ objects. We |
|
372 | 370 | are calling this new layer, ``tconfig``, as it will use a `Traits`_-like |
|
373 | 371 | validation model. We won't actually use `Traits`_, but will implement |
|
374 | 372 | something similar in pure Python. But, even in this new system, we will still |
|
375 | 373 | use `ConfigObj`_ and `.ini`_ files underneath the hood. Talk to Fernando if you |
|
376 | 374 | are interested in working on this part of IPython. The current prototype of |
|
377 | 375 | ``tconfig`` is located in the IPython sandbox. |
|
378 | 376 | |
|
379 | 377 | .. _.ini: http://docs.python.org/lib/module-ConfigParser.html |
|
380 | 378 | .. _ConfigObj: http://www.voidspace.org.uk/python/configobj.html |
|
381 | 379 | .. _Traits: http://code.enthought.com/traits/ |
|
382 | 380 | |
|
381 | ||
|
383 | 382 | Installation and testing scenarios |
|
384 | 383 | ================================== |
|
385 | 384 | |
|
386 | 385 | This section outlines the various scenarios that we need to test before we |
|
387 | 386 | release an IPython version. These scenarios represent different ways of |
|
388 | 387 | installing IPython and its dependencies. |
|
389 | 388 | |
|
390 | 389 | Installation scenarios under Linux and OS X |
|
391 | 390 | ------------------------------------------- |
|
392 | 391 | |
|
393 | 392 | 1. Install from tarball using ``python setup.py install``. |
|
394 | 393 | a. With only readline+nose dependencies installed. |
|
395 | 394 | b. With all dependencies installed (readline, zope.interface, Twisted, |
|
396 | 395 | foolscap, Sphinx, nose, pyOpenSSL). |
|
397 | 396 | |
|
398 | 397 | 2. Install using easy_install. |
|
399 | 398 | |
|
400 | 399 | a. With only readline+nose dependencies installed. |
|
401 | 400 | i. Default dependencies: ``easy_install ipython-0.9.beta3-py2.5.egg`` |
|
402 | 401 | ii. Optional dependency sets: ``easy_install -f ipython-0.9.beta3-py2.5.egg IPython[kernel,doc,test,security]`` |
|
403 | 402 | |
|
404 | 403 | b. With all dependencies already installed. |
|
405 | 404 | |
|
406 | 405 | |
|
407 | 406 | Installation scenarios under Win32 |
|
408 | 407 | ---------------------------------- |
|
409 | 408 | |
|
410 | 409 | 1. Install everything from .exe installers |
|
411 | 410 | 2. easy_install? |
|
412 | 411 | |
|
413 | 412 | |
|
414 | 413 | Tests to run for these scenarios |
|
415 | 414 | -------------------------------- |
|
416 | 415 | |
|
417 | 416 | 1. Run the full test suite. |
|
418 | 417 | 2. Start a controller and engines and try a few things by hand. |
|
419 | 418 | a. Using ipcluster. |
|
420 | 419 | b. Using ipcontroller/ipengine by hand. |
|
421 | 420 | |
|
422 | 421 | 3. Run a few of the parallel examples. |
|
423 | 422 | 4. Try the kernel with and without security with and without PyOpenSSL |
|
424 | 423 | installed. |
|
425 | 424 | 5. Beat on the IPython terminal a bunch. |
|
426 | 425 | 6. Make sure that furl files are being put in proper locations. |
|
426 | ||
|
427 | ||
|
428 | Release checklist | |
|
429 | ================= | |
|
430 | ||
|
431 | Most of the release process is automated by the :file:`release` script in the | |
|
432 | :file:`tools` directory. This is just a handy reminder for the release manager. | |
|
433 | ||
|
434 | #. Run the release script, which makes the tar.gz, eggs and Win32 .exe | |
|
435 | installer. It posts them to the site and registers the release with PyPI. | |
|
436 | ||
|
437 | #. Updating the website with announcements and links to the updated changes.txt | |
|
438 | in html form. Remember to put a short note both on the news page of the site | |
|
439 | and on launcphad. | |
|
440 | ||
|
441 | #. Drafting a short release announcement with i) highlights and ii) a link to | |
|
442 | the html changes.txt. | |
|
443 | ||
|
444 | #. Make sure that the released version of the docs is live on the site. | |
|
445 | ||
|
446 | #. Celebrate! |
@@ -1,32 +1,32 b'' | |||
|
1 | 1 | ===================== |
|
2 | 2 | IPython Documentation |
|
3 | 3 | ===================== |
|
4 | 4 | |
|
5 | 5 | .. htmlonly:: |
|
6 | 6 | |
|
7 |
:Release: | |
|
|
7 | :Release: |release| | |
|
8 | 8 | :Date: |today| |
|
9 | 9 | |
|
10 | 10 | Contents: |
|
11 | 11 | |
|
12 | 12 | .. toctree:: |
|
13 | 13 | :maxdepth: 2 |
|
14 | 14 | |
|
15 | 15 | overview.txt |
|
16 | 16 | install/index.txt |
|
17 | 17 | interactive/index.txt |
|
18 | 18 | parallel/index.txt |
|
19 | 19 | config/index.txt |
|
20 | 20 | changes.txt |
|
21 | 21 | development/index.txt |
|
22 | 22 | faq.txt |
|
23 | 23 | history.txt |
|
24 | 24 | license_and_copyright.txt |
|
25 | 25 | credits.txt |
|
26 | 26 | |
|
27 | 27 | |
|
28 | 28 | .. htmlonly:: |
|
29 | 29 | |
|
30 | 30 | * :ref:`genindex` |
|
31 | 31 | * :ref:`modindex` |
|
32 | 32 | * :ref:`search` |
General Comments 0
You need to be logged in to leave comments.
Login now