##// END OF EJS Templates
updated portability picture
Paul Ivanov -
Show More
@@ -1,286 +1,285 b''
1 .. _overview:
1 .. _overview:
2
2
3 ============
3 ============
4 Introduction
4 Introduction
5 ============
5 ============
6
6
7 Overview
7 Overview
8 ========
8 ========
9
9
10 One of Python's most useful features is its interactive interpreter.
10 One of Python's most useful features is its interactive interpreter.
11 It allows for very fast testing of ideas without the overhead of
11 It allows for very fast testing of ideas without the overhead of
12 creating test files as is typical in most programming languages.
12 creating test files as is typical in most programming languages.
13 However, the interpreter supplied with the standard Python distribution
13 However, the interpreter supplied with the standard Python distribution
14 is somewhat limited for extended interactive use.
14 is somewhat limited for extended interactive use.
15
15
16 The goal of IPython is to create a comprehensive environment for
16 The goal of IPython is to create a comprehensive environment for
17 interactive and exploratory computing. To support this goal, IPython
17 interactive and exploratory computing. To support this goal, IPython
18 has three main components:
18 has three main components:
19
19
20 * An enhanced interactive Python shell.
20 * An enhanced interactive Python shell.
21 * A decoupled two-process communication model, which allows for multiple
21 * A decoupled two-process communication model, which allows for multiple
22 clients to connect to a computation kernel, most notably the web-based
22 clients to connect to a computation kernel, most notably the web-based
23 :ref:`notebook <htmlnotebook>`
23 :ref:`notebook <htmlnotebook>`
24 * An architecture for interactive parallel computing.
24 * An architecture for interactive parallel computing.
25
25
26 All of IPython is open source (released under the revised BSD license).
26 All of IPython is open source (released under the revised BSD license).
27
27
28 Enhanced interactive Python shell
28 Enhanced interactive Python shell
29 =================================
29 =================================
30
30
31 IPython's interactive shell (:command:`ipython`), has the following goals,
31 IPython's interactive shell (:command:`ipython`), has the following goals,
32 amongst others:
32 amongst others:
33
33
34 1. Provide an interactive shell superior to Python's default. IPython
34 1. Provide an interactive shell superior to Python's default. IPython
35 has many features for tab-completion, object introspection, system shell
35 has many features for tab-completion, object introspection, system shell
36 access, command history retrieval across sessions, and its own special
36 access, command history retrieval across sessions, and its own special
37 command system for adding functionality when working interactively. It
37 command system for adding functionality when working interactively. It
38 tries to be a very efficient environment both for Python code development
38 tries to be a very efficient environment both for Python code development
39 and for exploration of problems using Python objects (in situations like
39 and for exploration of problems using Python objects (in situations like
40 data analysis).
40 data analysis).
41
41
42 2. Serve as an embeddable, ready to use interpreter for your own
42 2. Serve as an embeddable, ready to use interpreter for your own
43 programs. An interactive IPython shell can be started with a single call
43 programs. An interactive IPython shell can be started with a single call
44 from inside another program, providing access to the current namespace.
44 from inside another program, providing access to the current namespace.
45 This can be very useful both for debugging purposes and for situations
45 This can be very useful both for debugging purposes and for situations
46 where a blend of batch-processing and interactive exploration are needed.
46 where a blend of batch-processing and interactive exploration are needed.
47
47
48 3. Offer a flexible framework which can be used as the base
48 3. Offer a flexible framework which can be used as the base
49 environment for working with other systems, with Python as the underlying
49 environment for working with other systems, with Python as the underlying
50 bridge language. Specifically scientific environments like Mathematica,
50 bridge language. Specifically scientific environments like Mathematica,
51 IDL and Matlab inspired its design, but similar ideas can be
51 IDL and Matlab inspired its design, but similar ideas can be
52 useful in many fields.
52 useful in many fields.
53
53
54 4. Allow interactive testing of threaded graphical toolkits. IPython
54 4. Allow interactive testing of threaded graphical toolkits. IPython
55 has support for interactive, non-blocking control of GTK, Qt, WX, GLUT, and
55 has support for interactive, non-blocking control of GTK, Qt, WX, GLUT, and
56 OS X applications via special threading flags. The normal Python
56 OS X applications via special threading flags. The normal Python
57 shell can only do this for Tkinter applications.
57 shell can only do this for Tkinter applications.
58
58
59 Main features of the interactive shell
59 Main features of the interactive shell
60 --------------------------------------
60 --------------------------------------
61
61
62 * Dynamic object introspection. One can access docstrings, function
62 * Dynamic object introspection. One can access docstrings, function
63 definition prototypes, source code, source files and other details
63 definition prototypes, source code, source files and other details
64 of any object accessible to the interpreter with a single
64 of any object accessible to the interpreter with a single
65 keystroke (:samp:`?`, and using :samp:`??` provides additional detail).
65 keystroke (:samp:`?`, and using :samp:`??` provides additional detail).
66
66
67 * Searching through modules and namespaces with :samp:`*` wildcards, both
67 * Searching through modules and namespaces with :samp:`*` wildcards, both
68 when using the :samp:`?` system and via the :samp:`%psearch` command.
68 when using the :samp:`?` system and via the :samp:`%psearch` command.
69
69
70 * Completion in the local namespace, by typing :kbd:`TAB` at the prompt.
70 * Completion in the local namespace, by typing :kbd:`TAB` at the prompt.
71 This works for keywords, modules, methods, variables and files in the
71 This works for keywords, modules, methods, variables and files in the
72 current directory. This is supported via the readline library, and
72 current directory. This is supported via the readline library, and
73 full access to configuring readline's behavior is provided.
73 full access to configuring readline's behavior is provided.
74 Custom completers can be implemented easily for different purposes
74 Custom completers can be implemented easily for different purposes
75 (system commands, magic arguments etc.)
75 (system commands, magic arguments etc.)
76
76
77 * Numbered input/output prompts with command history (persistent
77 * Numbered input/output prompts with command history (persistent
78 across sessions and tied to each profile), full searching in this
78 across sessions and tied to each profile), full searching in this
79 history and caching of all input and output.
79 history and caching of all input and output.
80
80
81 * User-extensible 'magic' commands. A set of commands prefixed with
81 * User-extensible 'magic' commands. A set of commands prefixed with
82 :samp:`%` is available for controlling IPython itself and provides
82 :samp:`%` is available for controlling IPython itself and provides
83 directory control, namespace information and many aliases to
83 directory control, namespace information and many aliases to
84 common system shell commands.
84 common system shell commands.
85
85
86 * Alias facility for defining your own system aliases.
86 * Alias facility for defining your own system aliases.
87
87
88 * Complete system shell access. Lines starting with :samp:`!` are passed
88 * Complete system shell access. Lines starting with :samp:`!` are passed
89 directly to the system shell, and using :samp:`!!` or :samp:`var = !cmd`
89 directly to the system shell, and using :samp:`!!` or :samp:`var = !cmd`
90 captures shell output into python variables for further use.
90 captures shell output into python variables for further use.
91
91
92 * The ability to expand python variables when calling the system shell. In a
92 * The ability to expand python variables when calling the system shell. In a
93 shell command, any python variable prefixed with :samp:`$` is expanded. A
93 shell command, any python variable prefixed with :samp:`$` is expanded. A
94 double :samp:`$$` allows passing a literal :samp:`$` to the shell (for access
94 double :samp:`$$` allows passing a literal :samp:`$` to the shell (for access
95 to shell and environment variables like :envvar:`PATH`).
95 to shell and environment variables like :envvar:`PATH`).
96
96
97 * Filesystem navigation, via a magic :samp:`%cd` command, along with a
97 * Filesystem navigation, via a magic :samp:`%cd` command, along with a
98 persistent bookmark system (using :samp:`%bookmark`) for fast access to
98 persistent bookmark system (using :samp:`%bookmark`) for fast access to
99 frequently visited directories.
99 frequently visited directories.
100
100
101 * A lightweight persistence framework via the :samp:`%store` command, which
101 * A lightweight persistence framework via the :samp:`%store` command, which
102 allows you to save arbitrary Python variables. These get restored
102 allows you to save arbitrary Python variables. These get restored
103 when you run the :samp:`%store -r` command.
103 when you run the :samp:`%store -r` command.
104
104
105 * Automatic indentation (optional) of code as you type (through the
105 * Automatic indentation (optional) of code as you type (through the
106 readline library).
106 readline library).
107
107
108 * Macro system for quickly re-executing multiple lines of previous
108 * Macro system for quickly re-executing multiple lines of previous
109 input with a single name via the :samp:`%macro` command. Macros can be
109 input with a single name via the :samp:`%macro` command. Macros can be
110 stored persistently via :samp:`%store` and edited via :samp:`%edit`.
110 stored persistently via :samp:`%store` and edited via :samp:`%edit`.
111
111
112 * Session logging (you can then later use these logs as code in your
112 * Session logging (you can then later use these logs as code in your
113 programs). Logs can optionally timestamp all input, and also store
113 programs). Logs can optionally timestamp all input, and also store
114 session output (marked as comments, so the log remains valid
114 session output (marked as comments, so the log remains valid
115 Python source code).
115 Python source code).
116
116
117 * Session restoring: logs can be replayed to restore a previous
117 * Session restoring: logs can be replayed to restore a previous
118 session to the state where you left it.
118 session to the state where you left it.
119
119
120 * Verbose and colored exception traceback printouts. Easier to parse
120 * Verbose and colored exception traceback printouts. Easier to parse
121 visually, and in verbose mode they produce a lot of useful
121 visually, and in verbose mode they produce a lot of useful
122 debugging information (basically a terminal version of the cgitb
122 debugging information (basically a terminal version of the cgitb
123 module).
123 module).
124
124
125 * Auto-parentheses via the :samp:`%autocall` command: callable objects can be
125 * Auto-parentheses via the :samp:`%autocall` command: callable objects can be
126 executed without parentheses: :samp:`sin 3` is automatically converted to
126 executed without parentheses: :samp:`sin 3` is automatically converted to
127 :samp:`sin(3)`
127 :samp:`sin(3)`
128
128
129 * Auto-quoting: using :samp:`,`, or :samp:`;` as the first character forces
129 * Auto-quoting: using :samp:`,`, or :samp:`;` as the first character forces
130 auto-quoting of the rest of the line: :samp:`,my_function a b` becomes
130 auto-quoting of the rest of the line: :samp:`,my_function a b` becomes
131 automatically :samp:`my_function("a","b")`, while :samp:`;my_function a b`
131 automatically :samp:`my_function("a","b")`, while :samp:`;my_function a b`
132 becomes :samp:`my_function("a b")`.
132 becomes :samp:`my_function("a b")`.
133
133
134 * Extensible input syntax. You can define filters that pre-process
134 * Extensible input syntax. You can define filters that pre-process
135 user input to simplify input in special situations. This allows
135 user input to simplify input in special situations. This allows
136 for example pasting multi-line code fragments which start with
136 for example pasting multi-line code fragments which start with
137 :samp:`>>>` or :samp:`...` such as those from other python sessions or the
137 :samp:`>>>` or :samp:`...` such as those from other python sessions or the
138 standard Python documentation.
138 standard Python documentation.
139
139
140 * Flexible :ref:`configuration system <config_overview>`. It uses a
140 * Flexible :ref:`configuration system <config_overview>`. It uses a
141 configuration file which allows permanent setting of all command-line
141 configuration file which allows permanent setting of all command-line
142 options, module loading, code and file execution. The system allows
142 options, module loading, code and file execution. The system allows
143 recursive file inclusion, so you can have a base file with defaults and
143 recursive file inclusion, so you can have a base file with defaults and
144 layers which load other customizations for particular projects.
144 layers which load other customizations for particular projects.
145
145
146 * Embeddable. You can call IPython as a python shell inside your own
146 * Embeddable. You can call IPython as a python shell inside your own
147 python programs. This can be used both for debugging code or for
147 python programs. This can be used both for debugging code or for
148 providing interactive abilities to your programs with knowledge
148 providing interactive abilities to your programs with knowledge
149 about the local namespaces (very useful in debugging and data
149 about the local namespaces (very useful in debugging and data
150 analysis situations).
150 analysis situations).
151
151
152 * Easy debugger access. You can set IPython to call up an enhanced version of
152 * Easy debugger access. You can set IPython to call up an enhanced version of
153 the Python debugger (pdb) every time there is an uncaught exception. This
153 the Python debugger (pdb) every time there is an uncaught exception. This
154 drops you inside the code which triggered the exception with all the data
154 drops you inside the code which triggered the exception with all the data
155 live and it is possible to navigate the stack to rapidly isolate the source
155 live and it is possible to navigate the stack to rapidly isolate the source
156 of a bug. The :samp:`%run` magic command (with the :samp:`-d` option) can run
156 of a bug. The :samp:`%run` magic command (with the :samp:`-d` option) can run
157 any script under pdb's control, automatically setting initial breakpoints for
157 any script under pdb's control, automatically setting initial breakpoints for
158 you. This version of pdb has IPython-specific improvements, including
158 you. This version of pdb has IPython-specific improvements, including
159 tab-completion and traceback coloring support. For even easier debugger
159 tab-completion and traceback coloring support. For even easier debugger
160 access, try :samp:`%debug` after seeing an exception.
160 access, try :samp:`%debug` after seeing an exception.
161
161
162 * Profiler support. You can run single statements (similar to
162 * Profiler support. You can run single statements (similar to
163 :samp:`profile.run()`) or complete programs under the profiler's control.
163 :samp:`profile.run()`) or complete programs under the profiler's control.
164 While this is possible with standard cProfile or profile modules,
164 While this is possible with standard cProfile or profile modules,
165 IPython wraps this functionality with magic commands (see :samp:`%prun`
165 IPython wraps this functionality with magic commands (see :samp:`%prun`
166 and :samp:`%run -p`) convenient for rapid interactive work.
166 and :samp:`%run -p`) convenient for rapid interactive work.
167
167
168 * Simple timing information. You can use the :samp:`%timeit` command to get
168 * Simple timing information. You can use the :samp:`%timeit` command to get
169 the execution time of a Python statement or expression. This machinery is
169 the execution time of a Python statement or expression. This machinery is
170 intelligent enough to do more repetitions for commands that finish very
170 intelligent enough to do more repetitions for commands that finish very
171 quickly in order to get a better estimate of their running time.
171 quickly in order to get a better estimate of their running time.
172
172
173 .. sourcecode:: ipython
173 .. sourcecode:: ipython
174
174
175 In [5]: %timeit 1+1
175 In [5]: %timeit 1+1
176 10000000 loops, best of 3: 25.5 ns per loop
176 10000000 loops, best of 3: 25.5 ns per loop
177 .. _sourcecode:
177 .. _sourcecode:
178
178
179 In [2]: %timeit [math.sin(x) for x in range(5000)]
179 In [2]: %timeit [math.sin(x) for x in range(5000)]
180 1000 loops, best of 3: 719 Β΅s per loop
180 1000 loops, best of 3: 719 Β΅s per loop
181
181
182 ..
182 ..
183
183
184 To get the timing information for more than one expression, use the
184 To get the timing information for more than one expression, use the
185 :samp:`%%timeit` cell magic command.
185 :samp:`%%timeit` cell magic command.
186
186
187
187
188 * Doctest support. The special :samp:`%doctest_mode` command toggles a mode
188 * Doctest support. The special :samp:`%doctest_mode` command toggles a mode
189 uses doctest-compatible prompts, so you can use IPython sessions as doctest
189 uses doctest-compatible prompts, so you can use IPython sessions as doctest
190 code.. By default, IPython also allows you to paste existing doctests (with
190 code.. By default, IPython also allows you to paste existing doctests (with
191 leading :samp:`>>>` and :samp:`...` prompts in them
191 leading :samp:`>>>` and :samp:`...` prompts in them
192
192
193 .. _ipythonzmq:
193 .. _ipythonzmq:
194
194
195 Decoupled two-process model
195 Decoupled two-process model
196 ==============================
196 ==============================
197
197
198 IPython has abstracted and extended the notion of a traditional
198 IPython has abstracted and extended the notion of a traditional
199 *Read-Evaluate-Print Loop* (REPL) environment by decoupling the *evaluation*
199 *Read-Evaluate-Print Loop* (REPL) environment by decoupling the *evaluation*
200 into its own process. We call this process a kernel: it receives execution
200 into its own process. We call this process a kernel: it receives execution
201 instructions from clients and communicates the results back to them.
201 instructions from clients and communicates the results back to them.
202
202
203 This decoupling allows us to have several clients connected to the same
203 This decoupling allows us to have several clients connected to the same
204 kernel, and even allows clients and kernels to live on different machines.
204 kernel, and even allows clients and kernels to live on different machines.
205 With the exclusion of the traditional single process terminal-based IPython
205 With the exclusion of the traditional single process terminal-based IPython
206 (what you start if you run ``ipython`` without any subcommands), all
206 (what you start if you run ``ipython`` without any subcommands), all
207 other IPython machinery uses this two-process model. This includes ``ipython
207 other IPython machinery uses this two-process model. This includes ``ipython
208 console``, ``ipython qtconsole``, and ``ipython notebook``.
208 console``, ``ipython qtconsole``, and ``ipython notebook``.
209
209
210 As an example, this means that when you start ``ipython qtconsole``, you're
210 As an example, this means that when you start ``ipython qtconsole``, you're
211 really starting two processes, a kernel and a Qt-based client can send
211 really starting two processes, a kernel and a Qt-based client can send
212 commands to and receive results from that kernel. If there is already a kernel
212 commands to and receive results from that kernel. If there is already a kernel
213 running that you want to connect to, you can pass the ``--existing`` flag
213 running that you want to connect to, you can pass the ``--existing`` flag
214 which will skip initiating a new kernel and connect to the most recent kernel,
214 which will skip initiating a new kernel and connect to the most recent kernel,
215 instead. To connect to a specific kernel once you have several kernels
215 instead. To connect to a specific kernel once you have several kernels
216 running, use the ``%connect_info`` magic to get the unique connection file,
216 running, use the ``%connect_info`` magic to get the unique connection file,
217 which will be something like ``--existing kernel-19732.json`` but with
217 which will be something like ``--existing kernel-19732.json`` but with
218 different numbers which correspond to the Process ID of the kernel.
218 different numbers which correspond to the Process ID of the kernel.
219
219
220 You can read more about using :ref:`ipython qtconsole <qtconsole>`, and
220 You can read more about using :ref:`ipython qtconsole <qtconsole>`, and
221 :ref:`ipython notebook <htmlnotebook>`. There is also a :ref:`message spec
221 :ref:`ipython notebook <htmlnotebook>`. There is also a :ref:`message spec
222 <messaging>` which documents the protocol for communication between kernels
222 <messaging>` which documents the protocol for communication between kernels
223 and clients.
223 and clients.
224
224
225
225
226 Interactive parallel computing
226 Interactive parallel computing
227 ==============================
227 ==============================
228
228
229 Increasingly, parallel computer hardware, such as multicore CPUs, clusters and
229 Increasingly, parallel computer hardware, such as multicore CPUs, clusters and
230 supercomputers, is becoming ubiquitous. Over the last several years, we have
230 supercomputers, is becoming ubiquitous. Over the last several years, we have
231 developed an architecture within IPython that allows such hardware to be used
231 developed an architecture within IPython that allows such hardware to be used
232 quickly and easily from Python. Moreover, this architecture is designed to
232 quickly and easily from Python. Moreover, this architecture is designed to
233 support interactive and collaborative parallel computing.
233 support interactive and collaborative parallel computing.
234
234
235 The main features of this system are:
235 The main features of this system are:
236
236
237 * Quickly parallelize Python code from an interactive Python/IPython session.
237 * Quickly parallelize Python code from an interactive Python/IPython session.
238
238
239 * A flexible and dynamic process model that be deployed on anything from
239 * A flexible and dynamic process model that be deployed on anything from
240 multicore workstations to supercomputers.
240 multicore workstations to supercomputers.
241
241
242 * An architecture that supports many different styles of parallelism, from
242 * An architecture that supports many different styles of parallelism, from
243 message passing to task farming. And all of these styles can be handled
243 message passing to task farming. And all of these styles can be handled
244 interactively.
244 interactively.
245
245
246 * Both blocking and fully asynchronous interfaces.
246 * Both blocking and fully asynchronous interfaces.
247
247
248 * High level APIs that enable many things to be parallelized in a few lines
248 * High level APIs that enable many things to be parallelized in a few lines
249 of code.
249 of code.
250
250
251 * Write parallel code that will run unchanged on everything from multicore
251 * Write parallel code that will run unchanged on everything from multicore
252 workstations to supercomputers.
252 workstations to supercomputers.
253
253
254 * Full integration with Message Passing libraries (MPI).
254 * Full integration with Message Passing libraries (MPI).
255
255
256 * Capabilities based security model with full encryption of network connections.
256 * Capabilities based security model with full encryption of network connections.
257
257
258 * Share live parallel jobs with other users securely. We call this
258 * Share live parallel jobs with other users securely. We call this
259 collaborative parallel computing.
259 collaborative parallel computing.
260
260
261 * Dynamically load balanced task farming system.
261 * Dynamically load balanced task farming system.
262
262
263 * Robust error handling. Python exceptions raised in parallel execution are
263 * Robust error handling. Python exceptions raised in parallel execution are
264 gathered and presented to the top-level code.
264 gathered and presented to the top-level code.
265
265
266 For more information, see our :ref:`overview <parallel_index>` of using IPython
266 For more information, see our :ref:`overview <parallel_index>` of using IPython
267 for parallel computing.
267 for parallel computing.
268
268
269 Portability and Python requirements
269 Portability and Python requirements
270 -----------------------------------
270 -----------------------------------
271
271
272 As of the 0.11 release, IPython works with Python 2.6 and 2.7. Versions 0.9 and
272 As of the 1.0 release, IPython works with Python 2.6, 2.7, 3.2 and 3.3.
273 0.10 worked with Python 2.4 and above. IPython now also supports Python 3,
273 Version 0.12 introduced full support for Python 3. Version 0.11 worked with
274 although for now the code for this is separate, and kept up to date with the
274 Python 2.6 and 2.7 only. Versions 0.9 and 0.10 worked with Python 2.4 and
275 main IPython repository. In the future, these will converge to a single codebase
275 above (not including Python 3).
276 which can be automatically translated using 2to3.
277
276
278 IPython is known to work on the following operating systems:
277 IPython is known to work on the following operating systems:
279
278
280 * Linux
279 * Linux
281 * Most other Unix-like OSs (AIX, Solaris, BSD, etc.)
280 * Most other Unix-like OSs (AIX, Solaris, BSD, etc.)
282 * Mac OS X
281 * Mac OS X
283 * Windows (CygWin, XP, Vista, etc.)
282 * Windows (CygWin, XP, Vista, etc.)
284
283
285 See :ref:`here <install_index>` for instructions on how to install IPython.
284 See :ref:`here <install_index>` for instructions on how to install IPython.
286
285
General Comments 0
You need to be logged in to leave comments. Login now