##// END OF EJS Templates
updating docs, overview section one
Paul Ivanov -
Show More
@@ -1,271 +1,270 b''
1 .. _overview:
1 .. _overview:
2
2
3 ============
3 ============
4 Introduction
4 Introduction
5 ============
5 ============
6
6
7 Overview
7 Overview
8 ========
8 ========
9
9
10 One of Python's most useful features is its interactive interpreter.
10 One of Python's most useful features is its interactive interpreter.
11 This system allows very fast testing of ideas without the overhead of
11 It allows for very fast testing of ideas without the overhead of
12 creating test files as is typical in most programming languages.
12 creating test files as is typical in most programming languages.
13 However, the interpreter supplied with the standard Python distribution
13 However, the interpreter supplied with the standard Python distribution
14 is somewhat limited for extended interactive use.
14 is somewhat limited for extended interactive use.
15
15
16 The goal of IPython is to create a comprehensive environment for
16 The goal of IPython is to create a comprehensive environment for
17 interactive and exploratory computing. To support this goal, IPython
17 interactive and exploratory computing. To support this goal, IPython
18 has three main components:
18 has three main components:
19
19
20 * An enhanced interactive Python shell.
20 * An enhanced interactive Python shell.
21 * A decoupled two-process communication model, which allows for multiple
21 * A decoupled two-process communication model, which allows for multiple
22 clients to connect to a computation kernel, most notably the web-based
22 clients to connect to a computation kernel, most notably the web-based
23 :ref:`notebook <htmlnotebook>`
23 :ref:`notebook <htmlnotebook>`
24 * An architecture for interactive parallel computing.
24 * An architecture for interactive parallel computing.
25
25
26 All of IPython is open source (released under the revised BSD license).
26 All of IPython is open source (released under the revised BSD license).
27
27
28 Enhanced interactive Python shell
28 Enhanced interactive Python shell
29 =================================
29 =================================
30
30
31 IPython's interactive shell (:command:`ipython`), has the following goals,
31 IPython's interactive shell (:command:`ipython`), has the following goals,
32 amongst others:
32 amongst others:
33
33
34 1. Provide an interactive shell superior to Python's default. IPython
34 1. Provide an interactive shell superior to Python's default. IPython
35 has many features for object introspection, system shell access,
35 has many features for tab-completion, object introspection, system shell
36 and its own special command system for adding functionality when
36 access, command history retrieval across sessions, and its own special
37 working interactively. It tries to be a very efficient environment
37 command system for adding functionality when working interactively. It
38 both for Python code development and for exploration of problems
38 tries to be a very efficient environment both for Python code development
39 using Python objects (in situations like data analysis).
39 and for exploration of problems using Python objects (in situations like
40 data analysis).
40
41
41 2. Serve as an embeddable, ready to use interpreter for your own
42 2. Serve as an embeddable, ready to use interpreter for your own
42 programs. IPython can be started with a single call from inside
43 programs. An interactive IPython shell can be started with a single call
43 another program, providing access to the current namespace. This
44 from inside another program, providing access to the current namespace.
44 can be very useful both for debugging purposes and for situations
45 This can be very useful both for debugging purposes and for situations
45 where a blend of batch-processing and interactive exploration are
46 where a blend of batch-processing and interactive exploration are needed.
46 needed. New in the 0.9 version of IPython is a reusable wxPython
47 based IPython widget.
48
47
49 3. Offer a flexible framework which can be used as the base
48 3. Offer a flexible framework which can be used as the base
50 environment for other systems with Python as the underlying
49 environment for working with other systems, with Python as the underlying
51 language. Specifically scientific environments like Mathematica,
50 bridge language. Specifically scientific environments like Mathematica,
52 IDL and Matlab inspired its design, but similar ideas can be
51 IDL and Matlab inspired its design, but similar ideas can be
53 useful in many fields.
52 useful in many fields.
54
53
55 4. Allow interactive testing of threaded graphical toolkits. IPython
54 4. Allow interactive testing of threaded graphical toolkits. IPython
56 has support for interactive, non-blocking control of GTK, Qt and
55 has support for interactive, non-blocking control of GTK, Qt, WX, GLUT, and
57 WX applications via special threading flags. The normal Python
56 OS X applications via special threading flags. The normal Python
58 shell can only do this for Tkinter applications.
57 shell can only do this for Tkinter applications.
59
58
60 Main features of the interactive shell
59 Main features of the interactive shell
61 --------------------------------------
60 --------------------------------------
62
61
63 * Dynamic object introspection. One can access docstrings, function
62 * Dynamic object introspection. One can access docstrings, function
64 definition prototypes, source code, source files and other details
63 definition prototypes, source code, source files and other details
65 of any object accessible to the interpreter with a single
64 of any object accessible to the interpreter with a single
66 keystroke (:samp:`?`, and using :samp:`??` provides additional detail).
65 keystroke (:samp:`?`, and using :samp:`??` provides additional detail).
67
66
68 * Searching through modules and namespaces with :samp:`*` wildcards, both
67 * Searching through modules and namespaces with :samp:`*` wildcards, both
69 when using the :samp:`?` system and via the :samp:`%psearch` command.
68 when using the :samp:`?` system and via the :samp:`%psearch` command.
70
69
71 * Completion in the local namespace, by typing :kbd:`TAB` at the prompt.
70 * Completion in the local namespace, by typing :kbd:`TAB` at the prompt.
72 This works for keywords, modules, methods, variables and files in the
71 This works for keywords, modules, methods, variables and files in the
73 current directory. This is supported via the readline library, and
72 current directory. This is supported via the readline library, and
74 full access to configuring readline's behavior is provided.
73 full access to configuring readline's behavior is provided.
75 Custom completers can be implemented easily for different purposes
74 Custom completers can be implemented easily for different purposes
76 (system commands, magic arguments etc.)
75 (system commands, magic arguments etc.)
77
76
78 * Numbered input/output prompts with command history (persistent
77 * Numbered input/output prompts with command history (persistent
79 across sessions and tied to each profile), full searching in this
78 across sessions and tied to each profile), full searching in this
80 history and caching of all input and output.
79 history and caching of all input and output.
81
80
82 * User-extensible 'magic' commands. A set of commands prefixed with
81 * User-extensible 'magic' commands. A set of commands prefixed with
83 :samp:`%` is available for controlling IPython itself and provides
82 :samp:`%` is available for controlling IPython itself and provides
84 directory control, namespace information and many aliases to
83 directory control, namespace information and many aliases to
85 common system shell commands.
84 common system shell commands.
86
85
87 * Alias facility for defining your own system aliases.
86 * Alias facility for defining your own system aliases.
88
87
89 * Complete system shell access. Lines starting with :samp:`!` are passed
88 * Complete system shell access. Lines starting with :samp:`!` are passed
90 directly to the system shell, and using :samp:`!!` or :samp:`var = !cmd`
89 directly to the system shell, and using :samp:`!!` or :samp:`var = !cmd`
91 captures shell output into python variables for further use.
90 captures shell output into python variables for further use.
92
91
93 * Background execution of Python commands in a separate thread.
92 * Background execution of Python commands in a separate thread.
94 IPython has an internal job manager called jobs, and a
93 IPython has an internal job manager called jobs, and a
95 convenience backgrounding magic function called :samp:`%bg`.
94 convenience backgrounding magic function called :samp:`%bg`.
96
95
97 * The ability to expand python variables when calling the system shell. In a
96 * The ability to expand python variables when calling the system shell. In a
98 shell command, any python variable prefixed with :samp:`$` is expanded. A
97 shell command, any python variable prefixed with :samp:`$` is expanded. A
99 double :samp:`$$` allows passing a literal :samp:`$` to the shell (for access
98 double :samp:`$$` allows passing a literal :samp:`$` to the shell (for access
100 to shell and environment variables like :envvar:`PATH`).
99 to shell and environment variables like :envvar:`PATH`).
101
100
102 * Filesystem navigation, via a magic :samp:`%cd` command, along with a
101 * Filesystem navigation, via a magic :samp:`%cd` command, along with a
103 persistent bookmark system (using :samp:`%bookmark`) for fast access to
102 persistent bookmark system (using :samp:`%bookmark`) for fast access to
104 frequently visited directories.
103 frequently visited directories.
105
104
106 * A lightweight persistence framework via the :samp:`%store` command, which
105 * A lightweight persistence framework via the :samp:`%store` command, which
107 allows you to save arbitrary Python variables. These get restored
106 allows you to save arbitrary Python variables. These get restored
108 automatically when your session restarts.
107 automatically when your session restarts.
109
108
110 * Automatic indentation (optional) of code as you type (through the
109 * Automatic indentation (optional) of code as you type (through the
111 readline library).
110 readline library).
112
111
113 * Macro system for quickly re-executing multiple lines of previous
112 * Macro system for quickly re-executing multiple lines of previous
114 input with a single name. Macros can be stored persistently via
113 input with a single name. Macros can be stored persistently via
115 :samp:`%store` and edited via :samp:`%edit`.
114 :samp:`%store` and edited via :samp:`%edit`.
116
115
117 * Session logging (you can then later use these logs as code in your
116 * Session logging (you can then later use these logs as code in your
118 programs). Logs can optionally timestamp all input, and also store
117 programs). Logs can optionally timestamp all input, and also store
119 session output (marked as comments, so the log remains valid
118 session output (marked as comments, so the log remains valid
120 Python source code).
119 Python source code).
121
120
122 * Session restoring: logs can be replayed to restore a previous
121 * Session restoring: logs can be replayed to restore a previous
123 session to the state where you left it.
122 session to the state where you left it.
124
123
125 * Verbose and colored exception traceback printouts. Easier to parse
124 * Verbose and colored exception traceback printouts. Easier to parse
126 visually, and in verbose mode they produce a lot of useful
125 visually, and in verbose mode they produce a lot of useful
127 debugging information (basically a terminal version of the cgitb
126 debugging information (basically a terminal version of the cgitb
128 module).
127 module).
129
128
130 * Auto-parentheses: callable objects can be executed without
129 * Auto-parentheses: callable objects can be executed without
131 parentheses: :samp:`sin 3` is automatically converted to :samp:`sin(3)`.
130 parentheses: :samp:`sin 3` is automatically converted to :samp:`sin(3)`.
132
131
133 * Auto-quoting: using :samp:`,`, or :samp:`;` as the first character forces
132 * Auto-quoting: using :samp:`,`, or :samp:`;` as the first character forces
134 auto-quoting of the rest of the line: :samp:`,my_function a b` becomes
133 auto-quoting of the rest of the line: :samp:`,my_function a b` becomes
135 automatically :samp:`my_function("a","b")`, while :samp:`;my_function a b`
134 automatically :samp:`my_function("a","b")`, while :samp:`;my_function a b`
136 becomes :samp:`my_function("a b")`.
135 becomes :samp:`my_function("a b")`.
137
136
138 * Extensible input syntax. You can define filters that pre-process
137 * Extensible input syntax. You can define filters that pre-process
139 user input to simplify input in special situations. This allows
138 user input to simplify input in special situations. This allows
140 for example pasting multi-line code fragments which start with
139 for example pasting multi-line code fragments which start with
141 :samp:`>>>` or :samp:`...` such as those from other python sessions or the
140 :samp:`>>>` or :samp:`...` such as those from other python sessions or the
142 standard Python documentation.
141 standard Python documentation.
143
142
144 * Flexible configuration system. It uses a configuration file which
143 * Flexible configuration system. It uses a configuration file which
145 allows permanent setting of all command-line options, module
144 allows permanent setting of all command-line options, module
146 loading, code and file execution. The system allows recursive file
145 loading, code and file execution. The system allows recursive file
147 inclusion, so you can have a base file with defaults and layers
146 inclusion, so you can have a base file with defaults and layers
148 which load other customizations for particular projects.
147 which load other customizations for particular projects.
149
148
150 * Embeddable. You can call IPython as a python shell inside your own
149 * Embeddable. You can call IPython as a python shell inside your own
151 python programs. This can be used both for debugging code or for
150 python programs. This can be used both for debugging code or for
152 providing interactive abilities to your programs with knowledge
151 providing interactive abilities to your programs with knowledge
153 about the local namespaces (very useful in debugging and data
152 about the local namespaces (very useful in debugging and data
154 analysis situations).
153 analysis situations).
155
154
156 * Easy debugger access. You can set IPython to call up an enhanced version of
155 * Easy debugger access. You can set IPython to call up an enhanced version of
157 the Python debugger (pdb) every time there is an uncaught exception. This
156 the Python debugger (pdb) every time there is an uncaught exception. This
158 drops you inside the code which triggered the exception with all the data
157 drops you inside the code which triggered the exception with all the data
159 live and it is possible to navigate the stack to rapidly isolate the source
158 live and it is possible to navigate the stack to rapidly isolate the source
160 of a bug. The :samp:`%run` magic command (with the :samp:`-d` option) can run
159 of a bug. The :samp:`%run` magic command (with the :samp:`-d` option) can run
161 any script under pdb's control, automatically setting initial breakpoints for
160 any script under pdb's control, automatically setting initial breakpoints for
162 you. This version of pdb has IPython-specific improvements, including
161 you. This version of pdb has IPython-specific improvements, including
163 tab-completion and traceback coloring support. For even easier debugger
162 tab-completion and traceback coloring support. For even easier debugger
164 access, try :samp:`%debug` after seeing an exception. winpdb is also
163 access, try :samp:`%debug` after seeing an exception. winpdb is also
165 supported, see ipy_winpdb extension.
164 supported, see ipy_winpdb extension.
166
165
167 * Profiler support. You can run single statements (similar to
166 * Profiler support. You can run single statements (similar to
168 :samp:`profile.run()`) or complete programs under the profiler's control.
167 :samp:`profile.run()`) or complete programs under the profiler's control.
169 While this is possible with standard cProfile or profile modules,
168 While this is possible with standard cProfile or profile modules,
170 IPython wraps this functionality with magic commands (see :samp:`%prun`
169 IPython wraps this functionality with magic commands (see :samp:`%prun`
171 and :samp:`%run -p`) convenient for rapid interactive work.
170 and :samp:`%run -p`) convenient for rapid interactive work.
172
171
173 * Doctest support. The special :samp:`%doctest_mode` command toggles a mode
172 * Doctest support. The special :samp:`%doctest_mode` command toggles a mode
174 that allows you to paste existing doctests (with leading :samp:`>>>`
173 that allows you to paste existing doctests (with leading :samp:`>>>`
175 prompts and whitespace) and uses doctest-compatible prompts and
174 prompts and whitespace) and uses doctest-compatible prompts and
176 output, so you can use IPython sessions as doctest code.
175 output, so you can use IPython sessions as doctest code.
177
176
178 .. _ipythonzmq:
177 .. _ipythonzmq:
179
178
180 Decoupled two-process model
179 Decoupled two-process model
181 ==============================
180 ==============================
182
181
183 IPython has abstracted and extended the notion of a traditional
182 IPython has abstracted and extended the notion of a traditional
184 *Read-Evaluate-Print Loop* (REPL) environment by decoupling the *evaluation*
183 *Read-Evaluate-Print Loop* (REPL) environment by decoupling the *evaluation*
185 into its own process. We call this process a kernel: it receives execution
184 into its own process. We call this process a kernel: it receives execution
186 instructions from clients and communicates the results back to them.
185 instructions from clients and communicates the results back to them.
187
186
188 This decoupling allows us to have several clients connected to the same
187 This decoupling allows us to have several clients connected to the same
189 kernel, and even allows clients and kernels to live on different machines.
188 kernel, and even allows clients and kernels to live on different machines.
190 With the exclusion of the traditional single process terminal-based IPython
189 With the exclusion of the traditional single process terminal-based IPython
191 (what you start if you run ``ipython`` without any subcommands), all
190 (what you start if you run ``ipython`` without any subcommands), all
192 other IPython machinery uses this two-process model. This includes ``ipython
191 other IPython machinery uses this two-process model. This includes ``ipython
193 console``, ``ipython qtconsole``, and ``ipython notebook``.
192 console``, ``ipython qtconsole``, and ``ipython notebook``.
194
193
195 As an example, this means that when you start ``ipython qtconsole``, you're
194 As an example, this means that when you start ``ipython qtconsole``, you're
196 really starting two processes, a kernel and a Qt-based client can send
195 really starting two processes, a kernel and a Qt-based client can send
197 commands to and receive results from that kernel. If there is already a kernel
196 commands to and receive results from that kernel. If there is already a kernel
198 running that you want to connect to, you can pass the ``--existing`` flag
197 running that you want to connect to, you can pass the ``--existing`` flag
199 which will skip initiating a new kernel and connect to the most recent kernel,
198 which will skip initiating a new kernel and connect to the most recent kernel,
200 instead. To connect to a specific kernel once you have several kernels
199 instead. To connect to a specific kernel once you have several kernels
201 running, use the ``%connect_info`` magic to get the unique connection file,
200 running, use the ``%connect_info`` magic to get the unique connection file,
202 which will be something like ``--existing kernel-19732.json`` but with
201 which will be something like ``--existing kernel-19732.json`` but with
203 different numbers which correspond to the Process ID of the kernel.
202 different numbers which correspond to the Process ID of the kernel.
204
203
205 You can read more about using :ref:`ipython qtconsole <qtconsole>`, and
204 You can read more about using :ref:`ipython qtconsole <qtconsole>`, and
206 :ref:`ipython notebook <htmlnotebook>`. There is also a :ref:`message spec
205 :ref:`ipython notebook <htmlnotebook>`. There is also a :ref:`message spec
207 <messaging>` which documents the protocol for communication between kernels
206 <messaging>` which documents the protocol for communication between kernels
208 and clients.
207 and clients.
209
208
210
209
211 Interactive parallel computing
210 Interactive parallel computing
212 ==============================
211 ==============================
213
212
214 Increasingly, parallel computer hardware, such as multicore CPUs, clusters and
213 Increasingly, parallel computer hardware, such as multicore CPUs, clusters and
215 supercomputers, is becoming ubiquitous. Over the last several years, we have
214 supercomputers, is becoming ubiquitous. Over the last several years, we have
216 developed an architecture within IPython that allows such hardware to be used
215 developed an architecture within IPython that allows such hardware to be used
217 quickly and easily from Python. Moreover, this architecture is designed to
216 quickly and easily from Python. Moreover, this architecture is designed to
218 support interactive and collaborative parallel computing.
217 support interactive and collaborative parallel computing.
219
218
220 The main features of this system are:
219 The main features of this system are:
221
220
222 * Quickly parallelize Python code from an interactive Python/IPython session.
221 * Quickly parallelize Python code from an interactive Python/IPython session.
223
222
224 * A flexible and dynamic process model that be deployed on anything from
223 * A flexible and dynamic process model that be deployed on anything from
225 multicore workstations to supercomputers.
224 multicore workstations to supercomputers.
226
225
227 * An architecture that supports many different styles of parallelism, from
226 * An architecture that supports many different styles of parallelism, from
228 message passing to task farming. And all of these styles can be handled
227 message passing to task farming. And all of these styles can be handled
229 interactively.
228 interactively.
230
229
231 * Both blocking and fully asynchronous interfaces.
230 * Both blocking and fully asynchronous interfaces.
232
231
233 * High level APIs that enable many things to be parallelized in a few lines
232 * High level APIs that enable many things to be parallelized in a few lines
234 of code.
233 of code.
235
234
236 * Write parallel code that will run unchanged on everything from multicore
235 * Write parallel code that will run unchanged on everything from multicore
237 workstations to supercomputers.
236 workstations to supercomputers.
238
237
239 * Full integration with Message Passing libraries (MPI).
238 * Full integration with Message Passing libraries (MPI).
240
239
241 * Capabilities based security model with full encryption of network connections.
240 * Capabilities based security model with full encryption of network connections.
242
241
243 * Share live parallel jobs with other users securely. We call this
242 * Share live parallel jobs with other users securely. We call this
244 collaborative parallel computing.
243 collaborative parallel computing.
245
244
246 * Dynamically load balanced task farming system.
245 * Dynamically load balanced task farming system.
247
246
248 * Robust error handling. Python exceptions raised in parallel execution are
247 * Robust error handling. Python exceptions raised in parallel execution are
249 gathered and presented to the top-level code.
248 gathered and presented to the top-level code.
250
249
251 For more information, see our :ref:`overview <parallel_index>` of using IPython
250 For more information, see our :ref:`overview <parallel_index>` of using IPython
252 for parallel computing.
251 for parallel computing.
253
252
254 Portability and Python requirements
253 Portability and Python requirements
255 -----------------------------------
254 -----------------------------------
256
255
257 As of the 0.11 release, IPython works with Python 2.6 and 2.7. Versions 0.9 and
256 As of the 0.11 release, IPython works with Python 2.6 and 2.7. Versions 0.9 and
258 0.10 worked with Python 2.4 and above. IPython now also supports Python 3,
257 0.10 worked with Python 2.4 and above. IPython now also supports Python 3,
259 although for now the code for this is separate, and kept up to date with the
258 although for now the code for this is separate, and kept up to date with the
260 main IPython repository. In the future, these will converge to a single codebase
259 main IPython repository. In the future, these will converge to a single codebase
261 which can be automatically translated using 2to3.
260 which can be automatically translated using 2to3.
262
261
263 IPython is known to work on the following operating systems:
262 IPython is known to work on the following operating systems:
264
263
265 * Linux
264 * Linux
266 * Most other Unix-like OSs (AIX, Solaris, BSD, etc.)
265 * Most other Unix-like OSs (AIX, Solaris, BSD, etc.)
267 * Mac OS X
266 * Mac OS X
268 * Windows (CygWin, XP, Vista, etc.)
267 * Windows (CygWin, XP, Vista, etc.)
269
268
270 See :ref:`here <install_index>` for instructions on how to install IPython.
269 See :ref:`here <install_index>` for instructions on how to install IPython.
271
270
General Comments 0
You need to be logged in to leave comments. Login now