##// END OF EJS Templates
updating docs, overview section one
Paul Ivanov -
Show More
@@ -1,271 +1,270
1 1 .. _overview:
2 2
3 3 ============
4 4 Introduction
5 5 ============
6 6
7 7 Overview
8 8 ========
9 9
10 10 One of Python's most useful features is its interactive interpreter.
11 This system allows very fast testing of ideas without the overhead of
11 It allows for very fast testing of ideas without the overhead of
12 12 creating test files as is typical in most programming languages.
13 13 However, the interpreter supplied with the standard Python distribution
14 14 is somewhat limited for extended interactive use.
15 15
16 16 The goal of IPython is to create a comprehensive environment for
17 17 interactive and exploratory computing. To support this goal, IPython
18 18 has three main components:
19 19
20 20 * An enhanced interactive Python shell.
21 21 * A decoupled two-process communication model, which allows for multiple
22 22 clients to connect to a computation kernel, most notably the web-based
23 23 :ref:`notebook <htmlnotebook>`
24 24 * An architecture for interactive parallel computing.
25 25
26 26 All of IPython is open source (released under the revised BSD license).
27 27
28 28 Enhanced interactive Python shell
29 29 =================================
30 30
31 31 IPython's interactive shell (:command:`ipython`), has the following goals,
32 32 amongst others:
33 33
34 34 1. Provide an interactive shell superior to Python's default. IPython
35 has many features for object introspection, system shell access,
36 and its own special command system for adding functionality when
37 working interactively. It tries to be a very efficient environment
38 both for Python code development and for exploration of problems
39 using Python objects (in situations like data analysis).
35 has many features for tab-completion, object introspection, system shell
36 access, command history retrieval across sessions, and its own special
37 command system for adding functionality when working interactively. It
38 tries to be a very efficient environment both for Python code development
39 and for exploration of problems using Python objects (in situations like
40 data analysis).
40 41
41 42 2. Serve as an embeddable, ready to use interpreter for your own
42 programs. IPython can be started with a single call from inside
43 another program, providing access to the current namespace. This
44 can be very useful both for debugging purposes and for situations
45 where a blend of batch-processing and interactive exploration are
46 needed. New in the 0.9 version of IPython is a reusable wxPython
47 based IPython widget.
43 programs. An interactive IPython shell can be started with a single call
44 from inside another program, providing access to the current namespace.
45 This can be very useful both for debugging purposes and for situations
46 where a blend of batch-processing and interactive exploration are needed.
48 47
49 48 3. Offer a flexible framework which can be used as the base
50 environment for other systems with Python as the underlying
51 language. Specifically scientific environments like Mathematica,
49 environment for working with other systems, with Python as the underlying
50 bridge language. Specifically scientific environments like Mathematica,
52 51 IDL and Matlab inspired its design, but similar ideas can be
53 52 useful in many fields.
54 53
55 54 4. Allow interactive testing of threaded graphical toolkits. IPython
56 has support for interactive, non-blocking control of GTK, Qt and
57 WX applications via special threading flags. The normal Python
55 has support for interactive, non-blocking control of GTK, Qt, WX, GLUT, and
56 OS X applications via special threading flags. The normal Python
58 57 shell can only do this for Tkinter applications.
59 58
60 59 Main features of the interactive shell
61 60 --------------------------------------
62 61
63 62 * Dynamic object introspection. One can access docstrings, function
64 63 definition prototypes, source code, source files and other details
65 64 of any object accessible to the interpreter with a single
66 65 keystroke (:samp:`?`, and using :samp:`??` provides additional detail).
67 66
68 67 * Searching through modules and namespaces with :samp:`*` wildcards, both
69 68 when using the :samp:`?` system and via the :samp:`%psearch` command.
70 69
71 70 * Completion in the local namespace, by typing :kbd:`TAB` at the prompt.
72 71 This works for keywords, modules, methods, variables and files in the
73 72 current directory. This is supported via the readline library, and
74 73 full access to configuring readline's behavior is provided.
75 74 Custom completers can be implemented easily for different purposes
76 75 (system commands, magic arguments etc.)
77 76
78 77 * Numbered input/output prompts with command history (persistent
79 78 across sessions and tied to each profile), full searching in this
80 79 history and caching of all input and output.
81 80
82 81 * User-extensible 'magic' commands. A set of commands prefixed with
83 82 :samp:`%` is available for controlling IPython itself and provides
84 83 directory control, namespace information and many aliases to
85 84 common system shell commands.
86 85
87 86 * Alias facility for defining your own system aliases.
88 87
89 88 * Complete system shell access. Lines starting with :samp:`!` are passed
90 89 directly to the system shell, and using :samp:`!!` or :samp:`var = !cmd`
91 90 captures shell output into python variables for further use.
92 91
93 92 * Background execution of Python commands in a separate thread.
94 93 IPython has an internal job manager called jobs, and a
95 94 convenience backgrounding magic function called :samp:`%bg`.
96 95
97 96 * The ability to expand python variables when calling the system shell. In a
98 97 shell command, any python variable prefixed with :samp:`$` is expanded. A
99 98 double :samp:`$$` allows passing a literal :samp:`$` to the shell (for access
100 99 to shell and environment variables like :envvar:`PATH`).
101 100
102 101 * Filesystem navigation, via a magic :samp:`%cd` command, along with a
103 102 persistent bookmark system (using :samp:`%bookmark`) for fast access to
104 103 frequently visited directories.
105 104
106 105 * A lightweight persistence framework via the :samp:`%store` command, which
107 106 allows you to save arbitrary Python variables. These get restored
108 107 automatically when your session restarts.
109 108
110 109 * Automatic indentation (optional) of code as you type (through the
111 110 readline library).
112 111
113 112 * Macro system for quickly re-executing multiple lines of previous
114 113 input with a single name. Macros can be stored persistently via
115 114 :samp:`%store` and edited via :samp:`%edit`.
116 115
117 116 * Session logging (you can then later use these logs as code in your
118 117 programs). Logs can optionally timestamp all input, and also store
119 118 session output (marked as comments, so the log remains valid
120 119 Python source code).
121 120
122 121 * Session restoring: logs can be replayed to restore a previous
123 122 session to the state where you left it.
124 123
125 124 * Verbose and colored exception traceback printouts. Easier to parse
126 125 visually, and in verbose mode they produce a lot of useful
127 126 debugging information (basically a terminal version of the cgitb
128 127 module).
129 128
130 129 * Auto-parentheses: callable objects can be executed without
131 130 parentheses: :samp:`sin 3` is automatically converted to :samp:`sin(3)`.
132 131
133 132 * Auto-quoting: using :samp:`,`, or :samp:`;` as the first character forces
134 133 auto-quoting of the rest of the line: :samp:`,my_function a b` becomes
135 134 automatically :samp:`my_function("a","b")`, while :samp:`;my_function a b`
136 135 becomes :samp:`my_function("a b")`.
137 136
138 137 * Extensible input syntax. You can define filters that pre-process
139 138 user input to simplify input in special situations. This allows
140 139 for example pasting multi-line code fragments which start with
141 140 :samp:`>>>` or :samp:`...` such as those from other python sessions or the
142 141 standard Python documentation.
143 142
144 143 * Flexible configuration system. It uses a configuration file which
145 144 allows permanent setting of all command-line options, module
146 145 loading, code and file execution. The system allows recursive file
147 146 inclusion, so you can have a base file with defaults and layers
148 147 which load other customizations for particular projects.
149 148
150 149 * Embeddable. You can call IPython as a python shell inside your own
151 150 python programs. This can be used both for debugging code or for
152 151 providing interactive abilities to your programs with knowledge
153 152 about the local namespaces (very useful in debugging and data
154 153 analysis situations).
155 154
156 155 * Easy debugger access. You can set IPython to call up an enhanced version of
157 156 the Python debugger (pdb) every time there is an uncaught exception. This
158 157 drops you inside the code which triggered the exception with all the data
159 158 live and it is possible to navigate the stack to rapidly isolate the source
160 159 of a bug. The :samp:`%run` magic command (with the :samp:`-d` option) can run
161 160 any script under pdb's control, automatically setting initial breakpoints for
162 161 you. This version of pdb has IPython-specific improvements, including
163 162 tab-completion and traceback coloring support. For even easier debugger
164 163 access, try :samp:`%debug` after seeing an exception. winpdb is also
165 164 supported, see ipy_winpdb extension.
166 165
167 166 * Profiler support. You can run single statements (similar to
168 167 :samp:`profile.run()`) or complete programs under the profiler's control.
169 168 While this is possible with standard cProfile or profile modules,
170 169 IPython wraps this functionality with magic commands (see :samp:`%prun`
171 170 and :samp:`%run -p`) convenient for rapid interactive work.
172 171
173 172 * Doctest support. The special :samp:`%doctest_mode` command toggles a mode
174 173 that allows you to paste existing doctests (with leading :samp:`>>>`
175 174 prompts and whitespace) and uses doctest-compatible prompts and
176 175 output, so you can use IPython sessions as doctest code.
177 176
178 177 .. _ipythonzmq:
179 178
180 179 Decoupled two-process model
181 180 ==============================
182 181
183 182 IPython has abstracted and extended the notion of a traditional
184 183 *Read-Evaluate-Print Loop* (REPL) environment by decoupling the *evaluation*
185 184 into its own process. We call this process a kernel: it receives execution
186 185 instructions from clients and communicates the results back to them.
187 186
188 187 This decoupling allows us to have several clients connected to the same
189 188 kernel, and even allows clients and kernels to live on different machines.
190 189 With the exclusion of the traditional single process terminal-based IPython
191 190 (what you start if you run ``ipython`` without any subcommands), all
192 191 other IPython machinery uses this two-process model. This includes ``ipython
193 192 console``, ``ipython qtconsole``, and ``ipython notebook``.
194 193
195 194 As an example, this means that when you start ``ipython qtconsole``, you're
196 195 really starting two processes, a kernel and a Qt-based client can send
197 196 commands to and receive results from that kernel. If there is already a kernel
198 197 running that you want to connect to, you can pass the ``--existing`` flag
199 198 which will skip initiating a new kernel and connect to the most recent kernel,
200 199 instead. To connect to a specific kernel once you have several kernels
201 200 running, use the ``%connect_info`` magic to get the unique connection file,
202 201 which will be something like ``--existing kernel-19732.json`` but with
203 202 different numbers which correspond to the Process ID of the kernel.
204 203
205 204 You can read more about using :ref:`ipython qtconsole <qtconsole>`, and
206 205 :ref:`ipython notebook <htmlnotebook>`. There is also a :ref:`message spec
207 206 <messaging>` which documents the protocol for communication between kernels
208 207 and clients.
209 208
210 209
211 210 Interactive parallel computing
212 211 ==============================
213 212
214 213 Increasingly, parallel computer hardware, such as multicore CPUs, clusters and
215 214 supercomputers, is becoming ubiquitous. Over the last several years, we have
216 215 developed an architecture within IPython that allows such hardware to be used
217 216 quickly and easily from Python. Moreover, this architecture is designed to
218 217 support interactive and collaborative parallel computing.
219 218
220 219 The main features of this system are:
221 220
222 221 * Quickly parallelize Python code from an interactive Python/IPython session.
223 222
224 223 * A flexible and dynamic process model that be deployed on anything from
225 224 multicore workstations to supercomputers.
226 225
227 226 * An architecture that supports many different styles of parallelism, from
228 227 message passing to task farming. And all of these styles can be handled
229 228 interactively.
230 229
231 230 * Both blocking and fully asynchronous interfaces.
232 231
233 232 * High level APIs that enable many things to be parallelized in a few lines
234 233 of code.
235 234
236 235 * Write parallel code that will run unchanged on everything from multicore
237 236 workstations to supercomputers.
238 237
239 238 * Full integration with Message Passing libraries (MPI).
240 239
241 240 * Capabilities based security model with full encryption of network connections.
242 241
243 242 * Share live parallel jobs with other users securely. We call this
244 243 collaborative parallel computing.
245 244
246 245 * Dynamically load balanced task farming system.
247 246
248 247 * Robust error handling. Python exceptions raised in parallel execution are
249 248 gathered and presented to the top-level code.
250 249
251 250 For more information, see our :ref:`overview <parallel_index>` of using IPython
252 251 for parallel computing.
253 252
254 253 Portability and Python requirements
255 254 -----------------------------------
256 255
257 256 As of the 0.11 release, IPython works with Python 2.6 and 2.7. Versions 0.9 and
258 257 0.10 worked with Python 2.4 and above. IPython now also supports Python 3,
259 258 although for now the code for this is separate, and kept up to date with the
260 259 main IPython repository. In the future, these will converge to a single codebase
261 260 which can be automatically translated using 2to3.
262 261
263 262 IPython is known to work on the following operating systems:
264 263
265 264 * Linux
266 265 * Most other Unix-like OSs (AIX, Solaris, BSD, etc.)
267 266 * Mac OS X
268 267 * Windows (CygWin, XP, Vista, etc.)
269 268
270 269 See :ref:`here <install_index>` for instructions on how to install IPython.
271 270
General Comments 0
You need to be logged in to leave comments. Login now