##// END OF EJS Templates
First working version of cell magics in inputsplitter in line mode....
Fernando Perez -
Show More
@@ -1,808 +1,910 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 from __future__ import print_function
63 from __future__ import print_function
64
64
65 #-----------------------------------------------------------------------------
65 #-----------------------------------------------------------------------------
66 # Imports
66 # Imports
67 #-----------------------------------------------------------------------------
67 #-----------------------------------------------------------------------------
68 # stdlib
68 # stdlib
69 import ast
69 import ast
70 import codeop
70 import codeop
71 import re
71 import re
72 import sys
72 import sys
73 import tokenize
73 import tokenize
74 from StringIO import StringIO
74 from StringIO import StringIO
75
75
76 # IPython modules
76 # IPython modules
77 from IPython.core.splitinput import split_user_input, LineInfo
77 from IPython.core.splitinput import split_user_input, LineInfo
78 from IPython.utils.py3compat import cast_unicode
78 from IPython.utils.py3compat import cast_unicode
79
79
80 #-----------------------------------------------------------------------------
80 #-----------------------------------------------------------------------------
81 # Globals
81 # Globals
82 #-----------------------------------------------------------------------------
82 #-----------------------------------------------------------------------------
83
83
84 # The escape sequences that define the syntax transformations IPython will
84 # The escape sequences that define the syntax transformations IPython will
85 # apply to user input. These can NOT be just changed here: many regular
85 # apply to user input. These can NOT be just changed here: many regular
86 # expressions and other parts of the code may use their hardcoded values, and
86 # expressions and other parts of the code may use their hardcoded values, and
87 # for all intents and purposes they constitute the 'IPython syntax', so they
87 # for all intents and purposes they constitute the 'IPython syntax', so they
88 # should be considered fixed.
88 # should be considered fixed.
89
89
90 ESC_SHELL = '!' # Send line to underlying system shell
90 ESC_SHELL = '!' # Send line to underlying system shell
91 ESC_SH_CAP = '!!' # Send line to system shell and capture output
91 ESC_SH_CAP = '!!' # Send line to system shell and capture output
92 ESC_HELP = '?' # Find information about object
92 ESC_HELP = '?' # Find information about object
93 ESC_HELP2 = '??' # Find extra-detailed information about object
93 ESC_HELP2 = '??' # Find extra-detailed information about object
94 ESC_MAGIC = '%' # Call magic function
94 ESC_MAGIC = '%' # Call magic function
95 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
95 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
96 ESC_QUOTE2 = ';' # Quote all args as a single string, call
96 ESC_QUOTE2 = ';' # Quote all args as a single string, call
97 ESC_PAREN = '/' # Call first argument with rest of line as arguments
97 ESC_PAREN = '/' # Call first argument with rest of line as arguments
98
98
99 #-----------------------------------------------------------------------------
99 #-----------------------------------------------------------------------------
100 # Utilities
100 # Utilities
101 #-----------------------------------------------------------------------------
101 #-----------------------------------------------------------------------------
102
102
103 # FIXME: These are general-purpose utilities that later can be moved to the
103 # FIXME: These are general-purpose utilities that later can be moved to the
104 # general ward. Kept here for now because we're being very strict about test
104 # general ward. Kept here for now because we're being very strict about test
105 # coverage with this code, and this lets us ensure that we keep 100% coverage
105 # coverage with this code, and this lets us ensure that we keep 100% coverage
106 # while developing.
106 # while developing.
107
107
108 # compiled regexps for autoindent management
108 # compiled regexps for autoindent management
109 dedent_re = re.compile('|'.join([
109 dedent_re = re.compile('|'.join([
110 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
110 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
111 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
111 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
112 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
112 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
113 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
113 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
114 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
114 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
115 ]))
115 ]))
116 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
116 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
117
117
118 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
118 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
119 # before pure comments
119 # before pure comments
120 comment_line_re = re.compile('^\s*\#')
120 comment_line_re = re.compile('^\s*\#')
121
121
122
122
123 def num_ini_spaces(s):
123 def num_ini_spaces(s):
124 """Return the number of initial spaces in a string.
124 """Return the number of initial spaces in a string.
125
125
126 Note that tabs are counted as a single space. For now, we do *not* support
126 Note that tabs are counted as a single space. For now, we do *not* support
127 mixing of tabs and spaces in the user's input.
127 mixing of tabs and spaces in the user's input.
128
128
129 Parameters
129 Parameters
130 ----------
130 ----------
131 s : string
131 s : string
132
132
133 Returns
133 Returns
134 -------
134 -------
135 n : int
135 n : int
136 """
136 """
137
137
138 ini_spaces = ini_spaces_re.match(s)
138 ini_spaces = ini_spaces_re.match(s)
139 if ini_spaces:
139 if ini_spaces:
140 return ini_spaces.end()
140 return ini_spaces.end()
141 else:
141 else:
142 return 0
142 return 0
143
143
144
144
145 def last_blank(src):
146 """Determine if the input source ends in a blank.
147
148 A blank is either a newline or a line consisting of whitespace.
149
150 Parameters
151 ----------
152 src : string
153 A single or multiline string.
154 """
155 if not src: return False
156 ll = src.splitlines()[-1]
157 return (ll == '') or ll.isspace()
158
159
145 def remove_comments(src):
160 def remove_comments(src):
146 """Remove all comments from input source.
161 """Remove all comments from input source.
147
162
148 Note: comments are NOT recognized inside of strings!
163 Note: comments are NOT recognized inside of strings!
149
164
150 Parameters
165 Parameters
151 ----------
166 ----------
152 src : string
167 src : string
153 A single or multiline input string.
168 A single or multiline input string.
154
169
155 Returns
170 Returns
156 -------
171 -------
157 String with all Python comments removed.
172 String with all Python comments removed.
158 """
173 """
159
174
160 return re.sub('#.*', '', src)
175 return re.sub('#.*', '', src)
161
176
162 def has_comment(src):
177 def has_comment(src):
163 """Indicate whether an input line has (i.e. ends in, or is) a comment.
178 """Indicate whether an input line has (i.e. ends in, or is) a comment.
164
179
165 This uses tokenize, so it can distinguish comments from # inside strings.
180 This uses tokenize, so it can distinguish comments from # inside strings.
166
181
167 Parameters
182 Parameters
168 ----------
183 ----------
169 src : string
184 src : string
170 A single line input string.
185 A single line input string.
171
186
172 Returns
187 Returns
173 -------
188 -------
174 Boolean: True if source has a comment.
189 Boolean: True if source has a comment.
175 """
190 """
176 readline = StringIO(src).readline
191 readline = StringIO(src).readline
177 toktypes = set()
192 toktypes = set()
178 try:
193 try:
179 for t in tokenize.generate_tokens(readline):
194 for t in tokenize.generate_tokens(readline):
180 toktypes.add(t[0])
195 toktypes.add(t[0])
181 except tokenize.TokenError:
196 except tokenize.TokenError:
182 pass
197 pass
183 return(tokenize.COMMENT in toktypes)
198 return(tokenize.COMMENT in toktypes)
184
199
185
200
186 def get_input_encoding():
201 def get_input_encoding():
187 """Return the default standard input encoding.
202 """Return the default standard input encoding.
188
203
189 If sys.stdin has no encoding, 'ascii' is returned."""
204 If sys.stdin has no encoding, 'ascii' is returned."""
190 # There are strange environments for which sys.stdin.encoding is None. We
205 # There are strange environments for which sys.stdin.encoding is None. We
191 # ensure that a valid encoding is returned.
206 # ensure that a valid encoding is returned.
192 encoding = getattr(sys.stdin, 'encoding', None)
207 encoding = getattr(sys.stdin, 'encoding', None)
193 if encoding is None:
208 if encoding is None:
194 encoding = 'ascii'
209 encoding = 'ascii'
195 return encoding
210 return encoding
196
211
197 #-----------------------------------------------------------------------------
212 #-----------------------------------------------------------------------------
198 # Classes and functions for normal Python syntax handling
213 # Classes and functions for normal Python syntax handling
199 #-----------------------------------------------------------------------------
214 #-----------------------------------------------------------------------------
200
215
201 class InputSplitter(object):
216 class InputSplitter(object):
202 """An object that can accumulate lines of Python source before execution.
217 """An object that can accumulate lines of Python source before execution.
203
218
204 This object is designed to be fed python source line-by-line, using
219 This object is designed to be fed python source line-by-line, using
205 :meth:`push`. It will return on each push whether the currently pushed
220 :meth:`push`. It will return on each push whether the currently pushed
206 code could be executed already. In addition, it provides a method called
221 code could be executed already. In addition, it provides a method called
207 :meth:`push_accepts_more` that can be used to query whether more input
222 :meth:`push_accepts_more` that can be used to query whether more input
208 can be pushed into a single interactive block.
223 can be pushed into a single interactive block.
209
224
210 This is a simple example of how an interactive terminal-based client can use
225 This is a simple example of how an interactive terminal-based client can use
211 this tool::
226 this tool::
212
227
213 isp = InputSplitter()
228 isp = InputSplitter()
214 while isp.push_accepts_more():
229 while isp.push_accepts_more():
215 indent = ' '*isp.indent_spaces
230 indent = ' '*isp.indent_spaces
216 prompt = '>>> ' + indent
231 prompt = '>>> ' + indent
217 line = indent + raw_input(prompt)
232 line = indent + raw_input(prompt)
218 isp.push(line)
233 isp.push(line)
219 print 'Input source was:\n', isp.source_reset(),
234 print 'Input source was:\n', isp.source_reset(),
220 """
235 """
221 # Number of spaces of indentation computed from input that has been pushed
236 # Number of spaces of indentation computed from input that has been pushed
222 # so far. This is the attributes callers should query to get the current
237 # so far. This is the attributes callers should query to get the current
223 # indentation level, in order to provide auto-indent facilities.
238 # indentation level, in order to provide auto-indent facilities.
224 indent_spaces = 0
239 indent_spaces = 0
225 # String, indicating the default input encoding. It is computed by default
240 # String, indicating the default input encoding. It is computed by default
226 # at initialization time via get_input_encoding(), but it can be reset by a
241 # at initialization time via get_input_encoding(), but it can be reset by a
227 # client with specific knowledge of the encoding.
242 # client with specific knowledge of the encoding.
228 encoding = ''
243 encoding = ''
229 # String where the current full source input is stored, properly encoded.
244 # String where the current full source input is stored, properly encoded.
230 # Reading this attribute is the normal way of querying the currently pushed
245 # Reading this attribute is the normal way of querying the currently pushed
231 # source code, that has been properly encoded.
246 # source code, that has been properly encoded.
232 source = ''
247 source = ''
233 # Code object corresponding to the current source. It is automatically
248 # Code object corresponding to the current source. It is automatically
234 # synced to the source, so it can be queried at any time to obtain the code
249 # synced to the source, so it can be queried at any time to obtain the code
235 # object; it will be None if the source doesn't compile to valid Python.
250 # object; it will be None if the source doesn't compile to valid Python.
236 code = None
251 code = None
237 # Input mode
252 # Input mode
238 input_mode = 'line'
253 input_mode = 'line'
239
254
240 # Private attributes
255 # Private attributes
241
256
242 # List with lines of input accumulated so far
257 # List with lines of input accumulated so far
243 _buffer = None
258 _buffer = None
244 # Command compiler
259 # Command compiler
245 _compile = None
260 _compile = None
246 # Mark when input has changed indentation all the way back to flush-left
261 # Mark when input has changed indentation all the way back to flush-left
247 _full_dedent = False
262 _full_dedent = False
248 # Boolean indicating whether the current block is complete
263 # Boolean indicating whether the current block is complete
249 _is_complete = None
264 _is_complete = None
250
265
251 def __init__(self, input_mode=None):
266 def __init__(self, input_mode=None):
252 """Create a new InputSplitter instance.
267 """Create a new InputSplitter instance.
253
268
254 Parameters
269 Parameters
255 ----------
270 ----------
256 input_mode : str
271 input_mode : str
257
272
258 One of ['line', 'cell']; default is 'line'.
273 One of ['line', 'cell']; default is 'line'.
259
274
260 The input_mode parameter controls how new inputs are used when fed via
275 The input_mode parameter controls how new inputs are used when fed via
261 the :meth:`push` method:
276 the :meth:`push` method:
262
277
263 - 'line': meant for line-oriented clients, inputs are appended one at a
278 - 'line': meant for line-oriented clients, inputs are appended one at a
264 time to the internal buffer and the whole buffer is compiled.
279 time to the internal buffer and the whole buffer is compiled.
265
280
266 - 'cell': meant for clients that can edit multi-line 'cells' of text at
281 - 'cell': meant for clients that can edit multi-line 'cells' of text at
267 a time. A cell can contain one or more blocks that can be compile in
282 a time. A cell can contain one or more blocks that can be compile in
268 'single' mode by Python. In this mode, each new input new input
283 'single' mode by Python. In this mode, each new input new input
269 completely replaces all prior inputs. Cell mode is thus equivalent
284 completely replaces all prior inputs. Cell mode is thus equivalent
270 to prepending a full reset() to every push() call.
285 to prepending a full reset() to every push() call.
271 """
286 """
272 self._buffer = []
287 self._buffer = []
273 self._compile = codeop.CommandCompiler()
288 self._compile = codeop.CommandCompiler()
274 self.encoding = get_input_encoding()
289 self.encoding = get_input_encoding()
275 self.input_mode = InputSplitter.input_mode if input_mode is None \
290 self.input_mode = InputSplitter.input_mode if input_mode is None \
276 else input_mode
291 else input_mode
277
292
278 def reset(self):
293 def reset(self):
279 """Reset the input buffer and associated state."""
294 """Reset the input buffer and associated state."""
280 self.indent_spaces = 0
295 self.indent_spaces = 0
281 self._buffer[:] = []
296 self._buffer[:] = []
282 self.source = ''
297 self.source = ''
283 self.code = None
298 self.code = None
284 self._is_complete = False
299 self._is_complete = False
285 self._full_dedent = False
300 self._full_dedent = False
286
301
287 def source_reset(self):
302 def source_reset(self):
288 """Return the input source and perform a full reset.
303 """Return the input source and perform a full reset.
289 """
304 """
290 out = self.source
305 out = self.source
291 self.reset()
306 self.reset()
292 return out
307 return out
293
308
294 def push(self, lines):
309 def push(self, lines):
295 """Push one or more lines of input.
310 """Push one or more lines of input.
296
311
297 This stores the given lines and returns a status code indicating
312 This stores the given lines and returns a status code indicating
298 whether the code forms a complete Python block or not.
313 whether the code forms a complete Python block or not.
299
314
300 Any exceptions generated in compilation are swallowed, but if an
315 Any exceptions generated in compilation are swallowed, but if an
301 exception was produced, the method returns True.
316 exception was produced, the method returns True.
302
317
303 Parameters
318 Parameters
304 ----------
319 ----------
305 lines : string
320 lines : string
306 One or more lines of Python input.
321 One or more lines of Python input.
307
322
308 Returns
323 Returns
309 -------
324 -------
310 is_complete : boolean
325 is_complete : boolean
311 True if the current input source (the result of the current input
326 True if the current input source (the result of the current input
312 plus prior inputs) forms a complete Python execution block. Note that
327 plus prior inputs) forms a complete Python execution block. Note that
313 this value is also stored as a private attribute (_is_complete), so it
328 this value is also stored as a private attribute (_is_complete), so it
314 can be queried at any time.
329 can be queried at any time.
315 """
330 """
316 if self.input_mode == 'cell':
331 if self.input_mode == 'cell':
317 self.reset()
332 self.reset()
318
333
319 self._store(lines)
334 self._store(lines)
320 source = self.source
335 source = self.source
321
336
322 # Before calling _compile(), reset the code object to None so that if an
337 # Before calling _compile(), reset the code object to None so that if an
323 # exception is raised in compilation, we don't mislead by having
338 # exception is raised in compilation, we don't mislead by having
324 # inconsistent code/source attributes.
339 # inconsistent code/source attributes.
325 self.code, self._is_complete = None, None
340 self.code, self._is_complete = None, None
326
341
327 # Honor termination lines properly
342 # Honor termination lines properly
328 if source.rstrip().endswith('\\'):
343 if source.rstrip().endswith('\\'):
329 return False
344 return False
330
345
331 self._update_indent(lines)
346 self._update_indent(lines)
332 try:
347 try:
333 self.code = self._compile(source, symbol="exec")
348 self.code = self._compile(source, symbol="exec")
334 # Invalid syntax can produce any of a number of different errors from
349 # Invalid syntax can produce any of a number of different errors from
335 # inside the compiler, so we have to catch them all. Syntax errors
350 # inside the compiler, so we have to catch them all. Syntax errors
336 # immediately produce a 'ready' block, so the invalid Python can be
351 # immediately produce a 'ready' block, so the invalid Python can be
337 # sent to the kernel for evaluation with possible ipython
352 # sent to the kernel for evaluation with possible ipython
338 # special-syntax conversion.
353 # special-syntax conversion.
339 except (SyntaxError, OverflowError, ValueError, TypeError,
354 except (SyntaxError, OverflowError, ValueError, TypeError,
340 MemoryError):
355 MemoryError):
341 self._is_complete = True
356 self._is_complete = True
342 else:
357 else:
343 # Compilation didn't produce any exceptions (though it may not have
358 # Compilation didn't produce any exceptions (though it may not have
344 # given a complete code object)
359 # given a complete code object)
345 self._is_complete = self.code is not None
360 self._is_complete = self.code is not None
346
361
347 return self._is_complete
362 return self._is_complete
348
363
349 def push_accepts_more(self):
364 def push_accepts_more(self):
350 """Return whether a block of interactive input can accept more input.
365 """Return whether a block of interactive input can accept more input.
351
366
352 This method is meant to be used by line-oriented frontends, who need to
367 This method is meant to be used by line-oriented frontends, who need to
353 guess whether a block is complete or not based solely on prior and
368 guess whether a block is complete or not based solely on prior and
354 current input lines. The InputSplitter considers it has a complete
369 current input lines. The InputSplitter considers it has a complete
355 interactive block and will not accept more input only when either a
370 interactive block and will not accept more input only when either a
356 SyntaxError is raised, or *all* of the following are true:
371 SyntaxError is raised, or *all* of the following are true:
357
372
358 1. The input compiles to a complete statement.
373 1. The input compiles to a complete statement.
359
374
360 2. The indentation level is flush-left (because if we are indented,
375 2. The indentation level is flush-left (because if we are indented,
361 like inside a function definition or for loop, we need to keep
376 like inside a function definition or for loop, we need to keep
362 reading new input).
377 reading new input).
363
378
364 3. There is one extra line consisting only of whitespace.
379 3. There is one extra line consisting only of whitespace.
365
380
366 Because of condition #3, this method should be used only by
381 Because of condition #3, this method should be used only by
367 *line-oriented* frontends, since it means that intermediate blank lines
382 *line-oriented* frontends, since it means that intermediate blank lines
368 are not allowed in function definitions (or any other indented block).
383 are not allowed in function definitions (or any other indented block).
369
384
370 If the current input produces a syntax error, this method immediately
385 If the current input produces a syntax error, this method immediately
371 returns False but does *not* raise the syntax error exception, as
386 returns False but does *not* raise the syntax error exception, as
372 typically clients will want to send invalid syntax to an execution
387 typically clients will want to send invalid syntax to an execution
373 backend which might convert the invalid syntax into valid Python via
388 backend which might convert the invalid syntax into valid Python via
374 one of the dynamic IPython mechanisms.
389 one of the dynamic IPython mechanisms.
375 """
390 """
376
391
377 # With incomplete input, unconditionally accept more
392 # With incomplete input, unconditionally accept more
378 if not self._is_complete:
393 if not self._is_complete:
379 return True
394 return True
380
395
381 # If we already have complete input and we're flush left, the answer
396 # If we already have complete input and we're flush left, the answer
382 # depends. In line mode, if there hasn't been any indentation,
397 # depends. In line mode, if there hasn't been any indentation,
383 # that's it. If we've come back from some indentation, we need
398 # that's it. If we've come back from some indentation, we need
384 # the blank final line to finish.
399 # the blank final line to finish.
385 # In cell mode, we need to check how many blocks the input so far
400 # In cell mode, we need to check how many blocks the input so far
386 # compiles into, because if there's already more than one full
401 # compiles into, because if there's already more than one full
387 # independent block of input, then the client has entered full
402 # independent block of input, then the client has entered full
388 # 'cell' mode and is feeding lines that each is complete. In this
403 # 'cell' mode and is feeding lines that each is complete. In this
389 # case we should then keep accepting. The Qt terminal-like console
404 # case we should then keep accepting. The Qt terminal-like console
390 # does precisely this, to provide the convenience of terminal-like
405 # does precisely this, to provide the convenience of terminal-like
391 # input of single expressions, but allowing the user (with a
406 # input of single expressions, but allowing the user (with a
392 # separate keystroke) to switch to 'cell' mode and type multiple
407 # separate keystroke) to switch to 'cell' mode and type multiple
393 # expressions in one shot.
408 # expressions in one shot.
394 if self.indent_spaces==0:
409 if self.indent_spaces==0:
395 if self.input_mode=='line':
410 if self.input_mode=='line':
396 if not self._full_dedent:
411 if not self._full_dedent:
397 return False
412 return False
398 else:
413 else:
399 try:
414 try:
400 code_ast = ast.parse(u''.join(self._buffer))
415 code_ast = ast.parse(u''.join(self._buffer))
401 except Exception:
416 except Exception:
402 return False
417 return False
403 else:
418 else:
404 if len(code_ast.body) == 1:
419 if len(code_ast.body) == 1:
405 return False
420 return False
406
421
407 # When input is complete, then termination is marked by an extra blank
422 # When input is complete, then termination is marked by an extra blank
408 # line at the end.
423 # line at the end.
409 last_line = self.source.splitlines()[-1]
424 last_line = self.source.splitlines()[-1]
410 return bool(last_line and not last_line.isspace())
425 return bool(last_line and not last_line.isspace())
411
426
412 #------------------------------------------------------------------------
427 #------------------------------------------------------------------------
413 # Private interface
428 # Private interface
414 #------------------------------------------------------------------------
429 #------------------------------------------------------------------------
415
430
416 def _find_indent(self, line):
431 def _find_indent(self, line):
417 """Compute the new indentation level for a single line.
432 """Compute the new indentation level for a single line.
418
433
419 Parameters
434 Parameters
420 ----------
435 ----------
421 line : str
436 line : str
422 A single new line of non-whitespace, non-comment Python input.
437 A single new line of non-whitespace, non-comment Python input.
423
438
424 Returns
439 Returns
425 -------
440 -------
426 indent_spaces : int
441 indent_spaces : int
427 New value for the indent level (it may be equal to self.indent_spaces
442 New value for the indent level (it may be equal to self.indent_spaces
428 if indentation doesn't change.
443 if indentation doesn't change.
429
444
430 full_dedent : boolean
445 full_dedent : boolean
431 Whether the new line causes a full flush-left dedent.
446 Whether the new line causes a full flush-left dedent.
432 """
447 """
433 indent_spaces = self.indent_spaces
448 indent_spaces = self.indent_spaces
434 full_dedent = self._full_dedent
449 full_dedent = self._full_dedent
435
450
436 inisp = num_ini_spaces(line)
451 inisp = num_ini_spaces(line)
437 if inisp < indent_spaces:
452 if inisp < indent_spaces:
438 indent_spaces = inisp
453 indent_spaces = inisp
439 if indent_spaces <= 0:
454 if indent_spaces <= 0:
440 #print 'Full dedent in text',self.source # dbg
455 #print 'Full dedent in text',self.source # dbg
441 full_dedent = True
456 full_dedent = True
442
457
443 if line.rstrip()[-1] == ':':
458 if line.rstrip()[-1] == ':':
444 indent_spaces += 4
459 indent_spaces += 4
445 elif dedent_re.match(line):
460 elif dedent_re.match(line):
446 indent_spaces -= 4
461 indent_spaces -= 4
447 if indent_spaces <= 0:
462 if indent_spaces <= 0:
448 full_dedent = True
463 full_dedent = True
449
464
450 # Safety
465 # Safety
451 if indent_spaces < 0:
466 if indent_spaces < 0:
452 indent_spaces = 0
467 indent_spaces = 0
453 #print 'safety' # dbg
468 #print 'safety' # dbg
454
469
455 return indent_spaces, full_dedent
470 return indent_spaces, full_dedent
456
471
457 def _update_indent(self, lines):
472 def _update_indent(self, lines):
458 for line in remove_comments(lines).splitlines():
473 for line in remove_comments(lines).splitlines():
459 if line and not line.isspace():
474 if line and not line.isspace():
460 self.indent_spaces, self._full_dedent = self._find_indent(line)
475 self.indent_spaces, self._full_dedent = self._find_indent(line)
461
476
462 def _store(self, lines, buffer=None, store='source'):
477 def _store(self, lines, buffer=None, store='source'):
463 """Store one or more lines of input.
478 """Store one or more lines of input.
464
479
465 If input lines are not newline-terminated, a newline is automatically
480 If input lines are not newline-terminated, a newline is automatically
466 appended."""
481 appended."""
467
482
468 if buffer is None:
483 if buffer is None:
469 buffer = self._buffer
484 buffer = self._buffer
470
485
471 if lines.endswith('\n'):
486 if lines.endswith('\n'):
472 buffer.append(lines)
487 buffer.append(lines)
473 else:
488 else:
474 buffer.append(lines+'\n')
489 buffer.append(lines+'\n')
475 setattr(self, store, self._set_source(buffer))
490 setattr(self, store, self._set_source(buffer))
476
491
477 def _set_source(self, buffer):
492 def _set_source(self, buffer):
478 return u''.join(buffer)
493 return u''.join(buffer)
479
494
480
495
481 #-----------------------------------------------------------------------------
496 #-----------------------------------------------------------------------------
482 # Functions and classes for IPython-specific syntactic support
497 # Functions and classes for IPython-specific syntactic support
483 #-----------------------------------------------------------------------------
498 #-----------------------------------------------------------------------------
484
499
485 # The escaped translators ALL receive a line where their own escape has been
500 # The escaped translators ALL receive a line where their own escape has been
486 # stripped. Only '?' is valid at the end of the line, all others can only be
501 # stripped. Only '?' is valid at the end of the line, all others can only be
487 # placed at the start.
502 # placed at the start.
488
503
489 # Transformations of the special syntaxes that don't rely on an explicit escape
504 # Transformations of the special syntaxes that don't rely on an explicit escape
490 # character but instead on patterns on the input line
505 # character but instead on patterns on the input line
491
506
492 # The core transformations are implemented as standalone functions that can be
507 # The core transformations are implemented as standalone functions that can be
493 # tested and validated in isolation. Each of these uses a regexp, we
508 # tested and validated in isolation. Each of these uses a regexp, we
494 # pre-compile these and keep them close to each function definition for clarity
509 # pre-compile these and keep them close to each function definition for clarity
495
510
496 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
511 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
497 r'\s*=\s*!\s*(?P<cmd>.*)')
512 r'\s*=\s*!\s*(?P<cmd>.*)')
498
513
499 def transform_assign_system(line):
514 def transform_assign_system(line):
500 """Handle the `files = !ls` syntax."""
515 """Handle the `files = !ls` syntax."""
501 m = _assign_system_re.match(line)
516 m = _assign_system_re.match(line)
502 if m is not None:
517 if m is not None:
503 cmd = m.group('cmd')
518 cmd = m.group('cmd')
504 lhs = m.group('lhs')
519 lhs = m.group('lhs')
505 new_line = '%s = get_ipython().getoutput(%r)' % (lhs, cmd)
520 new_line = '%s = get_ipython().getoutput(%r)' % (lhs, cmd)
506 return new_line
521 return new_line
507 return line
522 return line
508
523
509
524
510 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
525 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
511 r'\s*=\s*%\s*(?P<cmd>.*)')
526 r'\s*=\s*%\s*(?P<cmd>.*)')
512
527
513 def transform_assign_magic(line):
528 def transform_assign_magic(line):
514 """Handle the `a = %who` syntax."""
529 """Handle the `a = %who` syntax."""
515 m = _assign_magic_re.match(line)
530 m = _assign_magic_re.match(line)
516 if m is not None:
531 if m is not None:
517 cmd = m.group('cmd')
532 cmd = m.group('cmd')
518 lhs = m.group('lhs')
533 lhs = m.group('lhs')
519 new_line = '%s = get_ipython().magic(%r)' % (lhs, cmd)
534 new_line = '%s = get_ipython().magic(%r)' % (lhs, cmd)
520 return new_line
535 return new_line
521 return line
536 return line
522
537
523
538
524 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
539 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
525
540
526 def transform_classic_prompt(line):
541 def transform_classic_prompt(line):
527 """Handle inputs that start with '>>> ' syntax."""
542 """Handle inputs that start with '>>> ' syntax."""
528
543
529 if not line or line.isspace():
544 if not line or line.isspace():
530 return line
545 return line
531 m = _classic_prompt_re.match(line)
546 m = _classic_prompt_re.match(line)
532 if m:
547 if m:
533 return line[len(m.group(0)):]
548 return line[len(m.group(0)):]
534 else:
549 else:
535 return line
550 return line
536
551
537
552
538 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
553 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
539
554
540 def transform_ipy_prompt(line):
555 def transform_ipy_prompt(line):
541 """Handle inputs that start classic IPython prompt syntax."""
556 """Handle inputs that start classic IPython prompt syntax."""
542
557
543 if not line or line.isspace():
558 if not line or line.isspace():
544 return line
559 return line
545 #print 'LINE: %r' % line # dbg
560 #print 'LINE: %r' % line # dbg
546 m = _ipy_prompt_re.match(line)
561 m = _ipy_prompt_re.match(line)
547 if m:
562 if m:
548 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
563 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
549 return line[len(m.group(0)):]
564 return line[len(m.group(0)):]
550 else:
565 else:
551 return line
566 return line
552
567
553
568
554 def _make_help_call(target, esc, lspace, next_input=None):
569 def _make_help_call(target, esc, lspace, next_input=None):
555 """Prepares a pinfo(2)/psearch call from a target name and the escape
570 """Prepares a pinfo(2)/psearch call from a target name and the escape
556 (i.e. ? or ??)"""
571 (i.e. ? or ??)"""
557 method = 'pinfo2' if esc == '??' \
572 method = 'pinfo2' if esc == '??' \
558 else 'psearch' if '*' in target \
573 else 'psearch' if '*' in target \
559 else 'pinfo'
574 else 'pinfo'
560 arg = " ".join([method, target])
575 arg = " ".join([method, target])
561
576
562 if next_input is None:
577 if next_input is None:
563 return '%sget_ipython().magic(%r)' % (lspace, arg)
578 return '%sget_ipython().magic(%r)' % (lspace, arg)
564 else:
579 else:
565 return '%sget_ipython().set_next_input(%r);get_ipython().magic(%r)' % \
580 return '%sget_ipython().set_next_input(%r);get_ipython().magic(%r)' % \
566 (lspace, next_input, arg)
581 (lspace, next_input, arg)
567
582
568
583
569 _initial_space_re = re.compile(r'\s*')
584 _initial_space_re = re.compile(r'\s*')
570
585
571 _help_end_re = re.compile(r"""(%?
586 _help_end_re = re.compile(r"""(%?
572 [a-zA-Z_*][\w*]* # Variable name
587 [a-zA-Z_*][\w*]* # Variable name
573 (\.[a-zA-Z_*][\w*]*)* # .etc.etc
588 (\.[a-zA-Z_*][\w*]*)* # .etc.etc
574 )
589 )
575 (\?\??)$ # ? or ??""",
590 (\?\??)$ # ? or ??""",
576 re.VERBOSE)
591 re.VERBOSE)
577
592
578
593
579 def transform_help_end(line):
594 def transform_help_end(line):
580 """Translate lines with ?/?? at the end"""
595 """Translate lines with ?/?? at the end"""
581 m = _help_end_re.search(line)
596 m = _help_end_re.search(line)
582 if m is None or has_comment(line):
597 if m is None or has_comment(line):
583 return line
598 return line
584 target = m.group(1)
599 target = m.group(1)
585 esc = m.group(3)
600 esc = m.group(3)
586 lspace = _initial_space_re.match(line).group(0)
601 lspace = _initial_space_re.match(line).group(0)
587
602
588 # If we're mid-command, put it back on the next prompt for the user.
603 # If we're mid-command, put it back on the next prompt for the user.
589 next_input = line.rstrip('?') if line.strip() != m.group(0) else None
604 next_input = line.rstrip('?') if line.strip() != m.group(0) else None
590
605
591 return _make_help_call(target, esc, lspace, next_input)
606 return _make_help_call(target, esc, lspace, next_input)
592
607
593
608
594 class EscapedTransformer(object):
609 class EscapedTransformer(object):
595 """Class to transform lines that are explicitly escaped out."""
610 """Class to transform lines that are explicitly escaped out."""
596
611
597 def __init__(self):
612 def __init__(self):
598 tr = { ESC_SHELL : self._tr_system,
613 tr = { ESC_SHELL : self._tr_system,
599 ESC_SH_CAP : self._tr_system2,
614 ESC_SH_CAP : self._tr_system2,
600 ESC_HELP : self._tr_help,
615 ESC_HELP : self._tr_help,
601 ESC_HELP2 : self._tr_help,
616 ESC_HELP2 : self._tr_help,
602 ESC_MAGIC : self._tr_magic,
617 ESC_MAGIC : self._tr_magic,
603 ESC_QUOTE : self._tr_quote,
618 ESC_QUOTE : self._tr_quote,
604 ESC_QUOTE2 : self._tr_quote2,
619 ESC_QUOTE2 : self._tr_quote2,
605 ESC_PAREN : self._tr_paren }
620 ESC_PAREN : self._tr_paren }
606 self.tr = tr
621 self.tr = tr
607
622
608 # Support for syntax transformations that use explicit escapes typed by the
623 # Support for syntax transformations that use explicit escapes typed by the
609 # user at the beginning of a line
624 # user at the beginning of a line
610 @staticmethod
625 @staticmethod
611 def _tr_system(line_info):
626 def _tr_system(line_info):
612 "Translate lines escaped with: !"
627 "Translate lines escaped with: !"
613 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
628 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
614 return '%sget_ipython().system(%r)' % (line_info.pre, cmd)
629 return '%sget_ipython().system(%r)' % (line_info.pre, cmd)
615
630
616 @staticmethod
631 @staticmethod
617 def _tr_system2(line_info):
632 def _tr_system2(line_info):
618 "Translate lines escaped with: !!"
633 "Translate lines escaped with: !!"
619 cmd = line_info.line.lstrip()[2:]
634 cmd = line_info.line.lstrip()[2:]
620 return '%sget_ipython().getoutput(%r)' % (line_info.pre, cmd)
635 return '%sget_ipython().getoutput(%r)' % (line_info.pre, cmd)
621
636
622 @staticmethod
637 @staticmethod
623 def _tr_help(line_info):
638 def _tr_help(line_info):
624 "Translate lines escaped with: ?/??"
639 "Translate lines escaped with: ?/??"
625 # A naked help line should just fire the intro help screen
640 # A naked help line should just fire the intro help screen
626 if not line_info.line[1:]:
641 if not line_info.line[1:]:
627 return 'get_ipython().show_usage()'
642 return 'get_ipython().show_usage()'
628
643
629 return _make_help_call(line_info.ifun, line_info.esc, line_info.pre)
644 return _make_help_call(line_info.ifun, line_info.esc, line_info.pre)
630
645
631 @staticmethod
646 @staticmethod
632 def _tr_magic(line_info):
647 def _tr_magic(line_info):
633 "Translate lines escaped with: %"
648 "Translate lines escaped with: %"
634 tpl = '%sget_ipython().magic(%r)'
649 tpl = '%sget_ipython().magic(%r)'
635 cmd = ' '.join([line_info.ifun, line_info.the_rest]).strip()
650 cmd = ' '.join([line_info.ifun, line_info.the_rest]).strip()
636 return tpl % (line_info.pre, cmd)
651 return tpl % (line_info.pre, cmd)
637
652
638 @staticmethod
653 @staticmethod
639 def _tr_quote(line_info):
654 def _tr_quote(line_info):
640 "Translate lines escaped with: ,"
655 "Translate lines escaped with: ,"
641 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
656 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
642 '", "'.join(line_info.the_rest.split()) )
657 '", "'.join(line_info.the_rest.split()) )
643
658
644 @staticmethod
659 @staticmethod
645 def _tr_quote2(line_info):
660 def _tr_quote2(line_info):
646 "Translate lines escaped with: ;"
661 "Translate lines escaped with: ;"
647 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
662 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
648 line_info.the_rest)
663 line_info.the_rest)
649
664
650 @staticmethod
665 @staticmethod
651 def _tr_paren(line_info):
666 def _tr_paren(line_info):
652 "Translate lines escaped with: /"
667 "Translate lines escaped with: /"
653 return '%s%s(%s)' % (line_info.pre, line_info.ifun,
668 return '%s%s(%s)' % (line_info.pre, line_info.ifun,
654 ", ".join(line_info.the_rest.split()))
669 ", ".join(line_info.the_rest.split()))
655
670
656 def __call__(self, line):
671 def __call__(self, line):
657 """Class to transform lines that are explicitly escaped out.
672 """Class to transform lines that are explicitly escaped out.
658
673
659 This calls the above _tr_* static methods for the actual line
674 This calls the above _tr_* static methods for the actual line
660 translations."""
675 translations."""
661
676
662 # Empty lines just get returned unmodified
677 # Empty lines just get returned unmodified
663 if not line or line.isspace():
678 if not line or line.isspace():
664 return line
679 return line
665
680
666 # Get line endpoints, where the escapes can be
681 # Get line endpoints, where the escapes can be
667 line_info = LineInfo(line)
682 line_info = LineInfo(line)
668
683
669 if not line_info.esc in self.tr:
684 if not line_info.esc in self.tr:
670 # If we don't recognize the escape, don't modify the line
685 # If we don't recognize the escape, don't modify the line
671 return line
686 return line
672
687
673 return self.tr[line_info.esc](line_info)
688 return self.tr[line_info.esc](line_info)
674
689
675
690
676 # A function-looking object to be used by the rest of the code. The purpose of
691 # A function-looking object to be used by the rest of the code. The purpose of
677 # the class in this case is to organize related functionality, more than to
692 # the class in this case is to organize related functionality, more than to
678 # manage state.
693 # manage state.
679 transform_escaped = EscapedTransformer()
694 transform_escaped = EscapedTransformer()
680
695
681
696
682 class IPythonInputSplitter(InputSplitter):
697 class IPythonInputSplitter(InputSplitter):
683 """An input splitter that recognizes all of IPython's special syntax."""
698 """An input splitter that recognizes all of IPython's special syntax."""
684
699
685 # String with raw, untransformed input.
700 # String with raw, untransformed input.
686 source_raw = ''
701 source_raw = ''
687
702
688 cell_magic_body = None
703 cell_magic_parts = []
704
705 cell_magic_mode = False
689
706
690 # Private attributes
707 # Private attributes
691
708
692 # List with lines of raw input accumulated so far.
709 # List with lines of raw input accumulated so far.
693 _buffer_raw = None
710 _buffer_raw = None
694
711
695 def __init__(self, input_mode=None):
712 def __init__(self, input_mode=None):
696 super(IPythonInputSplitter, self).__init__(input_mode)
713 super(IPythonInputSplitter, self).__init__(input_mode)
697 self._buffer_raw = []
714 self._buffer_raw = []
715 self._validate = True
698
716
699 def reset(self):
717 def reset(self):
700 """Reset the input buffer and associated state."""
718 """Reset the input buffer and associated state."""
701 super(IPythonInputSplitter, self).reset()
719 super(IPythonInputSplitter, self).reset()
702 self._buffer_raw[:] = []
720 self._buffer_raw[:] = []
703 self.source_raw = ''
721 self.source_raw = ''
704 self.cell_magic_body = None
722 self.cell_magic_parts = []
723 self.cell_magic_mode = False
705
724
706 def source_raw_reset(self):
725 def source_raw_reset(self):
707 """Return input and raw source and perform a full reset.
726 """Return input and raw source and perform a full reset.
708 """
727 """
709 out = self.source
728 out = self.source
710 out_r = self.source_raw
729 out_r = self.source_raw
711 self.reset()
730 self.reset()
712 return out, out_r
731 return out, out_r
713
732
733 def push_accepts_more(self):
734 if self.cell_magic_mode:
735 return not self._is_complete
736 else:
737 return super(IPythonInputSplitter, self).push_accepts_more()
738
739 def _push_line_mode(self, lines):
740 """Push in line mode.
741
742 This means that we only get individual 'lines' with each call, though
743 in practice each input may be multiline. But this is in contrast to
744 cell mode, which feeds the entirety of the cell from the start with
745 each call.
746 """
747 # cell magic support
748 #print('#'*10)
749 #print(lines+'\n---') # dbg
750 #print (repr(lines)+'\n+++')
751 #print('raw', self._buffer_raw, 'validate', self.cell_magic_mode)
752 # Only trigger this block if we're at a 'fresh' pumping start.
753 if lines.startswith('%%') and (not self.cell_magic_mode) and \
754 not self._buffer_raw:
755 # Cell magics bypass all further transformations
756 self.cell_magic_mode = True
757 first, _, body = lines.partition('\n')
758 magic_name, _, line = first.partition(' ')
759 magic_name = magic_name.lstrip(ESC_MAGIC)
760 # We store the body of the cell and create a call to a method that
761 # will use this stored value. This is ugly, but it's a first cut to
762 # get it all working, as right now changing the return API of our
763 # methods would require major refactoring.
764 self.cell_magic_parts = [body]
765 tpl = 'get_ipython()._cell_magic(%r, %r)'
766 tlines = tpl % (magic_name, line)
767 self._store(tlines)
768 self._store(lines, self._buffer_raw, 'source_raw')
769 self._is_complete = False
770 return False
771
772 if self.cell_magic_mode:
773 # Find out if the last stored block has a whitespace line as its
774 # last line and also this line is whitespace, case in which we're
775 # done (two contiguous blank lines signal termination). Note that
776 # the storage logic *enforces* that every stored block is
777 # newline-terminated, so we grab everything but the last character
778 # so we can have the body of the block alone.
779 last_block = self.cell_magic_parts[-1]
780 self._is_complete = last_blank(last_block) and lines.isspace()
781 # Only store the raw input. For lines beyond the first one, we
782 # only store them for history purposes, and for execution we want
783 # the caller to only receive the _cell_magic() call.
784 self._store(lines, self._buffer_raw, 'source_raw')
785 self.cell_magic_parts.append(lines)
786 return self._is_complete
787
788 lines_list = lines.splitlines()
789
790 transforms = [transform_ipy_prompt, transform_classic_prompt,
791 transform_help_end, transform_escaped,
792 transform_assign_system, transform_assign_magic]
793
794 # Transform logic
795 #
796 # We only apply the line transformers to the input if we have either no
797 # input yet, or complete input, or if the last line of the buffer ends
798 # with ':' (opening an indented block). This prevents the accidental
799 # transformation of escapes inside multiline expressions like
800 # triple-quoted strings or parenthesized expressions.
801 #
802 # The last heuristic, while ugly, ensures that the first line of an
803 # indented block is correctly transformed.
804 #
805 # FIXME: try to find a cleaner approach for this last bit.
806
807 # Store raw source before applying any transformations to it. Note
808 # that this must be done *after* the reset() call that would otherwise
809 # flush the buffer.
810 self._store(lines, self._buffer_raw, 'source_raw')
811
812 push = super(IPythonInputSplitter, self).push
813 buf = self._buffer
814 for line in lines_list:
815 if self._is_complete or not buf or \
816 (buf and buf[-1].rstrip().endswith((':', ','))):
817 for f in transforms:
818 line = f(line)
819
820 out = push(line)
821 return out
822
714 def push(self, lines):
823 def push(self, lines):
715 """Push one or more lines of IPython input.
824 """Push one or more lines of IPython input.
716
825
717 This stores the given lines and returns a status code indicating
826 This stores the given lines and returns a status code indicating
718 whether the code forms a complete Python block or not, after processing
827 whether the code forms a complete Python block or not, after processing
719 all input lines for special IPython syntax.
828 all input lines for special IPython syntax.
720
829
721 Any exceptions generated in compilation are swallowed, but if an
830 Any exceptions generated in compilation are swallowed, but if an
722 exception was produced, the method returns True.
831 exception was produced, the method returns True.
723
832
724 Parameters
833 Parameters
725 ----------
834 ----------
726 lines : string
835 lines : string
727 One or more lines of Python input.
836 One or more lines of Python input.
728
837
729 Returns
838 Returns
730 -------
839 -------
731 is_complete : boolean
840 is_complete : boolean
732 True if the current input source (the result of the current input
841 True if the current input source (the result of the current input
733 plus prior inputs) forms a complete Python execution block. Note that
842 plus prior inputs) forms a complete Python execution block. Note that
734 this value is also stored as a private attribute (_is_complete), so it
843 this value is also stored as a private attribute (_is_complete), so it
735 can be queried at any time.
844 can be queried at any time.
736 """
845 """
846 print('mode:', self.input_mode)
847 print('lines:',repr(lines))
737 if not lines:
848 if not lines:
738 return super(IPythonInputSplitter, self).push(lines)
849 return super(IPythonInputSplitter, self).push(lines)
739
850
740 # We must ensure all input is pure unicode
851 # We must ensure all input is pure unicode
741 lines = cast_unicode(lines, self.encoding)
852 lines = cast_unicode(lines, self.encoding)
742
853
743 # cell magic support
854 if self.input_mode == 'line':
744 #print('IM:', self.input_mode,'\n'+lines); print('---') # dbg
855 return self._push_line_mode(lines)
745 #if self.input_mode == 'cell' and lines.startswith('%%'):
856
746 if lines.startswith('%%'):
857 ## else:
747 # Cell magics bypass all further transformations
858 ## return self._push_cell_mode(lines)
748 self.reset()
749 self._is_complete = is_complete = True
750 first, _, body = lines.partition('\n')
751 magic_name, _, line = first.partition(' ')
752 magic_name = magic_name.lstrip(ESC_MAGIC)
753 self.cell_magic_body = body
754 tpl = 'get_ipython()._cell_magic(%r, %r)'
755 lines = tpl % (magic_name, line)
756
859
757 lines_list = lines.splitlines()
860 lines_list = lines.splitlines()
758
861
759 transforms = [transform_ipy_prompt, transform_classic_prompt,
862 transforms = [transform_ipy_prompt, transform_classic_prompt,
760 transform_help_end, transform_escaped,
863 transform_help_end, transform_escaped,
761 transform_assign_system, transform_assign_magic]
864 transform_assign_system, transform_assign_magic]
762
865
763 # Transform logic
866 # Transform logic
764 #
867 #
765 # We only apply the line transformers to the input if we have either no
868 # We only apply the line transformers to the input if we have either no
766 # input yet, or complete input, or if the last line of the buffer ends
869 # input yet, or complete input, or if the last line of the buffer ends
767 # with ':' (opening an indented block). This prevents the accidental
870 # with ':' (opening an indented block). This prevents the accidental
768 # transformation of escapes inside multiline expressions like
871 # transformation of escapes inside multiline expressions like
769 # triple-quoted strings or parenthesized expressions.
872 # triple-quoted strings or parenthesized expressions.
770 #
873 #
771 # The last heuristic, while ugly, ensures that the first line of an
874 # The last heuristic, while ugly, ensures that the first line of an
772 # indented block is correctly transformed.
875 # indented block is correctly transformed.
773 #
876 #
774 # FIXME: try to find a cleaner approach for this last bit.
877 # FIXME: try to find a cleaner approach for this last bit.
775
878
776 # If we were in 'block' mode, since we're going to pump the parent
879 # If we were in 'block' mode, since we're going to pump the parent
777 # class by hand line by line, we need to temporarily switch out to
880 # class by hand line by line, we need to temporarily switch out to
778 # 'line' mode, do a single manual reset and then feed the lines one
881 # 'line' mode, do a single manual reset and then feed the lines one
779 # by one. Note that this only matters if the input has more than one
882 # by one. Note that this only matters if the input has more than one
780 # line.
883 # line.
781 changed_input_mode = False
884 changed_input_mode = False
782
885
783 if self.input_mode == 'cell':
886 if self.input_mode == 'cell':
784 self.reset()
887 self.reset()
785 changed_input_mode = True
888 changed_input_mode = True
786 saved_input_mode = 'cell'
889 saved_input_mode = 'cell'
787 self.input_mode = 'line'
890 self.input_mode = 'line'
788
891
789 # Store raw source before applying any transformations to it. Note
892 # Store raw source before applying any transformations to it. Note
790 # that this must be done *after* the reset() call that would otherwise
893 # that this must be done *after* the reset() call that would otherwise
791 # flush the buffer.
894 # flush the buffer.
792 self._store(lines, self._buffer_raw, 'source_raw')
895 self._store(lines, self._buffer_raw, 'source_raw')
793
896
794 try:
897 try:
795 push = super(IPythonInputSplitter, self).push
898 push = super(IPythonInputSplitter, self).push
796 buf = self._buffer
899 buf = self._buffer
797 for line in lines_list:
900 for line in lines_list:
798 if self._is_complete or not buf or \
901 if self._is_complete or not buf or \
799 (buf and (buf[-1].rstrip().endswith(':') or
902 (buf and buf[-1].rstrip().endswith((':', ','))):
800 buf[-1].rstrip().endswith(',')) ):
801 for f in transforms:
903 for f in transforms:
802 line = f(line)
904 line = f(line)
803
905
804 out = push(line)
906 out = push(line)
805 finally:
907 finally:
806 if changed_input_mode:
908 if changed_input_mode:
807 self.input_mode = saved_input_mode
909 self.input_mode = saved_input_mode
808 return out
910 return out
@@ -1,2950 +1,2952 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """Main IPython class."""
2 """Main IPython class."""
3
3
4 #-----------------------------------------------------------------------------
4 #-----------------------------------------------------------------------------
5 # Copyright (C) 2001 Janko Hauser <jhauser@zscout.de>
5 # Copyright (C) 2001 Janko Hauser <jhauser@zscout.de>
6 # Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu>
6 # Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu>
7 # Copyright (C) 2008-2011 The IPython Development Team
7 # Copyright (C) 2008-2011 The IPython Development Team
8 #
8 #
9 # Distributed under the terms of the BSD License. The full license is in
9 # Distributed under the terms of the BSD License. The full license is in
10 # the file COPYING, distributed as part of this software.
10 # the file COPYING, distributed as part of this software.
11 #-----------------------------------------------------------------------------
11 #-----------------------------------------------------------------------------
12
12
13 #-----------------------------------------------------------------------------
13 #-----------------------------------------------------------------------------
14 # Imports
14 # Imports
15 #-----------------------------------------------------------------------------
15 #-----------------------------------------------------------------------------
16
16
17 from __future__ import with_statement
17 from __future__ import with_statement
18 from __future__ import absolute_import
18 from __future__ import absolute_import
19
19
20 import __builtin__ as builtin_mod
20 import __builtin__ as builtin_mod
21 import __future__
21 import __future__
22 import abc
22 import abc
23 import ast
23 import ast
24 import atexit
24 import atexit
25 import os
25 import os
26 import re
26 import re
27 import runpy
27 import runpy
28 import sys
28 import sys
29 import tempfile
29 import tempfile
30 import types
30 import types
31 import urllib
31 import urllib
32 from io import open as io_open
32 from io import open as io_open
33
33
34 from IPython.config.configurable import SingletonConfigurable
34 from IPython.config.configurable import SingletonConfigurable
35 from IPython.core import debugger, oinspect
35 from IPython.core import debugger, oinspect
36 from IPython.core import history as ipcorehist
36 from IPython.core import history as ipcorehist
37 from IPython.core import magic
37 from IPython.core import magic
38 from IPython.core import page
38 from IPython.core import page
39 from IPython.core import prefilter
39 from IPython.core import prefilter
40 from IPython.core import shadowns
40 from IPython.core import shadowns
41 from IPython.core import ultratb
41 from IPython.core import ultratb
42 from IPython.core.alias import AliasManager, AliasError
42 from IPython.core.alias import AliasManager, AliasError
43 from IPython.core.autocall import ExitAutocall
43 from IPython.core.autocall import ExitAutocall
44 from IPython.core.builtin_trap import BuiltinTrap
44 from IPython.core.builtin_trap import BuiltinTrap
45 from IPython.core.compilerop import CachingCompiler
45 from IPython.core.compilerop import CachingCompiler
46 from IPython.core.display_trap import DisplayTrap
46 from IPython.core.display_trap import DisplayTrap
47 from IPython.core.displayhook import DisplayHook
47 from IPython.core.displayhook import DisplayHook
48 from IPython.core.displaypub import DisplayPublisher
48 from IPython.core.displaypub import DisplayPublisher
49 from IPython.core.error import UsageError
49 from IPython.core.error import UsageError
50 from IPython.core.extensions import ExtensionManager
50 from IPython.core.extensions import ExtensionManager
51 from IPython.core.fakemodule import FakeModule, init_fakemod_dict
51 from IPython.core.fakemodule import FakeModule, init_fakemod_dict
52 from IPython.core.formatters import DisplayFormatter
52 from IPython.core.formatters import DisplayFormatter
53 from IPython.core.history import HistoryManager
53 from IPython.core.history import HistoryManager
54 from IPython.core.inputsplitter import IPythonInputSplitter
54 from IPython.core.inputsplitter import IPythonInputSplitter
55 from IPython.core.logger import Logger
55 from IPython.core.logger import Logger
56 from IPython.core.macro import Macro
56 from IPython.core.macro import Macro
57 from IPython.core.payload import PayloadManager
57 from IPython.core.payload import PayloadManager
58 from IPython.core.plugin import PluginManager
58 from IPython.core.plugin import PluginManager
59 from IPython.core.prefilter import PrefilterManager, ESC_MAGIC
59 from IPython.core.prefilter import PrefilterManager, ESC_MAGIC
60 from IPython.core.profiledir import ProfileDir
60 from IPython.core.profiledir import ProfileDir
61 from IPython.core.pylabtools import pylab_activate
61 from IPython.core.pylabtools import pylab_activate
62 from IPython.core.prompts import PromptManager
62 from IPython.core.prompts import PromptManager
63 from IPython.utils import PyColorize
63 from IPython.utils import PyColorize
64 from IPython.utils import io
64 from IPython.utils import io
65 from IPython.utils import py3compat
65 from IPython.utils import py3compat
66 from IPython.utils import openpy
66 from IPython.utils import openpy
67 from IPython.utils.doctestreload import doctest_reload
67 from IPython.utils.doctestreload import doctest_reload
68 from IPython.utils.io import ask_yes_no
68 from IPython.utils.io import ask_yes_no
69 from IPython.utils.ipstruct import Struct
69 from IPython.utils.ipstruct import Struct
70 from IPython.utils.path import get_home_dir, get_ipython_dir, get_py_filename, unquote_filename
70 from IPython.utils.path import get_home_dir, get_ipython_dir, get_py_filename, unquote_filename
71 from IPython.utils.pickleshare import PickleShareDB
71 from IPython.utils.pickleshare import PickleShareDB
72 from IPython.utils.process import system, getoutput
72 from IPython.utils.process import system, getoutput
73 from IPython.utils.strdispatch import StrDispatch
73 from IPython.utils.strdispatch import StrDispatch
74 from IPython.utils.syspathcontext import prepended_to_syspath
74 from IPython.utils.syspathcontext import prepended_to_syspath
75 from IPython.utils.text import (format_screen, LSString, SList,
75 from IPython.utils.text import (format_screen, LSString, SList,
76 DollarFormatter)
76 DollarFormatter)
77 from IPython.utils.traitlets import (Integer, CBool, CaselessStrEnum, Enum,
77 from IPython.utils.traitlets import (Integer, CBool, CaselessStrEnum, Enum,
78 List, Unicode, Instance, Type)
78 List, Unicode, Instance, Type)
79 from IPython.utils.warn import warn, error
79 from IPython.utils.warn import warn, error
80 import IPython.core.hooks
80 import IPython.core.hooks
81
81
82 #-----------------------------------------------------------------------------
82 #-----------------------------------------------------------------------------
83 # Globals
83 # Globals
84 #-----------------------------------------------------------------------------
84 #-----------------------------------------------------------------------------
85
85
86 # compiled regexps for autoindent management
86 # compiled regexps for autoindent management
87 dedent_re = re.compile(r'^\s+raise|^\s+return|^\s+pass')
87 dedent_re = re.compile(r'^\s+raise|^\s+return|^\s+pass')
88
88
89 #-----------------------------------------------------------------------------
89 #-----------------------------------------------------------------------------
90 # Utilities
90 # Utilities
91 #-----------------------------------------------------------------------------
91 #-----------------------------------------------------------------------------
92
92
93 def softspace(file, newvalue):
93 def softspace(file, newvalue):
94 """Copied from code.py, to remove the dependency"""
94 """Copied from code.py, to remove the dependency"""
95
95
96 oldvalue = 0
96 oldvalue = 0
97 try:
97 try:
98 oldvalue = file.softspace
98 oldvalue = file.softspace
99 except AttributeError:
99 except AttributeError:
100 pass
100 pass
101 try:
101 try:
102 file.softspace = newvalue
102 file.softspace = newvalue
103 except (AttributeError, TypeError):
103 except (AttributeError, TypeError):
104 # "attribute-less object" or "read-only attributes"
104 # "attribute-less object" or "read-only attributes"
105 pass
105 pass
106 return oldvalue
106 return oldvalue
107
107
108
108
109 def no_op(*a, **kw): pass
109 def no_op(*a, **kw): pass
110
110
111 class NoOpContext(object):
111 class NoOpContext(object):
112 def __enter__(self): pass
112 def __enter__(self): pass
113 def __exit__(self, type, value, traceback): pass
113 def __exit__(self, type, value, traceback): pass
114 no_op_context = NoOpContext()
114 no_op_context = NoOpContext()
115
115
116 class SpaceInInput(Exception): pass
116 class SpaceInInput(Exception): pass
117
117
118 class Bunch: pass
118 class Bunch: pass
119
119
120
120
121 def get_default_colors():
121 def get_default_colors():
122 if sys.platform=='darwin':
122 if sys.platform=='darwin':
123 return "LightBG"
123 return "LightBG"
124 elif os.name=='nt':
124 elif os.name=='nt':
125 return 'Linux'
125 return 'Linux'
126 else:
126 else:
127 return 'Linux'
127 return 'Linux'
128
128
129
129
130 class SeparateUnicode(Unicode):
130 class SeparateUnicode(Unicode):
131 """A Unicode subclass to validate separate_in, separate_out, etc.
131 """A Unicode subclass to validate separate_in, separate_out, etc.
132
132
133 This is a Unicode based trait that converts '0'->'' and '\\n'->'\n'.
133 This is a Unicode based trait that converts '0'->'' and '\\n'->'\n'.
134 """
134 """
135
135
136 def validate(self, obj, value):
136 def validate(self, obj, value):
137 if value == '0': value = ''
137 if value == '0': value = ''
138 value = value.replace('\\n','\n')
138 value = value.replace('\\n','\n')
139 return super(SeparateUnicode, self).validate(obj, value)
139 return super(SeparateUnicode, self).validate(obj, value)
140
140
141
141
142 class ReadlineNoRecord(object):
142 class ReadlineNoRecord(object):
143 """Context manager to execute some code, then reload readline history
143 """Context manager to execute some code, then reload readline history
144 so that interactive input to the code doesn't appear when pressing up."""
144 so that interactive input to the code doesn't appear when pressing up."""
145 def __init__(self, shell):
145 def __init__(self, shell):
146 self.shell = shell
146 self.shell = shell
147 self._nested_level = 0
147 self._nested_level = 0
148
148
149 def __enter__(self):
149 def __enter__(self):
150 if self._nested_level == 0:
150 if self._nested_level == 0:
151 try:
151 try:
152 self.orig_length = self.current_length()
152 self.orig_length = self.current_length()
153 self.readline_tail = self.get_readline_tail()
153 self.readline_tail = self.get_readline_tail()
154 except (AttributeError, IndexError): # Can fail with pyreadline
154 except (AttributeError, IndexError): # Can fail with pyreadline
155 self.orig_length, self.readline_tail = 999999, []
155 self.orig_length, self.readline_tail = 999999, []
156 self._nested_level += 1
156 self._nested_level += 1
157
157
158 def __exit__(self, type, value, traceback):
158 def __exit__(self, type, value, traceback):
159 self._nested_level -= 1
159 self._nested_level -= 1
160 if self._nested_level == 0:
160 if self._nested_level == 0:
161 # Try clipping the end if it's got longer
161 # Try clipping the end if it's got longer
162 try:
162 try:
163 e = self.current_length() - self.orig_length
163 e = self.current_length() - self.orig_length
164 if e > 0:
164 if e > 0:
165 for _ in range(e):
165 for _ in range(e):
166 self.shell.readline.remove_history_item(self.orig_length)
166 self.shell.readline.remove_history_item(self.orig_length)
167
167
168 # If it still doesn't match, just reload readline history.
168 # If it still doesn't match, just reload readline history.
169 if self.current_length() != self.orig_length \
169 if self.current_length() != self.orig_length \
170 or self.get_readline_tail() != self.readline_tail:
170 or self.get_readline_tail() != self.readline_tail:
171 self.shell.refill_readline_hist()
171 self.shell.refill_readline_hist()
172 except (AttributeError, IndexError):
172 except (AttributeError, IndexError):
173 pass
173 pass
174 # Returning False will cause exceptions to propagate
174 # Returning False will cause exceptions to propagate
175 return False
175 return False
176
176
177 def current_length(self):
177 def current_length(self):
178 return self.shell.readline.get_current_history_length()
178 return self.shell.readline.get_current_history_length()
179
179
180 def get_readline_tail(self, n=10):
180 def get_readline_tail(self, n=10):
181 """Get the last n items in readline history."""
181 """Get the last n items in readline history."""
182 end = self.shell.readline.get_current_history_length() + 1
182 end = self.shell.readline.get_current_history_length() + 1
183 start = max(end-n, 1)
183 start = max(end-n, 1)
184 ghi = self.shell.readline.get_history_item
184 ghi = self.shell.readline.get_history_item
185 return [ghi(x) for x in range(start, end)]
185 return [ghi(x) for x in range(start, end)]
186
186
187 #-----------------------------------------------------------------------------
187 #-----------------------------------------------------------------------------
188 # Main IPython class
188 # Main IPython class
189 #-----------------------------------------------------------------------------
189 #-----------------------------------------------------------------------------
190
190
191 class InteractiveShell(SingletonConfigurable):
191 class InteractiveShell(SingletonConfigurable):
192 """An enhanced, interactive shell for Python."""
192 """An enhanced, interactive shell for Python."""
193
193
194 _instance = None
194 _instance = None
195
195
196 autocall = Enum((0,1,2), default_value=0, config=True, help=
196 autocall = Enum((0,1,2), default_value=0, config=True, help=
197 """
197 """
198 Make IPython automatically call any callable object even if you didn't
198 Make IPython automatically call any callable object even if you didn't
199 type explicit parentheses. For example, 'str 43' becomes 'str(43)'
199 type explicit parentheses. For example, 'str 43' becomes 'str(43)'
200 automatically. The value can be '0' to disable the feature, '1' for
200 automatically. The value can be '0' to disable the feature, '1' for
201 'smart' autocall, where it is not applied if there are no more
201 'smart' autocall, where it is not applied if there are no more
202 arguments on the line, and '2' for 'full' autocall, where all callable
202 arguments on the line, and '2' for 'full' autocall, where all callable
203 objects are automatically called (even if no arguments are present).
203 objects are automatically called (even if no arguments are present).
204 """
204 """
205 )
205 )
206 # TODO: remove all autoindent logic and put into frontends.
206 # TODO: remove all autoindent logic and put into frontends.
207 # We can't do this yet because even runlines uses the autoindent.
207 # We can't do this yet because even runlines uses the autoindent.
208 autoindent = CBool(True, config=True, help=
208 autoindent = CBool(True, config=True, help=
209 """
209 """
210 Autoindent IPython code entered interactively.
210 Autoindent IPython code entered interactively.
211 """
211 """
212 )
212 )
213 automagic = CBool(True, config=True, help=
213 automagic = CBool(True, config=True, help=
214 """
214 """
215 Enable magic commands to be called without the leading %.
215 Enable magic commands to be called without the leading %.
216 """
216 """
217 )
217 )
218 cache_size = Integer(1000, config=True, help=
218 cache_size = Integer(1000, config=True, help=
219 """
219 """
220 Set the size of the output cache. The default is 1000, you can
220 Set the size of the output cache. The default is 1000, you can
221 change it permanently in your config file. Setting it to 0 completely
221 change it permanently in your config file. Setting it to 0 completely
222 disables the caching system, and the minimum value accepted is 20 (if
222 disables the caching system, and the minimum value accepted is 20 (if
223 you provide a value less than 20, it is reset to 0 and a warning is
223 you provide a value less than 20, it is reset to 0 and a warning is
224 issued). This limit is defined because otherwise you'll spend more
224 issued). This limit is defined because otherwise you'll spend more
225 time re-flushing a too small cache than working
225 time re-flushing a too small cache than working
226 """
226 """
227 )
227 )
228 color_info = CBool(True, config=True, help=
228 color_info = CBool(True, config=True, help=
229 """
229 """
230 Use colors for displaying information about objects. Because this
230 Use colors for displaying information about objects. Because this
231 information is passed through a pager (like 'less'), and some pagers
231 information is passed through a pager (like 'less'), and some pagers
232 get confused with color codes, this capability can be turned off.
232 get confused with color codes, this capability can be turned off.
233 """
233 """
234 )
234 )
235 colors = CaselessStrEnum(('NoColor','LightBG','Linux'),
235 colors = CaselessStrEnum(('NoColor','LightBG','Linux'),
236 default_value=get_default_colors(), config=True,
236 default_value=get_default_colors(), config=True,
237 help="Set the color scheme (NoColor, Linux, or LightBG)."
237 help="Set the color scheme (NoColor, Linux, or LightBG)."
238 )
238 )
239 colors_force = CBool(False, help=
239 colors_force = CBool(False, help=
240 """
240 """
241 Force use of ANSI color codes, regardless of OS and readline
241 Force use of ANSI color codes, regardless of OS and readline
242 availability.
242 availability.
243 """
243 """
244 # FIXME: This is essentially a hack to allow ZMQShell to show colors
244 # FIXME: This is essentially a hack to allow ZMQShell to show colors
245 # without readline on Win32. When the ZMQ formatting system is
245 # without readline on Win32. When the ZMQ formatting system is
246 # refactored, this should be removed.
246 # refactored, this should be removed.
247 )
247 )
248 debug = CBool(False, config=True)
248 debug = CBool(False, config=True)
249 deep_reload = CBool(False, config=True, help=
249 deep_reload = CBool(False, config=True, help=
250 """
250 """
251 Enable deep (recursive) reloading by default. IPython can use the
251 Enable deep (recursive) reloading by default. IPython can use the
252 deep_reload module which reloads changes in modules recursively (it
252 deep_reload module which reloads changes in modules recursively (it
253 replaces the reload() function, so you don't need to change anything to
253 replaces the reload() function, so you don't need to change anything to
254 use it). deep_reload() forces a full reload of modules whose code may
254 use it). deep_reload() forces a full reload of modules whose code may
255 have changed, which the default reload() function does not. When
255 have changed, which the default reload() function does not. When
256 deep_reload is off, IPython will use the normal reload(), but
256 deep_reload is off, IPython will use the normal reload(), but
257 deep_reload will still be available as dreload().
257 deep_reload will still be available as dreload().
258 """
258 """
259 )
259 )
260 disable_failing_post_execute = CBool(False, config=True,
260 disable_failing_post_execute = CBool(False, config=True,
261 help="Don't call post-execute functions that have failed in the past."""
261 help="Don't call post-execute functions that have failed in the past."""
262 )
262 )
263 display_formatter = Instance(DisplayFormatter)
263 display_formatter = Instance(DisplayFormatter)
264 displayhook_class = Type(DisplayHook)
264 displayhook_class = Type(DisplayHook)
265 display_pub_class = Type(DisplayPublisher)
265 display_pub_class = Type(DisplayPublisher)
266
266
267 exit_now = CBool(False)
267 exit_now = CBool(False)
268 exiter = Instance(ExitAutocall)
268 exiter = Instance(ExitAutocall)
269 def _exiter_default(self):
269 def _exiter_default(self):
270 return ExitAutocall(self)
270 return ExitAutocall(self)
271 # Monotonically increasing execution counter
271 # Monotonically increasing execution counter
272 execution_count = Integer(1)
272 execution_count = Integer(1)
273 filename = Unicode("<ipython console>")
273 filename = Unicode("<ipython console>")
274 ipython_dir= Unicode('', config=True) # Set to get_ipython_dir() in __init__
274 ipython_dir= Unicode('', config=True) # Set to get_ipython_dir() in __init__
275
275
276 # Input splitter, to split entire cells of input into either individual
276 # Input splitter, to split entire cells of input into either individual
277 # interactive statements or whole blocks.
277 # interactive statements or whole blocks.
278 input_splitter = Instance('IPython.core.inputsplitter.IPythonInputSplitter',
278 input_splitter = Instance('IPython.core.inputsplitter.IPythonInputSplitter',
279 (), {})
279 (), {})
280 logstart = CBool(False, config=True, help=
280 logstart = CBool(False, config=True, help=
281 """
281 """
282 Start logging to the default log file.
282 Start logging to the default log file.
283 """
283 """
284 )
284 )
285 logfile = Unicode('', config=True, help=
285 logfile = Unicode('', config=True, help=
286 """
286 """
287 The name of the logfile to use.
287 The name of the logfile to use.
288 """
288 """
289 )
289 )
290 logappend = Unicode('', config=True, help=
290 logappend = Unicode('', config=True, help=
291 """
291 """
292 Start logging to the given file in append mode.
292 Start logging to the given file in append mode.
293 """
293 """
294 )
294 )
295 object_info_string_level = Enum((0,1,2), default_value=0,
295 object_info_string_level = Enum((0,1,2), default_value=0,
296 config=True)
296 config=True)
297 pdb = CBool(False, config=True, help=
297 pdb = CBool(False, config=True, help=
298 """
298 """
299 Automatically call the pdb debugger after every exception.
299 Automatically call the pdb debugger after every exception.
300 """
300 """
301 )
301 )
302 multiline_history = CBool(sys.platform != 'win32', config=True,
302 multiline_history = CBool(sys.platform != 'win32', config=True,
303 help="Save multi-line entries as one entry in readline history"
303 help="Save multi-line entries as one entry in readline history"
304 )
304 )
305
305
306 # deprecated prompt traits:
306 # deprecated prompt traits:
307
307
308 prompt_in1 = Unicode('In [\\#]: ', config=True,
308 prompt_in1 = Unicode('In [\\#]: ', config=True,
309 help="Deprecated, use PromptManager.in_template")
309 help="Deprecated, use PromptManager.in_template")
310 prompt_in2 = Unicode(' .\\D.: ', config=True,
310 prompt_in2 = Unicode(' .\\D.: ', config=True,
311 help="Deprecated, use PromptManager.in2_template")
311 help="Deprecated, use PromptManager.in2_template")
312 prompt_out = Unicode('Out[\\#]: ', config=True,
312 prompt_out = Unicode('Out[\\#]: ', config=True,
313 help="Deprecated, use PromptManager.out_template")
313 help="Deprecated, use PromptManager.out_template")
314 prompts_pad_left = CBool(True, config=True,
314 prompts_pad_left = CBool(True, config=True,
315 help="Deprecated, use PromptManager.justify")
315 help="Deprecated, use PromptManager.justify")
316
316
317 def _prompt_trait_changed(self, name, old, new):
317 def _prompt_trait_changed(self, name, old, new):
318 table = {
318 table = {
319 'prompt_in1' : 'in_template',
319 'prompt_in1' : 'in_template',
320 'prompt_in2' : 'in2_template',
320 'prompt_in2' : 'in2_template',
321 'prompt_out' : 'out_template',
321 'prompt_out' : 'out_template',
322 'prompts_pad_left' : 'justify',
322 'prompts_pad_left' : 'justify',
323 }
323 }
324 warn("InteractiveShell.{name} is deprecated, use PromptManager.{newname}\n".format(
324 warn("InteractiveShell.{name} is deprecated, use PromptManager.{newname}\n".format(
325 name=name, newname=table[name])
325 name=name, newname=table[name])
326 )
326 )
327 # protect against weird cases where self.config may not exist:
327 # protect against weird cases where self.config may not exist:
328 if self.config is not None:
328 if self.config is not None:
329 # propagate to corresponding PromptManager trait
329 # propagate to corresponding PromptManager trait
330 setattr(self.config.PromptManager, table[name], new)
330 setattr(self.config.PromptManager, table[name], new)
331
331
332 _prompt_in1_changed = _prompt_trait_changed
332 _prompt_in1_changed = _prompt_trait_changed
333 _prompt_in2_changed = _prompt_trait_changed
333 _prompt_in2_changed = _prompt_trait_changed
334 _prompt_out_changed = _prompt_trait_changed
334 _prompt_out_changed = _prompt_trait_changed
335 _prompt_pad_left_changed = _prompt_trait_changed
335 _prompt_pad_left_changed = _prompt_trait_changed
336
336
337 show_rewritten_input = CBool(True, config=True,
337 show_rewritten_input = CBool(True, config=True,
338 help="Show rewritten input, e.g. for autocall."
338 help="Show rewritten input, e.g. for autocall."
339 )
339 )
340
340
341 quiet = CBool(False, config=True)
341 quiet = CBool(False, config=True)
342
342
343 history_length = Integer(10000, config=True)
343 history_length = Integer(10000, config=True)
344
344
345 # The readline stuff will eventually be moved to the terminal subclass
345 # The readline stuff will eventually be moved to the terminal subclass
346 # but for now, we can't do that as readline is welded in everywhere.
346 # but for now, we can't do that as readline is welded in everywhere.
347 readline_use = CBool(True, config=True)
347 readline_use = CBool(True, config=True)
348 readline_remove_delims = Unicode('-/~', config=True)
348 readline_remove_delims = Unicode('-/~', config=True)
349 # don't use \M- bindings by default, because they
349 # don't use \M- bindings by default, because they
350 # conflict with 8-bit encodings. See gh-58,gh-88
350 # conflict with 8-bit encodings. See gh-58,gh-88
351 readline_parse_and_bind = List([
351 readline_parse_and_bind = List([
352 'tab: complete',
352 'tab: complete',
353 '"\C-l": clear-screen',
353 '"\C-l": clear-screen',
354 'set show-all-if-ambiguous on',
354 'set show-all-if-ambiguous on',
355 '"\C-o": tab-insert',
355 '"\C-o": tab-insert',
356 '"\C-r": reverse-search-history',
356 '"\C-r": reverse-search-history',
357 '"\C-s": forward-search-history',
357 '"\C-s": forward-search-history',
358 '"\C-p": history-search-backward',
358 '"\C-p": history-search-backward',
359 '"\C-n": history-search-forward',
359 '"\C-n": history-search-forward',
360 '"\e[A": history-search-backward',
360 '"\e[A": history-search-backward',
361 '"\e[B": history-search-forward',
361 '"\e[B": history-search-forward',
362 '"\C-k": kill-line',
362 '"\C-k": kill-line',
363 '"\C-u": unix-line-discard',
363 '"\C-u": unix-line-discard',
364 ], allow_none=False, config=True)
364 ], allow_none=False, config=True)
365
365
366 # TODO: this part of prompt management should be moved to the frontends.
366 # TODO: this part of prompt management should be moved to the frontends.
367 # Use custom TraitTypes that convert '0'->'' and '\\n'->'\n'
367 # Use custom TraitTypes that convert '0'->'' and '\\n'->'\n'
368 separate_in = SeparateUnicode('\n', config=True)
368 separate_in = SeparateUnicode('\n', config=True)
369 separate_out = SeparateUnicode('', config=True)
369 separate_out = SeparateUnicode('', config=True)
370 separate_out2 = SeparateUnicode('', config=True)
370 separate_out2 = SeparateUnicode('', config=True)
371 wildcards_case_sensitive = CBool(True, config=True)
371 wildcards_case_sensitive = CBool(True, config=True)
372 xmode = CaselessStrEnum(('Context','Plain', 'Verbose'),
372 xmode = CaselessStrEnum(('Context','Plain', 'Verbose'),
373 default_value='Context', config=True)
373 default_value='Context', config=True)
374
374
375 # Subcomponents of InteractiveShell
375 # Subcomponents of InteractiveShell
376 alias_manager = Instance('IPython.core.alias.AliasManager')
376 alias_manager = Instance('IPython.core.alias.AliasManager')
377 prefilter_manager = Instance('IPython.core.prefilter.PrefilterManager')
377 prefilter_manager = Instance('IPython.core.prefilter.PrefilterManager')
378 builtin_trap = Instance('IPython.core.builtin_trap.BuiltinTrap')
378 builtin_trap = Instance('IPython.core.builtin_trap.BuiltinTrap')
379 display_trap = Instance('IPython.core.display_trap.DisplayTrap')
379 display_trap = Instance('IPython.core.display_trap.DisplayTrap')
380 extension_manager = Instance('IPython.core.extensions.ExtensionManager')
380 extension_manager = Instance('IPython.core.extensions.ExtensionManager')
381 plugin_manager = Instance('IPython.core.plugin.PluginManager')
381 plugin_manager = Instance('IPython.core.plugin.PluginManager')
382 payload_manager = Instance('IPython.core.payload.PayloadManager')
382 payload_manager = Instance('IPython.core.payload.PayloadManager')
383 history_manager = Instance('IPython.core.history.HistoryManager')
383 history_manager = Instance('IPython.core.history.HistoryManager')
384 magics_manager = Instance('IPython.core.magic.MagicsManager')
384 magics_manager = Instance('IPython.core.magic.MagicsManager')
385
385
386 profile_dir = Instance('IPython.core.application.ProfileDir')
386 profile_dir = Instance('IPython.core.application.ProfileDir')
387 @property
387 @property
388 def profile(self):
388 def profile(self):
389 if self.profile_dir is not None:
389 if self.profile_dir is not None:
390 name = os.path.basename(self.profile_dir.location)
390 name = os.path.basename(self.profile_dir.location)
391 return name.replace('profile_','')
391 return name.replace('profile_','')
392
392
393
393
394 # Private interface
394 # Private interface
395 _post_execute = Instance(dict)
395 _post_execute = Instance(dict)
396
396
397 def __init__(self, config=None, ipython_dir=None, profile_dir=None,
397 def __init__(self, config=None, ipython_dir=None, profile_dir=None,
398 user_module=None, user_ns=None,
398 user_module=None, user_ns=None,
399 custom_exceptions=((), None)):
399 custom_exceptions=((), None)):
400
400
401 # This is where traits with a config_key argument are updated
401 # This is where traits with a config_key argument are updated
402 # from the values on config.
402 # from the values on config.
403 super(InteractiveShell, self).__init__(config=config)
403 super(InteractiveShell, self).__init__(config=config)
404 self.configurables = [self]
404 self.configurables = [self]
405
405
406 # These are relatively independent and stateless
406 # These are relatively independent and stateless
407 self.init_ipython_dir(ipython_dir)
407 self.init_ipython_dir(ipython_dir)
408 self.init_profile_dir(profile_dir)
408 self.init_profile_dir(profile_dir)
409 self.init_instance_attrs()
409 self.init_instance_attrs()
410 self.init_environment()
410 self.init_environment()
411
411
412 # Check if we're in a virtualenv, and set up sys.path.
412 # Check if we're in a virtualenv, and set up sys.path.
413 self.init_virtualenv()
413 self.init_virtualenv()
414
414
415 # Create namespaces (user_ns, user_global_ns, etc.)
415 # Create namespaces (user_ns, user_global_ns, etc.)
416 self.init_create_namespaces(user_module, user_ns)
416 self.init_create_namespaces(user_module, user_ns)
417 # This has to be done after init_create_namespaces because it uses
417 # This has to be done after init_create_namespaces because it uses
418 # something in self.user_ns, but before init_sys_modules, which
418 # something in self.user_ns, but before init_sys_modules, which
419 # is the first thing to modify sys.
419 # is the first thing to modify sys.
420 # TODO: When we override sys.stdout and sys.stderr before this class
420 # TODO: When we override sys.stdout and sys.stderr before this class
421 # is created, we are saving the overridden ones here. Not sure if this
421 # is created, we are saving the overridden ones here. Not sure if this
422 # is what we want to do.
422 # is what we want to do.
423 self.save_sys_module_state()
423 self.save_sys_module_state()
424 self.init_sys_modules()
424 self.init_sys_modules()
425
425
426 # While we're trying to have each part of the code directly access what
426 # While we're trying to have each part of the code directly access what
427 # it needs without keeping redundant references to objects, we have too
427 # it needs without keeping redundant references to objects, we have too
428 # much legacy code that expects ip.db to exist.
428 # much legacy code that expects ip.db to exist.
429 self.db = PickleShareDB(os.path.join(self.profile_dir.location, 'db'))
429 self.db = PickleShareDB(os.path.join(self.profile_dir.location, 'db'))
430
430
431 self.init_history()
431 self.init_history()
432 self.init_encoding()
432 self.init_encoding()
433 self.init_prefilter()
433 self.init_prefilter()
434
434
435 self.init_syntax_highlighting()
435 self.init_syntax_highlighting()
436 self.init_hooks()
436 self.init_hooks()
437 self.init_pushd_popd_magic()
437 self.init_pushd_popd_magic()
438 # self.init_traceback_handlers use to be here, but we moved it below
438 # self.init_traceback_handlers use to be here, but we moved it below
439 # because it and init_io have to come after init_readline.
439 # because it and init_io have to come after init_readline.
440 self.init_user_ns()
440 self.init_user_ns()
441 self.init_logger()
441 self.init_logger()
442 self.init_alias()
442 self.init_alias()
443 self.init_builtins()
443 self.init_builtins()
444
444
445 # pre_config_initialization
445 # pre_config_initialization
446
446
447 # The next section should contain everything that was in ipmaker.
447 # The next section should contain everything that was in ipmaker.
448 self.init_logstart()
448 self.init_logstart()
449
449
450 # The following was in post_config_initialization
450 # The following was in post_config_initialization
451 self.init_inspector()
451 self.init_inspector()
452 # init_readline() must come before init_io(), because init_io uses
452 # init_readline() must come before init_io(), because init_io uses
453 # readline related things.
453 # readline related things.
454 self.init_readline()
454 self.init_readline()
455 # We save this here in case user code replaces raw_input, but it needs
455 # We save this here in case user code replaces raw_input, but it needs
456 # to be after init_readline(), because PyPy's readline works by replacing
456 # to be after init_readline(), because PyPy's readline works by replacing
457 # raw_input.
457 # raw_input.
458 if py3compat.PY3:
458 if py3compat.PY3:
459 self.raw_input_original = input
459 self.raw_input_original = input
460 else:
460 else:
461 self.raw_input_original = raw_input
461 self.raw_input_original = raw_input
462 # init_completer must come after init_readline, because it needs to
462 # init_completer must come after init_readline, because it needs to
463 # know whether readline is present or not system-wide to configure the
463 # know whether readline is present or not system-wide to configure the
464 # completers, since the completion machinery can now operate
464 # completers, since the completion machinery can now operate
465 # independently of readline (e.g. over the network)
465 # independently of readline (e.g. over the network)
466 self.init_completer()
466 self.init_completer()
467 # TODO: init_io() needs to happen before init_traceback handlers
467 # TODO: init_io() needs to happen before init_traceback handlers
468 # because the traceback handlers hardcode the stdout/stderr streams.
468 # because the traceback handlers hardcode the stdout/stderr streams.
469 # This logic in in debugger.Pdb and should eventually be changed.
469 # This logic in in debugger.Pdb and should eventually be changed.
470 self.init_io()
470 self.init_io()
471 self.init_traceback_handlers(custom_exceptions)
471 self.init_traceback_handlers(custom_exceptions)
472 self.init_prompts()
472 self.init_prompts()
473 self.init_display_formatter()
473 self.init_display_formatter()
474 self.init_display_pub()
474 self.init_display_pub()
475 self.init_displayhook()
475 self.init_displayhook()
476 self.init_reload_doctest()
476 self.init_reload_doctest()
477 self.init_magics()
477 self.init_magics()
478 self.init_pdb()
478 self.init_pdb()
479 self.init_extension_manager()
479 self.init_extension_manager()
480 self.init_plugin_manager()
480 self.init_plugin_manager()
481 self.init_payload()
481 self.init_payload()
482 self.hooks.late_startup_hook()
482 self.hooks.late_startup_hook()
483 atexit.register(self.atexit_operations)
483 atexit.register(self.atexit_operations)
484
484
485 def get_ipython(self):
485 def get_ipython(self):
486 """Return the currently running IPython instance."""
486 """Return the currently running IPython instance."""
487 return self
487 return self
488
488
489 #-------------------------------------------------------------------------
489 #-------------------------------------------------------------------------
490 # Trait changed handlers
490 # Trait changed handlers
491 #-------------------------------------------------------------------------
491 #-------------------------------------------------------------------------
492
492
493 def _ipython_dir_changed(self, name, new):
493 def _ipython_dir_changed(self, name, new):
494 if not os.path.isdir(new):
494 if not os.path.isdir(new):
495 os.makedirs(new, mode = 0777)
495 os.makedirs(new, mode = 0777)
496
496
497 def set_autoindent(self,value=None):
497 def set_autoindent(self,value=None):
498 """Set the autoindent flag, checking for readline support.
498 """Set the autoindent flag, checking for readline support.
499
499
500 If called with no arguments, it acts as a toggle."""
500 If called with no arguments, it acts as a toggle."""
501
501
502 if value != 0 and not self.has_readline:
502 if value != 0 and not self.has_readline:
503 if os.name == 'posix':
503 if os.name == 'posix':
504 warn("The auto-indent feature requires the readline library")
504 warn("The auto-indent feature requires the readline library")
505 self.autoindent = 0
505 self.autoindent = 0
506 return
506 return
507 if value is None:
507 if value is None:
508 self.autoindent = not self.autoindent
508 self.autoindent = not self.autoindent
509 else:
509 else:
510 self.autoindent = value
510 self.autoindent = value
511
511
512 #-------------------------------------------------------------------------
512 #-------------------------------------------------------------------------
513 # init_* methods called by __init__
513 # init_* methods called by __init__
514 #-------------------------------------------------------------------------
514 #-------------------------------------------------------------------------
515
515
516 def init_ipython_dir(self, ipython_dir):
516 def init_ipython_dir(self, ipython_dir):
517 if ipython_dir is not None:
517 if ipython_dir is not None:
518 self.ipython_dir = ipython_dir
518 self.ipython_dir = ipython_dir
519 return
519 return
520
520
521 self.ipython_dir = get_ipython_dir()
521 self.ipython_dir = get_ipython_dir()
522
522
523 def init_profile_dir(self, profile_dir):
523 def init_profile_dir(self, profile_dir):
524 if profile_dir is not None:
524 if profile_dir is not None:
525 self.profile_dir = profile_dir
525 self.profile_dir = profile_dir
526 return
526 return
527 self.profile_dir =\
527 self.profile_dir =\
528 ProfileDir.create_profile_dir_by_name(self.ipython_dir, 'default')
528 ProfileDir.create_profile_dir_by_name(self.ipython_dir, 'default')
529
529
530 def init_instance_attrs(self):
530 def init_instance_attrs(self):
531 self.more = False
531 self.more = False
532
532
533 # command compiler
533 # command compiler
534 self.compile = CachingCompiler()
534 self.compile = CachingCompiler()
535
535
536 # Make an empty namespace, which extension writers can rely on both
536 # Make an empty namespace, which extension writers can rely on both
537 # existing and NEVER being used by ipython itself. This gives them a
537 # existing and NEVER being used by ipython itself. This gives them a
538 # convenient location for storing additional information and state
538 # convenient location for storing additional information and state
539 # their extensions may require, without fear of collisions with other
539 # their extensions may require, without fear of collisions with other
540 # ipython names that may develop later.
540 # ipython names that may develop later.
541 self.meta = Struct()
541 self.meta = Struct()
542
542
543 # Temporary files used for various purposes. Deleted at exit.
543 # Temporary files used for various purposes. Deleted at exit.
544 self.tempfiles = []
544 self.tempfiles = []
545
545
546 # Keep track of readline usage (later set by init_readline)
546 # Keep track of readline usage (later set by init_readline)
547 self.has_readline = False
547 self.has_readline = False
548
548
549 # keep track of where we started running (mainly for crash post-mortem)
549 # keep track of where we started running (mainly for crash post-mortem)
550 # This is not being used anywhere currently.
550 # This is not being used anywhere currently.
551 self.starting_dir = os.getcwdu()
551 self.starting_dir = os.getcwdu()
552
552
553 # Indentation management
553 # Indentation management
554 self.indent_current_nsp = 0
554 self.indent_current_nsp = 0
555
555
556 # Dict to track post-execution functions that have been registered
556 # Dict to track post-execution functions that have been registered
557 self._post_execute = {}
557 self._post_execute = {}
558
558
559 def init_environment(self):
559 def init_environment(self):
560 """Any changes we need to make to the user's environment."""
560 """Any changes we need to make to the user's environment."""
561 pass
561 pass
562
562
563 def init_encoding(self):
563 def init_encoding(self):
564 # Get system encoding at startup time. Certain terminals (like Emacs
564 # Get system encoding at startup time. Certain terminals (like Emacs
565 # under Win32 have it set to None, and we need to have a known valid
565 # under Win32 have it set to None, and we need to have a known valid
566 # encoding to use in the raw_input() method
566 # encoding to use in the raw_input() method
567 try:
567 try:
568 self.stdin_encoding = sys.stdin.encoding or 'ascii'
568 self.stdin_encoding = sys.stdin.encoding or 'ascii'
569 except AttributeError:
569 except AttributeError:
570 self.stdin_encoding = 'ascii'
570 self.stdin_encoding = 'ascii'
571
571
572 def init_syntax_highlighting(self):
572 def init_syntax_highlighting(self):
573 # Python source parser/formatter for syntax highlighting
573 # Python source parser/formatter for syntax highlighting
574 pyformat = PyColorize.Parser().format
574 pyformat = PyColorize.Parser().format
575 self.pycolorize = lambda src: pyformat(src,'str',self.colors)
575 self.pycolorize = lambda src: pyformat(src,'str',self.colors)
576
576
577 def init_pushd_popd_magic(self):
577 def init_pushd_popd_magic(self):
578 # for pushd/popd management
578 # for pushd/popd management
579 self.home_dir = get_home_dir()
579 self.home_dir = get_home_dir()
580
580
581 self.dir_stack = []
581 self.dir_stack = []
582
582
583 def init_logger(self):
583 def init_logger(self):
584 self.logger = Logger(self.home_dir, logfname='ipython_log.py',
584 self.logger = Logger(self.home_dir, logfname='ipython_log.py',
585 logmode='rotate')
585 logmode='rotate')
586
586
587 def init_logstart(self):
587 def init_logstart(self):
588 """Initialize logging in case it was requested at the command line.
588 """Initialize logging in case it was requested at the command line.
589 """
589 """
590 if self.logappend:
590 if self.logappend:
591 self.magic('logstart %s append' % self.logappend)
591 self.magic('logstart %s append' % self.logappend)
592 elif self.logfile:
592 elif self.logfile:
593 self.magic('logstart %' % self.logfile)
593 self.magic('logstart %' % self.logfile)
594 elif self.logstart:
594 elif self.logstart:
595 self.magic('logstart')
595 self.magic('logstart')
596
596
597 def init_builtins(self):
597 def init_builtins(self):
598 # A single, static flag that we set to True. Its presence indicates
598 # A single, static flag that we set to True. Its presence indicates
599 # that an IPython shell has been created, and we make no attempts at
599 # that an IPython shell has been created, and we make no attempts at
600 # removing on exit or representing the existence of more than one
600 # removing on exit or representing the existence of more than one
601 # IPython at a time.
601 # IPython at a time.
602 builtin_mod.__dict__['__IPYTHON__'] = True
602 builtin_mod.__dict__['__IPYTHON__'] = True
603
603
604 # In 0.11 we introduced '__IPYTHON__active' as an integer we'd try to
604 # In 0.11 we introduced '__IPYTHON__active' as an integer we'd try to
605 # manage on enter/exit, but with all our shells it's virtually
605 # manage on enter/exit, but with all our shells it's virtually
606 # impossible to get all the cases right. We're leaving the name in for
606 # impossible to get all the cases right. We're leaving the name in for
607 # those who adapted their codes to check for this flag, but will
607 # those who adapted their codes to check for this flag, but will
608 # eventually remove it after a few more releases.
608 # eventually remove it after a few more releases.
609 builtin_mod.__dict__['__IPYTHON__active'] = \
609 builtin_mod.__dict__['__IPYTHON__active'] = \
610 'Deprecated, check for __IPYTHON__'
610 'Deprecated, check for __IPYTHON__'
611
611
612 self.builtin_trap = BuiltinTrap(shell=self)
612 self.builtin_trap = BuiltinTrap(shell=self)
613
613
614 def init_inspector(self):
614 def init_inspector(self):
615 # Object inspector
615 # Object inspector
616 self.inspector = oinspect.Inspector(oinspect.InspectColors,
616 self.inspector = oinspect.Inspector(oinspect.InspectColors,
617 PyColorize.ANSICodeColors,
617 PyColorize.ANSICodeColors,
618 'NoColor',
618 'NoColor',
619 self.object_info_string_level)
619 self.object_info_string_level)
620
620
621 def init_io(self):
621 def init_io(self):
622 # This will just use sys.stdout and sys.stderr. If you want to
622 # This will just use sys.stdout and sys.stderr. If you want to
623 # override sys.stdout and sys.stderr themselves, you need to do that
623 # override sys.stdout and sys.stderr themselves, you need to do that
624 # *before* instantiating this class, because io holds onto
624 # *before* instantiating this class, because io holds onto
625 # references to the underlying streams.
625 # references to the underlying streams.
626 if sys.platform == 'win32' and self.has_readline:
626 if sys.platform == 'win32' and self.has_readline:
627 io.stdout = io.stderr = io.IOStream(self.readline._outputfile)
627 io.stdout = io.stderr = io.IOStream(self.readline._outputfile)
628 else:
628 else:
629 io.stdout = io.IOStream(sys.stdout)
629 io.stdout = io.IOStream(sys.stdout)
630 io.stderr = io.IOStream(sys.stderr)
630 io.stderr = io.IOStream(sys.stderr)
631
631
632 def init_prompts(self):
632 def init_prompts(self):
633 self.prompt_manager = PromptManager(shell=self, config=self.config)
633 self.prompt_manager = PromptManager(shell=self, config=self.config)
634 self.configurables.append(self.prompt_manager)
634 self.configurables.append(self.prompt_manager)
635 # Set system prompts, so that scripts can decide if they are running
635 # Set system prompts, so that scripts can decide if they are running
636 # interactively.
636 # interactively.
637 sys.ps1 = 'In : '
637 sys.ps1 = 'In : '
638 sys.ps2 = '...: '
638 sys.ps2 = '...: '
639 sys.ps3 = 'Out: '
639 sys.ps3 = 'Out: '
640
640
641 def init_display_formatter(self):
641 def init_display_formatter(self):
642 self.display_formatter = DisplayFormatter(config=self.config)
642 self.display_formatter = DisplayFormatter(config=self.config)
643 self.configurables.append(self.display_formatter)
643 self.configurables.append(self.display_formatter)
644
644
645 def init_display_pub(self):
645 def init_display_pub(self):
646 self.display_pub = self.display_pub_class(config=self.config)
646 self.display_pub = self.display_pub_class(config=self.config)
647 self.configurables.append(self.display_pub)
647 self.configurables.append(self.display_pub)
648
648
649 def init_displayhook(self):
649 def init_displayhook(self):
650 # Initialize displayhook, set in/out prompts and printing system
650 # Initialize displayhook, set in/out prompts and printing system
651 self.displayhook = self.displayhook_class(
651 self.displayhook = self.displayhook_class(
652 config=self.config,
652 config=self.config,
653 shell=self,
653 shell=self,
654 cache_size=self.cache_size,
654 cache_size=self.cache_size,
655 )
655 )
656 self.configurables.append(self.displayhook)
656 self.configurables.append(self.displayhook)
657 # This is a context manager that installs/revmoes the displayhook at
657 # This is a context manager that installs/revmoes the displayhook at
658 # the appropriate time.
658 # the appropriate time.
659 self.display_trap = DisplayTrap(hook=self.displayhook)
659 self.display_trap = DisplayTrap(hook=self.displayhook)
660
660
661 def init_reload_doctest(self):
661 def init_reload_doctest(self):
662 # Do a proper resetting of doctest, including the necessary displayhook
662 # Do a proper resetting of doctest, including the necessary displayhook
663 # monkeypatching
663 # monkeypatching
664 try:
664 try:
665 doctest_reload()
665 doctest_reload()
666 except ImportError:
666 except ImportError:
667 warn("doctest module does not exist.")
667 warn("doctest module does not exist.")
668
668
669 def init_virtualenv(self):
669 def init_virtualenv(self):
670 """Add a virtualenv to sys.path so the user can import modules from it.
670 """Add a virtualenv to sys.path so the user can import modules from it.
671 This isn't perfect: it doesn't use the Python interpreter with which the
671 This isn't perfect: it doesn't use the Python interpreter with which the
672 virtualenv was built, and it ignores the --no-site-packages option. A
672 virtualenv was built, and it ignores the --no-site-packages option. A
673 warning will appear suggesting the user installs IPython in the
673 warning will appear suggesting the user installs IPython in the
674 virtualenv, but for many cases, it probably works well enough.
674 virtualenv, but for many cases, it probably works well enough.
675
675
676 Adapted from code snippets online.
676 Adapted from code snippets online.
677
677
678 http://blog.ufsoft.org/2009/1/29/ipython-and-virtualenv
678 http://blog.ufsoft.org/2009/1/29/ipython-and-virtualenv
679 """
679 """
680 if 'VIRTUAL_ENV' not in os.environ:
680 if 'VIRTUAL_ENV' not in os.environ:
681 # Not in a virtualenv
681 # Not in a virtualenv
682 return
682 return
683
683
684 if sys.executable.startswith(os.environ['VIRTUAL_ENV']):
684 if sys.executable.startswith(os.environ['VIRTUAL_ENV']):
685 # Running properly in the virtualenv, don't need to do anything
685 # Running properly in the virtualenv, don't need to do anything
686 return
686 return
687
687
688 warn("Attempting to work in a virtualenv. If you encounter problems, please "
688 warn("Attempting to work in a virtualenv. If you encounter problems, please "
689 "install IPython inside the virtualenv.\n")
689 "install IPython inside the virtualenv.\n")
690 if sys.platform == "win32":
690 if sys.platform == "win32":
691 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'Lib', 'site-packages')
691 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'Lib', 'site-packages')
692 else:
692 else:
693 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'lib',
693 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'lib',
694 'python%d.%d' % sys.version_info[:2], 'site-packages')
694 'python%d.%d' % sys.version_info[:2], 'site-packages')
695
695
696 import site
696 import site
697 sys.path.insert(0, virtual_env)
697 sys.path.insert(0, virtual_env)
698 site.addsitedir(virtual_env)
698 site.addsitedir(virtual_env)
699
699
700 #-------------------------------------------------------------------------
700 #-------------------------------------------------------------------------
701 # Things related to injections into the sys module
701 # Things related to injections into the sys module
702 #-------------------------------------------------------------------------
702 #-------------------------------------------------------------------------
703
703
704 def save_sys_module_state(self):
704 def save_sys_module_state(self):
705 """Save the state of hooks in the sys module.
705 """Save the state of hooks in the sys module.
706
706
707 This has to be called after self.user_module is created.
707 This has to be called after self.user_module is created.
708 """
708 """
709 self._orig_sys_module_state = {}
709 self._orig_sys_module_state = {}
710 self._orig_sys_module_state['stdin'] = sys.stdin
710 self._orig_sys_module_state['stdin'] = sys.stdin
711 self._orig_sys_module_state['stdout'] = sys.stdout
711 self._orig_sys_module_state['stdout'] = sys.stdout
712 self._orig_sys_module_state['stderr'] = sys.stderr
712 self._orig_sys_module_state['stderr'] = sys.stderr
713 self._orig_sys_module_state['excepthook'] = sys.excepthook
713 self._orig_sys_module_state['excepthook'] = sys.excepthook
714 self._orig_sys_modules_main_name = self.user_module.__name__
714 self._orig_sys_modules_main_name = self.user_module.__name__
715 self._orig_sys_modules_main_mod = sys.modules.get(self.user_module.__name__)
715 self._orig_sys_modules_main_mod = sys.modules.get(self.user_module.__name__)
716
716
717 def restore_sys_module_state(self):
717 def restore_sys_module_state(self):
718 """Restore the state of the sys module."""
718 """Restore the state of the sys module."""
719 try:
719 try:
720 for k, v in self._orig_sys_module_state.iteritems():
720 for k, v in self._orig_sys_module_state.iteritems():
721 setattr(sys, k, v)
721 setattr(sys, k, v)
722 except AttributeError:
722 except AttributeError:
723 pass
723 pass
724 # Reset what what done in self.init_sys_modules
724 # Reset what what done in self.init_sys_modules
725 if self._orig_sys_modules_main_mod is not None:
725 if self._orig_sys_modules_main_mod is not None:
726 sys.modules[self._orig_sys_modules_main_name] = self._orig_sys_modules_main_mod
726 sys.modules[self._orig_sys_modules_main_name] = self._orig_sys_modules_main_mod
727
727
728 #-------------------------------------------------------------------------
728 #-------------------------------------------------------------------------
729 # Things related to hooks
729 # Things related to hooks
730 #-------------------------------------------------------------------------
730 #-------------------------------------------------------------------------
731
731
732 def init_hooks(self):
732 def init_hooks(self):
733 # hooks holds pointers used for user-side customizations
733 # hooks holds pointers used for user-side customizations
734 self.hooks = Struct()
734 self.hooks = Struct()
735
735
736 self.strdispatchers = {}
736 self.strdispatchers = {}
737
737
738 # Set all default hooks, defined in the IPython.hooks module.
738 # Set all default hooks, defined in the IPython.hooks module.
739 hooks = IPython.core.hooks
739 hooks = IPython.core.hooks
740 for hook_name in hooks.__all__:
740 for hook_name in hooks.__all__:
741 # default hooks have priority 100, i.e. low; user hooks should have
741 # default hooks have priority 100, i.e. low; user hooks should have
742 # 0-100 priority
742 # 0-100 priority
743 self.set_hook(hook_name,getattr(hooks,hook_name), 100)
743 self.set_hook(hook_name,getattr(hooks,hook_name), 100)
744
744
745 def set_hook(self,name,hook, priority = 50, str_key = None, re_key = None):
745 def set_hook(self,name,hook, priority = 50, str_key = None, re_key = None):
746 """set_hook(name,hook) -> sets an internal IPython hook.
746 """set_hook(name,hook) -> sets an internal IPython hook.
747
747
748 IPython exposes some of its internal API as user-modifiable hooks. By
748 IPython exposes some of its internal API as user-modifiable hooks. By
749 adding your function to one of these hooks, you can modify IPython's
749 adding your function to one of these hooks, you can modify IPython's
750 behavior to call at runtime your own routines."""
750 behavior to call at runtime your own routines."""
751
751
752 # At some point in the future, this should validate the hook before it
752 # At some point in the future, this should validate the hook before it
753 # accepts it. Probably at least check that the hook takes the number
753 # accepts it. Probably at least check that the hook takes the number
754 # of args it's supposed to.
754 # of args it's supposed to.
755
755
756 f = types.MethodType(hook,self)
756 f = types.MethodType(hook,self)
757
757
758 # check if the hook is for strdispatcher first
758 # check if the hook is for strdispatcher first
759 if str_key is not None:
759 if str_key is not None:
760 sdp = self.strdispatchers.get(name, StrDispatch())
760 sdp = self.strdispatchers.get(name, StrDispatch())
761 sdp.add_s(str_key, f, priority )
761 sdp.add_s(str_key, f, priority )
762 self.strdispatchers[name] = sdp
762 self.strdispatchers[name] = sdp
763 return
763 return
764 if re_key is not None:
764 if re_key is not None:
765 sdp = self.strdispatchers.get(name, StrDispatch())
765 sdp = self.strdispatchers.get(name, StrDispatch())
766 sdp.add_re(re.compile(re_key), f, priority )
766 sdp.add_re(re.compile(re_key), f, priority )
767 self.strdispatchers[name] = sdp
767 self.strdispatchers[name] = sdp
768 return
768 return
769
769
770 dp = getattr(self.hooks, name, None)
770 dp = getattr(self.hooks, name, None)
771 if name not in IPython.core.hooks.__all__:
771 if name not in IPython.core.hooks.__all__:
772 print "Warning! Hook '%s' is not one of %s" % \
772 print "Warning! Hook '%s' is not one of %s" % \
773 (name, IPython.core.hooks.__all__ )
773 (name, IPython.core.hooks.__all__ )
774 if not dp:
774 if not dp:
775 dp = IPython.core.hooks.CommandChainDispatcher()
775 dp = IPython.core.hooks.CommandChainDispatcher()
776
776
777 try:
777 try:
778 dp.add(f,priority)
778 dp.add(f,priority)
779 except AttributeError:
779 except AttributeError:
780 # it was not commandchain, plain old func - replace
780 # it was not commandchain, plain old func - replace
781 dp = f
781 dp = f
782
782
783 setattr(self.hooks,name, dp)
783 setattr(self.hooks,name, dp)
784
784
785 def register_post_execute(self, func):
785 def register_post_execute(self, func):
786 """Register a function for calling after code execution.
786 """Register a function for calling after code execution.
787 """
787 """
788 if not callable(func):
788 if not callable(func):
789 raise ValueError('argument %s must be callable' % func)
789 raise ValueError('argument %s must be callable' % func)
790 self._post_execute[func] = True
790 self._post_execute[func] = True
791
791
792 #-------------------------------------------------------------------------
792 #-------------------------------------------------------------------------
793 # Things related to the "main" module
793 # Things related to the "main" module
794 #-------------------------------------------------------------------------
794 #-------------------------------------------------------------------------
795
795
796 def new_main_mod(self,ns=None):
796 def new_main_mod(self,ns=None):
797 """Return a new 'main' module object for user code execution.
797 """Return a new 'main' module object for user code execution.
798 """
798 """
799 main_mod = self._user_main_module
799 main_mod = self._user_main_module
800 init_fakemod_dict(main_mod,ns)
800 init_fakemod_dict(main_mod,ns)
801 return main_mod
801 return main_mod
802
802
803 def cache_main_mod(self,ns,fname):
803 def cache_main_mod(self,ns,fname):
804 """Cache a main module's namespace.
804 """Cache a main module's namespace.
805
805
806 When scripts are executed via %run, we must keep a reference to the
806 When scripts are executed via %run, we must keep a reference to the
807 namespace of their __main__ module (a FakeModule instance) around so
807 namespace of their __main__ module (a FakeModule instance) around so
808 that Python doesn't clear it, rendering objects defined therein
808 that Python doesn't clear it, rendering objects defined therein
809 useless.
809 useless.
810
810
811 This method keeps said reference in a private dict, keyed by the
811 This method keeps said reference in a private dict, keyed by the
812 absolute path of the module object (which corresponds to the script
812 absolute path of the module object (which corresponds to the script
813 path). This way, for multiple executions of the same script we only
813 path). This way, for multiple executions of the same script we only
814 keep one copy of the namespace (the last one), thus preventing memory
814 keep one copy of the namespace (the last one), thus preventing memory
815 leaks from old references while allowing the objects from the last
815 leaks from old references while allowing the objects from the last
816 execution to be accessible.
816 execution to be accessible.
817
817
818 Note: we can not allow the actual FakeModule instances to be deleted,
818 Note: we can not allow the actual FakeModule instances to be deleted,
819 because of how Python tears down modules (it hard-sets all their
819 because of how Python tears down modules (it hard-sets all their
820 references to None without regard for reference counts). This method
820 references to None without regard for reference counts). This method
821 must therefore make a *copy* of the given namespace, to allow the
821 must therefore make a *copy* of the given namespace, to allow the
822 original module's __dict__ to be cleared and reused.
822 original module's __dict__ to be cleared and reused.
823
823
824
824
825 Parameters
825 Parameters
826 ----------
826 ----------
827 ns : a namespace (a dict, typically)
827 ns : a namespace (a dict, typically)
828
828
829 fname : str
829 fname : str
830 Filename associated with the namespace.
830 Filename associated with the namespace.
831
831
832 Examples
832 Examples
833 --------
833 --------
834
834
835 In [10]: import IPython
835 In [10]: import IPython
836
836
837 In [11]: _ip.cache_main_mod(IPython.__dict__,IPython.__file__)
837 In [11]: _ip.cache_main_mod(IPython.__dict__,IPython.__file__)
838
838
839 In [12]: IPython.__file__ in _ip._main_ns_cache
839 In [12]: IPython.__file__ in _ip._main_ns_cache
840 Out[12]: True
840 Out[12]: True
841 """
841 """
842 self._main_ns_cache[os.path.abspath(fname)] = ns.copy()
842 self._main_ns_cache[os.path.abspath(fname)] = ns.copy()
843
843
844 def clear_main_mod_cache(self):
844 def clear_main_mod_cache(self):
845 """Clear the cache of main modules.
845 """Clear the cache of main modules.
846
846
847 Mainly for use by utilities like %reset.
847 Mainly for use by utilities like %reset.
848
848
849 Examples
849 Examples
850 --------
850 --------
851
851
852 In [15]: import IPython
852 In [15]: import IPython
853
853
854 In [16]: _ip.cache_main_mod(IPython.__dict__,IPython.__file__)
854 In [16]: _ip.cache_main_mod(IPython.__dict__,IPython.__file__)
855
855
856 In [17]: len(_ip._main_ns_cache) > 0
856 In [17]: len(_ip._main_ns_cache) > 0
857 Out[17]: True
857 Out[17]: True
858
858
859 In [18]: _ip.clear_main_mod_cache()
859 In [18]: _ip.clear_main_mod_cache()
860
860
861 In [19]: len(_ip._main_ns_cache) == 0
861 In [19]: len(_ip._main_ns_cache) == 0
862 Out[19]: True
862 Out[19]: True
863 """
863 """
864 self._main_ns_cache.clear()
864 self._main_ns_cache.clear()
865
865
866 #-------------------------------------------------------------------------
866 #-------------------------------------------------------------------------
867 # Things related to debugging
867 # Things related to debugging
868 #-------------------------------------------------------------------------
868 #-------------------------------------------------------------------------
869
869
870 def init_pdb(self):
870 def init_pdb(self):
871 # Set calling of pdb on exceptions
871 # Set calling of pdb on exceptions
872 # self.call_pdb is a property
872 # self.call_pdb is a property
873 self.call_pdb = self.pdb
873 self.call_pdb = self.pdb
874
874
875 def _get_call_pdb(self):
875 def _get_call_pdb(self):
876 return self._call_pdb
876 return self._call_pdb
877
877
878 def _set_call_pdb(self,val):
878 def _set_call_pdb(self,val):
879
879
880 if val not in (0,1,False,True):
880 if val not in (0,1,False,True):
881 raise ValueError,'new call_pdb value must be boolean'
881 raise ValueError,'new call_pdb value must be boolean'
882
882
883 # store value in instance
883 # store value in instance
884 self._call_pdb = val
884 self._call_pdb = val
885
885
886 # notify the actual exception handlers
886 # notify the actual exception handlers
887 self.InteractiveTB.call_pdb = val
887 self.InteractiveTB.call_pdb = val
888
888
889 call_pdb = property(_get_call_pdb,_set_call_pdb,None,
889 call_pdb = property(_get_call_pdb,_set_call_pdb,None,
890 'Control auto-activation of pdb at exceptions')
890 'Control auto-activation of pdb at exceptions')
891
891
892 def debugger(self,force=False):
892 def debugger(self,force=False):
893 """Call the pydb/pdb debugger.
893 """Call the pydb/pdb debugger.
894
894
895 Keywords:
895 Keywords:
896
896
897 - force(False): by default, this routine checks the instance call_pdb
897 - force(False): by default, this routine checks the instance call_pdb
898 flag and does not actually invoke the debugger if the flag is false.
898 flag and does not actually invoke the debugger if the flag is false.
899 The 'force' option forces the debugger to activate even if the flag
899 The 'force' option forces the debugger to activate even if the flag
900 is false.
900 is false.
901 """
901 """
902
902
903 if not (force or self.call_pdb):
903 if not (force or self.call_pdb):
904 return
904 return
905
905
906 if not hasattr(sys,'last_traceback'):
906 if not hasattr(sys,'last_traceback'):
907 error('No traceback has been produced, nothing to debug.')
907 error('No traceback has been produced, nothing to debug.')
908 return
908 return
909
909
910 # use pydb if available
910 # use pydb if available
911 if debugger.has_pydb:
911 if debugger.has_pydb:
912 from pydb import pm
912 from pydb import pm
913 else:
913 else:
914 # fallback to our internal debugger
914 # fallback to our internal debugger
915 pm = lambda : self.InteractiveTB.debugger(force=True)
915 pm = lambda : self.InteractiveTB.debugger(force=True)
916
916
917 with self.readline_no_record:
917 with self.readline_no_record:
918 pm()
918 pm()
919
919
920 #-------------------------------------------------------------------------
920 #-------------------------------------------------------------------------
921 # Things related to IPython's various namespaces
921 # Things related to IPython's various namespaces
922 #-------------------------------------------------------------------------
922 #-------------------------------------------------------------------------
923 default_user_namespaces = True
923 default_user_namespaces = True
924
924
925 def init_create_namespaces(self, user_module=None, user_ns=None):
925 def init_create_namespaces(self, user_module=None, user_ns=None):
926 # Create the namespace where the user will operate. user_ns is
926 # Create the namespace where the user will operate. user_ns is
927 # normally the only one used, and it is passed to the exec calls as
927 # normally the only one used, and it is passed to the exec calls as
928 # the locals argument. But we do carry a user_global_ns namespace
928 # the locals argument. But we do carry a user_global_ns namespace
929 # given as the exec 'globals' argument, This is useful in embedding
929 # given as the exec 'globals' argument, This is useful in embedding
930 # situations where the ipython shell opens in a context where the
930 # situations where the ipython shell opens in a context where the
931 # distinction between locals and globals is meaningful. For
931 # distinction between locals and globals is meaningful. For
932 # non-embedded contexts, it is just the same object as the user_ns dict.
932 # non-embedded contexts, it is just the same object as the user_ns dict.
933
933
934 # FIXME. For some strange reason, __builtins__ is showing up at user
934 # FIXME. For some strange reason, __builtins__ is showing up at user
935 # level as a dict instead of a module. This is a manual fix, but I
935 # level as a dict instead of a module. This is a manual fix, but I
936 # should really track down where the problem is coming from. Alex
936 # should really track down where the problem is coming from. Alex
937 # Schmolck reported this problem first.
937 # Schmolck reported this problem first.
938
938
939 # A useful post by Alex Martelli on this topic:
939 # A useful post by Alex Martelli on this topic:
940 # Re: inconsistent value from __builtins__
940 # Re: inconsistent value from __builtins__
941 # Von: Alex Martelli <aleaxit@yahoo.com>
941 # Von: Alex Martelli <aleaxit@yahoo.com>
942 # Datum: Freitag 01 Oktober 2004 04:45:34 nachmittags/abends
942 # Datum: Freitag 01 Oktober 2004 04:45:34 nachmittags/abends
943 # Gruppen: comp.lang.python
943 # Gruppen: comp.lang.python
944
944
945 # Michael Hohn <hohn@hooknose.lbl.gov> wrote:
945 # Michael Hohn <hohn@hooknose.lbl.gov> wrote:
946 # > >>> print type(builtin_check.get_global_binding('__builtins__'))
946 # > >>> print type(builtin_check.get_global_binding('__builtins__'))
947 # > <type 'dict'>
947 # > <type 'dict'>
948 # > >>> print type(__builtins__)
948 # > >>> print type(__builtins__)
949 # > <type 'module'>
949 # > <type 'module'>
950 # > Is this difference in return value intentional?
950 # > Is this difference in return value intentional?
951
951
952 # Well, it's documented that '__builtins__' can be either a dictionary
952 # Well, it's documented that '__builtins__' can be either a dictionary
953 # or a module, and it's been that way for a long time. Whether it's
953 # or a module, and it's been that way for a long time. Whether it's
954 # intentional (or sensible), I don't know. In any case, the idea is
954 # intentional (or sensible), I don't know. In any case, the idea is
955 # that if you need to access the built-in namespace directly, you
955 # that if you need to access the built-in namespace directly, you
956 # should start with "import __builtin__" (note, no 's') which will
956 # should start with "import __builtin__" (note, no 's') which will
957 # definitely give you a module. Yeah, it's somewhat confusing:-(.
957 # definitely give you a module. Yeah, it's somewhat confusing:-(.
958
958
959 # These routines return a properly built module and dict as needed by
959 # These routines return a properly built module and dict as needed by
960 # the rest of the code, and can also be used by extension writers to
960 # the rest of the code, and can also be used by extension writers to
961 # generate properly initialized namespaces.
961 # generate properly initialized namespaces.
962 if (user_ns is not None) or (user_module is not None):
962 if (user_ns is not None) or (user_module is not None):
963 self.default_user_namespaces = False
963 self.default_user_namespaces = False
964 self.user_module, self.user_ns = self.prepare_user_module(user_module, user_ns)
964 self.user_module, self.user_ns = self.prepare_user_module(user_module, user_ns)
965
965
966 # A record of hidden variables we have added to the user namespace, so
966 # A record of hidden variables we have added to the user namespace, so
967 # we can list later only variables defined in actual interactive use.
967 # we can list later only variables defined in actual interactive use.
968 self.user_ns_hidden = set()
968 self.user_ns_hidden = set()
969
969
970 # Now that FakeModule produces a real module, we've run into a nasty
970 # Now that FakeModule produces a real module, we've run into a nasty
971 # problem: after script execution (via %run), the module where the user
971 # problem: after script execution (via %run), the module where the user
972 # code ran is deleted. Now that this object is a true module (needed
972 # code ran is deleted. Now that this object is a true module (needed
973 # so docetst and other tools work correctly), the Python module
973 # so docetst and other tools work correctly), the Python module
974 # teardown mechanism runs over it, and sets to None every variable
974 # teardown mechanism runs over it, and sets to None every variable
975 # present in that module. Top-level references to objects from the
975 # present in that module. Top-level references to objects from the
976 # script survive, because the user_ns is updated with them. However,
976 # script survive, because the user_ns is updated with them. However,
977 # calling functions defined in the script that use other things from
977 # calling functions defined in the script that use other things from
978 # the script will fail, because the function's closure had references
978 # the script will fail, because the function's closure had references
979 # to the original objects, which are now all None. So we must protect
979 # to the original objects, which are now all None. So we must protect
980 # these modules from deletion by keeping a cache.
980 # these modules from deletion by keeping a cache.
981 #
981 #
982 # To avoid keeping stale modules around (we only need the one from the
982 # To avoid keeping stale modules around (we only need the one from the
983 # last run), we use a dict keyed with the full path to the script, so
983 # last run), we use a dict keyed with the full path to the script, so
984 # only the last version of the module is held in the cache. Note,
984 # only the last version of the module is held in the cache. Note,
985 # however, that we must cache the module *namespace contents* (their
985 # however, that we must cache the module *namespace contents* (their
986 # __dict__). Because if we try to cache the actual modules, old ones
986 # __dict__). Because if we try to cache the actual modules, old ones
987 # (uncached) could be destroyed while still holding references (such as
987 # (uncached) could be destroyed while still holding references (such as
988 # those held by GUI objects that tend to be long-lived)>
988 # those held by GUI objects that tend to be long-lived)>
989 #
989 #
990 # The %reset command will flush this cache. See the cache_main_mod()
990 # The %reset command will flush this cache. See the cache_main_mod()
991 # and clear_main_mod_cache() methods for details on use.
991 # and clear_main_mod_cache() methods for details on use.
992
992
993 # This is the cache used for 'main' namespaces
993 # This is the cache used for 'main' namespaces
994 self._main_ns_cache = {}
994 self._main_ns_cache = {}
995 # And this is the single instance of FakeModule whose __dict__ we keep
995 # And this is the single instance of FakeModule whose __dict__ we keep
996 # copying and clearing for reuse on each %run
996 # copying and clearing for reuse on each %run
997 self._user_main_module = FakeModule()
997 self._user_main_module = FakeModule()
998
998
999 # A table holding all the namespaces IPython deals with, so that
999 # A table holding all the namespaces IPython deals with, so that
1000 # introspection facilities can search easily.
1000 # introspection facilities can search easily.
1001 self.ns_table = {'user_global':self.user_module.__dict__,
1001 self.ns_table = {'user_global':self.user_module.__dict__,
1002 'user_local':self.user_ns,
1002 'user_local':self.user_ns,
1003 'builtin':builtin_mod.__dict__
1003 'builtin':builtin_mod.__dict__
1004 }
1004 }
1005
1005
1006 @property
1006 @property
1007 def user_global_ns(self):
1007 def user_global_ns(self):
1008 return self.user_module.__dict__
1008 return self.user_module.__dict__
1009
1009
1010 def prepare_user_module(self, user_module=None, user_ns=None):
1010 def prepare_user_module(self, user_module=None, user_ns=None):
1011 """Prepare the module and namespace in which user code will be run.
1011 """Prepare the module and namespace in which user code will be run.
1012
1012
1013 When IPython is started normally, both parameters are None: a new module
1013 When IPython is started normally, both parameters are None: a new module
1014 is created automatically, and its __dict__ used as the namespace.
1014 is created automatically, and its __dict__ used as the namespace.
1015
1015
1016 If only user_module is provided, its __dict__ is used as the namespace.
1016 If only user_module is provided, its __dict__ is used as the namespace.
1017 If only user_ns is provided, a dummy module is created, and user_ns
1017 If only user_ns is provided, a dummy module is created, and user_ns
1018 becomes the global namespace. If both are provided (as they may be
1018 becomes the global namespace. If both are provided (as they may be
1019 when embedding), user_ns is the local namespace, and user_module
1019 when embedding), user_ns is the local namespace, and user_module
1020 provides the global namespace.
1020 provides the global namespace.
1021
1021
1022 Parameters
1022 Parameters
1023 ----------
1023 ----------
1024 user_module : module, optional
1024 user_module : module, optional
1025 The current user module in which IPython is being run. If None,
1025 The current user module in which IPython is being run. If None,
1026 a clean module will be created.
1026 a clean module will be created.
1027 user_ns : dict, optional
1027 user_ns : dict, optional
1028 A namespace in which to run interactive commands.
1028 A namespace in which to run interactive commands.
1029
1029
1030 Returns
1030 Returns
1031 -------
1031 -------
1032 A tuple of user_module and user_ns, each properly initialised.
1032 A tuple of user_module and user_ns, each properly initialised.
1033 """
1033 """
1034 if user_module is None and user_ns is not None:
1034 if user_module is None and user_ns is not None:
1035 user_ns.setdefault("__name__", "__main__")
1035 user_ns.setdefault("__name__", "__main__")
1036 class DummyMod(object):
1036 class DummyMod(object):
1037 "A dummy module used for IPython's interactive namespace."
1037 "A dummy module used for IPython's interactive namespace."
1038 pass
1038 pass
1039 user_module = DummyMod()
1039 user_module = DummyMod()
1040 user_module.__dict__ = user_ns
1040 user_module.__dict__ = user_ns
1041
1041
1042 if user_module is None:
1042 if user_module is None:
1043 user_module = types.ModuleType("__main__",
1043 user_module = types.ModuleType("__main__",
1044 doc="Automatically created module for IPython interactive environment")
1044 doc="Automatically created module for IPython interactive environment")
1045
1045
1046 # We must ensure that __builtin__ (without the final 's') is always
1046 # We must ensure that __builtin__ (without the final 's') is always
1047 # available and pointing to the __builtin__ *module*. For more details:
1047 # available and pointing to the __builtin__ *module*. For more details:
1048 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1048 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1049 user_module.__dict__.setdefault('__builtin__', builtin_mod)
1049 user_module.__dict__.setdefault('__builtin__', builtin_mod)
1050 user_module.__dict__.setdefault('__builtins__', builtin_mod)
1050 user_module.__dict__.setdefault('__builtins__', builtin_mod)
1051
1051
1052 if user_ns is None:
1052 if user_ns is None:
1053 user_ns = user_module.__dict__
1053 user_ns = user_module.__dict__
1054
1054
1055 return user_module, user_ns
1055 return user_module, user_ns
1056
1056
1057 def init_sys_modules(self):
1057 def init_sys_modules(self):
1058 # We need to insert into sys.modules something that looks like a
1058 # We need to insert into sys.modules something that looks like a
1059 # module but which accesses the IPython namespace, for shelve and
1059 # module but which accesses the IPython namespace, for shelve and
1060 # pickle to work interactively. Normally they rely on getting
1060 # pickle to work interactively. Normally they rely on getting
1061 # everything out of __main__, but for embedding purposes each IPython
1061 # everything out of __main__, but for embedding purposes each IPython
1062 # instance has its own private namespace, so we can't go shoving
1062 # instance has its own private namespace, so we can't go shoving
1063 # everything into __main__.
1063 # everything into __main__.
1064
1064
1065 # note, however, that we should only do this for non-embedded
1065 # note, however, that we should only do this for non-embedded
1066 # ipythons, which really mimic the __main__.__dict__ with their own
1066 # ipythons, which really mimic the __main__.__dict__ with their own
1067 # namespace. Embedded instances, on the other hand, should not do
1067 # namespace. Embedded instances, on the other hand, should not do
1068 # this because they need to manage the user local/global namespaces
1068 # this because they need to manage the user local/global namespaces
1069 # only, but they live within a 'normal' __main__ (meaning, they
1069 # only, but they live within a 'normal' __main__ (meaning, they
1070 # shouldn't overtake the execution environment of the script they're
1070 # shouldn't overtake the execution environment of the script they're
1071 # embedded in).
1071 # embedded in).
1072
1072
1073 # This is overridden in the InteractiveShellEmbed subclass to a no-op.
1073 # This is overridden in the InteractiveShellEmbed subclass to a no-op.
1074 main_name = self.user_module.__name__
1074 main_name = self.user_module.__name__
1075 sys.modules[main_name] = self.user_module
1075 sys.modules[main_name] = self.user_module
1076
1076
1077 def init_user_ns(self):
1077 def init_user_ns(self):
1078 """Initialize all user-visible namespaces to their minimum defaults.
1078 """Initialize all user-visible namespaces to their minimum defaults.
1079
1079
1080 Certain history lists are also initialized here, as they effectively
1080 Certain history lists are also initialized here, as they effectively
1081 act as user namespaces.
1081 act as user namespaces.
1082
1082
1083 Notes
1083 Notes
1084 -----
1084 -----
1085 All data structures here are only filled in, they are NOT reset by this
1085 All data structures here are only filled in, they are NOT reset by this
1086 method. If they were not empty before, data will simply be added to
1086 method. If they were not empty before, data will simply be added to
1087 therm.
1087 therm.
1088 """
1088 """
1089 # This function works in two parts: first we put a few things in
1089 # This function works in two parts: first we put a few things in
1090 # user_ns, and we sync that contents into user_ns_hidden so that these
1090 # user_ns, and we sync that contents into user_ns_hidden so that these
1091 # initial variables aren't shown by %who. After the sync, we add the
1091 # initial variables aren't shown by %who. After the sync, we add the
1092 # rest of what we *do* want the user to see with %who even on a new
1092 # rest of what we *do* want the user to see with %who even on a new
1093 # session (probably nothing, so theye really only see their own stuff)
1093 # session (probably nothing, so theye really only see their own stuff)
1094
1094
1095 # The user dict must *always* have a __builtin__ reference to the
1095 # The user dict must *always* have a __builtin__ reference to the
1096 # Python standard __builtin__ namespace, which must be imported.
1096 # Python standard __builtin__ namespace, which must be imported.
1097 # This is so that certain operations in prompt evaluation can be
1097 # This is so that certain operations in prompt evaluation can be
1098 # reliably executed with builtins. Note that we can NOT use
1098 # reliably executed with builtins. Note that we can NOT use
1099 # __builtins__ (note the 's'), because that can either be a dict or a
1099 # __builtins__ (note the 's'), because that can either be a dict or a
1100 # module, and can even mutate at runtime, depending on the context
1100 # module, and can even mutate at runtime, depending on the context
1101 # (Python makes no guarantees on it). In contrast, __builtin__ is
1101 # (Python makes no guarantees on it). In contrast, __builtin__ is
1102 # always a module object, though it must be explicitly imported.
1102 # always a module object, though it must be explicitly imported.
1103
1103
1104 # For more details:
1104 # For more details:
1105 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1105 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1106 ns = dict()
1106 ns = dict()
1107
1107
1108 # Put 'help' in the user namespace
1108 # Put 'help' in the user namespace
1109 try:
1109 try:
1110 from site import _Helper
1110 from site import _Helper
1111 ns['help'] = _Helper()
1111 ns['help'] = _Helper()
1112 except ImportError:
1112 except ImportError:
1113 warn('help() not available - check site.py')
1113 warn('help() not available - check site.py')
1114
1114
1115 # make global variables for user access to the histories
1115 # make global variables for user access to the histories
1116 ns['_ih'] = self.history_manager.input_hist_parsed
1116 ns['_ih'] = self.history_manager.input_hist_parsed
1117 ns['_oh'] = self.history_manager.output_hist
1117 ns['_oh'] = self.history_manager.output_hist
1118 ns['_dh'] = self.history_manager.dir_hist
1118 ns['_dh'] = self.history_manager.dir_hist
1119
1119
1120 ns['_sh'] = shadowns
1120 ns['_sh'] = shadowns
1121
1121
1122 # user aliases to input and output histories. These shouldn't show up
1122 # user aliases to input and output histories. These shouldn't show up
1123 # in %who, as they can have very large reprs.
1123 # in %who, as they can have very large reprs.
1124 ns['In'] = self.history_manager.input_hist_parsed
1124 ns['In'] = self.history_manager.input_hist_parsed
1125 ns['Out'] = self.history_manager.output_hist
1125 ns['Out'] = self.history_manager.output_hist
1126
1126
1127 # Store myself as the public api!!!
1127 # Store myself as the public api!!!
1128 ns['get_ipython'] = self.get_ipython
1128 ns['get_ipython'] = self.get_ipython
1129
1129
1130 ns['exit'] = self.exiter
1130 ns['exit'] = self.exiter
1131 ns['quit'] = self.exiter
1131 ns['quit'] = self.exiter
1132
1132
1133 # Sync what we've added so far to user_ns_hidden so these aren't seen
1133 # Sync what we've added so far to user_ns_hidden so these aren't seen
1134 # by %who
1134 # by %who
1135 self.user_ns_hidden.update(ns)
1135 self.user_ns_hidden.update(ns)
1136
1136
1137 # Anything put into ns now would show up in %who. Think twice before
1137 # Anything put into ns now would show up in %who. Think twice before
1138 # putting anything here, as we really want %who to show the user their
1138 # putting anything here, as we really want %who to show the user their
1139 # stuff, not our variables.
1139 # stuff, not our variables.
1140
1140
1141 # Finally, update the real user's namespace
1141 # Finally, update the real user's namespace
1142 self.user_ns.update(ns)
1142 self.user_ns.update(ns)
1143
1143
1144 @property
1144 @property
1145 def all_ns_refs(self):
1145 def all_ns_refs(self):
1146 """Get a list of references to all the namespace dictionaries in which
1146 """Get a list of references to all the namespace dictionaries in which
1147 IPython might store a user-created object.
1147 IPython might store a user-created object.
1148
1148
1149 Note that this does not include the displayhook, which also caches
1149 Note that this does not include the displayhook, which also caches
1150 objects from the output."""
1150 objects from the output."""
1151 return [self.user_ns, self.user_global_ns,
1151 return [self.user_ns, self.user_global_ns,
1152 self._user_main_module.__dict__] + self._main_ns_cache.values()
1152 self._user_main_module.__dict__] + self._main_ns_cache.values()
1153
1153
1154 def reset(self, new_session=True):
1154 def reset(self, new_session=True):
1155 """Clear all internal namespaces, and attempt to release references to
1155 """Clear all internal namespaces, and attempt to release references to
1156 user objects.
1156 user objects.
1157
1157
1158 If new_session is True, a new history session will be opened.
1158 If new_session is True, a new history session will be opened.
1159 """
1159 """
1160 # Clear histories
1160 # Clear histories
1161 self.history_manager.reset(new_session)
1161 self.history_manager.reset(new_session)
1162 # Reset counter used to index all histories
1162 # Reset counter used to index all histories
1163 if new_session:
1163 if new_session:
1164 self.execution_count = 1
1164 self.execution_count = 1
1165
1165
1166 # Flush cached output items
1166 # Flush cached output items
1167 if self.displayhook.do_full_cache:
1167 if self.displayhook.do_full_cache:
1168 self.displayhook.flush()
1168 self.displayhook.flush()
1169
1169
1170 # The main execution namespaces must be cleared very carefully,
1170 # The main execution namespaces must be cleared very carefully,
1171 # skipping the deletion of the builtin-related keys, because doing so
1171 # skipping the deletion of the builtin-related keys, because doing so
1172 # would cause errors in many object's __del__ methods.
1172 # would cause errors in many object's __del__ methods.
1173 if self.user_ns is not self.user_global_ns:
1173 if self.user_ns is not self.user_global_ns:
1174 self.user_ns.clear()
1174 self.user_ns.clear()
1175 ns = self.user_global_ns
1175 ns = self.user_global_ns
1176 drop_keys = set(ns.keys())
1176 drop_keys = set(ns.keys())
1177 drop_keys.discard('__builtin__')
1177 drop_keys.discard('__builtin__')
1178 drop_keys.discard('__builtins__')
1178 drop_keys.discard('__builtins__')
1179 drop_keys.discard('__name__')
1179 drop_keys.discard('__name__')
1180 for k in drop_keys:
1180 for k in drop_keys:
1181 del ns[k]
1181 del ns[k]
1182
1182
1183 self.user_ns_hidden.clear()
1183 self.user_ns_hidden.clear()
1184
1184
1185 # Restore the user namespaces to minimal usability
1185 # Restore the user namespaces to minimal usability
1186 self.init_user_ns()
1186 self.init_user_ns()
1187
1187
1188 # Restore the default and user aliases
1188 # Restore the default and user aliases
1189 self.alias_manager.clear_aliases()
1189 self.alias_manager.clear_aliases()
1190 self.alias_manager.init_aliases()
1190 self.alias_manager.init_aliases()
1191
1191
1192 # Flush the private list of module references kept for script
1192 # Flush the private list of module references kept for script
1193 # execution protection
1193 # execution protection
1194 self.clear_main_mod_cache()
1194 self.clear_main_mod_cache()
1195
1195
1196 # Clear out the namespace from the last %run
1196 # Clear out the namespace from the last %run
1197 self.new_main_mod()
1197 self.new_main_mod()
1198
1198
1199 def del_var(self, varname, by_name=False):
1199 def del_var(self, varname, by_name=False):
1200 """Delete a variable from the various namespaces, so that, as
1200 """Delete a variable from the various namespaces, so that, as
1201 far as possible, we're not keeping any hidden references to it.
1201 far as possible, we're not keeping any hidden references to it.
1202
1202
1203 Parameters
1203 Parameters
1204 ----------
1204 ----------
1205 varname : str
1205 varname : str
1206 The name of the variable to delete.
1206 The name of the variable to delete.
1207 by_name : bool
1207 by_name : bool
1208 If True, delete variables with the given name in each
1208 If True, delete variables with the given name in each
1209 namespace. If False (default), find the variable in the user
1209 namespace. If False (default), find the variable in the user
1210 namespace, and delete references to it.
1210 namespace, and delete references to it.
1211 """
1211 """
1212 if varname in ('__builtin__', '__builtins__'):
1212 if varname in ('__builtin__', '__builtins__'):
1213 raise ValueError("Refusing to delete %s" % varname)
1213 raise ValueError("Refusing to delete %s" % varname)
1214
1214
1215 ns_refs = self.all_ns_refs
1215 ns_refs = self.all_ns_refs
1216
1216
1217 if by_name: # Delete by name
1217 if by_name: # Delete by name
1218 for ns in ns_refs:
1218 for ns in ns_refs:
1219 try:
1219 try:
1220 del ns[varname]
1220 del ns[varname]
1221 except KeyError:
1221 except KeyError:
1222 pass
1222 pass
1223 else: # Delete by object
1223 else: # Delete by object
1224 try:
1224 try:
1225 obj = self.user_ns[varname]
1225 obj = self.user_ns[varname]
1226 except KeyError:
1226 except KeyError:
1227 raise NameError("name '%s' is not defined" % varname)
1227 raise NameError("name '%s' is not defined" % varname)
1228 # Also check in output history
1228 # Also check in output history
1229 ns_refs.append(self.history_manager.output_hist)
1229 ns_refs.append(self.history_manager.output_hist)
1230 for ns in ns_refs:
1230 for ns in ns_refs:
1231 to_delete = [n for n, o in ns.iteritems() if o is obj]
1231 to_delete = [n for n, o in ns.iteritems() if o is obj]
1232 for name in to_delete:
1232 for name in to_delete:
1233 del ns[name]
1233 del ns[name]
1234
1234
1235 # displayhook keeps extra references, but not in a dictionary
1235 # displayhook keeps extra references, but not in a dictionary
1236 for name in ('_', '__', '___'):
1236 for name in ('_', '__', '___'):
1237 if getattr(self.displayhook, name) is obj:
1237 if getattr(self.displayhook, name) is obj:
1238 setattr(self.displayhook, name, None)
1238 setattr(self.displayhook, name, None)
1239
1239
1240 def reset_selective(self, regex=None):
1240 def reset_selective(self, regex=None):
1241 """Clear selective variables from internal namespaces based on a
1241 """Clear selective variables from internal namespaces based on a
1242 specified regular expression.
1242 specified regular expression.
1243
1243
1244 Parameters
1244 Parameters
1245 ----------
1245 ----------
1246 regex : string or compiled pattern, optional
1246 regex : string or compiled pattern, optional
1247 A regular expression pattern that will be used in searching
1247 A regular expression pattern that will be used in searching
1248 variable names in the users namespaces.
1248 variable names in the users namespaces.
1249 """
1249 """
1250 if regex is not None:
1250 if regex is not None:
1251 try:
1251 try:
1252 m = re.compile(regex)
1252 m = re.compile(regex)
1253 except TypeError:
1253 except TypeError:
1254 raise TypeError('regex must be a string or compiled pattern')
1254 raise TypeError('regex must be a string or compiled pattern')
1255 # Search for keys in each namespace that match the given regex
1255 # Search for keys in each namespace that match the given regex
1256 # If a match is found, delete the key/value pair.
1256 # If a match is found, delete the key/value pair.
1257 for ns in self.all_ns_refs:
1257 for ns in self.all_ns_refs:
1258 for var in ns:
1258 for var in ns:
1259 if m.search(var):
1259 if m.search(var):
1260 del ns[var]
1260 del ns[var]
1261
1261
1262 def push(self, variables, interactive=True):
1262 def push(self, variables, interactive=True):
1263 """Inject a group of variables into the IPython user namespace.
1263 """Inject a group of variables into the IPython user namespace.
1264
1264
1265 Parameters
1265 Parameters
1266 ----------
1266 ----------
1267 variables : dict, str or list/tuple of str
1267 variables : dict, str or list/tuple of str
1268 The variables to inject into the user's namespace. If a dict, a
1268 The variables to inject into the user's namespace. If a dict, a
1269 simple update is done. If a str, the string is assumed to have
1269 simple update is done. If a str, the string is assumed to have
1270 variable names separated by spaces. A list/tuple of str can also
1270 variable names separated by spaces. A list/tuple of str can also
1271 be used to give the variable names. If just the variable names are
1271 be used to give the variable names. If just the variable names are
1272 give (list/tuple/str) then the variable values looked up in the
1272 give (list/tuple/str) then the variable values looked up in the
1273 callers frame.
1273 callers frame.
1274 interactive : bool
1274 interactive : bool
1275 If True (default), the variables will be listed with the ``who``
1275 If True (default), the variables will be listed with the ``who``
1276 magic.
1276 magic.
1277 """
1277 """
1278 vdict = None
1278 vdict = None
1279
1279
1280 # We need a dict of name/value pairs to do namespace updates.
1280 # We need a dict of name/value pairs to do namespace updates.
1281 if isinstance(variables, dict):
1281 if isinstance(variables, dict):
1282 vdict = variables
1282 vdict = variables
1283 elif isinstance(variables, (basestring, list, tuple)):
1283 elif isinstance(variables, (basestring, list, tuple)):
1284 if isinstance(variables, basestring):
1284 if isinstance(variables, basestring):
1285 vlist = variables.split()
1285 vlist = variables.split()
1286 else:
1286 else:
1287 vlist = variables
1287 vlist = variables
1288 vdict = {}
1288 vdict = {}
1289 cf = sys._getframe(1)
1289 cf = sys._getframe(1)
1290 for name in vlist:
1290 for name in vlist:
1291 try:
1291 try:
1292 vdict[name] = eval(name, cf.f_globals, cf.f_locals)
1292 vdict[name] = eval(name, cf.f_globals, cf.f_locals)
1293 except:
1293 except:
1294 print ('Could not get variable %s from %s' %
1294 print ('Could not get variable %s from %s' %
1295 (name,cf.f_code.co_name))
1295 (name,cf.f_code.co_name))
1296 else:
1296 else:
1297 raise ValueError('variables must be a dict/str/list/tuple')
1297 raise ValueError('variables must be a dict/str/list/tuple')
1298
1298
1299 # Propagate variables to user namespace
1299 # Propagate variables to user namespace
1300 self.user_ns.update(vdict)
1300 self.user_ns.update(vdict)
1301
1301
1302 # And configure interactive visibility
1302 # And configure interactive visibility
1303 user_ns_hidden = self.user_ns_hidden
1303 user_ns_hidden = self.user_ns_hidden
1304 if interactive:
1304 if interactive:
1305 user_ns_hidden.difference_update(vdict)
1305 user_ns_hidden.difference_update(vdict)
1306 else:
1306 else:
1307 user_ns_hidden.update(vdict)
1307 user_ns_hidden.update(vdict)
1308
1308
1309 def drop_by_id(self, variables):
1309 def drop_by_id(self, variables):
1310 """Remove a dict of variables from the user namespace, if they are the
1310 """Remove a dict of variables from the user namespace, if they are the
1311 same as the values in the dictionary.
1311 same as the values in the dictionary.
1312
1312
1313 This is intended for use by extensions: variables that they've added can
1313 This is intended for use by extensions: variables that they've added can
1314 be taken back out if they are unloaded, without removing any that the
1314 be taken back out if they are unloaded, without removing any that the
1315 user has overwritten.
1315 user has overwritten.
1316
1316
1317 Parameters
1317 Parameters
1318 ----------
1318 ----------
1319 variables : dict
1319 variables : dict
1320 A dictionary mapping object names (as strings) to the objects.
1320 A dictionary mapping object names (as strings) to the objects.
1321 """
1321 """
1322 for name, obj in variables.iteritems():
1322 for name, obj in variables.iteritems():
1323 if name in self.user_ns and self.user_ns[name] is obj:
1323 if name in self.user_ns and self.user_ns[name] is obj:
1324 del self.user_ns[name]
1324 del self.user_ns[name]
1325 self.user_ns_hidden.discard(name)
1325 self.user_ns_hidden.discard(name)
1326
1326
1327 #-------------------------------------------------------------------------
1327 #-------------------------------------------------------------------------
1328 # Things related to object introspection
1328 # Things related to object introspection
1329 #-------------------------------------------------------------------------
1329 #-------------------------------------------------------------------------
1330
1330
1331 def _ofind(self, oname, namespaces=None):
1331 def _ofind(self, oname, namespaces=None):
1332 """Find an object in the available namespaces.
1332 """Find an object in the available namespaces.
1333
1333
1334 self._ofind(oname) -> dict with keys: found,obj,ospace,ismagic
1334 self._ofind(oname) -> dict with keys: found,obj,ospace,ismagic
1335
1335
1336 Has special code to detect magic functions.
1336 Has special code to detect magic functions.
1337 """
1337 """
1338 oname = oname.strip()
1338 oname = oname.strip()
1339 #print '1- oname: <%r>' % oname # dbg
1339 #print '1- oname: <%r>' % oname # dbg
1340 if not py3compat.isidentifier(oname.lstrip(ESC_MAGIC), dotted=True):
1340 if not py3compat.isidentifier(oname.lstrip(ESC_MAGIC), dotted=True):
1341 return dict(found=False)
1341 return dict(found=False)
1342
1342
1343 alias_ns = None
1343 alias_ns = None
1344 if namespaces is None:
1344 if namespaces is None:
1345 # Namespaces to search in:
1345 # Namespaces to search in:
1346 # Put them in a list. The order is important so that we
1346 # Put them in a list. The order is important so that we
1347 # find things in the same order that Python finds them.
1347 # find things in the same order that Python finds them.
1348 namespaces = [ ('Interactive', self.user_ns),
1348 namespaces = [ ('Interactive', self.user_ns),
1349 ('Interactive (global)', self.user_global_ns),
1349 ('Interactive (global)', self.user_global_ns),
1350 ('Python builtin', builtin_mod.__dict__),
1350 ('Python builtin', builtin_mod.__dict__),
1351 ('Alias', self.alias_manager.alias_table),
1351 ('Alias', self.alias_manager.alias_table),
1352 ]
1352 ]
1353 alias_ns = self.alias_manager.alias_table
1353 alias_ns = self.alias_manager.alias_table
1354
1354
1355 # initialize results to 'null'
1355 # initialize results to 'null'
1356 found = False; obj = None; ospace = None; ds = None;
1356 found = False; obj = None; ospace = None; ds = None;
1357 ismagic = False; isalias = False; parent = None
1357 ismagic = False; isalias = False; parent = None
1358
1358
1359 # We need to special-case 'print', which as of python2.6 registers as a
1359 # We need to special-case 'print', which as of python2.6 registers as a
1360 # function but should only be treated as one if print_function was
1360 # function but should only be treated as one if print_function was
1361 # loaded with a future import. In this case, just bail.
1361 # loaded with a future import. In this case, just bail.
1362 if (oname == 'print' and not py3compat.PY3 and not \
1362 if (oname == 'print' and not py3compat.PY3 and not \
1363 (self.compile.compiler_flags & __future__.CO_FUTURE_PRINT_FUNCTION)):
1363 (self.compile.compiler_flags & __future__.CO_FUTURE_PRINT_FUNCTION)):
1364 return {'found':found, 'obj':obj, 'namespace':ospace,
1364 return {'found':found, 'obj':obj, 'namespace':ospace,
1365 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1365 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1366
1366
1367 # Look for the given name by splitting it in parts. If the head is
1367 # Look for the given name by splitting it in parts. If the head is
1368 # found, then we look for all the remaining parts as members, and only
1368 # found, then we look for all the remaining parts as members, and only
1369 # declare success if we can find them all.
1369 # declare success if we can find them all.
1370 oname_parts = oname.split('.')
1370 oname_parts = oname.split('.')
1371 oname_head, oname_rest = oname_parts[0],oname_parts[1:]
1371 oname_head, oname_rest = oname_parts[0],oname_parts[1:]
1372 for nsname,ns in namespaces:
1372 for nsname,ns in namespaces:
1373 try:
1373 try:
1374 obj = ns[oname_head]
1374 obj = ns[oname_head]
1375 except KeyError:
1375 except KeyError:
1376 continue
1376 continue
1377 else:
1377 else:
1378 #print 'oname_rest:', oname_rest # dbg
1378 #print 'oname_rest:', oname_rest # dbg
1379 for part in oname_rest:
1379 for part in oname_rest:
1380 try:
1380 try:
1381 parent = obj
1381 parent = obj
1382 obj = getattr(obj,part)
1382 obj = getattr(obj,part)
1383 except:
1383 except:
1384 # Blanket except b/c some badly implemented objects
1384 # Blanket except b/c some badly implemented objects
1385 # allow __getattr__ to raise exceptions other than
1385 # allow __getattr__ to raise exceptions other than
1386 # AttributeError, which then crashes IPython.
1386 # AttributeError, which then crashes IPython.
1387 break
1387 break
1388 else:
1388 else:
1389 # If we finish the for loop (no break), we got all members
1389 # If we finish the for loop (no break), we got all members
1390 found = True
1390 found = True
1391 ospace = nsname
1391 ospace = nsname
1392 if ns == alias_ns:
1392 if ns == alias_ns:
1393 isalias = True
1393 isalias = True
1394 break # namespace loop
1394 break # namespace loop
1395
1395
1396 # Try to see if it's magic
1396 # Try to see if it's magic
1397 if not found:
1397 if not found:
1398 if oname.startswith(ESC_MAGIC):
1398 if oname.startswith(ESC_MAGIC):
1399 oname = oname[1:]
1399 oname = oname[1:]
1400 obj = self.find_magic(oname)
1400 obj = self.find_magic(oname)
1401 if obj is not None:
1401 if obj is not None:
1402 found = True
1402 found = True
1403 ospace = 'IPython internal'
1403 ospace = 'IPython internal'
1404 ismagic = True
1404 ismagic = True
1405
1405
1406 # Last try: special-case some literals like '', [], {}, etc:
1406 # Last try: special-case some literals like '', [], {}, etc:
1407 if not found and oname_head in ["''",'""','[]','{}','()']:
1407 if not found and oname_head in ["''",'""','[]','{}','()']:
1408 obj = eval(oname_head)
1408 obj = eval(oname_head)
1409 found = True
1409 found = True
1410 ospace = 'Interactive'
1410 ospace = 'Interactive'
1411
1411
1412 return {'found':found, 'obj':obj, 'namespace':ospace,
1412 return {'found':found, 'obj':obj, 'namespace':ospace,
1413 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1413 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1414
1414
1415 def _ofind_property(self, oname, info):
1415 def _ofind_property(self, oname, info):
1416 """Second part of object finding, to look for property details."""
1416 """Second part of object finding, to look for property details."""
1417 if info.found:
1417 if info.found:
1418 # Get the docstring of the class property if it exists.
1418 # Get the docstring of the class property if it exists.
1419 path = oname.split('.')
1419 path = oname.split('.')
1420 root = '.'.join(path[:-1])
1420 root = '.'.join(path[:-1])
1421 if info.parent is not None:
1421 if info.parent is not None:
1422 try:
1422 try:
1423 target = getattr(info.parent, '__class__')
1423 target = getattr(info.parent, '__class__')
1424 # The object belongs to a class instance.
1424 # The object belongs to a class instance.
1425 try:
1425 try:
1426 target = getattr(target, path[-1])
1426 target = getattr(target, path[-1])
1427 # The class defines the object.
1427 # The class defines the object.
1428 if isinstance(target, property):
1428 if isinstance(target, property):
1429 oname = root + '.__class__.' + path[-1]
1429 oname = root + '.__class__.' + path[-1]
1430 info = Struct(self._ofind(oname))
1430 info = Struct(self._ofind(oname))
1431 except AttributeError: pass
1431 except AttributeError: pass
1432 except AttributeError: pass
1432 except AttributeError: pass
1433
1433
1434 # We return either the new info or the unmodified input if the object
1434 # We return either the new info or the unmodified input if the object
1435 # hadn't been found
1435 # hadn't been found
1436 return info
1436 return info
1437
1437
1438 def _object_find(self, oname, namespaces=None):
1438 def _object_find(self, oname, namespaces=None):
1439 """Find an object and return a struct with info about it."""
1439 """Find an object and return a struct with info about it."""
1440 inf = Struct(self._ofind(oname, namespaces))
1440 inf = Struct(self._ofind(oname, namespaces))
1441 return Struct(self._ofind_property(oname, inf))
1441 return Struct(self._ofind_property(oname, inf))
1442
1442
1443 def _inspect(self, meth, oname, namespaces=None, **kw):
1443 def _inspect(self, meth, oname, namespaces=None, **kw):
1444 """Generic interface to the inspector system.
1444 """Generic interface to the inspector system.
1445
1445
1446 This function is meant to be called by pdef, pdoc & friends."""
1446 This function is meant to be called by pdef, pdoc & friends."""
1447 info = self._object_find(oname)
1447 info = self._object_find(oname)
1448 if info.found:
1448 if info.found:
1449 pmethod = getattr(self.inspector, meth)
1449 pmethod = getattr(self.inspector, meth)
1450 formatter = format_screen if info.ismagic else None
1450 formatter = format_screen if info.ismagic else None
1451 if meth == 'pdoc':
1451 if meth == 'pdoc':
1452 pmethod(info.obj, oname, formatter)
1452 pmethod(info.obj, oname, formatter)
1453 elif meth == 'pinfo':
1453 elif meth == 'pinfo':
1454 pmethod(info.obj, oname, formatter, info, **kw)
1454 pmethod(info.obj, oname, formatter, info, **kw)
1455 else:
1455 else:
1456 pmethod(info.obj, oname)
1456 pmethod(info.obj, oname)
1457 else:
1457 else:
1458 print 'Object `%s` not found.' % oname
1458 print 'Object `%s` not found.' % oname
1459 return 'not found' # so callers can take other action
1459 return 'not found' # so callers can take other action
1460
1460
1461 def object_inspect(self, oname, detail_level=0):
1461 def object_inspect(self, oname, detail_level=0):
1462 with self.builtin_trap:
1462 with self.builtin_trap:
1463 info = self._object_find(oname)
1463 info = self._object_find(oname)
1464 if info.found:
1464 if info.found:
1465 return self.inspector.info(info.obj, oname, info=info,
1465 return self.inspector.info(info.obj, oname, info=info,
1466 detail_level=detail_level
1466 detail_level=detail_level
1467 )
1467 )
1468 else:
1468 else:
1469 return oinspect.object_info(name=oname, found=False)
1469 return oinspect.object_info(name=oname, found=False)
1470
1470
1471 #-------------------------------------------------------------------------
1471 #-------------------------------------------------------------------------
1472 # Things related to history management
1472 # Things related to history management
1473 #-------------------------------------------------------------------------
1473 #-------------------------------------------------------------------------
1474
1474
1475 def init_history(self):
1475 def init_history(self):
1476 """Sets up the command history, and starts regular autosaves."""
1476 """Sets up the command history, and starts regular autosaves."""
1477 self.history_manager = HistoryManager(shell=self, config=self.config)
1477 self.history_manager = HistoryManager(shell=self, config=self.config)
1478 self.configurables.append(self.history_manager)
1478 self.configurables.append(self.history_manager)
1479
1479
1480 #-------------------------------------------------------------------------
1480 #-------------------------------------------------------------------------
1481 # Things related to exception handling and tracebacks (not debugging)
1481 # Things related to exception handling and tracebacks (not debugging)
1482 #-------------------------------------------------------------------------
1482 #-------------------------------------------------------------------------
1483
1483
1484 def init_traceback_handlers(self, custom_exceptions):
1484 def init_traceback_handlers(self, custom_exceptions):
1485 # Syntax error handler.
1485 # Syntax error handler.
1486 self.SyntaxTB = ultratb.SyntaxTB(color_scheme='NoColor')
1486 self.SyntaxTB = ultratb.SyntaxTB(color_scheme='NoColor')
1487
1487
1488 # The interactive one is initialized with an offset, meaning we always
1488 # The interactive one is initialized with an offset, meaning we always
1489 # want to remove the topmost item in the traceback, which is our own
1489 # want to remove the topmost item in the traceback, which is our own
1490 # internal code. Valid modes: ['Plain','Context','Verbose']
1490 # internal code. Valid modes: ['Plain','Context','Verbose']
1491 self.InteractiveTB = ultratb.AutoFormattedTB(mode = 'Plain',
1491 self.InteractiveTB = ultratb.AutoFormattedTB(mode = 'Plain',
1492 color_scheme='NoColor',
1492 color_scheme='NoColor',
1493 tb_offset = 1,
1493 tb_offset = 1,
1494 check_cache=self.compile.check_cache)
1494 check_cache=self.compile.check_cache)
1495
1495
1496 # The instance will store a pointer to the system-wide exception hook,
1496 # The instance will store a pointer to the system-wide exception hook,
1497 # so that runtime code (such as magics) can access it. This is because
1497 # so that runtime code (such as magics) can access it. This is because
1498 # during the read-eval loop, it may get temporarily overwritten.
1498 # during the read-eval loop, it may get temporarily overwritten.
1499 self.sys_excepthook = sys.excepthook
1499 self.sys_excepthook = sys.excepthook
1500
1500
1501 # and add any custom exception handlers the user may have specified
1501 # and add any custom exception handlers the user may have specified
1502 self.set_custom_exc(*custom_exceptions)
1502 self.set_custom_exc(*custom_exceptions)
1503
1503
1504 # Set the exception mode
1504 # Set the exception mode
1505 self.InteractiveTB.set_mode(mode=self.xmode)
1505 self.InteractiveTB.set_mode(mode=self.xmode)
1506
1506
1507 def set_custom_exc(self, exc_tuple, handler):
1507 def set_custom_exc(self, exc_tuple, handler):
1508 """set_custom_exc(exc_tuple,handler)
1508 """set_custom_exc(exc_tuple,handler)
1509
1509
1510 Set a custom exception handler, which will be called if any of the
1510 Set a custom exception handler, which will be called if any of the
1511 exceptions in exc_tuple occur in the mainloop (specifically, in the
1511 exceptions in exc_tuple occur in the mainloop (specifically, in the
1512 run_code() method).
1512 run_code() method).
1513
1513
1514 Parameters
1514 Parameters
1515 ----------
1515 ----------
1516
1516
1517 exc_tuple : tuple of exception classes
1517 exc_tuple : tuple of exception classes
1518 A *tuple* of exception classes, for which to call the defined
1518 A *tuple* of exception classes, for which to call the defined
1519 handler. It is very important that you use a tuple, and NOT A
1519 handler. It is very important that you use a tuple, and NOT A
1520 LIST here, because of the way Python's except statement works. If
1520 LIST here, because of the way Python's except statement works. If
1521 you only want to trap a single exception, use a singleton tuple::
1521 you only want to trap a single exception, use a singleton tuple::
1522
1522
1523 exc_tuple == (MyCustomException,)
1523 exc_tuple == (MyCustomException,)
1524
1524
1525 handler : callable
1525 handler : callable
1526 handler must have the following signature::
1526 handler must have the following signature::
1527
1527
1528 def my_handler(self, etype, value, tb, tb_offset=None):
1528 def my_handler(self, etype, value, tb, tb_offset=None):
1529 ...
1529 ...
1530 return structured_traceback
1530 return structured_traceback
1531
1531
1532 Your handler must return a structured traceback (a list of strings),
1532 Your handler must return a structured traceback (a list of strings),
1533 or None.
1533 or None.
1534
1534
1535 This will be made into an instance method (via types.MethodType)
1535 This will be made into an instance method (via types.MethodType)
1536 of IPython itself, and it will be called if any of the exceptions
1536 of IPython itself, and it will be called if any of the exceptions
1537 listed in the exc_tuple are caught. If the handler is None, an
1537 listed in the exc_tuple are caught. If the handler is None, an
1538 internal basic one is used, which just prints basic info.
1538 internal basic one is used, which just prints basic info.
1539
1539
1540 To protect IPython from crashes, if your handler ever raises an
1540 To protect IPython from crashes, if your handler ever raises an
1541 exception or returns an invalid result, it will be immediately
1541 exception or returns an invalid result, it will be immediately
1542 disabled.
1542 disabled.
1543
1543
1544 WARNING: by putting in your own exception handler into IPython's main
1544 WARNING: by putting in your own exception handler into IPython's main
1545 execution loop, you run a very good chance of nasty crashes. This
1545 execution loop, you run a very good chance of nasty crashes. This
1546 facility should only be used if you really know what you are doing."""
1546 facility should only be used if you really know what you are doing."""
1547
1547
1548 assert type(exc_tuple)==type(()) , \
1548 assert type(exc_tuple)==type(()) , \
1549 "The custom exceptions must be given AS A TUPLE."
1549 "The custom exceptions must be given AS A TUPLE."
1550
1550
1551 def dummy_handler(self,etype,value,tb,tb_offset=None):
1551 def dummy_handler(self,etype,value,tb,tb_offset=None):
1552 print '*** Simple custom exception handler ***'
1552 print '*** Simple custom exception handler ***'
1553 print 'Exception type :',etype
1553 print 'Exception type :',etype
1554 print 'Exception value:',value
1554 print 'Exception value:',value
1555 print 'Traceback :',tb
1555 print 'Traceback :',tb
1556 #print 'Source code :','\n'.join(self.buffer)
1556 #print 'Source code :','\n'.join(self.buffer)
1557
1557
1558 def validate_stb(stb):
1558 def validate_stb(stb):
1559 """validate structured traceback return type
1559 """validate structured traceback return type
1560
1560
1561 return type of CustomTB *should* be a list of strings, but allow
1561 return type of CustomTB *should* be a list of strings, but allow
1562 single strings or None, which are harmless.
1562 single strings or None, which are harmless.
1563
1563
1564 This function will *always* return a list of strings,
1564 This function will *always* return a list of strings,
1565 and will raise a TypeError if stb is inappropriate.
1565 and will raise a TypeError if stb is inappropriate.
1566 """
1566 """
1567 msg = "CustomTB must return list of strings, not %r" % stb
1567 msg = "CustomTB must return list of strings, not %r" % stb
1568 if stb is None:
1568 if stb is None:
1569 return []
1569 return []
1570 elif isinstance(stb, basestring):
1570 elif isinstance(stb, basestring):
1571 return [stb]
1571 return [stb]
1572 elif not isinstance(stb, list):
1572 elif not isinstance(stb, list):
1573 raise TypeError(msg)
1573 raise TypeError(msg)
1574 # it's a list
1574 # it's a list
1575 for line in stb:
1575 for line in stb:
1576 # check every element
1576 # check every element
1577 if not isinstance(line, basestring):
1577 if not isinstance(line, basestring):
1578 raise TypeError(msg)
1578 raise TypeError(msg)
1579 return stb
1579 return stb
1580
1580
1581 if handler is None:
1581 if handler is None:
1582 wrapped = dummy_handler
1582 wrapped = dummy_handler
1583 else:
1583 else:
1584 def wrapped(self,etype,value,tb,tb_offset=None):
1584 def wrapped(self,etype,value,tb,tb_offset=None):
1585 """wrap CustomTB handler, to protect IPython from user code
1585 """wrap CustomTB handler, to protect IPython from user code
1586
1586
1587 This makes it harder (but not impossible) for custom exception
1587 This makes it harder (but not impossible) for custom exception
1588 handlers to crash IPython.
1588 handlers to crash IPython.
1589 """
1589 """
1590 try:
1590 try:
1591 stb = handler(self,etype,value,tb,tb_offset=tb_offset)
1591 stb = handler(self,etype,value,tb,tb_offset=tb_offset)
1592 return validate_stb(stb)
1592 return validate_stb(stb)
1593 except:
1593 except:
1594 # clear custom handler immediately
1594 # clear custom handler immediately
1595 self.set_custom_exc((), None)
1595 self.set_custom_exc((), None)
1596 print >> io.stderr, "Custom TB Handler failed, unregistering"
1596 print >> io.stderr, "Custom TB Handler failed, unregistering"
1597 # show the exception in handler first
1597 # show the exception in handler first
1598 stb = self.InteractiveTB.structured_traceback(*sys.exc_info())
1598 stb = self.InteractiveTB.structured_traceback(*sys.exc_info())
1599 print >> io.stdout, self.InteractiveTB.stb2text(stb)
1599 print >> io.stdout, self.InteractiveTB.stb2text(stb)
1600 print >> io.stdout, "The original exception:"
1600 print >> io.stdout, "The original exception:"
1601 stb = self.InteractiveTB.structured_traceback(
1601 stb = self.InteractiveTB.structured_traceback(
1602 (etype,value,tb), tb_offset=tb_offset
1602 (etype,value,tb), tb_offset=tb_offset
1603 )
1603 )
1604 return stb
1604 return stb
1605
1605
1606 self.CustomTB = types.MethodType(wrapped,self)
1606 self.CustomTB = types.MethodType(wrapped,self)
1607 self.custom_exceptions = exc_tuple
1607 self.custom_exceptions = exc_tuple
1608
1608
1609 def excepthook(self, etype, value, tb):
1609 def excepthook(self, etype, value, tb):
1610 """One more defense for GUI apps that call sys.excepthook.
1610 """One more defense for GUI apps that call sys.excepthook.
1611
1611
1612 GUI frameworks like wxPython trap exceptions and call
1612 GUI frameworks like wxPython trap exceptions and call
1613 sys.excepthook themselves. I guess this is a feature that
1613 sys.excepthook themselves. I guess this is a feature that
1614 enables them to keep running after exceptions that would
1614 enables them to keep running after exceptions that would
1615 otherwise kill their mainloop. This is a bother for IPython
1615 otherwise kill their mainloop. This is a bother for IPython
1616 which excepts to catch all of the program exceptions with a try:
1616 which excepts to catch all of the program exceptions with a try:
1617 except: statement.
1617 except: statement.
1618
1618
1619 Normally, IPython sets sys.excepthook to a CrashHandler instance, so if
1619 Normally, IPython sets sys.excepthook to a CrashHandler instance, so if
1620 any app directly invokes sys.excepthook, it will look to the user like
1620 any app directly invokes sys.excepthook, it will look to the user like
1621 IPython crashed. In order to work around this, we can disable the
1621 IPython crashed. In order to work around this, we can disable the
1622 CrashHandler and replace it with this excepthook instead, which prints a
1622 CrashHandler and replace it with this excepthook instead, which prints a
1623 regular traceback using our InteractiveTB. In this fashion, apps which
1623 regular traceback using our InteractiveTB. In this fashion, apps which
1624 call sys.excepthook will generate a regular-looking exception from
1624 call sys.excepthook will generate a regular-looking exception from
1625 IPython, and the CrashHandler will only be triggered by real IPython
1625 IPython, and the CrashHandler will only be triggered by real IPython
1626 crashes.
1626 crashes.
1627
1627
1628 This hook should be used sparingly, only in places which are not likely
1628 This hook should be used sparingly, only in places which are not likely
1629 to be true IPython errors.
1629 to be true IPython errors.
1630 """
1630 """
1631 self.showtraceback((etype,value,tb),tb_offset=0)
1631 self.showtraceback((etype,value,tb),tb_offset=0)
1632
1632
1633 def _get_exc_info(self, exc_tuple=None):
1633 def _get_exc_info(self, exc_tuple=None):
1634 """get exc_info from a given tuple, sys.exc_info() or sys.last_type etc.
1634 """get exc_info from a given tuple, sys.exc_info() or sys.last_type etc.
1635
1635
1636 Ensures sys.last_type,value,traceback hold the exc_info we found,
1636 Ensures sys.last_type,value,traceback hold the exc_info we found,
1637 from whichever source.
1637 from whichever source.
1638
1638
1639 raises ValueError if none of these contain any information
1639 raises ValueError if none of these contain any information
1640 """
1640 """
1641 if exc_tuple is None:
1641 if exc_tuple is None:
1642 etype, value, tb = sys.exc_info()
1642 etype, value, tb = sys.exc_info()
1643 else:
1643 else:
1644 etype, value, tb = exc_tuple
1644 etype, value, tb = exc_tuple
1645
1645
1646 if etype is None:
1646 if etype is None:
1647 if hasattr(sys, 'last_type'):
1647 if hasattr(sys, 'last_type'):
1648 etype, value, tb = sys.last_type, sys.last_value, \
1648 etype, value, tb = sys.last_type, sys.last_value, \
1649 sys.last_traceback
1649 sys.last_traceback
1650
1650
1651 if etype is None:
1651 if etype is None:
1652 raise ValueError("No exception to find")
1652 raise ValueError("No exception to find")
1653
1653
1654 # Now store the exception info in sys.last_type etc.
1654 # Now store the exception info in sys.last_type etc.
1655 # WARNING: these variables are somewhat deprecated and not
1655 # WARNING: these variables are somewhat deprecated and not
1656 # necessarily safe to use in a threaded environment, but tools
1656 # necessarily safe to use in a threaded environment, but tools
1657 # like pdb depend on their existence, so let's set them. If we
1657 # like pdb depend on their existence, so let's set them. If we
1658 # find problems in the field, we'll need to revisit their use.
1658 # find problems in the field, we'll need to revisit their use.
1659 sys.last_type = etype
1659 sys.last_type = etype
1660 sys.last_value = value
1660 sys.last_value = value
1661 sys.last_traceback = tb
1661 sys.last_traceback = tb
1662
1662
1663 return etype, value, tb
1663 return etype, value, tb
1664
1664
1665
1665
1666 def showtraceback(self,exc_tuple = None,filename=None,tb_offset=None,
1666 def showtraceback(self,exc_tuple = None,filename=None,tb_offset=None,
1667 exception_only=False):
1667 exception_only=False):
1668 """Display the exception that just occurred.
1668 """Display the exception that just occurred.
1669
1669
1670 If nothing is known about the exception, this is the method which
1670 If nothing is known about the exception, this is the method which
1671 should be used throughout the code for presenting user tracebacks,
1671 should be used throughout the code for presenting user tracebacks,
1672 rather than directly invoking the InteractiveTB object.
1672 rather than directly invoking the InteractiveTB object.
1673
1673
1674 A specific showsyntaxerror() also exists, but this method can take
1674 A specific showsyntaxerror() also exists, but this method can take
1675 care of calling it if needed, so unless you are explicitly catching a
1675 care of calling it if needed, so unless you are explicitly catching a
1676 SyntaxError exception, don't try to analyze the stack manually and
1676 SyntaxError exception, don't try to analyze the stack manually and
1677 simply call this method."""
1677 simply call this method."""
1678
1678
1679 try:
1679 try:
1680 try:
1680 try:
1681 etype, value, tb = self._get_exc_info(exc_tuple)
1681 etype, value, tb = self._get_exc_info(exc_tuple)
1682 except ValueError:
1682 except ValueError:
1683 self.write_err('No traceback available to show.\n')
1683 self.write_err('No traceback available to show.\n')
1684 return
1684 return
1685
1685
1686 if etype is SyntaxError:
1686 if etype is SyntaxError:
1687 # Though this won't be called by syntax errors in the input
1687 # Though this won't be called by syntax errors in the input
1688 # line, there may be SyntaxError cases with imported code.
1688 # line, there may be SyntaxError cases with imported code.
1689 self.showsyntaxerror(filename)
1689 self.showsyntaxerror(filename)
1690 elif etype is UsageError:
1690 elif etype is UsageError:
1691 self.write_err("UsageError: %s" % value)
1691 self.write_err("UsageError: %s" % value)
1692 else:
1692 else:
1693 if etype in self.custom_exceptions:
1693 if etype in self.custom_exceptions:
1694 stb = self.CustomTB(etype, value, tb, tb_offset)
1694 stb = self.CustomTB(etype, value, tb, tb_offset)
1695 else:
1695 else:
1696 if exception_only:
1696 if exception_only:
1697 stb = ['An exception has occurred, use %tb to see '
1697 stb = ['An exception has occurred, use %tb to see '
1698 'the full traceback.\n']
1698 'the full traceback.\n']
1699 stb.extend(self.InteractiveTB.get_exception_only(etype,
1699 stb.extend(self.InteractiveTB.get_exception_only(etype,
1700 value))
1700 value))
1701 else:
1701 else:
1702 stb = self.InteractiveTB.structured_traceback(etype,
1702 stb = self.InteractiveTB.structured_traceback(etype,
1703 value, tb, tb_offset=tb_offset)
1703 value, tb, tb_offset=tb_offset)
1704
1704
1705 self._showtraceback(etype, value, stb)
1705 self._showtraceback(etype, value, stb)
1706 if self.call_pdb:
1706 if self.call_pdb:
1707 # drop into debugger
1707 # drop into debugger
1708 self.debugger(force=True)
1708 self.debugger(force=True)
1709 return
1709 return
1710
1710
1711 # Actually show the traceback
1711 # Actually show the traceback
1712 self._showtraceback(etype, value, stb)
1712 self._showtraceback(etype, value, stb)
1713
1713
1714 except KeyboardInterrupt:
1714 except KeyboardInterrupt:
1715 self.write_err("\nKeyboardInterrupt\n")
1715 self.write_err("\nKeyboardInterrupt\n")
1716
1716
1717 def _showtraceback(self, etype, evalue, stb):
1717 def _showtraceback(self, etype, evalue, stb):
1718 """Actually show a traceback.
1718 """Actually show a traceback.
1719
1719
1720 Subclasses may override this method to put the traceback on a different
1720 Subclasses may override this method to put the traceback on a different
1721 place, like a side channel.
1721 place, like a side channel.
1722 """
1722 """
1723 print >> io.stdout, self.InteractiveTB.stb2text(stb)
1723 print >> io.stdout, self.InteractiveTB.stb2text(stb)
1724
1724
1725 def showsyntaxerror(self, filename=None):
1725 def showsyntaxerror(self, filename=None):
1726 """Display the syntax error that just occurred.
1726 """Display the syntax error that just occurred.
1727
1727
1728 This doesn't display a stack trace because there isn't one.
1728 This doesn't display a stack trace because there isn't one.
1729
1729
1730 If a filename is given, it is stuffed in the exception instead
1730 If a filename is given, it is stuffed in the exception instead
1731 of what was there before (because Python's parser always uses
1731 of what was there before (because Python's parser always uses
1732 "<string>" when reading from a string).
1732 "<string>" when reading from a string).
1733 """
1733 """
1734 etype, value, last_traceback = self._get_exc_info()
1734 etype, value, last_traceback = self._get_exc_info()
1735
1735
1736 if filename and etype is SyntaxError:
1736 if filename and etype is SyntaxError:
1737 try:
1737 try:
1738 value.filename = filename
1738 value.filename = filename
1739 except:
1739 except:
1740 # Not the format we expect; leave it alone
1740 # Not the format we expect; leave it alone
1741 pass
1741 pass
1742
1742
1743 stb = self.SyntaxTB.structured_traceback(etype, value, [])
1743 stb = self.SyntaxTB.structured_traceback(etype, value, [])
1744 self._showtraceback(etype, value, stb)
1744 self._showtraceback(etype, value, stb)
1745
1745
1746 # This is overridden in TerminalInteractiveShell to show a message about
1746 # This is overridden in TerminalInteractiveShell to show a message about
1747 # the %paste magic.
1747 # the %paste magic.
1748 def showindentationerror(self):
1748 def showindentationerror(self):
1749 """Called by run_cell when there's an IndentationError in code entered
1749 """Called by run_cell when there's an IndentationError in code entered
1750 at the prompt.
1750 at the prompt.
1751
1751
1752 This is overridden in TerminalInteractiveShell to show a message about
1752 This is overridden in TerminalInteractiveShell to show a message about
1753 the %paste magic."""
1753 the %paste magic."""
1754 self.showsyntaxerror()
1754 self.showsyntaxerror()
1755
1755
1756 #-------------------------------------------------------------------------
1756 #-------------------------------------------------------------------------
1757 # Things related to readline
1757 # Things related to readline
1758 #-------------------------------------------------------------------------
1758 #-------------------------------------------------------------------------
1759
1759
1760 def init_readline(self):
1760 def init_readline(self):
1761 """Command history completion/saving/reloading."""
1761 """Command history completion/saving/reloading."""
1762
1762
1763 if self.readline_use:
1763 if self.readline_use:
1764 import IPython.utils.rlineimpl as readline
1764 import IPython.utils.rlineimpl as readline
1765
1765
1766 self.rl_next_input = None
1766 self.rl_next_input = None
1767 self.rl_do_indent = False
1767 self.rl_do_indent = False
1768
1768
1769 if not self.readline_use or not readline.have_readline:
1769 if not self.readline_use or not readline.have_readline:
1770 self.has_readline = False
1770 self.has_readline = False
1771 self.readline = None
1771 self.readline = None
1772 # Set a number of methods that depend on readline to be no-op
1772 # Set a number of methods that depend on readline to be no-op
1773 self.readline_no_record = no_op_context
1773 self.readline_no_record = no_op_context
1774 self.set_readline_completer = no_op
1774 self.set_readline_completer = no_op
1775 self.set_custom_completer = no_op
1775 self.set_custom_completer = no_op
1776 self.set_completer_frame = no_op
1776 self.set_completer_frame = no_op
1777 if self.readline_use:
1777 if self.readline_use:
1778 warn('Readline services not available or not loaded.')
1778 warn('Readline services not available or not loaded.')
1779 else:
1779 else:
1780 self.has_readline = True
1780 self.has_readline = True
1781 self.readline = readline
1781 self.readline = readline
1782 sys.modules['readline'] = readline
1782 sys.modules['readline'] = readline
1783
1783
1784 # Platform-specific configuration
1784 # Platform-specific configuration
1785 if os.name == 'nt':
1785 if os.name == 'nt':
1786 # FIXME - check with Frederick to see if we can harmonize
1786 # FIXME - check with Frederick to see if we can harmonize
1787 # naming conventions with pyreadline to avoid this
1787 # naming conventions with pyreadline to avoid this
1788 # platform-dependent check
1788 # platform-dependent check
1789 self.readline_startup_hook = readline.set_pre_input_hook
1789 self.readline_startup_hook = readline.set_pre_input_hook
1790 else:
1790 else:
1791 self.readline_startup_hook = readline.set_startup_hook
1791 self.readline_startup_hook = readline.set_startup_hook
1792
1792
1793 # Load user's initrc file (readline config)
1793 # Load user's initrc file (readline config)
1794 # Or if libedit is used, load editrc.
1794 # Or if libedit is used, load editrc.
1795 inputrc_name = os.environ.get('INPUTRC')
1795 inputrc_name = os.environ.get('INPUTRC')
1796 if inputrc_name is None:
1796 if inputrc_name is None:
1797 inputrc_name = '.inputrc'
1797 inputrc_name = '.inputrc'
1798 if readline.uses_libedit:
1798 if readline.uses_libedit:
1799 inputrc_name = '.editrc'
1799 inputrc_name = '.editrc'
1800 inputrc_name = os.path.join(self.home_dir, inputrc_name)
1800 inputrc_name = os.path.join(self.home_dir, inputrc_name)
1801 if os.path.isfile(inputrc_name):
1801 if os.path.isfile(inputrc_name):
1802 try:
1802 try:
1803 readline.read_init_file(inputrc_name)
1803 readline.read_init_file(inputrc_name)
1804 except:
1804 except:
1805 warn('Problems reading readline initialization file <%s>'
1805 warn('Problems reading readline initialization file <%s>'
1806 % inputrc_name)
1806 % inputrc_name)
1807
1807
1808 # Configure readline according to user's prefs
1808 # Configure readline according to user's prefs
1809 # This is only done if GNU readline is being used. If libedit
1809 # This is only done if GNU readline is being used. If libedit
1810 # is being used (as on Leopard) the readline config is
1810 # is being used (as on Leopard) the readline config is
1811 # not run as the syntax for libedit is different.
1811 # not run as the syntax for libedit is different.
1812 if not readline.uses_libedit:
1812 if not readline.uses_libedit:
1813 for rlcommand in self.readline_parse_and_bind:
1813 for rlcommand in self.readline_parse_and_bind:
1814 #print "loading rl:",rlcommand # dbg
1814 #print "loading rl:",rlcommand # dbg
1815 readline.parse_and_bind(rlcommand)
1815 readline.parse_and_bind(rlcommand)
1816
1816
1817 # Remove some chars from the delimiters list. If we encounter
1817 # Remove some chars from the delimiters list. If we encounter
1818 # unicode chars, discard them.
1818 # unicode chars, discard them.
1819 delims = readline.get_completer_delims()
1819 delims = readline.get_completer_delims()
1820 if not py3compat.PY3:
1820 if not py3compat.PY3:
1821 delims = delims.encode("ascii", "ignore")
1821 delims = delims.encode("ascii", "ignore")
1822 for d in self.readline_remove_delims:
1822 for d in self.readline_remove_delims:
1823 delims = delims.replace(d, "")
1823 delims = delims.replace(d, "")
1824 delims = delims.replace(ESC_MAGIC, '')
1824 delims = delims.replace(ESC_MAGIC, '')
1825 readline.set_completer_delims(delims)
1825 readline.set_completer_delims(delims)
1826 # otherwise we end up with a monster history after a while:
1826 # otherwise we end up with a monster history after a while:
1827 readline.set_history_length(self.history_length)
1827 readline.set_history_length(self.history_length)
1828
1828
1829 self.refill_readline_hist()
1829 self.refill_readline_hist()
1830 self.readline_no_record = ReadlineNoRecord(self)
1830 self.readline_no_record = ReadlineNoRecord(self)
1831
1831
1832 # Configure auto-indent for all platforms
1832 # Configure auto-indent for all platforms
1833 self.set_autoindent(self.autoindent)
1833 self.set_autoindent(self.autoindent)
1834
1834
1835 def refill_readline_hist(self):
1835 def refill_readline_hist(self):
1836 # Load the last 1000 lines from history
1836 # Load the last 1000 lines from history
1837 self.readline.clear_history()
1837 self.readline.clear_history()
1838 stdin_encoding = sys.stdin.encoding or "utf-8"
1838 stdin_encoding = sys.stdin.encoding or "utf-8"
1839 last_cell = u""
1839 last_cell = u""
1840 for _, _, cell in self.history_manager.get_tail(1000,
1840 for _, _, cell in self.history_manager.get_tail(1000,
1841 include_latest=True):
1841 include_latest=True):
1842 # Ignore blank lines and consecutive duplicates
1842 # Ignore blank lines and consecutive duplicates
1843 cell = cell.rstrip()
1843 cell = cell.rstrip()
1844 if cell and (cell != last_cell):
1844 if cell and (cell != last_cell):
1845 if self.multiline_history:
1845 if self.multiline_history:
1846 self.readline.add_history(py3compat.unicode_to_str(cell,
1846 self.readline.add_history(py3compat.unicode_to_str(cell,
1847 stdin_encoding))
1847 stdin_encoding))
1848 else:
1848 else:
1849 for line in cell.splitlines():
1849 for line in cell.splitlines():
1850 self.readline.add_history(py3compat.unicode_to_str(line,
1850 self.readline.add_history(py3compat.unicode_to_str(line,
1851 stdin_encoding))
1851 stdin_encoding))
1852 last_cell = cell
1852 last_cell = cell
1853
1853
1854 def set_next_input(self, s):
1854 def set_next_input(self, s):
1855 """ Sets the 'default' input string for the next command line.
1855 """ Sets the 'default' input string for the next command line.
1856
1856
1857 Requires readline.
1857 Requires readline.
1858
1858
1859 Example:
1859 Example:
1860
1860
1861 [D:\ipython]|1> _ip.set_next_input("Hello Word")
1861 [D:\ipython]|1> _ip.set_next_input("Hello Word")
1862 [D:\ipython]|2> Hello Word_ # cursor is here
1862 [D:\ipython]|2> Hello Word_ # cursor is here
1863 """
1863 """
1864 self.rl_next_input = py3compat.cast_bytes_py2(s)
1864 self.rl_next_input = py3compat.cast_bytes_py2(s)
1865
1865
1866 # Maybe move this to the terminal subclass?
1866 # Maybe move this to the terminal subclass?
1867 def pre_readline(self):
1867 def pre_readline(self):
1868 """readline hook to be used at the start of each line.
1868 """readline hook to be used at the start of each line.
1869
1869
1870 Currently it handles auto-indent only."""
1870 Currently it handles auto-indent only."""
1871
1871
1872 if self.rl_do_indent:
1872 if self.rl_do_indent:
1873 self.readline.insert_text(self._indent_current_str())
1873 self.readline.insert_text(self._indent_current_str())
1874 if self.rl_next_input is not None:
1874 if self.rl_next_input is not None:
1875 self.readline.insert_text(self.rl_next_input)
1875 self.readline.insert_text(self.rl_next_input)
1876 self.rl_next_input = None
1876 self.rl_next_input = None
1877
1877
1878 def _indent_current_str(self):
1878 def _indent_current_str(self):
1879 """return the current level of indentation as a string"""
1879 """return the current level of indentation as a string"""
1880 return self.input_splitter.indent_spaces * ' '
1880 return self.input_splitter.indent_spaces * ' '
1881
1881
1882 #-------------------------------------------------------------------------
1882 #-------------------------------------------------------------------------
1883 # Things related to text completion
1883 # Things related to text completion
1884 #-------------------------------------------------------------------------
1884 #-------------------------------------------------------------------------
1885
1885
1886 def init_completer(self):
1886 def init_completer(self):
1887 """Initialize the completion machinery.
1887 """Initialize the completion machinery.
1888
1888
1889 This creates completion machinery that can be used by client code,
1889 This creates completion machinery that can be used by client code,
1890 either interactively in-process (typically triggered by the readline
1890 either interactively in-process (typically triggered by the readline
1891 library), programatically (such as in test suites) or out-of-prcess
1891 library), programatically (such as in test suites) or out-of-prcess
1892 (typically over the network by remote frontends).
1892 (typically over the network by remote frontends).
1893 """
1893 """
1894 from IPython.core.completer import IPCompleter
1894 from IPython.core.completer import IPCompleter
1895 from IPython.core.completerlib import (module_completer,
1895 from IPython.core.completerlib import (module_completer,
1896 magic_run_completer, cd_completer, reset_completer)
1896 magic_run_completer, cd_completer, reset_completer)
1897
1897
1898 self.Completer = IPCompleter(shell=self,
1898 self.Completer = IPCompleter(shell=self,
1899 namespace=self.user_ns,
1899 namespace=self.user_ns,
1900 global_namespace=self.user_global_ns,
1900 global_namespace=self.user_global_ns,
1901 alias_table=self.alias_manager.alias_table,
1901 alias_table=self.alias_manager.alias_table,
1902 use_readline=self.has_readline,
1902 use_readline=self.has_readline,
1903 config=self.config,
1903 config=self.config,
1904 )
1904 )
1905 self.configurables.append(self.Completer)
1905 self.configurables.append(self.Completer)
1906
1906
1907 # Add custom completers to the basic ones built into IPCompleter
1907 # Add custom completers to the basic ones built into IPCompleter
1908 sdisp = self.strdispatchers.get('complete_command', StrDispatch())
1908 sdisp = self.strdispatchers.get('complete_command', StrDispatch())
1909 self.strdispatchers['complete_command'] = sdisp
1909 self.strdispatchers['complete_command'] = sdisp
1910 self.Completer.custom_completers = sdisp
1910 self.Completer.custom_completers = sdisp
1911
1911
1912 self.set_hook('complete_command', module_completer, str_key = 'import')
1912 self.set_hook('complete_command', module_completer, str_key = 'import')
1913 self.set_hook('complete_command', module_completer, str_key = 'from')
1913 self.set_hook('complete_command', module_completer, str_key = 'from')
1914 self.set_hook('complete_command', magic_run_completer, str_key = '%run')
1914 self.set_hook('complete_command', magic_run_completer, str_key = '%run')
1915 self.set_hook('complete_command', cd_completer, str_key = '%cd')
1915 self.set_hook('complete_command', cd_completer, str_key = '%cd')
1916 self.set_hook('complete_command', reset_completer, str_key = '%reset')
1916 self.set_hook('complete_command', reset_completer, str_key = '%reset')
1917
1917
1918 # Only configure readline if we truly are using readline. IPython can
1918 # Only configure readline if we truly are using readline. IPython can
1919 # do tab-completion over the network, in GUIs, etc, where readline
1919 # do tab-completion over the network, in GUIs, etc, where readline
1920 # itself may be absent
1920 # itself may be absent
1921 if self.has_readline:
1921 if self.has_readline:
1922 self.set_readline_completer()
1922 self.set_readline_completer()
1923
1923
1924 def complete(self, text, line=None, cursor_pos=None):
1924 def complete(self, text, line=None, cursor_pos=None):
1925 """Return the completed text and a list of completions.
1925 """Return the completed text and a list of completions.
1926
1926
1927 Parameters
1927 Parameters
1928 ----------
1928 ----------
1929
1929
1930 text : string
1930 text : string
1931 A string of text to be completed on. It can be given as empty and
1931 A string of text to be completed on. It can be given as empty and
1932 instead a line/position pair are given. In this case, the
1932 instead a line/position pair are given. In this case, the
1933 completer itself will split the line like readline does.
1933 completer itself will split the line like readline does.
1934
1934
1935 line : string, optional
1935 line : string, optional
1936 The complete line that text is part of.
1936 The complete line that text is part of.
1937
1937
1938 cursor_pos : int, optional
1938 cursor_pos : int, optional
1939 The position of the cursor on the input line.
1939 The position of the cursor on the input line.
1940
1940
1941 Returns
1941 Returns
1942 -------
1942 -------
1943 text : string
1943 text : string
1944 The actual text that was completed.
1944 The actual text that was completed.
1945
1945
1946 matches : list
1946 matches : list
1947 A sorted list with all possible completions.
1947 A sorted list with all possible completions.
1948
1948
1949 The optional arguments allow the completion to take more context into
1949 The optional arguments allow the completion to take more context into
1950 account, and are part of the low-level completion API.
1950 account, and are part of the low-level completion API.
1951
1951
1952 This is a wrapper around the completion mechanism, similar to what
1952 This is a wrapper around the completion mechanism, similar to what
1953 readline does at the command line when the TAB key is hit. By
1953 readline does at the command line when the TAB key is hit. By
1954 exposing it as a method, it can be used by other non-readline
1954 exposing it as a method, it can be used by other non-readline
1955 environments (such as GUIs) for text completion.
1955 environments (such as GUIs) for text completion.
1956
1956
1957 Simple usage example:
1957 Simple usage example:
1958
1958
1959 In [1]: x = 'hello'
1959 In [1]: x = 'hello'
1960
1960
1961 In [2]: _ip.complete('x.l')
1961 In [2]: _ip.complete('x.l')
1962 Out[2]: ('x.l', ['x.ljust', 'x.lower', 'x.lstrip'])
1962 Out[2]: ('x.l', ['x.ljust', 'x.lower', 'x.lstrip'])
1963 """
1963 """
1964
1964
1965 # Inject names into __builtin__ so we can complete on the added names.
1965 # Inject names into __builtin__ so we can complete on the added names.
1966 with self.builtin_trap:
1966 with self.builtin_trap:
1967 return self.Completer.complete(text, line, cursor_pos)
1967 return self.Completer.complete(text, line, cursor_pos)
1968
1968
1969 def set_custom_completer(self, completer, pos=0):
1969 def set_custom_completer(self, completer, pos=0):
1970 """Adds a new custom completer function.
1970 """Adds a new custom completer function.
1971
1971
1972 The position argument (defaults to 0) is the index in the completers
1972 The position argument (defaults to 0) is the index in the completers
1973 list where you want the completer to be inserted."""
1973 list where you want the completer to be inserted."""
1974
1974
1975 newcomp = types.MethodType(completer,self.Completer)
1975 newcomp = types.MethodType(completer,self.Completer)
1976 self.Completer.matchers.insert(pos,newcomp)
1976 self.Completer.matchers.insert(pos,newcomp)
1977
1977
1978 def set_readline_completer(self):
1978 def set_readline_completer(self):
1979 """Reset readline's completer to be our own."""
1979 """Reset readline's completer to be our own."""
1980 self.readline.set_completer(self.Completer.rlcomplete)
1980 self.readline.set_completer(self.Completer.rlcomplete)
1981
1981
1982 def set_completer_frame(self, frame=None):
1982 def set_completer_frame(self, frame=None):
1983 """Set the frame of the completer."""
1983 """Set the frame of the completer."""
1984 if frame:
1984 if frame:
1985 self.Completer.namespace = frame.f_locals
1985 self.Completer.namespace = frame.f_locals
1986 self.Completer.global_namespace = frame.f_globals
1986 self.Completer.global_namespace = frame.f_globals
1987 else:
1987 else:
1988 self.Completer.namespace = self.user_ns
1988 self.Completer.namespace = self.user_ns
1989 self.Completer.global_namespace = self.user_global_ns
1989 self.Completer.global_namespace = self.user_global_ns
1990
1990
1991 #-------------------------------------------------------------------------
1991 #-------------------------------------------------------------------------
1992 # Things related to magics
1992 # Things related to magics
1993 #-------------------------------------------------------------------------
1993 #-------------------------------------------------------------------------
1994
1994
1995 def init_magics(self):
1995 def init_magics(self):
1996 from IPython.core import magics as m
1996 from IPython.core import magics as m
1997 self.magics_manager = magic.MagicsManager(shell=self,
1997 self.magics_manager = magic.MagicsManager(shell=self,
1998 confg=self.config,
1998 confg=self.config,
1999 user_magics=m.UserMagics(self))
1999 user_magics=m.UserMagics(self))
2000 self.configurables.append(self.magics_manager)
2000 self.configurables.append(self.magics_manager)
2001
2001
2002 # Expose as public API from the magics manager
2002 # Expose as public API from the magics manager
2003 self.register_magics = self.magics_manager.register
2003 self.register_magics = self.magics_manager.register
2004 self.register_magic_function = self.magics_manager.register_function
2004 self.register_magic_function = self.magics_manager.register_function
2005 self.define_magic = self.magics_manager.define_magic
2005 self.define_magic = self.magics_manager.define_magic
2006
2006
2007 self.register_magics(m.AutoMagics, m.BasicMagics, m.CodeMagics,
2007 self.register_magics(m.AutoMagics, m.BasicMagics, m.CodeMagics,
2008 m.ConfigMagics, m.DeprecatedMagics, m.ExecutionMagics,
2008 m.ConfigMagics, m.DeprecatedMagics, m.ExecutionMagics,
2009 m.ExtensionMagics, m.HistoryMagics, m.LoggingMagics,
2009 m.ExtensionMagics, m.HistoryMagics, m.LoggingMagics,
2010 m.NamespaceMagics, m.OSMagics, m.PylabMagics )
2010 m.NamespaceMagics, m.OSMagics, m.PylabMagics )
2011
2011
2012 # FIXME: Move the color initialization to the DisplayHook, which
2012 # FIXME: Move the color initialization to the DisplayHook, which
2013 # should be split into a prompt manager and displayhook. We probably
2013 # should be split into a prompt manager and displayhook. We probably
2014 # even need a centralize colors management object.
2014 # even need a centralize colors management object.
2015 self.magic('colors %s' % self.colors)
2015 self.magic('colors %s' % self.colors)
2016
2016
2017 def line_magic(self, magic_name, line):
2017 def line_magic(self, magic_name, line):
2018 """Execute the given line magic.
2018 """Execute the given line magic.
2019
2019
2020 Parameters
2020 Parameters
2021 ----------
2021 ----------
2022 magic_name : str
2022 magic_name : str
2023 Name of the desired magic function, without '%' prefix.
2023 Name of the desired magic function, without '%' prefix.
2024
2024
2025 line : str
2025 line : str
2026 The rest of the input line as a single string.
2026 The rest of the input line as a single string.
2027 """
2027 """
2028 fn = self.find_line_magic(magic_name)
2028 fn = self.find_line_magic(magic_name)
2029 if fn is None:
2029 if fn is None:
2030 error("Line magic function `%%%s` not found." % magic_name)
2030 em = "Line magic function `%%%s` not found" % magic_name
2031 cm = self.find_cell_magic(magic_name)
2032 if cm is not None:
2033 em += (' (Did you by chance mean the cell magic `%%%%%s` '
2034 'instead?).')
2035 error()
2031 else:
2036 else:
2032 # Note: this is the distance in the stack to the user's frame.
2037 # Note: this is the distance in the stack to the user's frame.
2033 # This will need to be updated if the internal calling logic gets
2038 # This will need to be updated if the internal calling logic gets
2034 # refactored, or else we'll be expanding the wrong variables.
2039 # refactored, or else we'll be expanding the wrong variables.
2035 stack_depth = 2
2040 stack_depth = 2
2036 magic_arg_s = self.var_expand(line, stack_depth)
2041 magic_arg_s = self.var_expand(line, stack_depth)
2037 # Put magic args in a list so we can call with f(*a) syntax
2042 # Put magic args in a list so we can call with f(*a) syntax
2038 args = [magic_arg_s]
2043 args = [magic_arg_s]
2039 # Grab local namespace if we need it:
2044 # Grab local namespace if we need it:
2040 if getattr(fn, "needs_local_scope", False):
2045 if getattr(fn, "needs_local_scope", False):
2041 args.append(sys._getframe(stack_depth).f_locals)
2046 args.append(sys._getframe(stack_depth).f_locals)
2042 with self.builtin_trap:
2047 with self.builtin_trap:
2043 result = fn(*args)
2048 result = fn(*args)
2044 return result
2049 return result
2045
2050
2046 def cell_magic(self, magic_name, line, cell):
2051 def cell_magic(self, magic_name, line, cell):
2047 """Execute the given cell magic.
2052 """Execute the given cell magic.
2048 """
2053 """
2049 fn = self.find_cell_magic(magic_name)
2054 fn = self.find_cell_magic(magic_name)
2050 if fn is None:
2055 if fn is None:
2051 error("Cell magic function `%%%%%s` not found." % magic_name)
2056 error("Cell magic function `%%%%%s` not found." % magic_name)
2052 else:
2057 else:
2053 # Note: this is the distance in the stack to the user's frame.
2058 # Note: this is the distance in the stack to the user's frame.
2054 # This will need to be updated if the internal calling logic gets
2059 # This will need to be updated if the internal calling logic gets
2055 # refactored, or else we'll be expanding the wrong variables.
2060 # refactored, or else we'll be expanding the wrong variables.
2056 stack_depth = 2
2061 stack_depth = 2
2057 magic_arg_s = self.var_expand(line, stack_depth)
2062 magic_arg_s = self.var_expand(line, stack_depth)
2058 with self.builtin_trap:
2063 with self.builtin_trap:
2059 result = fn(line, cell)
2064 result = fn(line, cell)
2060 return result
2065 return result
2061
2066
2062 def find_line_magic(self, magic_name):
2067 def find_line_magic(self, magic_name):
2063 """Find and return a line magic by name.
2068 """Find and return a line magic by name.
2064
2069
2065 Returns None if the magic isn't found."""
2070 Returns None if the magic isn't found."""
2066 return self.magics_manager.magics['line'].get(magic_name)
2071 return self.magics_manager.magics['line'].get(magic_name)
2067
2072
2068 def find_cell_magic(self, magic_name):
2073 def find_cell_magic(self, magic_name):
2069 """Find and return a cell magic by name.
2074 """Find and return a cell magic by name.
2070
2075
2071 Returns None if the magic isn't found."""
2076 Returns None if the magic isn't found."""
2072 return self.magics_manager.magics['cell'].get(magic_name)
2077 return self.magics_manager.magics['cell'].get(magic_name)
2073
2078
2074 def find_magic(self, magic_name, magic_kind='line'):
2079 def find_magic(self, magic_name, magic_kind='line'):
2075 """Find and return a magic of the given type by name.
2080 """Find and return a magic of the given type by name.
2076
2081
2077 Returns None if the magic isn't found."""
2082 Returns None if the magic isn't found."""
2078 return self.magics_manager.magics[magic_kind].get(magic_name)
2083 return self.magics_manager.magics[magic_kind].get(magic_name)
2079
2084
2080 def magic(self, arg_s):
2085 def magic(self, arg_s):
2081 """DEPRECATED. Use line_magic() instead.
2086 """DEPRECATED. Use line_magic() instead.
2082
2087
2083 Call a magic function by name.
2088 Call a magic function by name.
2084
2089
2085 Input: a string containing the name of the magic function to call and
2090 Input: a string containing the name of the magic function to call and
2086 any additional arguments to be passed to the magic.
2091 any additional arguments to be passed to the magic.
2087
2092
2088 magic('name -opt foo bar') is equivalent to typing at the ipython
2093 magic('name -opt foo bar') is equivalent to typing at the ipython
2089 prompt:
2094 prompt:
2090
2095
2091 In[1]: %name -opt foo bar
2096 In[1]: %name -opt foo bar
2092
2097
2093 To call a magic without arguments, simply use magic('name').
2098 To call a magic without arguments, simply use magic('name').
2094
2099
2095 This provides a proper Python function to call IPython's magics in any
2100 This provides a proper Python function to call IPython's magics in any
2096 valid Python code you can type at the interpreter, including loops and
2101 valid Python code you can type at the interpreter, including loops and
2097 compound statements.
2102 compound statements.
2098 """
2103 """
2099 # TODO: should we issue a loud deprecation warning here?
2104 # TODO: should we issue a loud deprecation warning here?
2100 magic_name, _, magic_arg_s = arg_s.partition(' ')
2105 magic_name, _, magic_arg_s = arg_s.partition(' ')
2101 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
2106 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
2102 return self.line_magic(magic_name, magic_arg_s)
2107 return self.line_magic(magic_name, magic_arg_s)
2103
2108
2104 #-------------------------------------------------------------------------
2109 #-------------------------------------------------------------------------
2105 # Things related to macros
2110 # Things related to macros
2106 #-------------------------------------------------------------------------
2111 #-------------------------------------------------------------------------
2107
2112
2108 def define_macro(self, name, themacro):
2113 def define_macro(self, name, themacro):
2109 """Define a new macro
2114 """Define a new macro
2110
2115
2111 Parameters
2116 Parameters
2112 ----------
2117 ----------
2113 name : str
2118 name : str
2114 The name of the macro.
2119 The name of the macro.
2115 themacro : str or Macro
2120 themacro : str or Macro
2116 The action to do upon invoking the macro. If a string, a new
2121 The action to do upon invoking the macro. If a string, a new
2117 Macro object is created by passing the string to it.
2122 Macro object is created by passing the string to it.
2118 """
2123 """
2119
2124
2120 from IPython.core import macro
2125 from IPython.core import macro
2121
2126
2122 if isinstance(themacro, basestring):
2127 if isinstance(themacro, basestring):
2123 themacro = macro.Macro(themacro)
2128 themacro = macro.Macro(themacro)
2124 if not isinstance(themacro, macro.Macro):
2129 if not isinstance(themacro, macro.Macro):
2125 raise ValueError('A macro must be a string or a Macro instance.')
2130 raise ValueError('A macro must be a string or a Macro instance.')
2126 self.user_ns[name] = themacro
2131 self.user_ns[name] = themacro
2127
2132
2128 #-------------------------------------------------------------------------
2133 #-------------------------------------------------------------------------
2129 # Things related to the running of system commands
2134 # Things related to the running of system commands
2130 #-------------------------------------------------------------------------
2135 #-------------------------------------------------------------------------
2131
2136
2132 def system_piped(self, cmd):
2137 def system_piped(self, cmd):
2133 """Call the given cmd in a subprocess, piping stdout/err
2138 """Call the given cmd in a subprocess, piping stdout/err
2134
2139
2135 Parameters
2140 Parameters
2136 ----------
2141 ----------
2137 cmd : str
2142 cmd : str
2138 Command to execute (can not end in '&', as background processes are
2143 Command to execute (can not end in '&', as background processes are
2139 not supported. Should not be a command that expects input
2144 not supported. Should not be a command that expects input
2140 other than simple text.
2145 other than simple text.
2141 """
2146 """
2142 if cmd.rstrip().endswith('&'):
2147 if cmd.rstrip().endswith('&'):
2143 # this is *far* from a rigorous test
2148 # this is *far* from a rigorous test
2144 # We do not support backgrounding processes because we either use
2149 # We do not support backgrounding processes because we either use
2145 # pexpect or pipes to read from. Users can always just call
2150 # pexpect or pipes to read from. Users can always just call
2146 # os.system() or use ip.system=ip.system_raw
2151 # os.system() or use ip.system=ip.system_raw
2147 # if they really want a background process.
2152 # if they really want a background process.
2148 raise OSError("Background processes not supported.")
2153 raise OSError("Background processes not supported.")
2149
2154
2150 # we explicitly do NOT return the subprocess status code, because
2155 # we explicitly do NOT return the subprocess status code, because
2151 # a non-None value would trigger :func:`sys.displayhook` calls.
2156 # a non-None value would trigger :func:`sys.displayhook` calls.
2152 # Instead, we store the exit_code in user_ns.
2157 # Instead, we store the exit_code in user_ns.
2153 self.user_ns['_exit_code'] = system(self.var_expand(cmd, depth=2))
2158 self.user_ns['_exit_code'] = system(self.var_expand(cmd, depth=2))
2154
2159
2155 def system_raw(self, cmd):
2160 def system_raw(self, cmd):
2156 """Call the given cmd in a subprocess using os.system
2161 """Call the given cmd in a subprocess using os.system
2157
2162
2158 Parameters
2163 Parameters
2159 ----------
2164 ----------
2160 cmd : str
2165 cmd : str
2161 Command to execute.
2166 Command to execute.
2162 """
2167 """
2163 cmd = self.var_expand(cmd, depth=2)
2168 cmd = self.var_expand(cmd, depth=2)
2164 # protect os.system from UNC paths on Windows, which it can't handle:
2169 # protect os.system from UNC paths on Windows, which it can't handle:
2165 if sys.platform == 'win32':
2170 if sys.platform == 'win32':
2166 from IPython.utils._process_win32 import AvoidUNCPath
2171 from IPython.utils._process_win32 import AvoidUNCPath
2167 with AvoidUNCPath() as path:
2172 with AvoidUNCPath() as path:
2168 if path is not None:
2173 if path is not None:
2169 cmd = '"pushd %s &&"%s' % (path, cmd)
2174 cmd = '"pushd %s &&"%s' % (path, cmd)
2170 cmd = py3compat.unicode_to_str(cmd)
2175 cmd = py3compat.unicode_to_str(cmd)
2171 ec = os.system(cmd)
2176 ec = os.system(cmd)
2172 else:
2177 else:
2173 cmd = py3compat.unicode_to_str(cmd)
2178 cmd = py3compat.unicode_to_str(cmd)
2174 ec = os.system(cmd)
2179 ec = os.system(cmd)
2175
2180
2176 # We explicitly do NOT return the subprocess status code, because
2181 # We explicitly do NOT return the subprocess status code, because
2177 # a non-None value would trigger :func:`sys.displayhook` calls.
2182 # a non-None value would trigger :func:`sys.displayhook` calls.
2178 # Instead, we store the exit_code in user_ns.
2183 # Instead, we store the exit_code in user_ns.
2179 self.user_ns['_exit_code'] = ec
2184 self.user_ns['_exit_code'] = ec
2180
2185
2181 # use piped system by default, because it is better behaved
2186 # use piped system by default, because it is better behaved
2182 system = system_piped
2187 system = system_piped
2183
2188
2184 def getoutput(self, cmd, split=True):
2189 def getoutput(self, cmd, split=True):
2185 """Get output (possibly including stderr) from a subprocess.
2190 """Get output (possibly including stderr) from a subprocess.
2186
2191
2187 Parameters
2192 Parameters
2188 ----------
2193 ----------
2189 cmd : str
2194 cmd : str
2190 Command to execute (can not end in '&', as background processes are
2195 Command to execute (can not end in '&', as background processes are
2191 not supported.
2196 not supported.
2192 split : bool, optional
2197 split : bool, optional
2193
2198
2194 If True, split the output into an IPython SList. Otherwise, an
2199 If True, split the output into an IPython SList. Otherwise, an
2195 IPython LSString is returned. These are objects similar to normal
2200 IPython LSString is returned. These are objects similar to normal
2196 lists and strings, with a few convenience attributes for easier
2201 lists and strings, with a few convenience attributes for easier
2197 manipulation of line-based output. You can use '?' on them for
2202 manipulation of line-based output. You can use '?' on them for
2198 details.
2203 details.
2199 """
2204 """
2200 if cmd.rstrip().endswith('&'):
2205 if cmd.rstrip().endswith('&'):
2201 # this is *far* from a rigorous test
2206 # this is *far* from a rigorous test
2202 raise OSError("Background processes not supported.")
2207 raise OSError("Background processes not supported.")
2203 out = getoutput(self.var_expand(cmd, depth=2))
2208 out = getoutput(self.var_expand(cmd, depth=2))
2204 if split:
2209 if split:
2205 out = SList(out.splitlines())
2210 out = SList(out.splitlines())
2206 else:
2211 else:
2207 out = LSString(out)
2212 out = LSString(out)
2208 return out
2213 return out
2209
2214
2210 #-------------------------------------------------------------------------
2215 #-------------------------------------------------------------------------
2211 # Things related to aliases
2216 # Things related to aliases
2212 #-------------------------------------------------------------------------
2217 #-------------------------------------------------------------------------
2213
2218
2214 def init_alias(self):
2219 def init_alias(self):
2215 self.alias_manager = AliasManager(shell=self, config=self.config)
2220 self.alias_manager = AliasManager(shell=self, config=self.config)
2216 self.configurables.append(self.alias_manager)
2221 self.configurables.append(self.alias_manager)
2217 self.ns_table['alias'] = self.alias_manager.alias_table,
2222 self.ns_table['alias'] = self.alias_manager.alias_table,
2218
2223
2219 #-------------------------------------------------------------------------
2224 #-------------------------------------------------------------------------
2220 # Things related to extensions and plugins
2225 # Things related to extensions and plugins
2221 #-------------------------------------------------------------------------
2226 #-------------------------------------------------------------------------
2222
2227
2223 def init_extension_manager(self):
2228 def init_extension_manager(self):
2224 self.extension_manager = ExtensionManager(shell=self, config=self.config)
2229 self.extension_manager = ExtensionManager(shell=self, config=self.config)
2225 self.configurables.append(self.extension_manager)
2230 self.configurables.append(self.extension_manager)
2226
2231
2227 def init_plugin_manager(self):
2232 def init_plugin_manager(self):
2228 self.plugin_manager = PluginManager(config=self.config)
2233 self.plugin_manager = PluginManager(config=self.config)
2229 self.configurables.append(self.plugin_manager)
2234 self.configurables.append(self.plugin_manager)
2230
2235
2231
2236
2232 #-------------------------------------------------------------------------
2237 #-------------------------------------------------------------------------
2233 # Things related to payloads
2238 # Things related to payloads
2234 #-------------------------------------------------------------------------
2239 #-------------------------------------------------------------------------
2235
2240
2236 def init_payload(self):
2241 def init_payload(self):
2237 self.payload_manager = PayloadManager(config=self.config)
2242 self.payload_manager = PayloadManager(config=self.config)
2238 self.configurables.append(self.payload_manager)
2243 self.configurables.append(self.payload_manager)
2239
2244
2240 #-------------------------------------------------------------------------
2245 #-------------------------------------------------------------------------
2241 # Things related to the prefilter
2246 # Things related to the prefilter
2242 #-------------------------------------------------------------------------
2247 #-------------------------------------------------------------------------
2243
2248
2244 def init_prefilter(self):
2249 def init_prefilter(self):
2245 self.prefilter_manager = PrefilterManager(shell=self, config=self.config)
2250 self.prefilter_manager = PrefilterManager(shell=self, config=self.config)
2246 self.configurables.append(self.prefilter_manager)
2251 self.configurables.append(self.prefilter_manager)
2247 # Ultimately this will be refactored in the new interpreter code, but
2252 # Ultimately this will be refactored in the new interpreter code, but
2248 # for now, we should expose the main prefilter method (there's legacy
2253 # for now, we should expose the main prefilter method (there's legacy
2249 # code out there that may rely on this).
2254 # code out there that may rely on this).
2250 self.prefilter = self.prefilter_manager.prefilter_lines
2255 self.prefilter = self.prefilter_manager.prefilter_lines
2251
2256
2252 def auto_rewrite_input(self, cmd):
2257 def auto_rewrite_input(self, cmd):
2253 """Print to the screen the rewritten form of the user's command.
2258 """Print to the screen the rewritten form of the user's command.
2254
2259
2255 This shows visual feedback by rewriting input lines that cause
2260 This shows visual feedback by rewriting input lines that cause
2256 automatic calling to kick in, like::
2261 automatic calling to kick in, like::
2257
2262
2258 /f x
2263 /f x
2259
2264
2260 into::
2265 into::
2261
2266
2262 ------> f(x)
2267 ------> f(x)
2263
2268
2264 after the user's input prompt. This helps the user understand that the
2269 after the user's input prompt. This helps the user understand that the
2265 input line was transformed automatically by IPython.
2270 input line was transformed automatically by IPython.
2266 """
2271 """
2267 if not self.show_rewritten_input:
2272 if not self.show_rewritten_input:
2268 return
2273 return
2269
2274
2270 rw = self.prompt_manager.render('rewrite') + cmd
2275 rw = self.prompt_manager.render('rewrite') + cmd
2271
2276
2272 try:
2277 try:
2273 # plain ascii works better w/ pyreadline, on some machines, so
2278 # plain ascii works better w/ pyreadline, on some machines, so
2274 # we use it and only print uncolored rewrite if we have unicode
2279 # we use it and only print uncolored rewrite if we have unicode
2275 rw = str(rw)
2280 rw = str(rw)
2276 print >> io.stdout, rw
2281 print >> io.stdout, rw
2277 except UnicodeEncodeError:
2282 except UnicodeEncodeError:
2278 print "------> " + cmd
2283 print "------> " + cmd
2279
2284
2280 #-------------------------------------------------------------------------
2285 #-------------------------------------------------------------------------
2281 # Things related to extracting values/expressions from kernel and user_ns
2286 # Things related to extracting values/expressions from kernel and user_ns
2282 #-------------------------------------------------------------------------
2287 #-------------------------------------------------------------------------
2283
2288
2284 def _simple_error(self):
2289 def _simple_error(self):
2285 etype, value = sys.exc_info()[:2]
2290 etype, value = sys.exc_info()[:2]
2286 return u'[ERROR] {e.__name__}: {v}'.format(e=etype, v=value)
2291 return u'[ERROR] {e.__name__}: {v}'.format(e=etype, v=value)
2287
2292
2288 def user_variables(self, names):
2293 def user_variables(self, names):
2289 """Get a list of variable names from the user's namespace.
2294 """Get a list of variable names from the user's namespace.
2290
2295
2291 Parameters
2296 Parameters
2292 ----------
2297 ----------
2293 names : list of strings
2298 names : list of strings
2294 A list of names of variables to be read from the user namespace.
2299 A list of names of variables to be read from the user namespace.
2295
2300
2296 Returns
2301 Returns
2297 -------
2302 -------
2298 A dict, keyed by the input names and with the repr() of each value.
2303 A dict, keyed by the input names and with the repr() of each value.
2299 """
2304 """
2300 out = {}
2305 out = {}
2301 user_ns = self.user_ns
2306 user_ns = self.user_ns
2302 for varname in names:
2307 for varname in names:
2303 try:
2308 try:
2304 value = repr(user_ns[varname])
2309 value = repr(user_ns[varname])
2305 except:
2310 except:
2306 value = self._simple_error()
2311 value = self._simple_error()
2307 out[varname] = value
2312 out[varname] = value
2308 return out
2313 return out
2309
2314
2310 def user_expressions(self, expressions):
2315 def user_expressions(self, expressions):
2311 """Evaluate a dict of expressions in the user's namespace.
2316 """Evaluate a dict of expressions in the user's namespace.
2312
2317
2313 Parameters
2318 Parameters
2314 ----------
2319 ----------
2315 expressions : dict
2320 expressions : dict
2316 A dict with string keys and string values. The expression values
2321 A dict with string keys and string values. The expression values
2317 should be valid Python expressions, each of which will be evaluated
2322 should be valid Python expressions, each of which will be evaluated
2318 in the user namespace.
2323 in the user namespace.
2319
2324
2320 Returns
2325 Returns
2321 -------
2326 -------
2322 A dict, keyed like the input expressions dict, with the repr() of each
2327 A dict, keyed like the input expressions dict, with the repr() of each
2323 value.
2328 value.
2324 """
2329 """
2325 out = {}
2330 out = {}
2326 user_ns = self.user_ns
2331 user_ns = self.user_ns
2327 global_ns = self.user_global_ns
2332 global_ns = self.user_global_ns
2328 for key, expr in expressions.iteritems():
2333 for key, expr in expressions.iteritems():
2329 try:
2334 try:
2330 value = repr(eval(expr, global_ns, user_ns))
2335 value = repr(eval(expr, global_ns, user_ns))
2331 except:
2336 except:
2332 value = self._simple_error()
2337 value = self._simple_error()
2333 out[key] = value
2338 out[key] = value
2334 return out
2339 return out
2335
2340
2336 #-------------------------------------------------------------------------
2341 #-------------------------------------------------------------------------
2337 # Things related to the running of code
2342 # Things related to the running of code
2338 #-------------------------------------------------------------------------
2343 #-------------------------------------------------------------------------
2339
2344
2340 def ex(self, cmd):
2345 def ex(self, cmd):
2341 """Execute a normal python statement in user namespace."""
2346 """Execute a normal python statement in user namespace."""
2342 with self.builtin_trap:
2347 with self.builtin_trap:
2343 exec cmd in self.user_global_ns, self.user_ns
2348 exec cmd in self.user_global_ns, self.user_ns
2344
2349
2345 def ev(self, expr):
2350 def ev(self, expr):
2346 """Evaluate python expression expr in user namespace.
2351 """Evaluate python expression expr in user namespace.
2347
2352
2348 Returns the result of evaluation
2353 Returns the result of evaluation
2349 """
2354 """
2350 with self.builtin_trap:
2355 with self.builtin_trap:
2351 return eval(expr, self.user_global_ns, self.user_ns)
2356 return eval(expr, self.user_global_ns, self.user_ns)
2352
2357
2353 def safe_execfile(self, fname, *where, **kw):
2358 def safe_execfile(self, fname, *where, **kw):
2354 """A safe version of the builtin execfile().
2359 """A safe version of the builtin execfile().
2355
2360
2356 This version will never throw an exception, but instead print
2361 This version will never throw an exception, but instead print
2357 helpful error messages to the screen. This only works on pure
2362 helpful error messages to the screen. This only works on pure
2358 Python files with the .py extension.
2363 Python files with the .py extension.
2359
2364
2360 Parameters
2365 Parameters
2361 ----------
2366 ----------
2362 fname : string
2367 fname : string
2363 The name of the file to be executed.
2368 The name of the file to be executed.
2364 where : tuple
2369 where : tuple
2365 One or two namespaces, passed to execfile() as (globals,locals).
2370 One or two namespaces, passed to execfile() as (globals,locals).
2366 If only one is given, it is passed as both.
2371 If only one is given, it is passed as both.
2367 exit_ignore : bool (False)
2372 exit_ignore : bool (False)
2368 If True, then silence SystemExit for non-zero status (it is always
2373 If True, then silence SystemExit for non-zero status (it is always
2369 silenced for zero status, as it is so common).
2374 silenced for zero status, as it is so common).
2370 raise_exceptions : bool (False)
2375 raise_exceptions : bool (False)
2371 If True raise exceptions everywhere. Meant for testing.
2376 If True raise exceptions everywhere. Meant for testing.
2372
2377
2373 """
2378 """
2374 kw.setdefault('exit_ignore', False)
2379 kw.setdefault('exit_ignore', False)
2375 kw.setdefault('raise_exceptions', False)
2380 kw.setdefault('raise_exceptions', False)
2376
2381
2377 fname = os.path.abspath(os.path.expanduser(fname))
2382 fname = os.path.abspath(os.path.expanduser(fname))
2378
2383
2379 # Make sure we can open the file
2384 # Make sure we can open the file
2380 try:
2385 try:
2381 with open(fname) as thefile:
2386 with open(fname) as thefile:
2382 pass
2387 pass
2383 except:
2388 except:
2384 warn('Could not open file <%s> for safe execution.' % fname)
2389 warn('Could not open file <%s> for safe execution.' % fname)
2385 return
2390 return
2386
2391
2387 # Find things also in current directory. This is needed to mimic the
2392 # Find things also in current directory. This is needed to mimic the
2388 # behavior of running a script from the system command line, where
2393 # behavior of running a script from the system command line, where
2389 # Python inserts the script's directory into sys.path
2394 # Python inserts the script's directory into sys.path
2390 dname = os.path.dirname(fname)
2395 dname = os.path.dirname(fname)
2391
2396
2392 with prepended_to_syspath(dname):
2397 with prepended_to_syspath(dname):
2393 try:
2398 try:
2394 py3compat.execfile(fname,*where)
2399 py3compat.execfile(fname,*where)
2395 except SystemExit, status:
2400 except SystemExit, status:
2396 # If the call was made with 0 or None exit status (sys.exit(0)
2401 # If the call was made with 0 or None exit status (sys.exit(0)
2397 # or sys.exit() ), don't bother showing a traceback, as both of
2402 # or sys.exit() ), don't bother showing a traceback, as both of
2398 # these are considered normal by the OS:
2403 # these are considered normal by the OS:
2399 # > python -c'import sys;sys.exit(0)'; echo $?
2404 # > python -c'import sys;sys.exit(0)'; echo $?
2400 # 0
2405 # 0
2401 # > python -c'import sys;sys.exit()'; echo $?
2406 # > python -c'import sys;sys.exit()'; echo $?
2402 # 0
2407 # 0
2403 # For other exit status, we show the exception unless
2408 # For other exit status, we show the exception unless
2404 # explicitly silenced, but only in short form.
2409 # explicitly silenced, but only in short form.
2405 if kw['raise_exceptions']:
2410 if kw['raise_exceptions']:
2406 raise
2411 raise
2407 if status.code not in (0, None) and not kw['exit_ignore']:
2412 if status.code not in (0, None) and not kw['exit_ignore']:
2408 self.showtraceback(exception_only=True)
2413 self.showtraceback(exception_only=True)
2409 except:
2414 except:
2410 if kw['raise_exceptions']:
2415 if kw['raise_exceptions']:
2411 raise
2416 raise
2412 self.showtraceback()
2417 self.showtraceback()
2413
2418
2414 def safe_execfile_ipy(self, fname):
2419 def safe_execfile_ipy(self, fname):
2415 """Like safe_execfile, but for .ipy files with IPython syntax.
2420 """Like safe_execfile, but for .ipy files with IPython syntax.
2416
2421
2417 Parameters
2422 Parameters
2418 ----------
2423 ----------
2419 fname : str
2424 fname : str
2420 The name of the file to execute. The filename must have a
2425 The name of the file to execute. The filename must have a
2421 .ipy extension.
2426 .ipy extension.
2422 """
2427 """
2423 fname = os.path.abspath(os.path.expanduser(fname))
2428 fname = os.path.abspath(os.path.expanduser(fname))
2424
2429
2425 # Make sure we can open the file
2430 # Make sure we can open the file
2426 try:
2431 try:
2427 with open(fname) as thefile:
2432 with open(fname) as thefile:
2428 pass
2433 pass
2429 except:
2434 except:
2430 warn('Could not open file <%s> for safe execution.' % fname)
2435 warn('Could not open file <%s> for safe execution.' % fname)
2431 return
2436 return
2432
2437
2433 # Find things also in current directory. This is needed to mimic the
2438 # Find things also in current directory. This is needed to mimic the
2434 # behavior of running a script from the system command line, where
2439 # behavior of running a script from the system command line, where
2435 # Python inserts the script's directory into sys.path
2440 # Python inserts the script's directory into sys.path
2436 dname = os.path.dirname(fname)
2441 dname = os.path.dirname(fname)
2437
2442
2438 with prepended_to_syspath(dname):
2443 with prepended_to_syspath(dname):
2439 try:
2444 try:
2440 with open(fname) as thefile:
2445 with open(fname) as thefile:
2441 # self.run_cell currently captures all exceptions
2446 # self.run_cell currently captures all exceptions
2442 # raised in user code. It would be nice if there were
2447 # raised in user code. It would be nice if there were
2443 # versions of runlines, execfile that did raise, so
2448 # versions of runlines, execfile that did raise, so
2444 # we could catch the errors.
2449 # we could catch the errors.
2445 self.run_cell(thefile.read(), store_history=False)
2450 self.run_cell(thefile.read(), store_history=False)
2446 except:
2451 except:
2447 self.showtraceback()
2452 self.showtraceback()
2448 warn('Unknown failure executing file: <%s>' % fname)
2453 warn('Unknown failure executing file: <%s>' % fname)
2449
2454
2450 def safe_run_module(self, mod_name, where):
2455 def safe_run_module(self, mod_name, where):
2451 """A safe version of runpy.run_module().
2456 """A safe version of runpy.run_module().
2452
2457
2453 This version will never throw an exception, but instead print
2458 This version will never throw an exception, but instead print
2454 helpful error messages to the screen.
2459 helpful error messages to the screen.
2455
2460
2456 Parameters
2461 Parameters
2457 ----------
2462 ----------
2458 mod_name : string
2463 mod_name : string
2459 The name of the module to be executed.
2464 The name of the module to be executed.
2460 where : dict
2465 where : dict
2461 The globals namespace.
2466 The globals namespace.
2462 """
2467 """
2463 try:
2468 try:
2464 where.update(
2469 where.update(
2465 runpy.run_module(str(mod_name), run_name="__main__",
2470 runpy.run_module(str(mod_name), run_name="__main__",
2466 alter_sys=True)
2471 alter_sys=True)
2467 )
2472 )
2468 except:
2473 except:
2469 self.showtraceback()
2474 self.showtraceback()
2470 warn('Unknown failure executing module: <%s>' % mod_name)
2475 warn('Unknown failure executing module: <%s>' % mod_name)
2471
2476
2472 def call_cell_magic(self, raw_cell, store_history=False):
2473 line, _, cell = raw_cell.partition(os.linesep)
2474 magic_name, _, line = line.partition(' ')
2475 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
2476 return self.cell_magic(magic_name, line, cell)
2477
2478 def _cell_magic(self, magic_name, line):
2477 def _cell_magic(self, magic_name, line):
2478 """Special method to call a cell magic with the data stored in self.
2479 """
2479 cell = self._current_cell_magic_body
2480 cell = self._current_cell_magic_body
2480 self._current_cell_magic_body = None
2481 self._current_cell_magic_body = None
2481 return self.cell_magic(magic_name, line, cell)
2482 return self.cell_magic(magic_name, line, cell)
2482
2483
2483 def run_cell(self, raw_cell, store_history=False, silent=False):
2484 def run_cell(self, raw_cell, store_history=False, silent=False):
2484 """Run a complete IPython cell.
2485 """Run a complete IPython cell.
2485
2486
2486 Parameters
2487 Parameters
2487 ----------
2488 ----------
2488 raw_cell : str
2489 raw_cell : str
2489 The code (including IPython code such as %magic functions) to run.
2490 The code (including IPython code such as %magic functions) to run.
2490 store_history : bool
2491 store_history : bool
2491 If True, the raw and translated cell will be stored in IPython's
2492 If True, the raw and translated cell will be stored in IPython's
2492 history. For user code calling back into IPython's machinery, this
2493 history. For user code calling back into IPython's machinery, this
2493 should be set to False.
2494 should be set to False.
2494 silent : bool
2495 silent : bool
2495 If True, avoid side-effets, such as implicit displayhooks, history,
2496 If True, avoid side-effets, such as implicit displayhooks, history,
2496 and logging. silent=True forces store_history=False.
2497 and logging. silent=True forces store_history=False.
2497 """
2498 """
2498 if (not raw_cell) or raw_cell.isspace():
2499 if (not raw_cell) or raw_cell.isspace():
2499 return
2500 return
2500
2501
2501 if silent:
2502 if silent:
2502 store_history = False
2503 store_history = False
2503
2504
2504 self.input_splitter.push(raw_cell)
2505 self.input_splitter.push(raw_cell)
2505
2506
2506 # Check for cell magics, which leave state behind. This interface is
2507 # Check for cell magics, which leave state behind. This interface is
2507 # ugly, we need to do something cleaner later... Now the logic is
2508 # ugly, we need to do something cleaner later... Now the logic is
2508 # simply that the input_splitter remembers if there was a cell magic,
2509 # simply that the input_splitter remembers if there was a cell magic,
2509 # and in that case we grab the cell body.
2510 # and in that case we grab the cell body.
2510 if self.input_splitter.cell_magic_body is not None:
2511 if self.input_splitter.cell_magic_parts:
2511 self._current_cell_magic_body = self.input_splitter.cell_magic_body
2512 self._current_cell_magic_body = \
2513 ''.join(self.input_splitter.cell_magic_parts)
2512 cell = self.input_splitter.source_reset()
2514 cell = self.input_splitter.source_reset()
2513
2515
2514 with self.builtin_trap:
2516 with self.builtin_trap:
2515 prefilter_failed = False
2517 prefilter_failed = False
2516 if len(cell.splitlines()) == 1:
2518 if len(cell.splitlines()) == 1:
2517 try:
2519 try:
2518 # use prefilter_lines to handle trailing newlines
2520 # use prefilter_lines to handle trailing newlines
2519 # restore trailing newline for ast.parse
2521 # restore trailing newline for ast.parse
2520 cell = self.prefilter_manager.prefilter_lines(cell) + '\n'
2522 cell = self.prefilter_manager.prefilter_lines(cell) + '\n'
2521 except AliasError as e:
2523 except AliasError as e:
2522 error(e)
2524 error(e)
2523 prefilter_failed = True
2525 prefilter_failed = True
2524 except Exception:
2526 except Exception:
2525 # don't allow prefilter errors to crash IPython
2527 # don't allow prefilter errors to crash IPython
2526 self.showtraceback()
2528 self.showtraceback()
2527 prefilter_failed = True
2529 prefilter_failed = True
2528
2530
2529 # Store raw and processed history
2531 # Store raw and processed history
2530 if store_history:
2532 if store_history:
2531 self.history_manager.store_inputs(self.execution_count,
2533 self.history_manager.store_inputs(self.execution_count,
2532 cell, raw_cell)
2534 cell, raw_cell)
2533 if not silent:
2535 if not silent:
2534 self.logger.log(cell, raw_cell)
2536 self.logger.log(cell, raw_cell)
2535
2537
2536 if not prefilter_failed:
2538 if not prefilter_failed:
2537 # don't run if prefilter failed
2539 # don't run if prefilter failed
2538 cell_name = self.compile.cache(cell, self.execution_count)
2540 cell_name = self.compile.cache(cell, self.execution_count)
2539
2541
2540 with self.display_trap:
2542 with self.display_trap:
2541 try:
2543 try:
2542 code_ast = self.compile.ast_parse(cell,
2544 code_ast = self.compile.ast_parse(cell,
2543 filename=cell_name)
2545 filename=cell_name)
2544 except IndentationError:
2546 except IndentationError:
2545 self.showindentationerror()
2547 self.showindentationerror()
2546 if store_history:
2548 if store_history:
2547 self.execution_count += 1
2549 self.execution_count += 1
2548 return None
2550 return None
2549 except (OverflowError, SyntaxError, ValueError, TypeError,
2551 except (OverflowError, SyntaxError, ValueError, TypeError,
2550 MemoryError):
2552 MemoryError):
2551 self.showsyntaxerror()
2553 self.showsyntaxerror()
2552 if store_history:
2554 if store_history:
2553 self.execution_count += 1
2555 self.execution_count += 1
2554 return None
2556 return None
2555
2557
2556 interactivity = "none" if silent else "last_expr"
2558 interactivity = "none" if silent else "last_expr"
2557 self.run_ast_nodes(code_ast.body, cell_name,
2559 self.run_ast_nodes(code_ast.body, cell_name,
2558 interactivity=interactivity)
2560 interactivity=interactivity)
2559
2561
2560 # Execute any registered post-execution functions.
2562 # Execute any registered post-execution functions.
2561 # unless we are silent
2563 # unless we are silent
2562 post_exec = [] if silent else self._post_execute.iteritems()
2564 post_exec = [] if silent else self._post_execute.iteritems()
2563
2565
2564 for func, status in post_exec:
2566 for func, status in post_exec:
2565 if self.disable_failing_post_execute and not status:
2567 if self.disable_failing_post_execute and not status:
2566 continue
2568 continue
2567 try:
2569 try:
2568 func()
2570 func()
2569 except KeyboardInterrupt:
2571 except KeyboardInterrupt:
2570 print >> io.stderr, "\nKeyboardInterrupt"
2572 print >> io.stderr, "\nKeyboardInterrupt"
2571 except Exception:
2573 except Exception:
2572 # register as failing:
2574 # register as failing:
2573 self._post_execute[func] = False
2575 self._post_execute[func] = False
2574 self.showtraceback()
2576 self.showtraceback()
2575 print >> io.stderr, '\n'.join([
2577 print >> io.stderr, '\n'.join([
2576 "post-execution function %r produced an error." % func,
2578 "post-execution function %r produced an error." % func,
2577 "If this problem persists, you can disable failing post-exec functions with:",
2579 "If this problem persists, you can disable failing post-exec functions with:",
2578 "",
2580 "",
2579 " get_ipython().disable_failing_post_execute = True"
2581 " get_ipython().disable_failing_post_execute = True"
2580 ])
2582 ])
2581
2583
2582 if store_history:
2584 if store_history:
2583 # Write output to the database. Does nothing unless
2585 # Write output to the database. Does nothing unless
2584 # history output logging is enabled.
2586 # history output logging is enabled.
2585 self.history_manager.store_output(self.execution_count)
2587 self.history_manager.store_output(self.execution_count)
2586 # Each cell is a *single* input, regardless of how many lines it has
2588 # Each cell is a *single* input, regardless of how many lines it has
2587 self.execution_count += 1
2589 self.execution_count += 1
2588
2590
2589 def run_ast_nodes(self, nodelist, cell_name, interactivity='last_expr'):
2591 def run_ast_nodes(self, nodelist, cell_name, interactivity='last_expr'):
2590 """Run a sequence of AST nodes. The execution mode depends on the
2592 """Run a sequence of AST nodes. The execution mode depends on the
2591 interactivity parameter.
2593 interactivity parameter.
2592
2594
2593 Parameters
2595 Parameters
2594 ----------
2596 ----------
2595 nodelist : list
2597 nodelist : list
2596 A sequence of AST nodes to run.
2598 A sequence of AST nodes to run.
2597 cell_name : str
2599 cell_name : str
2598 Will be passed to the compiler as the filename of the cell. Typically
2600 Will be passed to the compiler as the filename of the cell. Typically
2599 the value returned by ip.compile.cache(cell).
2601 the value returned by ip.compile.cache(cell).
2600 interactivity : str
2602 interactivity : str
2601 'all', 'last', 'last_expr' or 'none', specifying which nodes should be
2603 'all', 'last', 'last_expr' or 'none', specifying which nodes should be
2602 run interactively (displaying output from expressions). 'last_expr'
2604 run interactively (displaying output from expressions). 'last_expr'
2603 will run the last node interactively only if it is an expression (i.e.
2605 will run the last node interactively only if it is an expression (i.e.
2604 expressions in loops or other blocks are not displayed. Other values
2606 expressions in loops or other blocks are not displayed. Other values
2605 for this parameter will raise a ValueError.
2607 for this parameter will raise a ValueError.
2606 """
2608 """
2607 if not nodelist:
2609 if not nodelist:
2608 return
2610 return
2609
2611
2610 if interactivity == 'last_expr':
2612 if interactivity == 'last_expr':
2611 if isinstance(nodelist[-1], ast.Expr):
2613 if isinstance(nodelist[-1], ast.Expr):
2612 interactivity = "last"
2614 interactivity = "last"
2613 else:
2615 else:
2614 interactivity = "none"
2616 interactivity = "none"
2615
2617
2616 if interactivity == 'none':
2618 if interactivity == 'none':
2617 to_run_exec, to_run_interactive = nodelist, []
2619 to_run_exec, to_run_interactive = nodelist, []
2618 elif interactivity == 'last':
2620 elif interactivity == 'last':
2619 to_run_exec, to_run_interactive = nodelist[:-1], nodelist[-1:]
2621 to_run_exec, to_run_interactive = nodelist[:-1], nodelist[-1:]
2620 elif interactivity == 'all':
2622 elif interactivity == 'all':
2621 to_run_exec, to_run_interactive = [], nodelist
2623 to_run_exec, to_run_interactive = [], nodelist
2622 else:
2624 else:
2623 raise ValueError("Interactivity was %r" % interactivity)
2625 raise ValueError("Interactivity was %r" % interactivity)
2624
2626
2625 exec_count = self.execution_count
2627 exec_count = self.execution_count
2626
2628
2627 try:
2629 try:
2628 for i, node in enumerate(to_run_exec):
2630 for i, node in enumerate(to_run_exec):
2629 mod = ast.Module([node])
2631 mod = ast.Module([node])
2630 code = self.compile(mod, cell_name, "exec")
2632 code = self.compile(mod, cell_name, "exec")
2631 if self.run_code(code):
2633 if self.run_code(code):
2632 return True
2634 return True
2633
2635
2634 for i, node in enumerate(to_run_interactive):
2636 for i, node in enumerate(to_run_interactive):
2635 mod = ast.Interactive([node])
2637 mod = ast.Interactive([node])
2636 code = self.compile(mod, cell_name, "single")
2638 code = self.compile(mod, cell_name, "single")
2637 if self.run_code(code):
2639 if self.run_code(code):
2638 return True
2640 return True
2639
2641
2640 # Flush softspace
2642 # Flush softspace
2641 if softspace(sys.stdout, 0):
2643 if softspace(sys.stdout, 0):
2642 print
2644 print
2643
2645
2644 except:
2646 except:
2645 # It's possible to have exceptions raised here, typically by
2647 # It's possible to have exceptions raised here, typically by
2646 # compilation of odd code (such as a naked 'return' outside a
2648 # compilation of odd code (such as a naked 'return' outside a
2647 # function) that did parse but isn't valid. Typically the exception
2649 # function) that did parse but isn't valid. Typically the exception
2648 # is a SyntaxError, but it's safest just to catch anything and show
2650 # is a SyntaxError, but it's safest just to catch anything and show
2649 # the user a traceback.
2651 # the user a traceback.
2650
2652
2651 # We do only one try/except outside the loop to minimize the impact
2653 # We do only one try/except outside the loop to minimize the impact
2652 # on runtime, and also because if any node in the node list is
2654 # on runtime, and also because if any node in the node list is
2653 # broken, we should stop execution completely.
2655 # broken, we should stop execution completely.
2654 self.showtraceback()
2656 self.showtraceback()
2655
2657
2656 return False
2658 return False
2657
2659
2658 def run_code(self, code_obj):
2660 def run_code(self, code_obj):
2659 """Execute a code object.
2661 """Execute a code object.
2660
2662
2661 When an exception occurs, self.showtraceback() is called to display a
2663 When an exception occurs, self.showtraceback() is called to display a
2662 traceback.
2664 traceback.
2663
2665
2664 Parameters
2666 Parameters
2665 ----------
2667 ----------
2666 code_obj : code object
2668 code_obj : code object
2667 A compiled code object, to be executed
2669 A compiled code object, to be executed
2668
2670
2669 Returns
2671 Returns
2670 -------
2672 -------
2671 False : successful execution.
2673 False : successful execution.
2672 True : an error occurred.
2674 True : an error occurred.
2673 """
2675 """
2674
2676
2675 # Set our own excepthook in case the user code tries to call it
2677 # Set our own excepthook in case the user code tries to call it
2676 # directly, so that the IPython crash handler doesn't get triggered
2678 # directly, so that the IPython crash handler doesn't get triggered
2677 old_excepthook,sys.excepthook = sys.excepthook, self.excepthook
2679 old_excepthook,sys.excepthook = sys.excepthook, self.excepthook
2678
2680
2679 # we save the original sys.excepthook in the instance, in case config
2681 # we save the original sys.excepthook in the instance, in case config
2680 # code (such as magics) needs access to it.
2682 # code (such as magics) needs access to it.
2681 self.sys_excepthook = old_excepthook
2683 self.sys_excepthook = old_excepthook
2682 outflag = 1 # happens in more places, so it's easier as default
2684 outflag = 1 # happens in more places, so it's easier as default
2683 try:
2685 try:
2684 try:
2686 try:
2685 self.hooks.pre_run_code_hook()
2687 self.hooks.pre_run_code_hook()
2686 #rprint('Running code', repr(code_obj)) # dbg
2688 #rprint('Running code', repr(code_obj)) # dbg
2687 exec code_obj in self.user_global_ns, self.user_ns
2689 exec code_obj in self.user_global_ns, self.user_ns
2688 finally:
2690 finally:
2689 # Reset our crash handler in place
2691 # Reset our crash handler in place
2690 sys.excepthook = old_excepthook
2692 sys.excepthook = old_excepthook
2691 except SystemExit:
2693 except SystemExit:
2692 self.showtraceback(exception_only=True)
2694 self.showtraceback(exception_only=True)
2693 warn("To exit: use 'exit', 'quit', or Ctrl-D.", level=1)
2695 warn("To exit: use 'exit', 'quit', or Ctrl-D.", level=1)
2694 except self.custom_exceptions:
2696 except self.custom_exceptions:
2695 etype,value,tb = sys.exc_info()
2697 etype,value,tb = sys.exc_info()
2696 self.CustomTB(etype,value,tb)
2698 self.CustomTB(etype,value,tb)
2697 except:
2699 except:
2698 self.showtraceback()
2700 self.showtraceback()
2699 else:
2701 else:
2700 outflag = 0
2702 outflag = 0
2701 return outflag
2703 return outflag
2702
2704
2703 # For backwards compatibility
2705 # For backwards compatibility
2704 runcode = run_code
2706 runcode = run_code
2705
2707
2706 #-------------------------------------------------------------------------
2708 #-------------------------------------------------------------------------
2707 # Things related to GUI support and pylab
2709 # Things related to GUI support and pylab
2708 #-------------------------------------------------------------------------
2710 #-------------------------------------------------------------------------
2709
2711
2710 def enable_gui(self, gui=None):
2712 def enable_gui(self, gui=None):
2711 raise NotImplementedError('Implement enable_gui in a subclass')
2713 raise NotImplementedError('Implement enable_gui in a subclass')
2712
2714
2713 def enable_pylab(self, gui=None, import_all=True):
2715 def enable_pylab(self, gui=None, import_all=True):
2714 """Activate pylab support at runtime.
2716 """Activate pylab support at runtime.
2715
2717
2716 This turns on support for matplotlib, preloads into the interactive
2718 This turns on support for matplotlib, preloads into the interactive
2717 namespace all of numpy and pylab, and configures IPython to correctly
2719 namespace all of numpy and pylab, and configures IPython to correctly
2718 interact with the GUI event loop. The GUI backend to be used can be
2720 interact with the GUI event loop. The GUI backend to be used can be
2719 optionally selected with the optional :param:`gui` argument.
2721 optionally selected with the optional :param:`gui` argument.
2720
2722
2721 Parameters
2723 Parameters
2722 ----------
2724 ----------
2723 gui : optional, string
2725 gui : optional, string
2724
2726
2725 If given, dictates the choice of matplotlib GUI backend to use
2727 If given, dictates the choice of matplotlib GUI backend to use
2726 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
2728 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
2727 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
2729 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
2728 matplotlib (as dictated by the matplotlib build-time options plus the
2730 matplotlib (as dictated by the matplotlib build-time options plus the
2729 user's matplotlibrc configuration file). Note that not all backends
2731 user's matplotlibrc configuration file). Note that not all backends
2730 make sense in all contexts, for example a terminal ipython can't
2732 make sense in all contexts, for example a terminal ipython can't
2731 display figures inline.
2733 display figures inline.
2732 """
2734 """
2733 from IPython.core.pylabtools import mpl_runner
2735 from IPython.core.pylabtools import mpl_runner
2734 # We want to prevent the loading of pylab to pollute the user's
2736 # We want to prevent the loading of pylab to pollute the user's
2735 # namespace as shown by the %who* magics, so we execute the activation
2737 # namespace as shown by the %who* magics, so we execute the activation
2736 # code in an empty namespace, and we update *both* user_ns and
2738 # code in an empty namespace, and we update *both* user_ns and
2737 # user_ns_hidden with this information.
2739 # user_ns_hidden with this information.
2738 ns = {}
2740 ns = {}
2739 try:
2741 try:
2740 gui = pylab_activate(ns, gui, import_all, self)
2742 gui = pylab_activate(ns, gui, import_all, self)
2741 except KeyError:
2743 except KeyError:
2742 error("Backend %r not supported" % gui)
2744 error("Backend %r not supported" % gui)
2743 return
2745 return
2744 self.user_ns.update(ns)
2746 self.user_ns.update(ns)
2745 self.user_ns_hidden.update(ns)
2747 self.user_ns_hidden.update(ns)
2746 # Now we must activate the gui pylab wants to use, and fix %run to take
2748 # Now we must activate the gui pylab wants to use, and fix %run to take
2747 # plot updates into account
2749 # plot updates into account
2748 self.enable_gui(gui)
2750 self.enable_gui(gui)
2749 self.magics_manager.registry['ExecutionMagics'].default_runner = \
2751 self.magics_manager.registry['ExecutionMagics'].default_runner = \
2750 mpl_runner(self.safe_execfile)
2752 mpl_runner(self.safe_execfile)
2751
2753
2752 #-------------------------------------------------------------------------
2754 #-------------------------------------------------------------------------
2753 # Utilities
2755 # Utilities
2754 #-------------------------------------------------------------------------
2756 #-------------------------------------------------------------------------
2755
2757
2756 def var_expand(self, cmd, depth=0, formatter=DollarFormatter()):
2758 def var_expand(self, cmd, depth=0, formatter=DollarFormatter()):
2757 """Expand python variables in a string.
2759 """Expand python variables in a string.
2758
2760
2759 The depth argument indicates how many frames above the caller should
2761 The depth argument indicates how many frames above the caller should
2760 be walked to look for the local namespace where to expand variables.
2762 be walked to look for the local namespace where to expand variables.
2761
2763
2762 The global namespace for expansion is always the user's interactive
2764 The global namespace for expansion is always the user's interactive
2763 namespace.
2765 namespace.
2764 """
2766 """
2765 ns = self.user_ns.copy()
2767 ns = self.user_ns.copy()
2766 ns.update(sys._getframe(depth+1).f_locals)
2768 ns.update(sys._getframe(depth+1).f_locals)
2767 ns.pop('self', None)
2769 ns.pop('self', None)
2768 try:
2770 try:
2769 cmd = formatter.format(cmd, **ns)
2771 cmd = formatter.format(cmd, **ns)
2770 except Exception:
2772 except Exception:
2771 # if formatter couldn't format, just let it go untransformed
2773 # if formatter couldn't format, just let it go untransformed
2772 pass
2774 pass
2773 return cmd
2775 return cmd
2774
2776
2775 def mktempfile(self, data=None, prefix='ipython_edit_'):
2777 def mktempfile(self, data=None, prefix='ipython_edit_'):
2776 """Make a new tempfile and return its filename.
2778 """Make a new tempfile and return its filename.
2777
2779
2778 This makes a call to tempfile.mktemp, but it registers the created
2780 This makes a call to tempfile.mktemp, but it registers the created
2779 filename internally so ipython cleans it up at exit time.
2781 filename internally so ipython cleans it up at exit time.
2780
2782
2781 Optional inputs:
2783 Optional inputs:
2782
2784
2783 - data(None): if data is given, it gets written out to the temp file
2785 - data(None): if data is given, it gets written out to the temp file
2784 immediately, and the file is closed again."""
2786 immediately, and the file is closed again."""
2785
2787
2786 filename = tempfile.mktemp('.py', prefix)
2788 filename = tempfile.mktemp('.py', prefix)
2787 self.tempfiles.append(filename)
2789 self.tempfiles.append(filename)
2788
2790
2789 if data:
2791 if data:
2790 tmp_file = open(filename,'w')
2792 tmp_file = open(filename,'w')
2791 tmp_file.write(data)
2793 tmp_file.write(data)
2792 tmp_file.close()
2794 tmp_file.close()
2793 return filename
2795 return filename
2794
2796
2795 # TODO: This should be removed when Term is refactored.
2797 # TODO: This should be removed when Term is refactored.
2796 def write(self,data):
2798 def write(self,data):
2797 """Write a string to the default output"""
2799 """Write a string to the default output"""
2798 io.stdout.write(data)
2800 io.stdout.write(data)
2799
2801
2800 # TODO: This should be removed when Term is refactored.
2802 # TODO: This should be removed when Term is refactored.
2801 def write_err(self,data):
2803 def write_err(self,data):
2802 """Write a string to the default error output"""
2804 """Write a string to the default error output"""
2803 io.stderr.write(data)
2805 io.stderr.write(data)
2804
2806
2805 def ask_yes_no(self, prompt, default=None):
2807 def ask_yes_no(self, prompt, default=None):
2806 if self.quiet:
2808 if self.quiet:
2807 return True
2809 return True
2808 return ask_yes_no(prompt,default)
2810 return ask_yes_no(prompt,default)
2809
2811
2810 def show_usage(self):
2812 def show_usage(self):
2811 """Show a usage message"""
2813 """Show a usage message"""
2812 page.page(IPython.core.usage.interactive_usage)
2814 page.page(IPython.core.usage.interactive_usage)
2813
2815
2814 def extract_input_lines(self, range_str, raw=False):
2816 def extract_input_lines(self, range_str, raw=False):
2815 """Return as a string a set of input history slices.
2817 """Return as a string a set of input history slices.
2816
2818
2817 Parameters
2819 Parameters
2818 ----------
2820 ----------
2819 range_str : string
2821 range_str : string
2820 The set of slices is given as a string, like "~5/6-~4/2 4:8 9",
2822 The set of slices is given as a string, like "~5/6-~4/2 4:8 9",
2821 since this function is for use by magic functions which get their
2823 since this function is for use by magic functions which get their
2822 arguments as strings. The number before the / is the session
2824 arguments as strings. The number before the / is the session
2823 number: ~n goes n back from the current session.
2825 number: ~n goes n back from the current session.
2824
2826
2825 Optional Parameters:
2827 Optional Parameters:
2826 - raw(False): by default, the processed input is used. If this is
2828 - raw(False): by default, the processed input is used. If this is
2827 true, the raw input history is used instead.
2829 true, the raw input history is used instead.
2828
2830
2829 Note that slices can be called with two notations:
2831 Note that slices can be called with two notations:
2830
2832
2831 N:M -> standard python form, means including items N...(M-1).
2833 N:M -> standard python form, means including items N...(M-1).
2832
2834
2833 N-M -> include items N..M (closed endpoint)."""
2835 N-M -> include items N..M (closed endpoint)."""
2834 lines = self.history_manager.get_range_by_str(range_str, raw=raw)
2836 lines = self.history_manager.get_range_by_str(range_str, raw=raw)
2835 return "\n".join(x for _, _, x in lines)
2837 return "\n".join(x for _, _, x in lines)
2836
2838
2837 def find_user_code(self, target, raw=True, py_only=False):
2839 def find_user_code(self, target, raw=True, py_only=False):
2838 """Get a code string from history, file, url, or a string or macro.
2840 """Get a code string from history, file, url, or a string or macro.
2839
2841
2840 This is mainly used by magic functions.
2842 This is mainly used by magic functions.
2841
2843
2842 Parameters
2844 Parameters
2843 ----------
2845 ----------
2844
2846
2845 target : str
2847 target : str
2846
2848
2847 A string specifying code to retrieve. This will be tried respectively
2849 A string specifying code to retrieve. This will be tried respectively
2848 as: ranges of input history (see %history for syntax), url,
2850 as: ranges of input history (see %history for syntax), url,
2849 correspnding .py file, filename, or an expression evaluating to a
2851 correspnding .py file, filename, or an expression evaluating to a
2850 string or Macro in the user namespace.
2852 string or Macro in the user namespace.
2851
2853
2852 raw : bool
2854 raw : bool
2853 If true (default), retrieve raw history. Has no effect on the other
2855 If true (default), retrieve raw history. Has no effect on the other
2854 retrieval mechanisms.
2856 retrieval mechanisms.
2855
2857
2856 py_only : bool (default False)
2858 py_only : bool (default False)
2857 Only try to fetch python code, do not try alternative methods to decode file
2859 Only try to fetch python code, do not try alternative methods to decode file
2858 if unicode fails.
2860 if unicode fails.
2859
2861
2860 Returns
2862 Returns
2861 -------
2863 -------
2862 A string of code.
2864 A string of code.
2863
2865
2864 ValueError is raised if nothing is found, and TypeError if it evaluates
2866 ValueError is raised if nothing is found, and TypeError if it evaluates
2865 to an object of another type. In each case, .args[0] is a printable
2867 to an object of another type. In each case, .args[0] is a printable
2866 message.
2868 message.
2867 """
2869 """
2868 code = self.extract_input_lines(target, raw=raw) # Grab history
2870 code = self.extract_input_lines(target, raw=raw) # Grab history
2869 if code:
2871 if code:
2870 return code
2872 return code
2871 utarget = unquote_filename(target)
2873 utarget = unquote_filename(target)
2872 try:
2874 try:
2873 if utarget.startswith(('http://', 'https://')):
2875 if utarget.startswith(('http://', 'https://')):
2874 return openpy.read_py_url(utarget, skip_encoding_cookie=True)
2876 return openpy.read_py_url(utarget, skip_encoding_cookie=True)
2875 except UnicodeDecodeError:
2877 except UnicodeDecodeError:
2876 if not py_only :
2878 if not py_only :
2877 response = urllib.urlopen(target)
2879 response = urllib.urlopen(target)
2878 return response.read().decode('latin1')
2880 return response.read().decode('latin1')
2879 raise ValueError(("'%s' seem to be unreadable.") % utarget)
2881 raise ValueError(("'%s' seem to be unreadable.") % utarget)
2880
2882
2881 potential_target = [target]
2883 potential_target = [target]
2882 try :
2884 try :
2883 potential_target.insert(0,get_py_filename(target))
2885 potential_target.insert(0,get_py_filename(target))
2884 except IOError:
2886 except IOError:
2885 pass
2887 pass
2886
2888
2887 for tgt in potential_target :
2889 for tgt in potential_target :
2888 if os.path.isfile(tgt): # Read file
2890 if os.path.isfile(tgt): # Read file
2889 try :
2891 try :
2890 return openpy.read_py_file(tgt, skip_encoding_cookie=True)
2892 return openpy.read_py_file(tgt, skip_encoding_cookie=True)
2891 except UnicodeDecodeError :
2893 except UnicodeDecodeError :
2892 if not py_only :
2894 if not py_only :
2893 with io_open(tgt,'r', encoding='latin1') as f :
2895 with io_open(tgt,'r', encoding='latin1') as f :
2894 return f.read()
2896 return f.read()
2895 raise ValueError(("'%s' seem to be unreadable.") % target)
2897 raise ValueError(("'%s' seem to be unreadable.") % target)
2896
2898
2897 try: # User namespace
2899 try: # User namespace
2898 codeobj = eval(target, self.user_ns)
2900 codeobj = eval(target, self.user_ns)
2899 except Exception:
2901 except Exception:
2900 raise ValueError(("'%s' was not found in history, as a file, url, "
2902 raise ValueError(("'%s' was not found in history, as a file, url, "
2901 "nor in the user namespace.") % target)
2903 "nor in the user namespace.") % target)
2902 if isinstance(codeobj, basestring):
2904 if isinstance(codeobj, basestring):
2903 return codeobj
2905 return codeobj
2904 elif isinstance(codeobj, Macro):
2906 elif isinstance(codeobj, Macro):
2905 return codeobj.value
2907 return codeobj.value
2906
2908
2907 raise TypeError("%s is neither a string nor a macro." % target,
2909 raise TypeError("%s is neither a string nor a macro." % target,
2908 codeobj)
2910 codeobj)
2909
2911
2910 #-------------------------------------------------------------------------
2912 #-------------------------------------------------------------------------
2911 # Things related to IPython exiting
2913 # Things related to IPython exiting
2912 #-------------------------------------------------------------------------
2914 #-------------------------------------------------------------------------
2913 def atexit_operations(self):
2915 def atexit_operations(self):
2914 """This will be executed at the time of exit.
2916 """This will be executed at the time of exit.
2915
2917
2916 Cleanup operations and saving of persistent data that is done
2918 Cleanup operations and saving of persistent data that is done
2917 unconditionally by IPython should be performed here.
2919 unconditionally by IPython should be performed here.
2918
2920
2919 For things that may depend on startup flags or platform specifics (such
2921 For things that may depend on startup flags or platform specifics (such
2920 as having readline or not), register a separate atexit function in the
2922 as having readline or not), register a separate atexit function in the
2921 code that has the appropriate information, rather than trying to
2923 code that has the appropriate information, rather than trying to
2922 clutter
2924 clutter
2923 """
2925 """
2924 # Close the history session (this stores the end time and line count)
2926 # Close the history session (this stores the end time and line count)
2925 # this must be *before* the tempfile cleanup, in case of temporary
2927 # this must be *before* the tempfile cleanup, in case of temporary
2926 # history db
2928 # history db
2927 self.history_manager.end_session()
2929 self.history_manager.end_session()
2928
2930
2929 # Cleanup all tempfiles left around
2931 # Cleanup all tempfiles left around
2930 for tfile in self.tempfiles:
2932 for tfile in self.tempfiles:
2931 try:
2933 try:
2932 os.unlink(tfile)
2934 os.unlink(tfile)
2933 except OSError:
2935 except OSError:
2934 pass
2936 pass
2935
2937
2936 # Clear all user namespaces to release all references cleanly.
2938 # Clear all user namespaces to release all references cleanly.
2937 self.reset(new_session=False)
2939 self.reset(new_session=False)
2938
2940
2939 # Run user hooks
2941 # Run user hooks
2940 self.hooks.shutdown_hook()
2942 self.hooks.shutdown_hook()
2941
2943
2942 def cleanup(self):
2944 def cleanup(self):
2943 self.restore_sys_module_state()
2945 self.restore_sys_module_state()
2944
2946
2945
2947
2946 class InteractiveShellABC(object):
2948 class InteractiveShellABC(object):
2947 """An abstract base class for InteractiveShell."""
2949 """An abstract base class for InteractiveShell."""
2948 __metaclass__ = abc.ABCMeta
2950 __metaclass__ = abc.ABCMeta
2949
2951
2950 InteractiveShellABC.register(InteractiveShell)
2952 InteractiveShellABC.register(InteractiveShell)
@@ -1,710 +1,745 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """Tests for the inputsplitter module.
2 """Tests for the inputsplitter module.
3
3
4 Authors
4 Authors
5 -------
5 -------
6 * Fernando Perez
6 * Fernando Perez
7 * Robert Kern
7 * Robert Kern
8 """
8 """
9 #-----------------------------------------------------------------------------
9 #-----------------------------------------------------------------------------
10 # Copyright (C) 2010-2011 The IPython Development Team
10 # Copyright (C) 2010-2011 The IPython Development Team
11 #
11 #
12 # Distributed under the terms of the BSD License. The full license is in
12 # Distributed under the terms of the BSD License. The full license is in
13 # the file COPYING, distributed as part of this software.
13 # the file COPYING, distributed as part of this software.
14 #-----------------------------------------------------------------------------
14 #-----------------------------------------------------------------------------
15
15
16 #-----------------------------------------------------------------------------
16 #-----------------------------------------------------------------------------
17 # Imports
17 # Imports
18 #-----------------------------------------------------------------------------
18 #-----------------------------------------------------------------------------
19 # stdlib
19 # stdlib
20 import unittest
20 import unittest
21 import sys
21 import sys
22
22
23 # Third party
23 # Third party
24 import nose.tools as nt
24 import nose.tools as nt
25
25
26 # Our own
26 # Our own
27 from IPython.core import inputsplitter as isp
27 from IPython.core import inputsplitter as isp
28 from IPython.testing import tools as tt
28 from IPython.testing import tools as tt
29 from IPython.utils import py3compat
29 from IPython.utils import py3compat
30
30
31 #-----------------------------------------------------------------------------
31 #-----------------------------------------------------------------------------
32 # Semi-complete examples (also used as tests)
32 # Semi-complete examples (also used as tests)
33 #-----------------------------------------------------------------------------
33 #-----------------------------------------------------------------------------
34
34
35 # Note: at the bottom, there's a slightly more complete version of this that
35 # Note: at the bottom, there's a slightly more complete version of this that
36 # can be useful during development of code here.
36 # can be useful during development of code here.
37
37
38 def mini_interactive_loop(input_func):
38 def mini_interactive_loop(input_func):
39 """Minimal example of the logic of an interactive interpreter loop.
39 """Minimal example of the logic of an interactive interpreter loop.
40
40
41 This serves as an example, and it is used by the test system with a fake
41 This serves as an example, and it is used by the test system with a fake
42 raw_input that simulates interactive input."""
42 raw_input that simulates interactive input."""
43
43
44 from IPython.core.inputsplitter import InputSplitter
44 from IPython.core.inputsplitter import InputSplitter
45
45
46 isp = InputSplitter()
46 isp = InputSplitter()
47 # In practice, this input loop would be wrapped in an outside loop to read
47 # In practice, this input loop would be wrapped in an outside loop to read
48 # input indefinitely, until some exit/quit command was issued. Here we
48 # input indefinitely, until some exit/quit command was issued. Here we
49 # only illustrate the basic inner loop.
49 # only illustrate the basic inner loop.
50 while isp.push_accepts_more():
50 while isp.push_accepts_more():
51 indent = ' '*isp.indent_spaces
51 indent = ' '*isp.indent_spaces
52 prompt = '>>> ' + indent
52 prompt = '>>> ' + indent
53 line = indent + input_func(prompt)
53 line = indent + input_func(prompt)
54 isp.push(line)
54 isp.push(line)
55
55
56 # Here we just return input so we can use it in a test suite, but a real
56 # Here we just return input so we can use it in a test suite, but a real
57 # interpreter would instead send it for execution somewhere.
57 # interpreter would instead send it for execution somewhere.
58 src = isp.source_reset()
58 src = isp.source_reset()
59 #print 'Input source was:\n', src # dbg
59 #print 'Input source was:\n', src # dbg
60 return src
60 return src
61
61
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 # Test utilities, just for local use
63 # Test utilities, just for local use
64 #-----------------------------------------------------------------------------
64 #-----------------------------------------------------------------------------
65
65
66 def assemble(block):
66 def assemble(block):
67 """Assemble a block into multi-line sub-blocks."""
67 """Assemble a block into multi-line sub-blocks."""
68 return ['\n'.join(sub_block)+'\n' for sub_block in block]
68 return ['\n'.join(sub_block)+'\n' for sub_block in block]
69
69
70
70
71 def pseudo_input(lines):
71 def pseudo_input(lines):
72 """Return a function that acts like raw_input but feeds the input list."""
72 """Return a function that acts like raw_input but feeds the input list."""
73 ilines = iter(lines)
73 ilines = iter(lines)
74 def raw_in(prompt):
74 def raw_in(prompt):
75 try:
75 try:
76 return next(ilines)
76 return next(ilines)
77 except StopIteration:
77 except StopIteration:
78 return ''
78 return ''
79 return raw_in
79 return raw_in
80
80
81 #-----------------------------------------------------------------------------
81 #-----------------------------------------------------------------------------
82 # Tests
82 # Tests
83 #-----------------------------------------------------------------------------
83 #-----------------------------------------------------------------------------
84 def test_spaces():
84 def test_spaces():
85 tests = [('', 0),
85 tests = [('', 0),
86 (' ', 1),
86 (' ', 1),
87 ('\n', 0),
87 ('\n', 0),
88 (' \n', 1),
88 (' \n', 1),
89 ('x', 0),
89 ('x', 0),
90 (' x', 1),
90 (' x', 1),
91 (' x',2),
91 (' x',2),
92 (' x',4),
92 (' x',4),
93 # Note: tabs are counted as a single whitespace!
93 # Note: tabs are counted as a single whitespace!
94 ('\tx', 1),
94 ('\tx', 1),
95 ('\t x', 2),
95 ('\t x', 2),
96 ]
96 ]
97 tt.check_pairs(isp.num_ini_spaces, tests)
97 tt.check_pairs(isp.num_ini_spaces, tests)
98
98
99
99
100 def test_remove_comments():
100 def test_remove_comments():
101 tests = [('text', 'text'),
101 tests = [('text', 'text'),
102 ('text # comment', 'text '),
102 ('text # comment', 'text '),
103 ('text # comment\n', 'text \n'),
103 ('text # comment\n', 'text \n'),
104 ('text # comment \n', 'text \n'),
104 ('text # comment \n', 'text \n'),
105 ('line # c \nline\n','line \nline\n'),
105 ('line # c \nline\n','line \nline\n'),
106 ('line # c \nline#c2 \nline\nline #c\n\n',
106 ('line # c \nline#c2 \nline\nline #c\n\n',
107 'line \nline\nline\nline \n\n'),
107 'line \nline\nline\nline \n\n'),
108 ]
108 ]
109 tt.check_pairs(isp.remove_comments, tests)
109 tt.check_pairs(isp.remove_comments, tests)
110
110
111 def test_has_comment():
111 def test_has_comment():
112 tests = [('text', False),
112 tests = [('text', False),
113 ('text #comment', True),
113 ('text #comment', True),
114 ('text #comment\n', True),
114 ('text #comment\n', True),
115 ('#comment', True),
115 ('#comment', True),
116 ('#comment\n', True),
116 ('#comment\n', True),
117 ('a = "#string"', False),
117 ('a = "#string"', False),
118 ('a = "#string" # comment', True),
118 ('a = "#string" # comment', True),
119 ('a #comment not "string"', True),
119 ('a #comment not "string"', True),
120 ]
120 ]
121 tt.check_pairs(isp.has_comment, tests)
121 tt.check_pairs(isp.has_comment, tests)
122
122
123
123
124 def test_get_input_encoding():
124 def test_get_input_encoding():
125 encoding = isp.get_input_encoding()
125 encoding = isp.get_input_encoding()
126 nt.assert_true(isinstance(encoding, basestring))
126 nt.assert_true(isinstance(encoding, basestring))
127 # simple-minded check that at least encoding a simple string works with the
127 # simple-minded check that at least encoding a simple string works with the
128 # encoding we got.
128 # encoding we got.
129 nt.assert_equal(u'test'.encode(encoding), b'test')
129 nt.assert_equal(u'test'.encode(encoding), b'test')
130
130
131
131
132 class NoInputEncodingTestCase(unittest.TestCase):
132 class NoInputEncodingTestCase(unittest.TestCase):
133 def setUp(self):
133 def setUp(self):
134 self.old_stdin = sys.stdin
134 self.old_stdin = sys.stdin
135 class X: pass
135 class X: pass
136 fake_stdin = X()
136 fake_stdin = X()
137 sys.stdin = fake_stdin
137 sys.stdin = fake_stdin
138
138
139 def test(self):
139 def test(self):
140 # Verify that if sys.stdin has no 'encoding' attribute we do the right
140 # Verify that if sys.stdin has no 'encoding' attribute we do the right
141 # thing
141 # thing
142 enc = isp.get_input_encoding()
142 enc = isp.get_input_encoding()
143 self.assertEqual(enc, 'ascii')
143 self.assertEqual(enc, 'ascii')
144
144
145 def tearDown(self):
145 def tearDown(self):
146 sys.stdin = self.old_stdin
146 sys.stdin = self.old_stdin
147
147
148
148
149 class InputSplitterTestCase(unittest.TestCase):
149 class InputSplitterTestCase(unittest.TestCase):
150 def setUp(self):
150 def setUp(self):
151 self.isp = isp.InputSplitter()
151 self.isp = isp.InputSplitter()
152
152
153 def test_reset(self):
153 def test_reset(self):
154 isp = self.isp
154 isp = self.isp
155 isp.push('x=1')
155 isp.push('x=1')
156 isp.reset()
156 isp.reset()
157 self.assertEqual(isp._buffer, [])
157 self.assertEqual(isp._buffer, [])
158 self.assertEqual(isp.indent_spaces, 0)
158 self.assertEqual(isp.indent_spaces, 0)
159 self.assertEqual(isp.source, '')
159 self.assertEqual(isp.source, '')
160 self.assertEqual(isp.code, None)
160 self.assertEqual(isp.code, None)
161 self.assertEqual(isp._is_complete, False)
161 self.assertEqual(isp._is_complete, False)
162
162
163 def test_source(self):
163 def test_source(self):
164 self.isp._store('1')
164 self.isp._store('1')
165 self.isp._store('2')
165 self.isp._store('2')
166 self.assertEqual(self.isp.source, '1\n2\n')
166 self.assertEqual(self.isp.source, '1\n2\n')
167 self.assertTrue(len(self.isp._buffer)>0)
167 self.assertTrue(len(self.isp._buffer)>0)
168 self.assertEqual(self.isp.source_reset(), '1\n2\n')
168 self.assertEqual(self.isp.source_reset(), '1\n2\n')
169 self.assertEqual(self.isp._buffer, [])
169 self.assertEqual(self.isp._buffer, [])
170 self.assertEqual(self.isp.source, '')
170 self.assertEqual(self.isp.source, '')
171
171
172 def test_indent(self):
172 def test_indent(self):
173 isp = self.isp # shorthand
173 isp = self.isp # shorthand
174 isp.push('x=1')
174 isp.push('x=1')
175 self.assertEqual(isp.indent_spaces, 0)
175 self.assertEqual(isp.indent_spaces, 0)
176 isp.push('if 1:\n x=1')
176 isp.push('if 1:\n x=1')
177 self.assertEqual(isp.indent_spaces, 4)
177 self.assertEqual(isp.indent_spaces, 4)
178 isp.push('y=2\n')
178 isp.push('y=2\n')
179 self.assertEqual(isp.indent_spaces, 0)
179 self.assertEqual(isp.indent_spaces, 0)
180
180
181 def test_indent2(self):
181 def test_indent2(self):
182 # In cell mode, inputs must be fed in whole blocks, so skip this test
182 # In cell mode, inputs must be fed in whole blocks, so skip this test
183 if self.isp.input_mode == 'cell': return
183 if self.isp.input_mode == 'cell': return
184
184
185 isp = self.isp
185 isp = self.isp
186 isp.push('if 1:')
186 isp.push('if 1:')
187 self.assertEqual(isp.indent_spaces, 4)
187 self.assertEqual(isp.indent_spaces, 4)
188 isp.push(' x=1')
188 isp.push(' x=1')
189 self.assertEqual(isp.indent_spaces, 4)
189 self.assertEqual(isp.indent_spaces, 4)
190 # Blank lines shouldn't change the indent level
190 # Blank lines shouldn't change the indent level
191 isp.push(' '*2)
191 isp.push(' '*2)
192 self.assertEqual(isp.indent_spaces, 4)
192 self.assertEqual(isp.indent_spaces, 4)
193
193
194 def test_indent3(self):
194 def test_indent3(self):
195 # In cell mode, inputs must be fed in whole blocks, so skip this test
195 # In cell mode, inputs must be fed in whole blocks, so skip this test
196 if self.isp.input_mode == 'cell': return
196 if self.isp.input_mode == 'cell': return
197
197
198 isp = self.isp
198 isp = self.isp
199 # When a multiline statement contains parens or multiline strings, we
199 # When a multiline statement contains parens or multiline strings, we
200 # shouldn't get confused.
200 # shouldn't get confused.
201 isp.push("if 1:")
201 isp.push("if 1:")
202 isp.push(" x = (1+\n 2)")
202 isp.push(" x = (1+\n 2)")
203 self.assertEqual(isp.indent_spaces, 4)
203 self.assertEqual(isp.indent_spaces, 4)
204
204
205 def test_indent4(self):
205 def test_indent4(self):
206 # In cell mode, inputs must be fed in whole blocks, so skip this test
206 # In cell mode, inputs must be fed in whole blocks, so skip this test
207 if self.isp.input_mode == 'cell': return
207 if self.isp.input_mode == 'cell': return
208
208
209 isp = self.isp
209 isp = self.isp
210 # whitespace after ':' should not screw up indent level
210 # whitespace after ':' should not screw up indent level
211 isp.push('if 1: \n x=1')
211 isp.push('if 1: \n x=1')
212 self.assertEqual(isp.indent_spaces, 4)
212 self.assertEqual(isp.indent_spaces, 4)
213 isp.push('y=2\n')
213 isp.push('y=2\n')
214 self.assertEqual(isp.indent_spaces, 0)
214 self.assertEqual(isp.indent_spaces, 0)
215 isp.push('if 1:\t\n x=1')
215 isp.push('if 1:\t\n x=1')
216 self.assertEqual(isp.indent_spaces, 4)
216 self.assertEqual(isp.indent_spaces, 4)
217 isp.push('y=2\n')
217 isp.push('y=2\n')
218 self.assertEqual(isp.indent_spaces, 0)
218 self.assertEqual(isp.indent_spaces, 0)
219
219
220 def test_dedent_pass(self):
220 def test_dedent_pass(self):
221 isp = self.isp # shorthand
221 isp = self.isp # shorthand
222 # should NOT cause dedent
222 # should NOT cause dedent
223 isp.push('if 1:\n passes = 5')
223 isp.push('if 1:\n passes = 5')
224 self.assertEqual(isp.indent_spaces, 4)
224 self.assertEqual(isp.indent_spaces, 4)
225 isp.push('if 1:\n pass')
225 isp.push('if 1:\n pass')
226 self.assertEqual(isp.indent_spaces, 0)
226 self.assertEqual(isp.indent_spaces, 0)
227 isp.push('if 1:\n pass ')
227 isp.push('if 1:\n pass ')
228 self.assertEqual(isp.indent_spaces, 0)
228 self.assertEqual(isp.indent_spaces, 0)
229
229
230 def test_dedent_raise(self):
230 def test_dedent_raise(self):
231 isp = self.isp # shorthand
231 isp = self.isp # shorthand
232 # should NOT cause dedent
232 # should NOT cause dedent
233 isp.push('if 1:\n raised = 4')
233 isp.push('if 1:\n raised = 4')
234 self.assertEqual(isp.indent_spaces, 4)
234 self.assertEqual(isp.indent_spaces, 4)
235 isp.push('if 1:\n raise TypeError()')
235 isp.push('if 1:\n raise TypeError()')
236 self.assertEqual(isp.indent_spaces, 0)
236 self.assertEqual(isp.indent_spaces, 0)
237 isp.push('if 1:\n raise')
237 isp.push('if 1:\n raise')
238 self.assertEqual(isp.indent_spaces, 0)
238 self.assertEqual(isp.indent_spaces, 0)
239 isp.push('if 1:\n raise ')
239 isp.push('if 1:\n raise ')
240 self.assertEqual(isp.indent_spaces, 0)
240 self.assertEqual(isp.indent_spaces, 0)
241
241
242 def test_dedent_return(self):
242 def test_dedent_return(self):
243 isp = self.isp # shorthand
243 isp = self.isp # shorthand
244 # should NOT cause dedent
244 # should NOT cause dedent
245 isp.push('if 1:\n returning = 4')
245 isp.push('if 1:\n returning = 4')
246 self.assertEqual(isp.indent_spaces, 4)
246 self.assertEqual(isp.indent_spaces, 4)
247 isp.push('if 1:\n return 5 + 493')
247 isp.push('if 1:\n return 5 + 493')
248 self.assertEqual(isp.indent_spaces, 0)
248 self.assertEqual(isp.indent_spaces, 0)
249 isp.push('if 1:\n return')
249 isp.push('if 1:\n return')
250 self.assertEqual(isp.indent_spaces, 0)
250 self.assertEqual(isp.indent_spaces, 0)
251 isp.push('if 1:\n return ')
251 isp.push('if 1:\n return ')
252 self.assertEqual(isp.indent_spaces, 0)
252 self.assertEqual(isp.indent_spaces, 0)
253 isp.push('if 1:\n return(0)')
253 isp.push('if 1:\n return(0)')
254 self.assertEqual(isp.indent_spaces, 0)
254 self.assertEqual(isp.indent_spaces, 0)
255
255
256 def test_push(self):
256 def test_push(self):
257 isp = self.isp
257 isp = self.isp
258 self.assertTrue(isp.push('x=1'))
258 self.assertTrue(isp.push('x=1'))
259
259
260 def test_push2(self):
260 def test_push2(self):
261 isp = self.isp
261 isp = self.isp
262 self.assertFalse(isp.push('if 1:'))
262 self.assertFalse(isp.push('if 1:'))
263 for line in [' x=1', '# a comment', ' y=2']:
263 for line in [' x=1', '# a comment', ' y=2']:
264 self.assertTrue(isp.push(line))
264 self.assertTrue(isp.push(line))
265
265
266 def test_push3(self):
266 def test_push3(self):
267 isp = self.isp
267 isp = self.isp
268 isp.push('if True:')
268 isp.push('if True:')
269 isp.push(' a = 1')
269 isp.push(' a = 1')
270 self.assertFalse(isp.push('b = [1,'))
270 self.assertFalse(isp.push('b = [1,'))
271
271
272 def test_replace_mode(self):
272 def test_replace_mode(self):
273 isp = self.isp
273 isp = self.isp
274 isp.input_mode = 'cell'
274 isp.input_mode = 'cell'
275 isp.push('x=1')
275 isp.push('x=1')
276 self.assertEqual(isp.source, 'x=1\n')
276 self.assertEqual(isp.source, 'x=1\n')
277 isp.push('x=2')
277 isp.push('x=2')
278 self.assertEqual(isp.source, 'x=2\n')
278 self.assertEqual(isp.source, 'x=2\n')
279
279
280 def test_push_accepts_more(self):
280 def test_push_accepts_more(self):
281 isp = self.isp
281 isp = self.isp
282 isp.push('x=1')
282 isp.push('x=1')
283 self.assertFalse(isp.push_accepts_more())
283 self.assertFalse(isp.push_accepts_more())
284
284
285 def test_push_accepts_more2(self):
285 def test_push_accepts_more2(self):
286 # In cell mode, inputs must be fed in whole blocks, so skip this test
286 # In cell mode, inputs must be fed in whole blocks, so skip this test
287 if self.isp.input_mode == 'cell': return
287 if self.isp.input_mode == 'cell': return
288
288
289 isp = self.isp
289 isp = self.isp
290 isp.push('if 1:')
290 isp.push('if 1:')
291 self.assertTrue(isp.push_accepts_more())
291 self.assertTrue(isp.push_accepts_more())
292 isp.push(' x=1')
292 isp.push(' x=1')
293 self.assertTrue(isp.push_accepts_more())
293 self.assertTrue(isp.push_accepts_more())
294 isp.push('')
294 isp.push('')
295 self.assertFalse(isp.push_accepts_more())
295 self.assertFalse(isp.push_accepts_more())
296
296
297 def test_push_accepts_more3(self):
297 def test_push_accepts_more3(self):
298 isp = self.isp
298 isp = self.isp
299 isp.push("x = (2+\n3)")
299 isp.push("x = (2+\n3)")
300 self.assertFalse(isp.push_accepts_more())
300 self.assertFalse(isp.push_accepts_more())
301
301
302 def test_push_accepts_more4(self):
302 def test_push_accepts_more4(self):
303 # In cell mode, inputs must be fed in whole blocks, so skip this test
303 # In cell mode, inputs must be fed in whole blocks, so skip this test
304 if self.isp.input_mode == 'cell': return
304 if self.isp.input_mode == 'cell': return
305
305
306 isp = self.isp
306 isp = self.isp
307 # When a multiline statement contains parens or multiline strings, we
307 # When a multiline statement contains parens or multiline strings, we
308 # shouldn't get confused.
308 # shouldn't get confused.
309 # FIXME: we should be able to better handle de-dents in statements like
309 # FIXME: we should be able to better handle de-dents in statements like
310 # multiline strings and multiline expressions (continued with \ or
310 # multiline strings and multiline expressions (continued with \ or
311 # parens). Right now we aren't handling the indentation tracking quite
311 # parens). Right now we aren't handling the indentation tracking quite
312 # correctly with this, though in practice it may not be too much of a
312 # correctly with this, though in practice it may not be too much of a
313 # problem. We'll need to see.
313 # problem. We'll need to see.
314 isp.push("if 1:")
314 isp.push("if 1:")
315 isp.push(" x = (2+")
315 isp.push(" x = (2+")
316 isp.push(" 3)")
316 isp.push(" 3)")
317 self.assertTrue(isp.push_accepts_more())
317 self.assertTrue(isp.push_accepts_more())
318 isp.push(" y = 3")
318 isp.push(" y = 3")
319 self.assertTrue(isp.push_accepts_more())
319 self.assertTrue(isp.push_accepts_more())
320 isp.push('')
320 isp.push('')
321 self.assertFalse(isp.push_accepts_more())
321 self.assertFalse(isp.push_accepts_more())
322
322
323 def test_push_accepts_more5(self):
323 def test_push_accepts_more5(self):
324 # In cell mode, inputs must be fed in whole blocks, so skip this test
324 # In cell mode, inputs must be fed in whole blocks, so skip this test
325 if self.isp.input_mode == 'cell': return
325 if self.isp.input_mode == 'cell': return
326
326
327 isp = self.isp
327 isp = self.isp
328 isp.push('try:')
328 isp.push('try:')
329 isp.push(' a = 5')
329 isp.push(' a = 5')
330 isp.push('except:')
330 isp.push('except:')
331 isp.push(' raise')
331 isp.push(' raise')
332 self.assertTrue(isp.push_accepts_more())
332 self.assertTrue(isp.push_accepts_more())
333
333
334 def test_continuation(self):
334 def test_continuation(self):
335 isp = self.isp
335 isp = self.isp
336 isp.push("import os, \\")
336 isp.push("import os, \\")
337 self.assertTrue(isp.push_accepts_more())
337 self.assertTrue(isp.push_accepts_more())
338 isp.push("sys")
338 isp.push("sys")
339 self.assertFalse(isp.push_accepts_more())
339 self.assertFalse(isp.push_accepts_more())
340
340
341 def test_syntax_error(self):
341 def test_syntax_error(self):
342 isp = self.isp
342 isp = self.isp
343 # Syntax errors immediately produce a 'ready' block, so the invalid
343 # Syntax errors immediately produce a 'ready' block, so the invalid
344 # Python can be sent to the kernel for evaluation with possible ipython
344 # Python can be sent to the kernel for evaluation with possible ipython
345 # special-syntax conversion.
345 # special-syntax conversion.
346 isp.push('run foo')
346 isp.push('run foo')
347 self.assertFalse(isp.push_accepts_more())
347 self.assertFalse(isp.push_accepts_more())
348
348
349 def test_unicode(self):
349 def test_unicode(self):
350 self.isp.push(u"PΓ©rez")
350 self.isp.push(u"PΓ©rez")
351 self.isp.push(u'\xc3\xa9')
351 self.isp.push(u'\xc3\xa9')
352 self.isp.push(u"u'\xc3\xa9'")
352 self.isp.push(u"u'\xc3\xa9'")
353
353
354 class InteractiveLoopTestCase(unittest.TestCase):
354 class InteractiveLoopTestCase(unittest.TestCase):
355 """Tests for an interactive loop like a python shell.
355 """Tests for an interactive loop like a python shell.
356 """
356 """
357 def check_ns(self, lines, ns):
357 def check_ns(self, lines, ns):
358 """Validate that the given input lines produce the resulting namespace.
358 """Validate that the given input lines produce the resulting namespace.
359
359
360 Note: the input lines are given exactly as they would be typed in an
360 Note: the input lines are given exactly as they would be typed in an
361 auto-indenting environment, as mini_interactive_loop above already does
361 auto-indenting environment, as mini_interactive_loop above already does
362 auto-indenting and prepends spaces to the input.
362 auto-indenting and prepends spaces to the input.
363 """
363 """
364 src = mini_interactive_loop(pseudo_input(lines))
364 src = mini_interactive_loop(pseudo_input(lines))
365 test_ns = {}
365 test_ns = {}
366 exec src in test_ns
366 exec src in test_ns
367 # We can't check that the provided ns is identical to the test_ns,
367 # We can't check that the provided ns is identical to the test_ns,
368 # because Python fills test_ns with extra keys (copyright, etc). But
368 # because Python fills test_ns with extra keys (copyright, etc). But
369 # we can check that the given dict is *contained* in test_ns
369 # we can check that the given dict is *contained* in test_ns
370 for k,v in ns.iteritems():
370 for k,v in ns.iteritems():
371 self.assertEqual(test_ns[k], v)
371 self.assertEqual(test_ns[k], v)
372
372
373 def test_simple(self):
373 def test_simple(self):
374 self.check_ns(['x=1'], dict(x=1))
374 self.check_ns(['x=1'], dict(x=1))
375
375
376 def test_simple2(self):
376 def test_simple2(self):
377 self.check_ns(['if 1:', 'x=2'], dict(x=2))
377 self.check_ns(['if 1:', 'x=2'], dict(x=2))
378
378
379 def test_xy(self):
379 def test_xy(self):
380 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
380 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
381
381
382 def test_abc(self):
382 def test_abc(self):
383 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
383 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
384
384
385 def test_multi(self):
385 def test_multi(self):
386 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
386 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
387
387
388
388
389 def test_LineInfo():
389 def test_LineInfo():
390 """Simple test for LineInfo construction and str()"""
390 """Simple test for LineInfo construction and str()"""
391 linfo = isp.LineInfo(' %cd /home')
391 linfo = isp.LineInfo(' %cd /home')
392 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
392 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
393
393
394 # Transformer tests
394 # Transformer tests
395 def transform_checker(tests, func):
395 def transform_checker(tests, func):
396 """Utility to loop over test inputs"""
396 """Utility to loop over test inputs"""
397 for inp, tr in tests:
397 for inp, tr in tests:
398 nt.assert_equals(func(inp), tr)
398 nt.assert_equals(func(inp), tr)
399
399
400 # Data for all the syntax tests in the form of lists of pairs of
400 # Data for all the syntax tests in the form of lists of pairs of
401 # raw/transformed input. We store it here as a global dict so that we can use
401 # raw/transformed input. We store it here as a global dict so that we can use
402 # it both within single-function tests and also to validate the behavior of the
402 # it both within single-function tests and also to validate the behavior of the
403 # larger objects
403 # larger objects
404
404
405 syntax = \
405 syntax = \
406 dict(assign_system =
406 dict(assign_system =
407 [(i,py3compat.u_format(o)) for i,o in \
407 [(i,py3compat.u_format(o)) for i,o in \
408 [(u'a =! ls', "a = get_ipython().getoutput({u}'ls')"),
408 [(u'a =! ls', "a = get_ipython().getoutput({u}'ls')"),
409 (u'b = !ls', "b = get_ipython().getoutput({u}'ls')"),
409 (u'b = !ls', "b = get_ipython().getoutput({u}'ls')"),
410 ('x=1', 'x=1'), # normal input is unmodified
410 ('x=1', 'x=1'), # normal input is unmodified
411 (' ',' '), # blank lines are kept intact
411 (' ',' '), # blank lines are kept intact
412 ]],
412 ]],
413
413
414 assign_magic =
414 assign_magic =
415 [(i,py3compat.u_format(o)) for i,o in \
415 [(i,py3compat.u_format(o)) for i,o in \
416 [(u'a =% who', "a = get_ipython().magic({u}'who')"),
416 [(u'a =% who', "a = get_ipython().magic({u}'who')"),
417 (u'b = %who', "b = get_ipython().magic({u}'who')"),
417 (u'b = %who', "b = get_ipython().magic({u}'who')"),
418 ('x=1', 'x=1'), # normal input is unmodified
418 ('x=1', 'x=1'), # normal input is unmodified
419 (' ',' '), # blank lines are kept intact
419 (' ',' '), # blank lines are kept intact
420 ]],
420 ]],
421
421
422 classic_prompt =
422 classic_prompt =
423 [('>>> x=1', 'x=1'),
423 [('>>> x=1', 'x=1'),
424 ('x=1', 'x=1'), # normal input is unmodified
424 ('x=1', 'x=1'), # normal input is unmodified
425 (' ', ' '), # blank lines are kept intact
425 (' ', ' '), # blank lines are kept intact
426 ('... ', ''), # continuation prompts
426 ('... ', ''), # continuation prompts
427 ],
427 ],
428
428
429 ipy_prompt =
429 ipy_prompt =
430 [('In [1]: x=1', 'x=1'),
430 [('In [1]: x=1', 'x=1'),
431 ('x=1', 'x=1'), # normal input is unmodified
431 ('x=1', 'x=1'), # normal input is unmodified
432 (' ',' '), # blank lines are kept intact
432 (' ',' '), # blank lines are kept intact
433 (' ....: ', ''), # continuation prompts
433 (' ....: ', ''), # continuation prompts
434 ],
434 ],
435
435
436 # Tests for the escape transformer to leave normal code alone
436 # Tests for the escape transformer to leave normal code alone
437 escaped_noesc =
437 escaped_noesc =
438 [ (' ', ' '),
438 [ (' ', ' '),
439 ('x=1', 'x=1'),
439 ('x=1', 'x=1'),
440 ],
440 ],
441
441
442 # System calls
442 # System calls
443 escaped_shell =
443 escaped_shell =
444 [(i,py3compat.u_format(o)) for i,o in \
444 [(i,py3compat.u_format(o)) for i,o in \
445 [ (u'!ls', "get_ipython().system({u}'ls')"),
445 [ (u'!ls', "get_ipython().system({u}'ls')"),
446 # Double-escape shell, this means to capture the output of the
446 # Double-escape shell, this means to capture the output of the
447 # subprocess and return it
447 # subprocess and return it
448 (u'!!ls', "get_ipython().getoutput({u}'ls')"),
448 (u'!!ls', "get_ipython().getoutput({u}'ls')"),
449 ]],
449 ]],
450
450
451 # Help/object info
451 # Help/object info
452 escaped_help =
452 escaped_help =
453 [(i,py3compat.u_format(o)) for i,o in \
453 [(i,py3compat.u_format(o)) for i,o in \
454 [ (u'?', 'get_ipython().show_usage()'),
454 [ (u'?', 'get_ipython().show_usage()'),
455 (u'?x1', "get_ipython().magic({u}'pinfo x1')"),
455 (u'?x1', "get_ipython().magic({u}'pinfo x1')"),
456 (u'??x2', "get_ipython().magic({u}'pinfo2 x2')"),
456 (u'??x2', "get_ipython().magic({u}'pinfo2 x2')"),
457 (u'?a.*s', "get_ipython().magic({u}'psearch a.*s')"),
457 (u'?a.*s', "get_ipython().magic({u}'psearch a.*s')"),
458 (u'?%hist', "get_ipython().magic({u}'pinfo %hist')"),
458 (u'?%hist', "get_ipython().magic({u}'pinfo %hist')"),
459 (u'?abc = qwe', "get_ipython().magic({u}'pinfo abc')"),
459 (u'?abc = qwe', "get_ipython().magic({u}'pinfo abc')"),
460 ]],
460 ]],
461
461
462 end_help =
462 end_help =
463 [(i,py3compat.u_format(o)) for i,o in \
463 [(i,py3compat.u_format(o)) for i,o in \
464 [ (u'x3?', "get_ipython().magic({u}'pinfo x3')"),
464 [ (u'x3?', "get_ipython().magic({u}'pinfo x3')"),
465 (u'x4??', "get_ipython().magic({u}'pinfo2 x4')"),
465 (u'x4??', "get_ipython().magic({u}'pinfo2 x4')"),
466 (u'%hist?', "get_ipython().magic({u}'pinfo %hist')"),
466 (u'%hist?', "get_ipython().magic({u}'pinfo %hist')"),
467 (u'f*?', "get_ipython().magic({u}'psearch f*')"),
467 (u'f*?', "get_ipython().magic({u}'psearch f*')"),
468 (u'ax.*aspe*?', "get_ipython().magic({u}'psearch ax.*aspe*')"),
468 (u'ax.*aspe*?', "get_ipython().magic({u}'psearch ax.*aspe*')"),
469 (u'a = abc?', "get_ipython().set_next_input({u}'a = abc');"
469 (u'a = abc?', "get_ipython().set_next_input({u}'a = abc');"
470 "get_ipython().magic({u}'pinfo abc')"),
470 "get_ipython().magic({u}'pinfo abc')"),
471 (u'a = abc.qe??', "get_ipython().set_next_input({u}'a = abc.qe');"
471 (u'a = abc.qe??', "get_ipython().set_next_input({u}'a = abc.qe');"
472 "get_ipython().magic({u}'pinfo2 abc.qe')"),
472 "get_ipython().magic({u}'pinfo2 abc.qe')"),
473 (u'a = *.items?', "get_ipython().set_next_input({u}'a = *.items');"
473 (u'a = *.items?', "get_ipython().set_next_input({u}'a = *.items');"
474 "get_ipython().magic({u}'psearch *.items')"),
474 "get_ipython().magic({u}'psearch *.items')"),
475 (u'plot(a?', "get_ipython().set_next_input({u}'plot(a');"
475 (u'plot(a?', "get_ipython().set_next_input({u}'plot(a');"
476 "get_ipython().magic({u}'pinfo a')"),
476 "get_ipython().magic({u}'pinfo a')"),
477 (u'a*2 #comment?', 'a*2 #comment?'),
477 (u'a*2 #comment?', 'a*2 #comment?'),
478 ]],
478 ]],
479
479
480 # Explicit magic calls
480 # Explicit magic calls
481 escaped_magic =
481 escaped_magic =
482 [(i,py3compat.u_format(o)) for i,o in \
482 [(i,py3compat.u_format(o)) for i,o in \
483 [ (u'%cd', "get_ipython().magic({u}'cd')"),
483 [ (u'%cd', "get_ipython().magic({u}'cd')"),
484 (u'%cd /home', "get_ipython().magic({u}'cd /home')"),
484 (u'%cd /home', "get_ipython().magic({u}'cd /home')"),
485 # Backslashes need to be escaped.
485 # Backslashes need to be escaped.
486 (u'%cd C:\\User', "get_ipython().magic({u}'cd C:\\\\User')"),
486 (u'%cd C:\\User', "get_ipython().magic({u}'cd C:\\\\User')"),
487 (u' %magic', " get_ipython().magic({u}'magic')"),
487 (u' %magic', " get_ipython().magic({u}'magic')"),
488 ]],
488 ]],
489
489
490 # Quoting with separate arguments
490 # Quoting with separate arguments
491 escaped_quote =
491 escaped_quote =
492 [ (',f', 'f("")'),
492 [ (',f', 'f("")'),
493 (',f x', 'f("x")'),
493 (',f x', 'f("x")'),
494 (' ,f y', ' f("y")'),
494 (' ,f y', ' f("y")'),
495 (',f a b', 'f("a", "b")'),
495 (',f a b', 'f("a", "b")'),
496 ],
496 ],
497
497
498 # Quoting with single argument
498 # Quoting with single argument
499 escaped_quote2 =
499 escaped_quote2 =
500 [ (';f', 'f("")'),
500 [ (';f', 'f("")'),
501 (';f x', 'f("x")'),
501 (';f x', 'f("x")'),
502 (' ;f y', ' f("y")'),
502 (' ;f y', ' f("y")'),
503 (';f a b', 'f("a b")'),
503 (';f a b', 'f("a b")'),
504 ],
504 ],
505
505
506 # Simply apply parens
506 # Simply apply parens
507 escaped_paren =
507 escaped_paren =
508 [ ('/f', 'f()'),
508 [ ('/f', 'f()'),
509 ('/f x', 'f(x)'),
509 ('/f x', 'f(x)'),
510 (' /f y', ' f(y)'),
510 (' /f y', ' f(y)'),
511 ('/f a b', 'f(a, b)'),
511 ('/f a b', 'f(a, b)'),
512 ],
512 ],
513
513
514 # Check that we transform prompts before other transforms
514 # Check that we transform prompts before other transforms
515 mixed =
515 mixed =
516 [(i,py3compat.u_format(o)) for i,o in \
516 [(i,py3compat.u_format(o)) for i,o in \
517 [ (u'In [1]: %lsmagic', "get_ipython().magic({u}'lsmagic')"),
517 [ (u'In [1]: %lsmagic', "get_ipython().magic({u}'lsmagic')"),
518 (u'>>> %lsmagic', "get_ipython().magic({u}'lsmagic')"),
518 (u'>>> %lsmagic', "get_ipython().magic({u}'lsmagic')"),
519 (u'In [2]: !ls', "get_ipython().system({u}'ls')"),
519 (u'In [2]: !ls', "get_ipython().system({u}'ls')"),
520 (u'In [3]: abs?', "get_ipython().magic({u}'pinfo abs')"),
520 (u'In [3]: abs?', "get_ipython().magic({u}'pinfo abs')"),
521 (u'In [4]: b = %who', "b = get_ipython().magic({u}'who')"),
521 (u'In [4]: b = %who', "b = get_ipython().magic({u}'who')"),
522 ]],
522 ]],
523 )
523 )
524
524
525 # multiline syntax examples. Each of these should be a list of lists, with
525 # multiline syntax examples. Each of these should be a list of lists, with
526 # each entry itself having pairs of raw/transformed input. The union (with
526 # each entry itself having pairs of raw/transformed input. The union (with
527 # '\n'.join() of the transformed inputs is what the splitter should produce
527 # '\n'.join() of the transformed inputs is what the splitter should produce
528 # when fed the raw lines one at a time via push.
528 # when fed the raw lines one at a time via push.
529 syntax_ml = \
529 syntax_ml = \
530 dict(classic_prompt =
530 dict(classic_prompt =
531 [ [('>>> for i in range(10):','for i in range(10):'),
531 [ [('>>> for i in range(10):','for i in range(10):'),
532 ('... print i',' print i'),
532 ('... print i',' print i'),
533 ('... ', ''),
533 ('... ', ''),
534 ],
534 ],
535 ],
535 ],
536
536
537 ipy_prompt =
537 ipy_prompt =
538 [ [('In [24]: for i in range(10):','for i in range(10):'),
538 [ [('In [24]: for i in range(10):','for i in range(10):'),
539 (' ....: print i',' print i'),
539 (' ....: print i',' print i'),
540 (' ....: ', ''),
540 (' ....: ', ''),
541 ],
541 ],
542 ],
542 ],
543
543
544 multiline_datastructure =
544 multiline_datastructure =
545 [ [('>>> a = [1,','a = [1,'),
545 [ [('>>> a = [1,','a = [1,'),
546 ('... 2]','2]'),
546 ('... 2]','2]'),
547 ],
547 ],
548 ],
548 ],
549 )
549 )
550
550
551
551
552 def test_assign_system():
552 def test_assign_system():
553 tt.check_pairs(isp.transform_assign_system, syntax['assign_system'])
553 tt.check_pairs(isp.transform_assign_system, syntax['assign_system'])
554
554
555
555
556 def test_assign_magic():
556 def test_assign_magic():
557 tt.check_pairs(isp.transform_assign_magic, syntax['assign_magic'])
557 tt.check_pairs(isp.transform_assign_magic, syntax['assign_magic'])
558
558
559
559
560 def test_classic_prompt():
560 def test_classic_prompt():
561 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
561 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
562 for example in syntax_ml['classic_prompt']:
562 for example in syntax_ml['classic_prompt']:
563 transform_checker(example, isp.transform_classic_prompt)
563 transform_checker(example, isp.transform_classic_prompt)
564
564
565
565
566 def test_ipy_prompt():
566 def test_ipy_prompt():
567 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
567 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
568 for example in syntax_ml['ipy_prompt']:
568 for example in syntax_ml['ipy_prompt']:
569 transform_checker(example, isp.transform_ipy_prompt)
569 transform_checker(example, isp.transform_ipy_prompt)
570
570
571 def test_end_help():
571 def test_end_help():
572 tt.check_pairs(isp.transform_help_end, syntax['end_help'])
572 tt.check_pairs(isp.transform_help_end, syntax['end_help'])
573
573
574 def test_escaped_noesc():
574 def test_escaped_noesc():
575 tt.check_pairs(isp.transform_escaped, syntax['escaped_noesc'])
575 tt.check_pairs(isp.transform_escaped, syntax['escaped_noesc'])
576
576
577
577
578 def test_escaped_shell():
578 def test_escaped_shell():
579 tt.check_pairs(isp.transform_escaped, syntax['escaped_shell'])
579 tt.check_pairs(isp.transform_escaped, syntax['escaped_shell'])
580
580
581
581
582 def test_escaped_help():
582 def test_escaped_help():
583 tt.check_pairs(isp.transform_escaped, syntax['escaped_help'])
583 tt.check_pairs(isp.transform_escaped, syntax['escaped_help'])
584
584
585
585
586 def test_escaped_magic():
586 def test_escaped_magic():
587 tt.check_pairs(isp.transform_escaped, syntax['escaped_magic'])
587 tt.check_pairs(isp.transform_escaped, syntax['escaped_magic'])
588
588
589
589
590 def test_escaped_quote():
590 def test_escaped_quote():
591 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote'])
591 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote'])
592
592
593
593
594 def test_escaped_quote2():
594 def test_escaped_quote2():
595 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote2'])
595 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote2'])
596
596
597
597
598 def test_escaped_paren():
598 def test_escaped_paren():
599 tt.check_pairs(isp.transform_escaped, syntax['escaped_paren'])
599 tt.check_pairs(isp.transform_escaped, syntax['escaped_paren'])
600
600
601
601
602 def test_last_blank():
603 nt.assert_false(isp.last_blank(''))
604 nt.assert_false(isp.last_blank('abc'))
605 nt.assert_false(isp.last_blank('abc\n'))
606 nt.assert_false(isp.last_blank('abc\na'))
607 nt.assert_true(isp.last_blank('\n'))
608 nt.assert_true(isp.last_blank('\n '))
609 nt.assert_true(isp.last_blank('abc\n '))
610 nt.assert_true(isp.last_blank('abc\n\n'))
611
612
613 def test_cell_magics():
614 from IPython.core import magic
615
616 cell = """\
617 %%cellm line
618 body
619 """
620 sp = isp.IPythonInputSplitter(input_mode='line')
621 sp.push(cell)
622 nt.assert_equal(sp.cell_magic_parts, ['body\n'])
623 out = sp.source
624 ref = u"get_ipython()._cell_magic(u'cellm', u'line')\n"
625 nt.assert_equal(out, ref)
626
627 sp.reset()
628
629 sp.push('%%cellm line2\n')
630 nt.assert_true(sp.push_accepts_more()) #1
631 sp.push('\n')
632 nt.assert_true(sp.push_accepts_more()) #2
633 sp.push('\n')
634 nt.assert_false(sp.push_accepts_more()) #3
635
636
602 class IPythonInputTestCase(InputSplitterTestCase):
637 class IPythonInputTestCase(InputSplitterTestCase):
603 """By just creating a new class whose .isp is a different instance, we
638 """By just creating a new class whose .isp is a different instance, we
604 re-run the same test battery on the new input splitter.
639 re-run the same test battery on the new input splitter.
605
640
606 In addition, this runs the tests over the syntax and syntax_ml dicts that
641 In addition, this runs the tests over the syntax and syntax_ml dicts that
607 were tested by individual functions, as part of the OO interface.
642 were tested by individual functions, as part of the OO interface.
608
643
609 It also makes some checks on the raw buffer storage.
644 It also makes some checks on the raw buffer storage.
610 """
645 """
611
646
612 def setUp(self):
647 def setUp(self):
613 self.isp = isp.IPythonInputSplitter(input_mode='line')
648 self.isp = isp.IPythonInputSplitter(input_mode='line')
614
649
615 def test_syntax(self):
650 def test_syntax(self):
616 """Call all single-line syntax tests from the main object"""
651 """Call all single-line syntax tests from the main object"""
617 isp = self.isp
652 isp = self.isp
618 for example in syntax.itervalues():
653 for example in syntax.itervalues():
619 for raw, out_t in example:
654 for raw, out_t in example:
620 if raw.startswith(' '):
655 if raw.startswith(' '):
621 continue
656 continue
622
657
623 isp.push(raw)
658 isp.push(raw)
624 out, out_raw = isp.source_raw_reset()
659 out, out_raw = isp.source_raw_reset()
625 self.assertEqual(out.rstrip(), out_t,
660 self.assertEqual(out.rstrip(), out_t,
626 tt.pair_fail_msg.format("inputsplitter",raw, out_t, out))
661 tt.pair_fail_msg.format("inputsplitter",raw, out_t, out))
627 self.assertEqual(out_raw.rstrip(), raw.rstrip())
662 self.assertEqual(out_raw.rstrip(), raw.rstrip())
628
663
629 def test_syntax_multiline(self):
664 def test_syntax_multiline(self):
630 isp = self.isp
665 isp = self.isp
631 for example in syntax_ml.itervalues():
666 for example in syntax_ml.itervalues():
632 out_t_parts = []
667 out_t_parts = []
633 raw_parts = []
668 raw_parts = []
634 for line_pairs in example:
669 for line_pairs in example:
635 for lraw, out_t_part in line_pairs:
670 for lraw, out_t_part in line_pairs:
636 isp.push(lraw)
671 isp.push(lraw)
637 out_t_parts.append(out_t_part)
672 out_t_parts.append(out_t_part)
638 raw_parts.append(lraw)
673 raw_parts.append(lraw)
639
674
640 out, out_raw = isp.source_raw_reset()
675 out, out_raw = isp.source_raw_reset()
641 out_t = '\n'.join(out_t_parts).rstrip()
676 out_t = '\n'.join(out_t_parts).rstrip()
642 raw = '\n'.join(raw_parts).rstrip()
677 raw = '\n'.join(raw_parts).rstrip()
643 self.assertEqual(out.rstrip(), out_t)
678 self.assertEqual(out.rstrip(), out_t)
644 self.assertEqual(out_raw.rstrip(), raw)
679 self.assertEqual(out_raw.rstrip(), raw)
645
680
646
681
647 class BlockIPythonInputTestCase(IPythonInputTestCase):
682 class BlockIPythonInputTestCase(IPythonInputTestCase):
648
683
649 # Deactivate tests that don't make sense for the block mode
684 # Deactivate tests that don't make sense for the block mode
650 test_push3 = test_split = lambda s: None
685 test_push3 = test_split = lambda s: None
651
686
652 def setUp(self):
687 def setUp(self):
653 self.isp = isp.IPythonInputSplitter(input_mode='cell')
688 self.isp = isp.IPythonInputSplitter(input_mode='cell')
654
689
655 def test_syntax_multiline(self):
690 def test_syntax_multiline(self):
656 isp = self.isp
691 isp = self.isp
657 for example in syntax_ml.itervalues():
692 for example in syntax_ml.itervalues():
658 raw_parts = []
693 raw_parts = []
659 out_t_parts = []
694 out_t_parts = []
660 for line_pairs in example:
695 for line_pairs in example:
661 for raw, out_t_part in line_pairs:
696 for raw, out_t_part in line_pairs:
662 raw_parts.append(raw)
697 raw_parts.append(raw)
663 out_t_parts.append(out_t_part)
698 out_t_parts.append(out_t_part)
664
699
665 raw = '\n'.join(raw_parts)
700 raw = '\n'.join(raw_parts)
666 out_t = '\n'.join(out_t_parts)
701 out_t = '\n'.join(out_t_parts)
667
702
668 isp.push(raw)
703 isp.push(raw)
669 out, out_raw = isp.source_raw_reset()
704 out, out_raw = isp.source_raw_reset()
670 # Match ignoring trailing whitespace
705 # Match ignoring trailing whitespace
671 self.assertEqual(out.rstrip(), out_t.rstrip())
706 self.assertEqual(out.rstrip(), out_t.rstrip())
672 self.assertEqual(out_raw.rstrip(), raw.rstrip())
707 self.assertEqual(out_raw.rstrip(), raw.rstrip())
673
708
674
709
675 #-----------------------------------------------------------------------------
710 #-----------------------------------------------------------------------------
676 # Main - use as a script, mostly for developer experiments
711 # Main - use as a script, mostly for developer experiments
677 #-----------------------------------------------------------------------------
712 #-----------------------------------------------------------------------------
678
713
679 if __name__ == '__main__':
714 if __name__ == '__main__':
680 # A simple demo for interactive experimentation. This code will not get
715 # A simple demo for interactive experimentation. This code will not get
681 # picked up by any test suite.
716 # picked up by any test suite.
682 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
717 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
683
718
684 # configure here the syntax to use, prompt and whether to autoindent
719 # configure here the syntax to use, prompt and whether to autoindent
685 #isp, start_prompt = InputSplitter(), '>>> '
720 #isp, start_prompt = InputSplitter(), '>>> '
686 isp, start_prompt = IPythonInputSplitter(), 'In> '
721 isp, start_prompt = IPythonInputSplitter(), 'In> '
687
722
688 autoindent = True
723 autoindent = True
689 #autoindent = False
724 #autoindent = False
690
725
691 try:
726 try:
692 while True:
727 while True:
693 prompt = start_prompt
728 prompt = start_prompt
694 while isp.push_accepts_more():
729 while isp.push_accepts_more():
695 indent = ' '*isp.indent_spaces
730 indent = ' '*isp.indent_spaces
696 if autoindent:
731 if autoindent:
697 line = indent + raw_input(prompt+indent)
732 line = indent + raw_input(prompt+indent)
698 else:
733 else:
699 line = raw_input(prompt)
734 line = raw_input(prompt)
700 isp.push(line)
735 isp.push(line)
701 prompt = '... '
736 prompt = '... '
702
737
703 # Here we just return input so we can use it in a test suite, but a
738 # Here we just return input so we can use it in a test suite, but a
704 # real interpreter would instead send it for execution somewhere.
739 # real interpreter would instead send it for execution somewhere.
705 #src = isp.source; raise EOFError # dbg
740 #src = isp.source; raise EOFError # dbg
706 src, raw = isp.source_raw_reset()
741 src, raw = isp.source_raw_reset()
707 print 'Input source was:\n', src
742 print 'Input source was:\n', src
708 print 'Raw source was:\n', raw
743 print 'Raw source was:\n', raw
709 except EOFError:
744 except EOFError:
710 print 'Bye'
745 print 'Bye'
General Comments 0
You need to be logged in to leave comments. Login now