##// END OF EJS Templates
Miscellaneous docs fixes
Thomas Kluyver -
Show More
@@ -1,661 +1,661 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63
63
64 #-----------------------------------------------------------------------------
64 #-----------------------------------------------------------------------------
65 # Imports
65 # Imports
66 #-----------------------------------------------------------------------------
66 #-----------------------------------------------------------------------------
67 # stdlib
67 # stdlib
68 import ast
68 import ast
69 import codeop
69 import codeop
70 import re
70 import re
71 import sys
71 import sys
72
72
73 # IPython modules
73 # IPython modules
74 from IPython.utils.py3compat import cast_unicode
74 from IPython.utils.py3compat import cast_unicode
75 from IPython.core.inputtransformer import (leading_indent,
75 from IPython.core.inputtransformer import (leading_indent,
76 classic_prompt,
76 classic_prompt,
77 ipy_prompt,
77 ipy_prompt,
78 strip_encoding_cookie,
78 strip_encoding_cookie,
79 cellmagic,
79 cellmagic,
80 assemble_logical_lines,
80 assemble_logical_lines,
81 help_end,
81 help_end,
82 escaped_commands,
82 escaped_commands,
83 assign_from_magic,
83 assign_from_magic,
84 assign_from_system,
84 assign_from_system,
85 assemble_python_lines,
85 assemble_python_lines,
86 )
86 )
87
87
88 # These are available in this module for backwards compatibility.
88 # These are available in this module for backwards compatibility.
89 from IPython.core.inputtransformer import (ESC_SHELL, ESC_SH_CAP, ESC_HELP,
89 from IPython.core.inputtransformer import (ESC_SHELL, ESC_SH_CAP, ESC_HELP,
90 ESC_HELP2, ESC_MAGIC, ESC_MAGIC2,
90 ESC_HELP2, ESC_MAGIC, ESC_MAGIC2,
91 ESC_QUOTE, ESC_QUOTE2, ESC_PAREN, ESC_SEQUENCES)
91 ESC_QUOTE, ESC_QUOTE2, ESC_PAREN, ESC_SEQUENCES)
92
92
93 #-----------------------------------------------------------------------------
93 #-----------------------------------------------------------------------------
94 # Utilities
94 # Utilities
95 #-----------------------------------------------------------------------------
95 #-----------------------------------------------------------------------------
96
96
97 # FIXME: These are general-purpose utilities that later can be moved to the
97 # FIXME: These are general-purpose utilities that later can be moved to the
98 # general ward. Kept here for now because we're being very strict about test
98 # general ward. Kept here for now because we're being very strict about test
99 # coverage with this code, and this lets us ensure that we keep 100% coverage
99 # coverage with this code, and this lets us ensure that we keep 100% coverage
100 # while developing.
100 # while developing.
101
101
102 # compiled regexps for autoindent management
102 # compiled regexps for autoindent management
103 dedent_re = re.compile('|'.join([
103 dedent_re = re.compile('|'.join([
104 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
104 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
105 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
105 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
106 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
106 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
107 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
107 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
108 r'^\s+pass\s*$', # pass (optionally followed by trailing spaces)
108 r'^\s+pass\s*$', # pass (optionally followed by trailing spaces)
109 r'^\s+break\s*$', # break (optionally followed by trailing spaces)
109 r'^\s+break\s*$', # break (optionally followed by trailing spaces)
110 r'^\s+continue\s*$', # continue (optionally followed by trailing spaces)
110 r'^\s+continue\s*$', # continue (optionally followed by trailing spaces)
111 ]))
111 ]))
112 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
112 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
113
113
114 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
114 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
115 # before pure comments
115 # before pure comments
116 comment_line_re = re.compile('^\s*\#')
116 comment_line_re = re.compile('^\s*\#')
117
117
118
118
119 def num_ini_spaces(s):
119 def num_ini_spaces(s):
120 """Return the number of initial spaces in a string.
120 """Return the number of initial spaces in a string.
121
121
122 Note that tabs are counted as a single space. For now, we do *not* support
122 Note that tabs are counted as a single space. For now, we do *not* support
123 mixing of tabs and spaces in the user's input.
123 mixing of tabs and spaces in the user's input.
124
124
125 Parameters
125 Parameters
126 ----------
126 ----------
127 s : string
127 s : string
128
128
129 Returns
129 Returns
130 -------
130 -------
131 n : int
131 n : int
132 """
132 """
133
133
134 ini_spaces = ini_spaces_re.match(s)
134 ini_spaces = ini_spaces_re.match(s)
135 if ini_spaces:
135 if ini_spaces:
136 return ini_spaces.end()
136 return ini_spaces.end()
137 else:
137 else:
138 return 0
138 return 0
139
139
140 def last_blank(src):
140 def last_blank(src):
141 """Determine if the input source ends in a blank.
141 """Determine if the input source ends in a blank.
142
142
143 A blank is either a newline or a line consisting of whitespace.
143 A blank is either a newline or a line consisting of whitespace.
144
144
145 Parameters
145 Parameters
146 ----------
146 ----------
147 src : string
147 src : string
148 A single or multiline string.
148 A single or multiline string.
149 """
149 """
150 if not src: return False
150 if not src: return False
151 ll = src.splitlines()[-1]
151 ll = src.splitlines()[-1]
152 return (ll == '') or ll.isspace()
152 return (ll == '') or ll.isspace()
153
153
154
154
155 last_two_blanks_re = re.compile(r'\n\s*\n\s*$', re.MULTILINE)
155 last_two_blanks_re = re.compile(r'\n\s*\n\s*$', re.MULTILINE)
156 last_two_blanks_re2 = re.compile(r'.+\n\s*\n\s+$', re.MULTILINE)
156 last_two_blanks_re2 = re.compile(r'.+\n\s*\n\s+$', re.MULTILINE)
157
157
158 def last_two_blanks(src):
158 def last_two_blanks(src):
159 """Determine if the input source ends in two blanks.
159 """Determine if the input source ends in two blanks.
160
160
161 A blank is either a newline or a line consisting of whitespace.
161 A blank is either a newline or a line consisting of whitespace.
162
162
163 Parameters
163 Parameters
164 ----------
164 ----------
165 src : string
165 src : string
166 A single or multiline string.
166 A single or multiline string.
167 """
167 """
168 if not src: return False
168 if not src: return False
169 # The logic here is tricky: I couldn't get a regexp to work and pass all
169 # The logic here is tricky: I couldn't get a regexp to work and pass all
170 # the tests, so I took a different approach: split the source by lines,
170 # the tests, so I took a different approach: split the source by lines,
171 # grab the last two and prepend '###\n' as a stand-in for whatever was in
171 # grab the last two and prepend '###\n' as a stand-in for whatever was in
172 # the body before the last two lines. Then, with that structure, it's
172 # the body before the last two lines. Then, with that structure, it's
173 # possible to analyze with two regexps. Not the most elegant solution, but
173 # possible to analyze with two regexps. Not the most elegant solution, but
174 # it works. If anyone tries to change this logic, make sure to validate
174 # it works. If anyone tries to change this logic, make sure to validate
175 # the whole test suite first!
175 # the whole test suite first!
176 new_src = '\n'.join(['###\n'] + src.splitlines()[-2:])
176 new_src = '\n'.join(['###\n'] + src.splitlines()[-2:])
177 return (bool(last_two_blanks_re.match(new_src)) or
177 return (bool(last_two_blanks_re.match(new_src)) or
178 bool(last_two_blanks_re2.match(new_src)) )
178 bool(last_two_blanks_re2.match(new_src)) )
179
179
180
180
181 def remove_comments(src):
181 def remove_comments(src):
182 """Remove all comments from input source.
182 """Remove all comments from input source.
183
183
184 Note: comments are NOT recognized inside of strings!
184 Note: comments are NOT recognized inside of strings!
185
185
186 Parameters
186 Parameters
187 ----------
187 ----------
188 src : string
188 src : string
189 A single or multiline input string.
189 A single or multiline input string.
190
190
191 Returns
191 Returns
192 -------
192 -------
193 String with all Python comments removed.
193 String with all Python comments removed.
194 """
194 """
195
195
196 return re.sub('#.*', '', src)
196 return re.sub('#.*', '', src)
197
197
198
198
199 def get_input_encoding():
199 def get_input_encoding():
200 """Return the default standard input encoding.
200 """Return the default standard input encoding.
201
201
202 If sys.stdin has no encoding, 'ascii' is returned."""
202 If sys.stdin has no encoding, 'ascii' is returned."""
203 # There are strange environments for which sys.stdin.encoding is None. We
203 # There are strange environments for which sys.stdin.encoding is None. We
204 # ensure that a valid encoding is returned.
204 # ensure that a valid encoding is returned.
205 encoding = getattr(sys.stdin, 'encoding', None)
205 encoding = getattr(sys.stdin, 'encoding', None)
206 if encoding is None:
206 if encoding is None:
207 encoding = 'ascii'
207 encoding = 'ascii'
208 return encoding
208 return encoding
209
209
210 #-----------------------------------------------------------------------------
210 #-----------------------------------------------------------------------------
211 # Classes and functions for normal Python syntax handling
211 # Classes and functions for normal Python syntax handling
212 #-----------------------------------------------------------------------------
212 #-----------------------------------------------------------------------------
213
213
214 class InputSplitter(object):
214 class InputSplitter(object):
215 """An object that can accumulate lines of Python source before execution.
215 r"""An object that can accumulate lines of Python source before execution.
216
216
217 This object is designed to be fed python source line-by-line, using
217 This object is designed to be fed python source line-by-line, using
218 :meth:`push`. It will return on each push whether the currently pushed
218 :meth:`push`. It will return on each push whether the currently pushed
219 code could be executed already. In addition, it provides a method called
219 code could be executed already. In addition, it provides a method called
220 :meth:`push_accepts_more` that can be used to query whether more input
220 :meth:`push_accepts_more` that can be used to query whether more input
221 can be pushed into a single interactive block.
221 can be pushed into a single interactive block.
222
222
223 This is a simple example of how an interactive terminal-based client can use
223 This is a simple example of how an interactive terminal-based client can use
224 this tool::
224 this tool::
225
225
226 isp = InputSplitter()
226 isp = InputSplitter()
227 while isp.push_accepts_more():
227 while isp.push_accepts_more():
228 indent = ' '*isp.indent_spaces
228 indent = ' '*isp.indent_spaces
229 prompt = '>>> ' + indent
229 prompt = '>>> ' + indent
230 line = indent + raw_input(prompt)
230 line = indent + raw_input(prompt)
231 isp.push(line)
231 isp.push(line)
232 print 'Input source was:\n', isp.source_reset(),
232 print 'Input source was:\n', isp.source_reset(),
233 """
233 """
234 # Number of spaces of indentation computed from input that has been pushed
234 # Number of spaces of indentation computed from input that has been pushed
235 # so far. This is the attributes callers should query to get the current
235 # so far. This is the attributes callers should query to get the current
236 # indentation level, in order to provide auto-indent facilities.
236 # indentation level, in order to provide auto-indent facilities.
237 indent_spaces = 0
237 indent_spaces = 0
238 # String, indicating the default input encoding. It is computed by default
238 # String, indicating the default input encoding. It is computed by default
239 # at initialization time via get_input_encoding(), but it can be reset by a
239 # at initialization time via get_input_encoding(), but it can be reset by a
240 # client with specific knowledge of the encoding.
240 # client with specific knowledge of the encoding.
241 encoding = ''
241 encoding = ''
242 # String where the current full source input is stored, properly encoded.
242 # String where the current full source input is stored, properly encoded.
243 # Reading this attribute is the normal way of querying the currently pushed
243 # Reading this attribute is the normal way of querying the currently pushed
244 # source code, that has been properly encoded.
244 # source code, that has been properly encoded.
245 source = ''
245 source = ''
246 # Code object corresponding to the current source. It is automatically
246 # Code object corresponding to the current source. It is automatically
247 # synced to the source, so it can be queried at any time to obtain the code
247 # synced to the source, so it can be queried at any time to obtain the code
248 # object; it will be None if the source doesn't compile to valid Python.
248 # object; it will be None if the source doesn't compile to valid Python.
249 code = None
249 code = None
250
250
251 # Private attributes
251 # Private attributes
252
252
253 # List with lines of input accumulated so far
253 # List with lines of input accumulated so far
254 _buffer = None
254 _buffer = None
255 # Command compiler
255 # Command compiler
256 _compile = None
256 _compile = None
257 # Mark when input has changed indentation all the way back to flush-left
257 # Mark when input has changed indentation all the way back to flush-left
258 _full_dedent = False
258 _full_dedent = False
259 # Boolean indicating whether the current block is complete
259 # Boolean indicating whether the current block is complete
260 _is_complete = None
260 _is_complete = None
261
261
262 def __init__(self):
262 def __init__(self):
263 """Create a new InputSplitter instance.
263 """Create a new InputSplitter instance.
264 """
264 """
265 self._buffer = []
265 self._buffer = []
266 self._compile = codeop.CommandCompiler()
266 self._compile = codeop.CommandCompiler()
267 self.encoding = get_input_encoding()
267 self.encoding = get_input_encoding()
268
268
269 def reset(self):
269 def reset(self):
270 """Reset the input buffer and associated state."""
270 """Reset the input buffer and associated state."""
271 self.indent_spaces = 0
271 self.indent_spaces = 0
272 self._buffer[:] = []
272 self._buffer[:] = []
273 self.source = ''
273 self.source = ''
274 self.code = None
274 self.code = None
275 self._is_complete = False
275 self._is_complete = False
276 self._full_dedent = False
276 self._full_dedent = False
277
277
278 def source_reset(self):
278 def source_reset(self):
279 """Return the input source and perform a full reset.
279 """Return the input source and perform a full reset.
280 """
280 """
281 out = self.source
281 out = self.source
282 self.reset()
282 self.reset()
283 return out
283 return out
284
284
285 def push(self, lines):
285 def push(self, lines):
286 """Push one or more lines of input.
286 """Push one or more lines of input.
287
287
288 This stores the given lines and returns a status code indicating
288 This stores the given lines and returns a status code indicating
289 whether the code forms a complete Python block or not.
289 whether the code forms a complete Python block or not.
290
290
291 Any exceptions generated in compilation are swallowed, but if an
291 Any exceptions generated in compilation are swallowed, but if an
292 exception was produced, the method returns True.
292 exception was produced, the method returns True.
293
293
294 Parameters
294 Parameters
295 ----------
295 ----------
296 lines : string
296 lines : string
297 One or more lines of Python input.
297 One or more lines of Python input.
298
298
299 Returns
299 Returns
300 -------
300 -------
301 is_complete : boolean
301 is_complete : boolean
302 True if the current input source (the result of the current input
302 True if the current input source (the result of the current input
303 plus prior inputs) forms a complete Python execution block. Note that
303 plus prior inputs) forms a complete Python execution block. Note that
304 this value is also stored as a private attribute (``_is_complete``), so it
304 this value is also stored as a private attribute (``_is_complete``), so it
305 can be queried at any time.
305 can be queried at any time.
306 """
306 """
307 self._store(lines)
307 self._store(lines)
308 source = self.source
308 source = self.source
309
309
310 # Before calling _compile(), reset the code object to None so that if an
310 # Before calling _compile(), reset the code object to None so that if an
311 # exception is raised in compilation, we don't mislead by having
311 # exception is raised in compilation, we don't mislead by having
312 # inconsistent code/source attributes.
312 # inconsistent code/source attributes.
313 self.code, self._is_complete = None, None
313 self.code, self._is_complete = None, None
314
314
315 # Honor termination lines properly
315 # Honor termination lines properly
316 if source.endswith('\\\n'):
316 if source.endswith('\\\n'):
317 return False
317 return False
318
318
319 self._update_indent(lines)
319 self._update_indent(lines)
320 try:
320 try:
321 self.code = self._compile(source, symbol="exec")
321 self.code = self._compile(source, symbol="exec")
322 # Invalid syntax can produce any of a number of different errors from
322 # Invalid syntax can produce any of a number of different errors from
323 # inside the compiler, so we have to catch them all. Syntax errors
323 # inside the compiler, so we have to catch them all. Syntax errors
324 # immediately produce a 'ready' block, so the invalid Python can be
324 # immediately produce a 'ready' block, so the invalid Python can be
325 # sent to the kernel for evaluation with possible ipython
325 # sent to the kernel for evaluation with possible ipython
326 # special-syntax conversion.
326 # special-syntax conversion.
327 except (SyntaxError, OverflowError, ValueError, TypeError,
327 except (SyntaxError, OverflowError, ValueError, TypeError,
328 MemoryError):
328 MemoryError):
329 self._is_complete = True
329 self._is_complete = True
330 else:
330 else:
331 # Compilation didn't produce any exceptions (though it may not have
331 # Compilation didn't produce any exceptions (though it may not have
332 # given a complete code object)
332 # given a complete code object)
333 self._is_complete = self.code is not None
333 self._is_complete = self.code is not None
334
334
335 return self._is_complete
335 return self._is_complete
336
336
337 def push_accepts_more(self):
337 def push_accepts_more(self):
338 """Return whether a block of interactive input can accept more input.
338 """Return whether a block of interactive input can accept more input.
339
339
340 This method is meant to be used by line-oriented frontends, who need to
340 This method is meant to be used by line-oriented frontends, who need to
341 guess whether a block is complete or not based solely on prior and
341 guess whether a block is complete or not based solely on prior and
342 current input lines. The InputSplitter considers it has a complete
342 current input lines. The InputSplitter considers it has a complete
343 interactive block and will not accept more input when either:
343 interactive block and will not accept more input when either:
344
344
345 * A SyntaxError is raised
345 * A SyntaxError is raised
346
346
347 * The code is complete and consists of a single line or a single
347 * The code is complete and consists of a single line or a single
348 non-compound statement
348 non-compound statement
349
349
350 * The code is complete and has a blank line at the end
350 * The code is complete and has a blank line at the end
351
351
352 If the current input produces a syntax error, this method immediately
352 If the current input produces a syntax error, this method immediately
353 returns False but does *not* raise the syntax error exception, as
353 returns False but does *not* raise the syntax error exception, as
354 typically clients will want to send invalid syntax to an execution
354 typically clients will want to send invalid syntax to an execution
355 backend which might convert the invalid syntax into valid Python via
355 backend which might convert the invalid syntax into valid Python via
356 one of the dynamic IPython mechanisms.
356 one of the dynamic IPython mechanisms.
357 """
357 """
358
358
359 # With incomplete input, unconditionally accept more
359 # With incomplete input, unconditionally accept more
360 # A syntax error also sets _is_complete to True - see push()
360 # A syntax error also sets _is_complete to True - see push()
361 if not self._is_complete:
361 if not self._is_complete:
362 #print("Not complete") # debug
362 #print("Not complete") # debug
363 return True
363 return True
364
364
365 # The user can make any (complete) input execute by leaving a blank line
365 # The user can make any (complete) input execute by leaving a blank line
366 last_line = self.source.splitlines()[-1]
366 last_line = self.source.splitlines()[-1]
367 if (not last_line) or last_line.isspace():
367 if (not last_line) or last_line.isspace():
368 #print("Blank line") # debug
368 #print("Blank line") # debug
369 return False
369 return False
370
370
371 # If there's just a single line or AST node, and we're flush left, as is
371 # If there's just a single line or AST node, and we're flush left, as is
372 # the case after a simple statement such as 'a=1', we want to execute it
372 # the case after a simple statement such as 'a=1', we want to execute it
373 # straight away.
373 # straight away.
374 if self.indent_spaces==0:
374 if self.indent_spaces==0:
375 if len(self.source.splitlines()) <= 1:
375 if len(self.source.splitlines()) <= 1:
376 return False
376 return False
377
377
378 try:
378 try:
379 code_ast = ast.parse(u''.join(self._buffer))
379 code_ast = ast.parse(u''.join(self._buffer))
380 except Exception:
380 except Exception:
381 #print("Can't parse AST") # debug
381 #print("Can't parse AST") # debug
382 return False
382 return False
383 else:
383 else:
384 if len(code_ast.body) == 1 and \
384 if len(code_ast.body) == 1 and \
385 not hasattr(code_ast.body[0], 'body'):
385 not hasattr(code_ast.body[0], 'body'):
386 #print("Simple statement") # debug
386 #print("Simple statement") # debug
387 return False
387 return False
388
388
389 # General fallback - accept more code
389 # General fallback - accept more code
390 return True
390 return True
391
391
392 #------------------------------------------------------------------------
392 #------------------------------------------------------------------------
393 # Private interface
393 # Private interface
394 #------------------------------------------------------------------------
394 #------------------------------------------------------------------------
395
395
396 def _find_indent(self, line):
396 def _find_indent(self, line):
397 """Compute the new indentation level for a single line.
397 """Compute the new indentation level for a single line.
398
398
399 Parameters
399 Parameters
400 ----------
400 ----------
401 line : str
401 line : str
402 A single new line of non-whitespace, non-comment Python input.
402 A single new line of non-whitespace, non-comment Python input.
403
403
404 Returns
404 Returns
405 -------
405 -------
406 indent_spaces : int
406 indent_spaces : int
407 New value for the indent level (it may be equal to self.indent_spaces
407 New value for the indent level (it may be equal to self.indent_spaces
408 if indentation doesn't change.
408 if indentation doesn't change.
409
409
410 full_dedent : boolean
410 full_dedent : boolean
411 Whether the new line causes a full flush-left dedent.
411 Whether the new line causes a full flush-left dedent.
412 """
412 """
413 indent_spaces = self.indent_spaces
413 indent_spaces = self.indent_spaces
414 full_dedent = self._full_dedent
414 full_dedent = self._full_dedent
415
415
416 inisp = num_ini_spaces(line)
416 inisp = num_ini_spaces(line)
417 if inisp < indent_spaces:
417 if inisp < indent_spaces:
418 indent_spaces = inisp
418 indent_spaces = inisp
419 if indent_spaces <= 0:
419 if indent_spaces <= 0:
420 #print 'Full dedent in text',self.source # dbg
420 #print 'Full dedent in text',self.source # dbg
421 full_dedent = True
421 full_dedent = True
422
422
423 if line.rstrip()[-1] == ':':
423 if line.rstrip()[-1] == ':':
424 indent_spaces += 4
424 indent_spaces += 4
425 elif dedent_re.match(line):
425 elif dedent_re.match(line):
426 indent_spaces -= 4
426 indent_spaces -= 4
427 if indent_spaces <= 0:
427 if indent_spaces <= 0:
428 full_dedent = True
428 full_dedent = True
429
429
430 # Safety
430 # Safety
431 if indent_spaces < 0:
431 if indent_spaces < 0:
432 indent_spaces = 0
432 indent_spaces = 0
433 #print 'safety' # dbg
433 #print 'safety' # dbg
434
434
435 return indent_spaces, full_dedent
435 return indent_spaces, full_dedent
436
436
437 def _update_indent(self, lines):
437 def _update_indent(self, lines):
438 for line in remove_comments(lines).splitlines():
438 for line in remove_comments(lines).splitlines():
439 if line and not line.isspace():
439 if line and not line.isspace():
440 self.indent_spaces, self._full_dedent = self._find_indent(line)
440 self.indent_spaces, self._full_dedent = self._find_indent(line)
441
441
442 def _store(self, lines, buffer=None, store='source'):
442 def _store(self, lines, buffer=None, store='source'):
443 """Store one or more lines of input.
443 """Store one or more lines of input.
444
444
445 If input lines are not newline-terminated, a newline is automatically
445 If input lines are not newline-terminated, a newline is automatically
446 appended."""
446 appended."""
447
447
448 if buffer is None:
448 if buffer is None:
449 buffer = self._buffer
449 buffer = self._buffer
450
450
451 if lines.endswith('\n'):
451 if lines.endswith('\n'):
452 buffer.append(lines)
452 buffer.append(lines)
453 else:
453 else:
454 buffer.append(lines+'\n')
454 buffer.append(lines+'\n')
455 setattr(self, store, self._set_source(buffer))
455 setattr(self, store, self._set_source(buffer))
456
456
457 def _set_source(self, buffer):
457 def _set_source(self, buffer):
458 return u''.join(buffer)
458 return u''.join(buffer)
459
459
460
460
461 class IPythonInputSplitter(InputSplitter):
461 class IPythonInputSplitter(InputSplitter):
462 """An input splitter that recognizes all of IPython's special syntax."""
462 """An input splitter that recognizes all of IPython's special syntax."""
463
463
464 # String with raw, untransformed input.
464 # String with raw, untransformed input.
465 source_raw = ''
465 source_raw = ''
466
466
467 # Flag to track when a transformer has stored input that it hasn't given
467 # Flag to track when a transformer has stored input that it hasn't given
468 # back yet.
468 # back yet.
469 transformer_accumulating = False
469 transformer_accumulating = False
470
470
471 # Flag to track when assemble_python_lines has stored input that it hasn't
471 # Flag to track when assemble_python_lines has stored input that it hasn't
472 # given back yet.
472 # given back yet.
473 within_python_line = False
473 within_python_line = False
474
474
475 # Private attributes
475 # Private attributes
476
476
477 # List with lines of raw input accumulated so far.
477 # List with lines of raw input accumulated so far.
478 _buffer_raw = None
478 _buffer_raw = None
479
479
480 def __init__(self, line_input_checker=True, physical_line_transforms=None,
480 def __init__(self, line_input_checker=True, physical_line_transforms=None,
481 logical_line_transforms=None, python_line_transforms=None):
481 logical_line_transforms=None, python_line_transforms=None):
482 super(IPythonInputSplitter, self).__init__()
482 super(IPythonInputSplitter, self).__init__()
483 self._buffer_raw = []
483 self._buffer_raw = []
484 self._validate = True
484 self._validate = True
485
485
486 if physical_line_transforms is not None:
486 if physical_line_transforms is not None:
487 self.physical_line_transforms = physical_line_transforms
487 self.physical_line_transforms = physical_line_transforms
488 else:
488 else:
489 self.physical_line_transforms = [
489 self.physical_line_transforms = [
490 leading_indent(),
490 leading_indent(),
491 classic_prompt(),
491 classic_prompt(),
492 ipy_prompt(),
492 ipy_prompt(),
493 strip_encoding_cookie(),
493 strip_encoding_cookie(),
494 cellmagic(end_on_blank_line=line_input_checker),
494 cellmagic(end_on_blank_line=line_input_checker),
495 ]
495 ]
496
496
497 self.assemble_logical_lines = assemble_logical_lines()
497 self.assemble_logical_lines = assemble_logical_lines()
498 if logical_line_transforms is not None:
498 if logical_line_transforms is not None:
499 self.logical_line_transforms = logical_line_transforms
499 self.logical_line_transforms = logical_line_transforms
500 else:
500 else:
501 self.logical_line_transforms = [
501 self.logical_line_transforms = [
502 help_end(),
502 help_end(),
503 escaped_commands(),
503 escaped_commands(),
504 assign_from_magic(),
504 assign_from_magic(),
505 assign_from_system(),
505 assign_from_system(),
506 ]
506 ]
507
507
508 self.assemble_python_lines = assemble_python_lines()
508 self.assemble_python_lines = assemble_python_lines()
509 if python_line_transforms is not None:
509 if python_line_transforms is not None:
510 self.python_line_transforms = python_line_transforms
510 self.python_line_transforms = python_line_transforms
511 else:
511 else:
512 # We don't use any of these at present
512 # We don't use any of these at present
513 self.python_line_transforms = []
513 self.python_line_transforms = []
514
514
515 @property
515 @property
516 def transforms(self):
516 def transforms(self):
517 "Quick access to all transformers."
517 "Quick access to all transformers."
518 return self.physical_line_transforms + \
518 return self.physical_line_transforms + \
519 [self.assemble_logical_lines] + self.logical_line_transforms + \
519 [self.assemble_logical_lines] + self.logical_line_transforms + \
520 [self.assemble_python_lines] + self.python_line_transforms
520 [self.assemble_python_lines] + self.python_line_transforms
521
521
522 @property
522 @property
523 def transforms_in_use(self):
523 def transforms_in_use(self):
524 """Transformers, excluding logical line transformers if we're in a
524 """Transformers, excluding logical line transformers if we're in a
525 Python line."""
525 Python line."""
526 t = self.physical_line_transforms[:]
526 t = self.physical_line_transforms[:]
527 if not self.within_python_line:
527 if not self.within_python_line:
528 t += [self.assemble_logical_lines] + self.logical_line_transforms
528 t += [self.assemble_logical_lines] + self.logical_line_transforms
529 return t + [self.assemble_python_lines] + self.python_line_transforms
529 return t + [self.assemble_python_lines] + self.python_line_transforms
530
530
531 def reset(self):
531 def reset(self):
532 """Reset the input buffer and associated state."""
532 """Reset the input buffer and associated state."""
533 super(IPythonInputSplitter, self).reset()
533 super(IPythonInputSplitter, self).reset()
534 self._buffer_raw[:] = []
534 self._buffer_raw[:] = []
535 self.source_raw = ''
535 self.source_raw = ''
536 self.transformer_accumulating = False
536 self.transformer_accumulating = False
537 self.within_python_line = False
537 self.within_python_line = False
538 for t in self.transforms:
538 for t in self.transforms:
539 t.reset()
539 t.reset()
540
540
541 def flush_transformers(self):
541 def flush_transformers(self):
542 def _flush(transform, out):
542 def _flush(transform, out):
543 if out is not None:
543 if out is not None:
544 tmp = transform.push(out)
544 tmp = transform.push(out)
545 return tmp or transform.reset() or None
545 return tmp or transform.reset() or None
546 else:
546 else:
547 return transform.reset() or None
547 return transform.reset() or None
548
548
549 out = None
549 out = None
550 for t in self.transforms_in_use:
550 for t in self.transforms_in_use:
551 out = _flush(t, out)
551 out = _flush(t, out)
552
552
553 if out is not None:
553 if out is not None:
554 self._store(out)
554 self._store(out)
555
555
556 def source_raw_reset(self):
556 def source_raw_reset(self):
557 """Return input and raw source and perform a full reset.
557 """Return input and raw source and perform a full reset.
558 """
558 """
559 self.flush_transformers()
559 self.flush_transformers()
560 out = self.source
560 out = self.source
561 out_r = self.source_raw
561 out_r = self.source_raw
562 self.reset()
562 self.reset()
563 return out, out_r
563 return out, out_r
564
564
565 def source_reset(self):
565 def source_reset(self):
566 self.flush_transformers()
566 self.flush_transformers()
567 return super(IPythonInputSplitter, self).source_reset()
567 return super(IPythonInputSplitter, self).source_reset()
568
568
569 def push_accepts_more(self):
569 def push_accepts_more(self):
570 if self.transformer_accumulating:
570 if self.transformer_accumulating:
571 return True
571 return True
572 else:
572 else:
573 return super(IPythonInputSplitter, self).push_accepts_more()
573 return super(IPythonInputSplitter, self).push_accepts_more()
574
574
575 def transform_cell(self, cell):
575 def transform_cell(self, cell):
576 """Process and translate a cell of input.
576 """Process and translate a cell of input.
577 """
577 """
578 self.reset()
578 self.reset()
579 self.push(cell)
579 self.push(cell)
580 return self.source_reset()
580 return self.source_reset()
581
581
582 def push(self, lines):
582 def push(self, lines):
583 """Push one or more lines of IPython input.
583 """Push one or more lines of IPython input.
584
584
585 This stores the given lines and returns a status code indicating
585 This stores the given lines and returns a status code indicating
586 whether the code forms a complete Python block or not, after processing
586 whether the code forms a complete Python block or not, after processing
587 all input lines for special IPython syntax.
587 all input lines for special IPython syntax.
588
588
589 Any exceptions generated in compilation are swallowed, but if an
589 Any exceptions generated in compilation are swallowed, but if an
590 exception was produced, the method returns True.
590 exception was produced, the method returns True.
591
591
592 Parameters
592 Parameters
593 ----------
593 ----------
594 lines : string
594 lines : string
595 One or more lines of Python input.
595 One or more lines of Python input.
596
596
597 Returns
597 Returns
598 -------
598 -------
599 is_complete : boolean
599 is_complete : boolean
600 True if the current input source (the result of the current input
600 True if the current input source (the result of the current input
601 plus prior inputs) forms a complete Python execution block. Note that
601 plus prior inputs) forms a complete Python execution block. Note that
602 this value is also stored as a private attribute (_is_complete), so it
602 this value is also stored as a private attribute (_is_complete), so it
603 can be queried at any time.
603 can be queried at any time.
604 """
604 """
605
605
606 # We must ensure all input is pure unicode
606 # We must ensure all input is pure unicode
607 lines = cast_unicode(lines, self.encoding)
607 lines = cast_unicode(lines, self.encoding)
608
608
609 # ''.splitlines() --> [], but we need to push the empty line to transformers
609 # ''.splitlines() --> [], but we need to push the empty line to transformers
610 lines_list = lines.splitlines()
610 lines_list = lines.splitlines()
611 if not lines_list:
611 if not lines_list:
612 lines_list = ['']
612 lines_list = ['']
613
613
614 # Store raw source before applying any transformations to it. Note
614 # Store raw source before applying any transformations to it. Note
615 # that this must be done *after* the reset() call that would otherwise
615 # that this must be done *after* the reset() call that would otherwise
616 # flush the buffer.
616 # flush the buffer.
617 self._store(lines, self._buffer_raw, 'source_raw')
617 self._store(lines, self._buffer_raw, 'source_raw')
618
618
619 for line in lines_list:
619 for line in lines_list:
620 out = self.push_line(line)
620 out = self.push_line(line)
621
621
622 return out
622 return out
623
623
624 def push_line(self, line):
624 def push_line(self, line):
625 buf = self._buffer
625 buf = self._buffer
626
626
627 def _accumulating(dbg):
627 def _accumulating(dbg):
628 #print(dbg)
628 #print(dbg)
629 self.transformer_accumulating = True
629 self.transformer_accumulating = True
630 return False
630 return False
631
631
632 for transformer in self.physical_line_transforms:
632 for transformer in self.physical_line_transforms:
633 line = transformer.push(line)
633 line = transformer.push(line)
634 if line is None:
634 if line is None:
635 return _accumulating(transformer)
635 return _accumulating(transformer)
636
636
637 if not self.within_python_line:
637 if not self.within_python_line:
638 line = self.assemble_logical_lines.push(line)
638 line = self.assemble_logical_lines.push(line)
639 if line is None:
639 if line is None:
640 return _accumulating('acc logical line')
640 return _accumulating('acc logical line')
641
641
642 for transformer in self.logical_line_transforms:
642 for transformer in self.logical_line_transforms:
643 line = transformer.push(line)
643 line = transformer.push(line)
644 if line is None:
644 if line is None:
645 return _accumulating(transformer)
645 return _accumulating(transformer)
646
646
647 line = self.assemble_python_lines.push(line)
647 line = self.assemble_python_lines.push(line)
648 if line is None:
648 if line is None:
649 self.within_python_line = True
649 self.within_python_line = True
650 return _accumulating('acc python line')
650 return _accumulating('acc python line')
651 else:
651 else:
652 self.within_python_line = False
652 self.within_python_line = False
653
653
654 for transformer in self.python_line_transforms:
654 for transformer in self.python_line_transforms:
655 line = transformer.push(line)
655 line = transformer.push(line)
656 if line is None:
656 if line is None:
657 return _accumulating(transformer)
657 return _accumulating(transformer)
658
658
659 #print("transformers clear") #debug
659 #print("transformers clear") #debug
660 self.transformer_accumulating = False
660 self.transformer_accumulating = False
661 return super(IPythonInputSplitter, self).push(line)
661 return super(IPythonInputSplitter, self).push(line)
@@ -1,3168 +1,3168 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """Main IPython class."""
2 """Main IPython class."""
3
3
4 #-----------------------------------------------------------------------------
4 #-----------------------------------------------------------------------------
5 # Copyright (C) 2001 Janko Hauser <jhauser@zscout.de>
5 # Copyright (C) 2001 Janko Hauser <jhauser@zscout.de>
6 # Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu>
6 # Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu>
7 # Copyright (C) 2008-2011 The IPython Development Team
7 # Copyright (C) 2008-2011 The IPython Development Team
8 #
8 #
9 # Distributed under the terms of the BSD License. The full license is in
9 # Distributed under the terms of the BSD License. The full license is in
10 # the file COPYING, distributed as part of this software.
10 # the file COPYING, distributed as part of this software.
11 #-----------------------------------------------------------------------------
11 #-----------------------------------------------------------------------------
12
12
13 #-----------------------------------------------------------------------------
13 #-----------------------------------------------------------------------------
14 # Imports
14 # Imports
15 #-----------------------------------------------------------------------------
15 #-----------------------------------------------------------------------------
16
16
17 from __future__ import absolute_import
17 from __future__ import absolute_import
18 from __future__ import print_function
18 from __future__ import print_function
19
19
20 import __future__
20 import __future__
21 import abc
21 import abc
22 import ast
22 import ast
23 import atexit
23 import atexit
24 import functools
24 import functools
25 import os
25 import os
26 import re
26 import re
27 import runpy
27 import runpy
28 import sys
28 import sys
29 import tempfile
29 import tempfile
30 import types
30 import types
31 import subprocess
31 import subprocess
32 from io import open as io_open
32 from io import open as io_open
33
33
34 from IPython.config.configurable import SingletonConfigurable
34 from IPython.config.configurable import SingletonConfigurable
35 from IPython.core import debugger, oinspect
35 from IPython.core import debugger, oinspect
36 from IPython.core import magic
36 from IPython.core import magic
37 from IPython.core import page
37 from IPython.core import page
38 from IPython.core import prefilter
38 from IPython.core import prefilter
39 from IPython.core import shadowns
39 from IPython.core import shadowns
40 from IPython.core import ultratb
40 from IPython.core import ultratb
41 from IPython.core.alias import AliasManager, AliasError
41 from IPython.core.alias import AliasManager, AliasError
42 from IPython.core.autocall import ExitAutocall
42 from IPython.core.autocall import ExitAutocall
43 from IPython.core.builtin_trap import BuiltinTrap
43 from IPython.core.builtin_trap import BuiltinTrap
44 from IPython.core.compilerop import CachingCompiler, check_linecache_ipython
44 from IPython.core.compilerop import CachingCompiler, check_linecache_ipython
45 from IPython.core.display_trap import DisplayTrap
45 from IPython.core.display_trap import DisplayTrap
46 from IPython.core.displayhook import DisplayHook
46 from IPython.core.displayhook import DisplayHook
47 from IPython.core.displaypub import DisplayPublisher
47 from IPython.core.displaypub import DisplayPublisher
48 from IPython.core.error import UsageError
48 from IPython.core.error import UsageError
49 from IPython.core.extensions import ExtensionManager
49 from IPython.core.extensions import ExtensionManager
50 from IPython.core.formatters import DisplayFormatter
50 from IPython.core.formatters import DisplayFormatter
51 from IPython.core.history import HistoryManager
51 from IPython.core.history import HistoryManager
52 from IPython.core.inputsplitter import IPythonInputSplitter, ESC_MAGIC, ESC_MAGIC2
52 from IPython.core.inputsplitter import IPythonInputSplitter, ESC_MAGIC, ESC_MAGIC2
53 from IPython.core.logger import Logger
53 from IPython.core.logger import Logger
54 from IPython.core.macro import Macro
54 from IPython.core.macro import Macro
55 from IPython.core.payload import PayloadManager
55 from IPython.core.payload import PayloadManager
56 from IPython.core.prefilter import PrefilterManager
56 from IPython.core.prefilter import PrefilterManager
57 from IPython.core.profiledir import ProfileDir
57 from IPython.core.profiledir import ProfileDir
58 from IPython.core.prompts import PromptManager
58 from IPython.core.prompts import PromptManager
59 from IPython.lib.latextools import LaTeXTool
59 from IPython.lib.latextools import LaTeXTool
60 from IPython.testing.skipdoctest import skip_doctest
60 from IPython.testing.skipdoctest import skip_doctest
61 from IPython.utils import PyColorize
61 from IPython.utils import PyColorize
62 from IPython.utils import io
62 from IPython.utils import io
63 from IPython.utils import py3compat
63 from IPython.utils import py3compat
64 from IPython.utils import openpy
64 from IPython.utils import openpy
65 from IPython.utils.decorators import undoc
65 from IPython.utils.decorators import undoc
66 from IPython.utils.io import ask_yes_no
66 from IPython.utils.io import ask_yes_no
67 from IPython.utils.ipstruct import Struct
67 from IPython.utils.ipstruct import Struct
68 from IPython.utils.path import get_home_dir, get_ipython_dir, get_py_filename, unquote_filename
68 from IPython.utils.path import get_home_dir, get_ipython_dir, get_py_filename, unquote_filename
69 from IPython.utils.pickleshare import PickleShareDB
69 from IPython.utils.pickleshare import PickleShareDB
70 from IPython.utils.process import system, getoutput
70 from IPython.utils.process import system, getoutput
71 from IPython.utils.py3compat import (builtin_mod, unicode_type, string_types,
71 from IPython.utils.py3compat import (builtin_mod, unicode_type, string_types,
72 with_metaclass, iteritems)
72 with_metaclass, iteritems)
73 from IPython.utils.strdispatch import StrDispatch
73 from IPython.utils.strdispatch import StrDispatch
74 from IPython.utils.syspathcontext import prepended_to_syspath
74 from IPython.utils.syspathcontext import prepended_to_syspath
75 from IPython.utils.text import (format_screen, LSString, SList,
75 from IPython.utils.text import (format_screen, LSString, SList,
76 DollarFormatter)
76 DollarFormatter)
77 from IPython.utils.traitlets import (Integer, CBool, CaselessStrEnum, Enum,
77 from IPython.utils.traitlets import (Integer, CBool, CaselessStrEnum, Enum,
78 List, Unicode, Instance, Type)
78 List, Unicode, Instance, Type)
79 from IPython.utils.warn import warn, error
79 from IPython.utils.warn import warn, error
80 import IPython.core.hooks
80 import IPython.core.hooks
81
81
82 #-----------------------------------------------------------------------------
82 #-----------------------------------------------------------------------------
83 # Globals
83 # Globals
84 #-----------------------------------------------------------------------------
84 #-----------------------------------------------------------------------------
85
85
86 # compiled regexps for autoindent management
86 # compiled regexps for autoindent management
87 dedent_re = re.compile(r'^\s+raise|^\s+return|^\s+pass')
87 dedent_re = re.compile(r'^\s+raise|^\s+return|^\s+pass')
88
88
89 #-----------------------------------------------------------------------------
89 #-----------------------------------------------------------------------------
90 # Utilities
90 # Utilities
91 #-----------------------------------------------------------------------------
91 #-----------------------------------------------------------------------------
92
92
93 @undoc
93 @undoc
94 def softspace(file, newvalue):
94 def softspace(file, newvalue):
95 """Copied from code.py, to remove the dependency"""
95 """Copied from code.py, to remove the dependency"""
96
96
97 oldvalue = 0
97 oldvalue = 0
98 try:
98 try:
99 oldvalue = file.softspace
99 oldvalue = file.softspace
100 except AttributeError:
100 except AttributeError:
101 pass
101 pass
102 try:
102 try:
103 file.softspace = newvalue
103 file.softspace = newvalue
104 except (AttributeError, TypeError):
104 except (AttributeError, TypeError):
105 # "attribute-less object" or "read-only attributes"
105 # "attribute-less object" or "read-only attributes"
106 pass
106 pass
107 return oldvalue
107 return oldvalue
108
108
109 @undoc
109 @undoc
110 def no_op(*a, **kw): pass
110 def no_op(*a, **kw): pass
111
111
112 @undoc
112 @undoc
113 class NoOpContext(object):
113 class NoOpContext(object):
114 def __enter__(self): pass
114 def __enter__(self): pass
115 def __exit__(self, type, value, traceback): pass
115 def __exit__(self, type, value, traceback): pass
116 no_op_context = NoOpContext()
116 no_op_context = NoOpContext()
117
117
118 class SpaceInInput(Exception): pass
118 class SpaceInInput(Exception): pass
119
119
120 @undoc
120 @undoc
121 class Bunch: pass
121 class Bunch: pass
122
122
123
123
124 def get_default_colors():
124 def get_default_colors():
125 if sys.platform=='darwin':
125 if sys.platform=='darwin':
126 return "LightBG"
126 return "LightBG"
127 elif os.name=='nt':
127 elif os.name=='nt':
128 return 'Linux'
128 return 'Linux'
129 else:
129 else:
130 return 'Linux'
130 return 'Linux'
131
131
132
132
133 class SeparateUnicode(Unicode):
133 class SeparateUnicode(Unicode):
134 """A Unicode subclass to validate separate_in, separate_out, etc.
134 r"""A Unicode subclass to validate separate_in, separate_out, etc.
135
135
136 This is a Unicode based trait that converts '0'->'' and '\\n'->'\n'.
136 This is a Unicode based trait that converts '0'->'' and ``'\\n'->'\n'``.
137 """
137 """
138
138
139 def validate(self, obj, value):
139 def validate(self, obj, value):
140 if value == '0': value = ''
140 if value == '0': value = ''
141 value = value.replace('\\n','\n')
141 value = value.replace('\\n','\n')
142 return super(SeparateUnicode, self).validate(obj, value)
142 return super(SeparateUnicode, self).validate(obj, value)
143
143
144
144
145 class ReadlineNoRecord(object):
145 class ReadlineNoRecord(object):
146 """Context manager to execute some code, then reload readline history
146 """Context manager to execute some code, then reload readline history
147 so that interactive input to the code doesn't appear when pressing up."""
147 so that interactive input to the code doesn't appear when pressing up."""
148 def __init__(self, shell):
148 def __init__(self, shell):
149 self.shell = shell
149 self.shell = shell
150 self._nested_level = 0
150 self._nested_level = 0
151
151
152 def __enter__(self):
152 def __enter__(self):
153 if self._nested_level == 0:
153 if self._nested_level == 0:
154 try:
154 try:
155 self.orig_length = self.current_length()
155 self.orig_length = self.current_length()
156 self.readline_tail = self.get_readline_tail()
156 self.readline_tail = self.get_readline_tail()
157 except (AttributeError, IndexError): # Can fail with pyreadline
157 except (AttributeError, IndexError): # Can fail with pyreadline
158 self.orig_length, self.readline_tail = 999999, []
158 self.orig_length, self.readline_tail = 999999, []
159 self._nested_level += 1
159 self._nested_level += 1
160
160
161 def __exit__(self, type, value, traceback):
161 def __exit__(self, type, value, traceback):
162 self._nested_level -= 1
162 self._nested_level -= 1
163 if self._nested_level == 0:
163 if self._nested_level == 0:
164 # Try clipping the end if it's got longer
164 # Try clipping the end if it's got longer
165 try:
165 try:
166 e = self.current_length() - self.orig_length
166 e = self.current_length() - self.orig_length
167 if e > 0:
167 if e > 0:
168 for _ in range(e):
168 for _ in range(e):
169 self.shell.readline.remove_history_item(self.orig_length)
169 self.shell.readline.remove_history_item(self.orig_length)
170
170
171 # If it still doesn't match, just reload readline history.
171 # If it still doesn't match, just reload readline history.
172 if self.current_length() != self.orig_length \
172 if self.current_length() != self.orig_length \
173 or self.get_readline_tail() != self.readline_tail:
173 or self.get_readline_tail() != self.readline_tail:
174 self.shell.refill_readline_hist()
174 self.shell.refill_readline_hist()
175 except (AttributeError, IndexError):
175 except (AttributeError, IndexError):
176 pass
176 pass
177 # Returning False will cause exceptions to propagate
177 # Returning False will cause exceptions to propagate
178 return False
178 return False
179
179
180 def current_length(self):
180 def current_length(self):
181 return self.shell.readline.get_current_history_length()
181 return self.shell.readline.get_current_history_length()
182
182
183 def get_readline_tail(self, n=10):
183 def get_readline_tail(self, n=10):
184 """Get the last n items in readline history."""
184 """Get the last n items in readline history."""
185 end = self.shell.readline.get_current_history_length() + 1
185 end = self.shell.readline.get_current_history_length() + 1
186 start = max(end-n, 1)
186 start = max(end-n, 1)
187 ghi = self.shell.readline.get_history_item
187 ghi = self.shell.readline.get_history_item
188 return [ghi(x) for x in range(start, end)]
188 return [ghi(x) for x in range(start, end)]
189
189
190
190
191 @undoc
191 @undoc
192 class DummyMod(object):
192 class DummyMod(object):
193 """A dummy module used for IPython's interactive module when
193 """A dummy module used for IPython's interactive module when
194 a namespace must be assigned to the module's __dict__."""
194 a namespace must be assigned to the module's __dict__."""
195 pass
195 pass
196
196
197 #-----------------------------------------------------------------------------
197 #-----------------------------------------------------------------------------
198 # Main IPython class
198 # Main IPython class
199 #-----------------------------------------------------------------------------
199 #-----------------------------------------------------------------------------
200
200
201 class InteractiveShell(SingletonConfigurable):
201 class InteractiveShell(SingletonConfigurable):
202 """An enhanced, interactive shell for Python."""
202 """An enhanced, interactive shell for Python."""
203
203
204 _instance = None
204 _instance = None
205
205
206 ast_transformers = List([], config=True, help=
206 ast_transformers = List([], config=True, help=
207 """
207 """
208 A list of ast.NodeTransformer subclass instances, which will be applied
208 A list of ast.NodeTransformer subclass instances, which will be applied
209 to user input before code is run.
209 to user input before code is run.
210 """
210 """
211 )
211 )
212
212
213 autocall = Enum((0,1,2), default_value=0, config=True, help=
213 autocall = Enum((0,1,2), default_value=0, config=True, help=
214 """
214 """
215 Make IPython automatically call any callable object even if you didn't
215 Make IPython automatically call any callable object even if you didn't
216 type explicit parentheses. For example, 'str 43' becomes 'str(43)'
216 type explicit parentheses. For example, 'str 43' becomes 'str(43)'
217 automatically. The value can be '0' to disable the feature, '1' for
217 automatically. The value can be '0' to disable the feature, '1' for
218 'smart' autocall, where it is not applied if there are no more
218 'smart' autocall, where it is not applied if there are no more
219 arguments on the line, and '2' for 'full' autocall, where all callable
219 arguments on the line, and '2' for 'full' autocall, where all callable
220 objects are automatically called (even if no arguments are present).
220 objects are automatically called (even if no arguments are present).
221 """
221 """
222 )
222 )
223 # TODO: remove all autoindent logic and put into frontends.
223 # TODO: remove all autoindent logic and put into frontends.
224 # We can't do this yet because even runlines uses the autoindent.
224 # We can't do this yet because even runlines uses the autoindent.
225 autoindent = CBool(True, config=True, help=
225 autoindent = CBool(True, config=True, help=
226 """
226 """
227 Autoindent IPython code entered interactively.
227 Autoindent IPython code entered interactively.
228 """
228 """
229 )
229 )
230 automagic = CBool(True, config=True, help=
230 automagic = CBool(True, config=True, help=
231 """
231 """
232 Enable magic commands to be called without the leading %.
232 Enable magic commands to be called without the leading %.
233 """
233 """
234 )
234 )
235 cache_size = Integer(1000, config=True, help=
235 cache_size = Integer(1000, config=True, help=
236 """
236 """
237 Set the size of the output cache. The default is 1000, you can
237 Set the size of the output cache. The default is 1000, you can
238 change it permanently in your config file. Setting it to 0 completely
238 change it permanently in your config file. Setting it to 0 completely
239 disables the caching system, and the minimum value accepted is 20 (if
239 disables the caching system, and the minimum value accepted is 20 (if
240 you provide a value less than 20, it is reset to 0 and a warning is
240 you provide a value less than 20, it is reset to 0 and a warning is
241 issued). This limit is defined because otherwise you'll spend more
241 issued). This limit is defined because otherwise you'll spend more
242 time re-flushing a too small cache than working
242 time re-flushing a too small cache than working
243 """
243 """
244 )
244 )
245 color_info = CBool(True, config=True, help=
245 color_info = CBool(True, config=True, help=
246 """
246 """
247 Use colors for displaying information about objects. Because this
247 Use colors for displaying information about objects. Because this
248 information is passed through a pager (like 'less'), and some pagers
248 information is passed through a pager (like 'less'), and some pagers
249 get confused with color codes, this capability can be turned off.
249 get confused with color codes, this capability can be turned off.
250 """
250 """
251 )
251 )
252 colors = CaselessStrEnum(('NoColor','LightBG','Linux'),
252 colors = CaselessStrEnum(('NoColor','LightBG','Linux'),
253 default_value=get_default_colors(), config=True,
253 default_value=get_default_colors(), config=True,
254 help="Set the color scheme (NoColor, Linux, or LightBG)."
254 help="Set the color scheme (NoColor, Linux, or LightBG)."
255 )
255 )
256 colors_force = CBool(False, help=
256 colors_force = CBool(False, help=
257 """
257 """
258 Force use of ANSI color codes, regardless of OS and readline
258 Force use of ANSI color codes, regardless of OS and readline
259 availability.
259 availability.
260 """
260 """
261 # FIXME: This is essentially a hack to allow ZMQShell to show colors
261 # FIXME: This is essentially a hack to allow ZMQShell to show colors
262 # without readline on Win32. When the ZMQ formatting system is
262 # without readline on Win32. When the ZMQ formatting system is
263 # refactored, this should be removed.
263 # refactored, this should be removed.
264 )
264 )
265 debug = CBool(False, config=True)
265 debug = CBool(False, config=True)
266 deep_reload = CBool(False, config=True, help=
266 deep_reload = CBool(False, config=True, help=
267 """
267 """
268 Enable deep (recursive) reloading by default. IPython can use the
268 Enable deep (recursive) reloading by default. IPython can use the
269 deep_reload module which reloads changes in modules recursively (it
269 deep_reload module which reloads changes in modules recursively (it
270 replaces the reload() function, so you don't need to change anything to
270 replaces the reload() function, so you don't need to change anything to
271 use it). deep_reload() forces a full reload of modules whose code may
271 use it). deep_reload() forces a full reload of modules whose code may
272 have changed, which the default reload() function does not. When
272 have changed, which the default reload() function does not. When
273 deep_reload is off, IPython will use the normal reload(), but
273 deep_reload is off, IPython will use the normal reload(), but
274 deep_reload will still be available as dreload().
274 deep_reload will still be available as dreload().
275 """
275 """
276 )
276 )
277 disable_failing_post_execute = CBool(False, config=True,
277 disable_failing_post_execute = CBool(False, config=True,
278 help="Don't call post-execute functions that have failed in the past."
278 help="Don't call post-execute functions that have failed in the past."
279 )
279 )
280 display_formatter = Instance(DisplayFormatter)
280 display_formatter = Instance(DisplayFormatter)
281 displayhook_class = Type(DisplayHook)
281 displayhook_class = Type(DisplayHook)
282 display_pub_class = Type(DisplayPublisher)
282 display_pub_class = Type(DisplayPublisher)
283 data_pub_class = None
283 data_pub_class = None
284
284
285 exit_now = CBool(False)
285 exit_now = CBool(False)
286 exiter = Instance(ExitAutocall)
286 exiter = Instance(ExitAutocall)
287 def _exiter_default(self):
287 def _exiter_default(self):
288 return ExitAutocall(self)
288 return ExitAutocall(self)
289 # Monotonically increasing execution counter
289 # Monotonically increasing execution counter
290 execution_count = Integer(1)
290 execution_count = Integer(1)
291 filename = Unicode("<ipython console>")
291 filename = Unicode("<ipython console>")
292 ipython_dir= Unicode('', config=True) # Set to get_ipython_dir() in __init__
292 ipython_dir= Unicode('', config=True) # Set to get_ipython_dir() in __init__
293
293
294 # Input splitter, to transform input line by line and detect when a block
294 # Input splitter, to transform input line by line and detect when a block
295 # is ready to be executed.
295 # is ready to be executed.
296 input_splitter = Instance('IPython.core.inputsplitter.IPythonInputSplitter',
296 input_splitter = Instance('IPython.core.inputsplitter.IPythonInputSplitter',
297 (), {'line_input_checker': True})
297 (), {'line_input_checker': True})
298
298
299 # This InputSplitter instance is used to transform completed cells before
299 # This InputSplitter instance is used to transform completed cells before
300 # running them. It allows cell magics to contain blank lines.
300 # running them. It allows cell magics to contain blank lines.
301 input_transformer_manager = Instance('IPython.core.inputsplitter.IPythonInputSplitter',
301 input_transformer_manager = Instance('IPython.core.inputsplitter.IPythonInputSplitter',
302 (), {'line_input_checker': False})
302 (), {'line_input_checker': False})
303
303
304 logstart = CBool(False, config=True, help=
304 logstart = CBool(False, config=True, help=
305 """
305 """
306 Start logging to the default log file.
306 Start logging to the default log file.
307 """
307 """
308 )
308 )
309 logfile = Unicode('', config=True, help=
309 logfile = Unicode('', config=True, help=
310 """
310 """
311 The name of the logfile to use.
311 The name of the logfile to use.
312 """
312 """
313 )
313 )
314 logappend = Unicode('', config=True, help=
314 logappend = Unicode('', config=True, help=
315 """
315 """
316 Start logging to the given file in append mode.
316 Start logging to the given file in append mode.
317 """
317 """
318 )
318 )
319 object_info_string_level = Enum((0,1,2), default_value=0,
319 object_info_string_level = Enum((0,1,2), default_value=0,
320 config=True)
320 config=True)
321 pdb = CBool(False, config=True, help=
321 pdb = CBool(False, config=True, help=
322 """
322 """
323 Automatically call the pdb debugger after every exception.
323 Automatically call the pdb debugger after every exception.
324 """
324 """
325 )
325 )
326 multiline_history = CBool(sys.platform != 'win32', config=True,
326 multiline_history = CBool(sys.platform != 'win32', config=True,
327 help="Save multi-line entries as one entry in readline history"
327 help="Save multi-line entries as one entry in readline history"
328 )
328 )
329
329
330 # deprecated prompt traits:
330 # deprecated prompt traits:
331
331
332 prompt_in1 = Unicode('In [\\#]: ', config=True,
332 prompt_in1 = Unicode('In [\\#]: ', config=True,
333 help="Deprecated, use PromptManager.in_template")
333 help="Deprecated, use PromptManager.in_template")
334 prompt_in2 = Unicode(' .\\D.: ', config=True,
334 prompt_in2 = Unicode(' .\\D.: ', config=True,
335 help="Deprecated, use PromptManager.in2_template")
335 help="Deprecated, use PromptManager.in2_template")
336 prompt_out = Unicode('Out[\\#]: ', config=True,
336 prompt_out = Unicode('Out[\\#]: ', config=True,
337 help="Deprecated, use PromptManager.out_template")
337 help="Deprecated, use PromptManager.out_template")
338 prompts_pad_left = CBool(True, config=True,
338 prompts_pad_left = CBool(True, config=True,
339 help="Deprecated, use PromptManager.justify")
339 help="Deprecated, use PromptManager.justify")
340
340
341 def _prompt_trait_changed(self, name, old, new):
341 def _prompt_trait_changed(self, name, old, new):
342 table = {
342 table = {
343 'prompt_in1' : 'in_template',
343 'prompt_in1' : 'in_template',
344 'prompt_in2' : 'in2_template',
344 'prompt_in2' : 'in2_template',
345 'prompt_out' : 'out_template',
345 'prompt_out' : 'out_template',
346 'prompts_pad_left' : 'justify',
346 'prompts_pad_left' : 'justify',
347 }
347 }
348 warn("InteractiveShell.{name} is deprecated, use PromptManager.{newname}".format(
348 warn("InteractiveShell.{name} is deprecated, use PromptManager.{newname}".format(
349 name=name, newname=table[name])
349 name=name, newname=table[name])
350 )
350 )
351 # protect against weird cases where self.config may not exist:
351 # protect against weird cases where self.config may not exist:
352 if self.config is not None:
352 if self.config is not None:
353 # propagate to corresponding PromptManager trait
353 # propagate to corresponding PromptManager trait
354 setattr(self.config.PromptManager, table[name], new)
354 setattr(self.config.PromptManager, table[name], new)
355
355
356 _prompt_in1_changed = _prompt_trait_changed
356 _prompt_in1_changed = _prompt_trait_changed
357 _prompt_in2_changed = _prompt_trait_changed
357 _prompt_in2_changed = _prompt_trait_changed
358 _prompt_out_changed = _prompt_trait_changed
358 _prompt_out_changed = _prompt_trait_changed
359 _prompt_pad_left_changed = _prompt_trait_changed
359 _prompt_pad_left_changed = _prompt_trait_changed
360
360
361 show_rewritten_input = CBool(True, config=True,
361 show_rewritten_input = CBool(True, config=True,
362 help="Show rewritten input, e.g. for autocall."
362 help="Show rewritten input, e.g. for autocall."
363 )
363 )
364
364
365 quiet = CBool(False, config=True)
365 quiet = CBool(False, config=True)
366
366
367 history_length = Integer(10000, config=True)
367 history_length = Integer(10000, config=True)
368
368
369 # The readline stuff will eventually be moved to the terminal subclass
369 # The readline stuff will eventually be moved to the terminal subclass
370 # but for now, we can't do that as readline is welded in everywhere.
370 # but for now, we can't do that as readline is welded in everywhere.
371 readline_use = CBool(True, config=True)
371 readline_use = CBool(True, config=True)
372 readline_remove_delims = Unicode('-/~', config=True)
372 readline_remove_delims = Unicode('-/~', config=True)
373 readline_delims = Unicode() # set by init_readline()
373 readline_delims = Unicode() # set by init_readline()
374 # don't use \M- bindings by default, because they
374 # don't use \M- bindings by default, because they
375 # conflict with 8-bit encodings. See gh-58,gh-88
375 # conflict with 8-bit encodings. See gh-58,gh-88
376 readline_parse_and_bind = List([
376 readline_parse_and_bind = List([
377 'tab: complete',
377 'tab: complete',
378 '"\C-l": clear-screen',
378 '"\C-l": clear-screen',
379 'set show-all-if-ambiguous on',
379 'set show-all-if-ambiguous on',
380 '"\C-o": tab-insert',
380 '"\C-o": tab-insert',
381 '"\C-r": reverse-search-history',
381 '"\C-r": reverse-search-history',
382 '"\C-s": forward-search-history',
382 '"\C-s": forward-search-history',
383 '"\C-p": history-search-backward',
383 '"\C-p": history-search-backward',
384 '"\C-n": history-search-forward',
384 '"\C-n": history-search-forward',
385 '"\e[A": history-search-backward',
385 '"\e[A": history-search-backward',
386 '"\e[B": history-search-forward',
386 '"\e[B": history-search-forward',
387 '"\C-k": kill-line',
387 '"\C-k": kill-line',
388 '"\C-u": unix-line-discard',
388 '"\C-u": unix-line-discard',
389 ], allow_none=False, config=True)
389 ], allow_none=False, config=True)
390
390
391 ast_node_interactivity = Enum(['all', 'last', 'last_expr', 'none'],
391 ast_node_interactivity = Enum(['all', 'last', 'last_expr', 'none'],
392 default_value='last_expr', config=True,
392 default_value='last_expr', config=True,
393 help="""
393 help="""
394 'all', 'last', 'last_expr' or 'none', specifying which nodes should be
394 'all', 'last', 'last_expr' or 'none', specifying which nodes should be
395 run interactively (displaying output from expressions).""")
395 run interactively (displaying output from expressions).""")
396
396
397 # TODO: this part of prompt management should be moved to the frontends.
397 # TODO: this part of prompt management should be moved to the frontends.
398 # Use custom TraitTypes that convert '0'->'' and '\\n'->'\n'
398 # Use custom TraitTypes that convert '0'->'' and '\\n'->'\n'
399 separate_in = SeparateUnicode('\n', config=True)
399 separate_in = SeparateUnicode('\n', config=True)
400 separate_out = SeparateUnicode('', config=True)
400 separate_out = SeparateUnicode('', config=True)
401 separate_out2 = SeparateUnicode('', config=True)
401 separate_out2 = SeparateUnicode('', config=True)
402 wildcards_case_sensitive = CBool(True, config=True)
402 wildcards_case_sensitive = CBool(True, config=True)
403 xmode = CaselessStrEnum(('Context','Plain', 'Verbose'),
403 xmode = CaselessStrEnum(('Context','Plain', 'Verbose'),
404 default_value='Context', config=True)
404 default_value='Context', config=True)
405
405
406 # Subcomponents of InteractiveShell
406 # Subcomponents of InteractiveShell
407 alias_manager = Instance('IPython.core.alias.AliasManager')
407 alias_manager = Instance('IPython.core.alias.AliasManager')
408 prefilter_manager = Instance('IPython.core.prefilter.PrefilterManager')
408 prefilter_manager = Instance('IPython.core.prefilter.PrefilterManager')
409 builtin_trap = Instance('IPython.core.builtin_trap.BuiltinTrap')
409 builtin_trap = Instance('IPython.core.builtin_trap.BuiltinTrap')
410 display_trap = Instance('IPython.core.display_trap.DisplayTrap')
410 display_trap = Instance('IPython.core.display_trap.DisplayTrap')
411 extension_manager = Instance('IPython.core.extensions.ExtensionManager')
411 extension_manager = Instance('IPython.core.extensions.ExtensionManager')
412 payload_manager = Instance('IPython.core.payload.PayloadManager')
412 payload_manager = Instance('IPython.core.payload.PayloadManager')
413 history_manager = Instance('IPython.core.history.HistoryManager')
413 history_manager = Instance('IPython.core.history.HistoryManager')
414 magics_manager = Instance('IPython.core.magic.MagicsManager')
414 magics_manager = Instance('IPython.core.magic.MagicsManager')
415
415
416 profile_dir = Instance('IPython.core.application.ProfileDir')
416 profile_dir = Instance('IPython.core.application.ProfileDir')
417 @property
417 @property
418 def profile(self):
418 def profile(self):
419 if self.profile_dir is not None:
419 if self.profile_dir is not None:
420 name = os.path.basename(self.profile_dir.location)
420 name = os.path.basename(self.profile_dir.location)
421 return name.replace('profile_','')
421 return name.replace('profile_','')
422
422
423
423
424 # Private interface
424 # Private interface
425 _post_execute = Instance(dict)
425 _post_execute = Instance(dict)
426
426
427 # Tracks any GUI loop loaded for pylab
427 # Tracks any GUI loop loaded for pylab
428 pylab_gui_select = None
428 pylab_gui_select = None
429
429
430 def __init__(self, ipython_dir=None, profile_dir=None,
430 def __init__(self, ipython_dir=None, profile_dir=None,
431 user_module=None, user_ns=None,
431 user_module=None, user_ns=None,
432 custom_exceptions=((), None), **kwargs):
432 custom_exceptions=((), None), **kwargs):
433
433
434 # This is where traits with a config_key argument are updated
434 # This is where traits with a config_key argument are updated
435 # from the values on config.
435 # from the values on config.
436 super(InteractiveShell, self).__init__(**kwargs)
436 super(InteractiveShell, self).__init__(**kwargs)
437 self.configurables = [self]
437 self.configurables = [self]
438
438
439 # These are relatively independent and stateless
439 # These are relatively independent and stateless
440 self.init_ipython_dir(ipython_dir)
440 self.init_ipython_dir(ipython_dir)
441 self.init_profile_dir(profile_dir)
441 self.init_profile_dir(profile_dir)
442 self.init_instance_attrs()
442 self.init_instance_attrs()
443 self.init_environment()
443 self.init_environment()
444
444
445 # Check if we're in a virtualenv, and set up sys.path.
445 # Check if we're in a virtualenv, and set up sys.path.
446 self.init_virtualenv()
446 self.init_virtualenv()
447
447
448 # Create namespaces (user_ns, user_global_ns, etc.)
448 # Create namespaces (user_ns, user_global_ns, etc.)
449 self.init_create_namespaces(user_module, user_ns)
449 self.init_create_namespaces(user_module, user_ns)
450 # This has to be done after init_create_namespaces because it uses
450 # This has to be done after init_create_namespaces because it uses
451 # something in self.user_ns, but before init_sys_modules, which
451 # something in self.user_ns, but before init_sys_modules, which
452 # is the first thing to modify sys.
452 # is the first thing to modify sys.
453 # TODO: When we override sys.stdout and sys.stderr before this class
453 # TODO: When we override sys.stdout and sys.stderr before this class
454 # is created, we are saving the overridden ones here. Not sure if this
454 # is created, we are saving the overridden ones here. Not sure if this
455 # is what we want to do.
455 # is what we want to do.
456 self.save_sys_module_state()
456 self.save_sys_module_state()
457 self.init_sys_modules()
457 self.init_sys_modules()
458
458
459 # While we're trying to have each part of the code directly access what
459 # While we're trying to have each part of the code directly access what
460 # it needs without keeping redundant references to objects, we have too
460 # it needs without keeping redundant references to objects, we have too
461 # much legacy code that expects ip.db to exist.
461 # much legacy code that expects ip.db to exist.
462 self.db = PickleShareDB(os.path.join(self.profile_dir.location, 'db'))
462 self.db = PickleShareDB(os.path.join(self.profile_dir.location, 'db'))
463
463
464 self.init_history()
464 self.init_history()
465 self.init_encoding()
465 self.init_encoding()
466 self.init_prefilter()
466 self.init_prefilter()
467
467
468 self.init_syntax_highlighting()
468 self.init_syntax_highlighting()
469 self.init_hooks()
469 self.init_hooks()
470 self.init_pushd_popd_magic()
470 self.init_pushd_popd_magic()
471 # self.init_traceback_handlers use to be here, but we moved it below
471 # self.init_traceback_handlers use to be here, but we moved it below
472 # because it and init_io have to come after init_readline.
472 # because it and init_io have to come after init_readline.
473 self.init_user_ns()
473 self.init_user_ns()
474 self.init_logger()
474 self.init_logger()
475 self.init_builtins()
475 self.init_builtins()
476
476
477 # The following was in post_config_initialization
477 # The following was in post_config_initialization
478 self.init_inspector()
478 self.init_inspector()
479 # init_readline() must come before init_io(), because init_io uses
479 # init_readline() must come before init_io(), because init_io uses
480 # readline related things.
480 # readline related things.
481 self.init_readline()
481 self.init_readline()
482 # We save this here in case user code replaces raw_input, but it needs
482 # We save this here in case user code replaces raw_input, but it needs
483 # to be after init_readline(), because PyPy's readline works by replacing
483 # to be after init_readline(), because PyPy's readline works by replacing
484 # raw_input.
484 # raw_input.
485 if py3compat.PY3:
485 if py3compat.PY3:
486 self.raw_input_original = input
486 self.raw_input_original = input
487 else:
487 else:
488 self.raw_input_original = raw_input
488 self.raw_input_original = raw_input
489 # init_completer must come after init_readline, because it needs to
489 # init_completer must come after init_readline, because it needs to
490 # know whether readline is present or not system-wide to configure the
490 # know whether readline is present or not system-wide to configure the
491 # completers, since the completion machinery can now operate
491 # completers, since the completion machinery can now operate
492 # independently of readline (e.g. over the network)
492 # independently of readline (e.g. over the network)
493 self.init_completer()
493 self.init_completer()
494 # TODO: init_io() needs to happen before init_traceback handlers
494 # TODO: init_io() needs to happen before init_traceback handlers
495 # because the traceback handlers hardcode the stdout/stderr streams.
495 # because the traceback handlers hardcode the stdout/stderr streams.
496 # This logic in in debugger.Pdb and should eventually be changed.
496 # This logic in in debugger.Pdb and should eventually be changed.
497 self.init_io()
497 self.init_io()
498 self.init_traceback_handlers(custom_exceptions)
498 self.init_traceback_handlers(custom_exceptions)
499 self.init_prompts()
499 self.init_prompts()
500 self.init_display_formatter()
500 self.init_display_formatter()
501 self.init_display_pub()
501 self.init_display_pub()
502 self.init_data_pub()
502 self.init_data_pub()
503 self.init_displayhook()
503 self.init_displayhook()
504 self.init_latextool()
504 self.init_latextool()
505 self.init_magics()
505 self.init_magics()
506 self.init_alias()
506 self.init_alias()
507 self.init_logstart()
507 self.init_logstart()
508 self.init_pdb()
508 self.init_pdb()
509 self.init_extension_manager()
509 self.init_extension_manager()
510 self.init_payload()
510 self.init_payload()
511 self.init_comms()
511 self.init_comms()
512 self.hooks.late_startup_hook()
512 self.hooks.late_startup_hook()
513 atexit.register(self.atexit_operations)
513 atexit.register(self.atexit_operations)
514
514
515 def get_ipython(self):
515 def get_ipython(self):
516 """Return the currently running IPython instance."""
516 """Return the currently running IPython instance."""
517 return self
517 return self
518
518
519 #-------------------------------------------------------------------------
519 #-------------------------------------------------------------------------
520 # Trait changed handlers
520 # Trait changed handlers
521 #-------------------------------------------------------------------------
521 #-------------------------------------------------------------------------
522
522
523 def _ipython_dir_changed(self, name, new):
523 def _ipython_dir_changed(self, name, new):
524 if not os.path.isdir(new):
524 if not os.path.isdir(new):
525 os.makedirs(new, mode = 0o777)
525 os.makedirs(new, mode = 0o777)
526
526
527 def set_autoindent(self,value=None):
527 def set_autoindent(self,value=None):
528 """Set the autoindent flag, checking for readline support.
528 """Set the autoindent flag, checking for readline support.
529
529
530 If called with no arguments, it acts as a toggle."""
530 If called with no arguments, it acts as a toggle."""
531
531
532 if value != 0 and not self.has_readline:
532 if value != 0 and not self.has_readline:
533 if os.name == 'posix':
533 if os.name == 'posix':
534 warn("The auto-indent feature requires the readline library")
534 warn("The auto-indent feature requires the readline library")
535 self.autoindent = 0
535 self.autoindent = 0
536 return
536 return
537 if value is None:
537 if value is None:
538 self.autoindent = not self.autoindent
538 self.autoindent = not self.autoindent
539 else:
539 else:
540 self.autoindent = value
540 self.autoindent = value
541
541
542 #-------------------------------------------------------------------------
542 #-------------------------------------------------------------------------
543 # init_* methods called by __init__
543 # init_* methods called by __init__
544 #-------------------------------------------------------------------------
544 #-------------------------------------------------------------------------
545
545
546 def init_ipython_dir(self, ipython_dir):
546 def init_ipython_dir(self, ipython_dir):
547 if ipython_dir is not None:
547 if ipython_dir is not None:
548 self.ipython_dir = ipython_dir
548 self.ipython_dir = ipython_dir
549 return
549 return
550
550
551 self.ipython_dir = get_ipython_dir()
551 self.ipython_dir = get_ipython_dir()
552
552
553 def init_profile_dir(self, profile_dir):
553 def init_profile_dir(self, profile_dir):
554 if profile_dir is not None:
554 if profile_dir is not None:
555 self.profile_dir = profile_dir
555 self.profile_dir = profile_dir
556 return
556 return
557 self.profile_dir =\
557 self.profile_dir =\
558 ProfileDir.create_profile_dir_by_name(self.ipython_dir, 'default')
558 ProfileDir.create_profile_dir_by_name(self.ipython_dir, 'default')
559
559
560 def init_instance_attrs(self):
560 def init_instance_attrs(self):
561 self.more = False
561 self.more = False
562
562
563 # command compiler
563 # command compiler
564 self.compile = CachingCompiler()
564 self.compile = CachingCompiler()
565
565
566 # Make an empty namespace, which extension writers can rely on both
566 # Make an empty namespace, which extension writers can rely on both
567 # existing and NEVER being used by ipython itself. This gives them a
567 # existing and NEVER being used by ipython itself. This gives them a
568 # convenient location for storing additional information and state
568 # convenient location for storing additional information and state
569 # their extensions may require, without fear of collisions with other
569 # their extensions may require, without fear of collisions with other
570 # ipython names that may develop later.
570 # ipython names that may develop later.
571 self.meta = Struct()
571 self.meta = Struct()
572
572
573 # Temporary files used for various purposes. Deleted at exit.
573 # Temporary files used for various purposes. Deleted at exit.
574 self.tempfiles = []
574 self.tempfiles = []
575
575
576 # Keep track of readline usage (later set by init_readline)
576 # Keep track of readline usage (later set by init_readline)
577 self.has_readline = False
577 self.has_readline = False
578
578
579 # keep track of where we started running (mainly for crash post-mortem)
579 # keep track of where we started running (mainly for crash post-mortem)
580 # This is not being used anywhere currently.
580 # This is not being used anywhere currently.
581 self.starting_dir = py3compat.getcwd()
581 self.starting_dir = py3compat.getcwd()
582
582
583 # Indentation management
583 # Indentation management
584 self.indent_current_nsp = 0
584 self.indent_current_nsp = 0
585
585
586 # Dict to track post-execution functions that have been registered
586 # Dict to track post-execution functions that have been registered
587 self._post_execute = {}
587 self._post_execute = {}
588
588
589 def init_environment(self):
589 def init_environment(self):
590 """Any changes we need to make to the user's environment."""
590 """Any changes we need to make to the user's environment."""
591 pass
591 pass
592
592
593 def init_encoding(self):
593 def init_encoding(self):
594 # Get system encoding at startup time. Certain terminals (like Emacs
594 # Get system encoding at startup time. Certain terminals (like Emacs
595 # under Win32 have it set to None, and we need to have a known valid
595 # under Win32 have it set to None, and we need to have a known valid
596 # encoding to use in the raw_input() method
596 # encoding to use in the raw_input() method
597 try:
597 try:
598 self.stdin_encoding = sys.stdin.encoding or 'ascii'
598 self.stdin_encoding = sys.stdin.encoding or 'ascii'
599 except AttributeError:
599 except AttributeError:
600 self.stdin_encoding = 'ascii'
600 self.stdin_encoding = 'ascii'
601
601
602 def init_syntax_highlighting(self):
602 def init_syntax_highlighting(self):
603 # Python source parser/formatter for syntax highlighting
603 # Python source parser/formatter for syntax highlighting
604 pyformat = PyColorize.Parser().format
604 pyformat = PyColorize.Parser().format
605 self.pycolorize = lambda src: pyformat(src,'str',self.colors)
605 self.pycolorize = lambda src: pyformat(src,'str',self.colors)
606
606
607 def init_pushd_popd_magic(self):
607 def init_pushd_popd_magic(self):
608 # for pushd/popd management
608 # for pushd/popd management
609 self.home_dir = get_home_dir()
609 self.home_dir = get_home_dir()
610
610
611 self.dir_stack = []
611 self.dir_stack = []
612
612
613 def init_logger(self):
613 def init_logger(self):
614 self.logger = Logger(self.home_dir, logfname='ipython_log.py',
614 self.logger = Logger(self.home_dir, logfname='ipython_log.py',
615 logmode='rotate')
615 logmode='rotate')
616
616
617 def init_logstart(self):
617 def init_logstart(self):
618 """Initialize logging in case it was requested at the command line.
618 """Initialize logging in case it was requested at the command line.
619 """
619 """
620 if self.logappend:
620 if self.logappend:
621 self.magic('logstart %s append' % self.logappend)
621 self.magic('logstart %s append' % self.logappend)
622 elif self.logfile:
622 elif self.logfile:
623 self.magic('logstart %s' % self.logfile)
623 self.magic('logstart %s' % self.logfile)
624 elif self.logstart:
624 elif self.logstart:
625 self.magic('logstart')
625 self.magic('logstart')
626
626
627 def init_builtins(self):
627 def init_builtins(self):
628 # A single, static flag that we set to True. Its presence indicates
628 # A single, static flag that we set to True. Its presence indicates
629 # that an IPython shell has been created, and we make no attempts at
629 # that an IPython shell has been created, and we make no attempts at
630 # removing on exit or representing the existence of more than one
630 # removing on exit or representing the existence of more than one
631 # IPython at a time.
631 # IPython at a time.
632 builtin_mod.__dict__['__IPYTHON__'] = True
632 builtin_mod.__dict__['__IPYTHON__'] = True
633
633
634 # In 0.11 we introduced '__IPYTHON__active' as an integer we'd try to
634 # In 0.11 we introduced '__IPYTHON__active' as an integer we'd try to
635 # manage on enter/exit, but with all our shells it's virtually
635 # manage on enter/exit, but with all our shells it's virtually
636 # impossible to get all the cases right. We're leaving the name in for
636 # impossible to get all the cases right. We're leaving the name in for
637 # those who adapted their codes to check for this flag, but will
637 # those who adapted their codes to check for this flag, but will
638 # eventually remove it after a few more releases.
638 # eventually remove it after a few more releases.
639 builtin_mod.__dict__['__IPYTHON__active'] = \
639 builtin_mod.__dict__['__IPYTHON__active'] = \
640 'Deprecated, check for __IPYTHON__'
640 'Deprecated, check for __IPYTHON__'
641
641
642 self.builtin_trap = BuiltinTrap(shell=self)
642 self.builtin_trap = BuiltinTrap(shell=self)
643
643
644 def init_inspector(self):
644 def init_inspector(self):
645 # Object inspector
645 # Object inspector
646 self.inspector = oinspect.Inspector(oinspect.InspectColors,
646 self.inspector = oinspect.Inspector(oinspect.InspectColors,
647 PyColorize.ANSICodeColors,
647 PyColorize.ANSICodeColors,
648 'NoColor',
648 'NoColor',
649 self.object_info_string_level)
649 self.object_info_string_level)
650
650
651 def init_io(self):
651 def init_io(self):
652 # This will just use sys.stdout and sys.stderr. If you want to
652 # This will just use sys.stdout and sys.stderr. If you want to
653 # override sys.stdout and sys.stderr themselves, you need to do that
653 # override sys.stdout and sys.stderr themselves, you need to do that
654 # *before* instantiating this class, because io holds onto
654 # *before* instantiating this class, because io holds onto
655 # references to the underlying streams.
655 # references to the underlying streams.
656 if (sys.platform == 'win32' or sys.platform == 'cli') and self.has_readline:
656 if (sys.platform == 'win32' or sys.platform == 'cli') and self.has_readline:
657 io.stdout = io.stderr = io.IOStream(self.readline._outputfile)
657 io.stdout = io.stderr = io.IOStream(self.readline._outputfile)
658 else:
658 else:
659 io.stdout = io.IOStream(sys.stdout)
659 io.stdout = io.IOStream(sys.stdout)
660 io.stderr = io.IOStream(sys.stderr)
660 io.stderr = io.IOStream(sys.stderr)
661
661
662 def init_prompts(self):
662 def init_prompts(self):
663 self.prompt_manager = PromptManager(shell=self, parent=self)
663 self.prompt_manager = PromptManager(shell=self, parent=self)
664 self.configurables.append(self.prompt_manager)
664 self.configurables.append(self.prompt_manager)
665 # Set system prompts, so that scripts can decide if they are running
665 # Set system prompts, so that scripts can decide if they are running
666 # interactively.
666 # interactively.
667 sys.ps1 = 'In : '
667 sys.ps1 = 'In : '
668 sys.ps2 = '...: '
668 sys.ps2 = '...: '
669 sys.ps3 = 'Out: '
669 sys.ps3 = 'Out: '
670
670
671 def init_display_formatter(self):
671 def init_display_formatter(self):
672 self.display_formatter = DisplayFormatter(parent=self)
672 self.display_formatter = DisplayFormatter(parent=self)
673 self.configurables.append(self.display_formatter)
673 self.configurables.append(self.display_formatter)
674
674
675 def init_display_pub(self):
675 def init_display_pub(self):
676 self.display_pub = self.display_pub_class(parent=self)
676 self.display_pub = self.display_pub_class(parent=self)
677 self.configurables.append(self.display_pub)
677 self.configurables.append(self.display_pub)
678
678
679 def init_data_pub(self):
679 def init_data_pub(self):
680 if not self.data_pub_class:
680 if not self.data_pub_class:
681 self.data_pub = None
681 self.data_pub = None
682 return
682 return
683 self.data_pub = self.data_pub_class(parent=self)
683 self.data_pub = self.data_pub_class(parent=self)
684 self.configurables.append(self.data_pub)
684 self.configurables.append(self.data_pub)
685
685
686 def init_displayhook(self):
686 def init_displayhook(self):
687 # Initialize displayhook, set in/out prompts and printing system
687 # Initialize displayhook, set in/out prompts and printing system
688 self.displayhook = self.displayhook_class(
688 self.displayhook = self.displayhook_class(
689 parent=self,
689 parent=self,
690 shell=self,
690 shell=self,
691 cache_size=self.cache_size,
691 cache_size=self.cache_size,
692 )
692 )
693 self.configurables.append(self.displayhook)
693 self.configurables.append(self.displayhook)
694 # This is a context manager that installs/revmoes the displayhook at
694 # This is a context manager that installs/revmoes the displayhook at
695 # the appropriate time.
695 # the appropriate time.
696 self.display_trap = DisplayTrap(hook=self.displayhook)
696 self.display_trap = DisplayTrap(hook=self.displayhook)
697
697
698 def init_latextool(self):
698 def init_latextool(self):
699 """Configure LaTeXTool."""
699 """Configure LaTeXTool."""
700 cfg = LaTeXTool.instance(parent=self)
700 cfg = LaTeXTool.instance(parent=self)
701 if cfg not in self.configurables:
701 if cfg not in self.configurables:
702 self.configurables.append(cfg)
702 self.configurables.append(cfg)
703
703
704 def init_virtualenv(self):
704 def init_virtualenv(self):
705 """Add a virtualenv to sys.path so the user can import modules from it.
705 """Add a virtualenv to sys.path so the user can import modules from it.
706 This isn't perfect: it doesn't use the Python interpreter with which the
706 This isn't perfect: it doesn't use the Python interpreter with which the
707 virtualenv was built, and it ignores the --no-site-packages option. A
707 virtualenv was built, and it ignores the --no-site-packages option. A
708 warning will appear suggesting the user installs IPython in the
708 warning will appear suggesting the user installs IPython in the
709 virtualenv, but for many cases, it probably works well enough.
709 virtualenv, but for many cases, it probably works well enough.
710
710
711 Adapted from code snippets online.
711 Adapted from code snippets online.
712
712
713 http://blog.ufsoft.org/2009/1/29/ipython-and-virtualenv
713 http://blog.ufsoft.org/2009/1/29/ipython-and-virtualenv
714 """
714 """
715 if 'VIRTUAL_ENV' not in os.environ:
715 if 'VIRTUAL_ENV' not in os.environ:
716 # Not in a virtualenv
716 # Not in a virtualenv
717 return
717 return
718
718
719 if os.path.realpath(sys.executable).startswith(
719 if os.path.realpath(sys.executable).startswith(
720 os.path.realpath(os.environ['VIRTUAL_ENV'])
720 os.path.realpath(os.environ['VIRTUAL_ENV'])
721 ):
721 ):
722 # Running properly in the virtualenv, don't need to do anything
722 # Running properly in the virtualenv, don't need to do anything
723 return
723 return
724
724
725 warn("Attempting to work in a virtualenv. If you encounter problems, please "
725 warn("Attempting to work in a virtualenv. If you encounter problems, please "
726 "install IPython inside the virtualenv.")
726 "install IPython inside the virtualenv.")
727 if sys.platform == "win32":
727 if sys.platform == "win32":
728 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'Lib', 'site-packages')
728 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'Lib', 'site-packages')
729 else:
729 else:
730 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'lib',
730 virtual_env = os.path.join(os.environ['VIRTUAL_ENV'], 'lib',
731 'python%d.%d' % sys.version_info[:2], 'site-packages')
731 'python%d.%d' % sys.version_info[:2], 'site-packages')
732
732
733 import site
733 import site
734 sys.path.insert(0, virtual_env)
734 sys.path.insert(0, virtual_env)
735 site.addsitedir(virtual_env)
735 site.addsitedir(virtual_env)
736
736
737 #-------------------------------------------------------------------------
737 #-------------------------------------------------------------------------
738 # Things related to injections into the sys module
738 # Things related to injections into the sys module
739 #-------------------------------------------------------------------------
739 #-------------------------------------------------------------------------
740
740
741 def save_sys_module_state(self):
741 def save_sys_module_state(self):
742 """Save the state of hooks in the sys module.
742 """Save the state of hooks in the sys module.
743
743
744 This has to be called after self.user_module is created.
744 This has to be called after self.user_module is created.
745 """
745 """
746 self._orig_sys_module_state = {}
746 self._orig_sys_module_state = {}
747 self._orig_sys_module_state['stdin'] = sys.stdin
747 self._orig_sys_module_state['stdin'] = sys.stdin
748 self._orig_sys_module_state['stdout'] = sys.stdout
748 self._orig_sys_module_state['stdout'] = sys.stdout
749 self._orig_sys_module_state['stderr'] = sys.stderr
749 self._orig_sys_module_state['stderr'] = sys.stderr
750 self._orig_sys_module_state['excepthook'] = sys.excepthook
750 self._orig_sys_module_state['excepthook'] = sys.excepthook
751 self._orig_sys_modules_main_name = self.user_module.__name__
751 self._orig_sys_modules_main_name = self.user_module.__name__
752 self._orig_sys_modules_main_mod = sys.modules.get(self.user_module.__name__)
752 self._orig_sys_modules_main_mod = sys.modules.get(self.user_module.__name__)
753
753
754 def restore_sys_module_state(self):
754 def restore_sys_module_state(self):
755 """Restore the state of the sys module."""
755 """Restore the state of the sys module."""
756 try:
756 try:
757 for k, v in iteritems(self._orig_sys_module_state):
757 for k, v in iteritems(self._orig_sys_module_state):
758 setattr(sys, k, v)
758 setattr(sys, k, v)
759 except AttributeError:
759 except AttributeError:
760 pass
760 pass
761 # Reset what what done in self.init_sys_modules
761 # Reset what what done in self.init_sys_modules
762 if self._orig_sys_modules_main_mod is not None:
762 if self._orig_sys_modules_main_mod is not None:
763 sys.modules[self._orig_sys_modules_main_name] = self._orig_sys_modules_main_mod
763 sys.modules[self._orig_sys_modules_main_name] = self._orig_sys_modules_main_mod
764
764
765 #-------------------------------------------------------------------------
765 #-------------------------------------------------------------------------
766 # Things related to hooks
766 # Things related to hooks
767 #-------------------------------------------------------------------------
767 #-------------------------------------------------------------------------
768
768
769 def init_hooks(self):
769 def init_hooks(self):
770 # hooks holds pointers used for user-side customizations
770 # hooks holds pointers used for user-side customizations
771 self.hooks = Struct()
771 self.hooks = Struct()
772
772
773 self.strdispatchers = {}
773 self.strdispatchers = {}
774
774
775 # Set all default hooks, defined in the IPython.hooks module.
775 # Set all default hooks, defined in the IPython.hooks module.
776 hooks = IPython.core.hooks
776 hooks = IPython.core.hooks
777 for hook_name in hooks.__all__:
777 for hook_name in hooks.__all__:
778 # default hooks have priority 100, i.e. low; user hooks should have
778 # default hooks have priority 100, i.e. low; user hooks should have
779 # 0-100 priority
779 # 0-100 priority
780 self.set_hook(hook_name,getattr(hooks,hook_name), 100)
780 self.set_hook(hook_name,getattr(hooks,hook_name), 100)
781
781
782 def set_hook(self,name,hook, priority = 50, str_key = None, re_key = None):
782 def set_hook(self,name,hook, priority = 50, str_key = None, re_key = None):
783 """set_hook(name,hook) -> sets an internal IPython hook.
783 """set_hook(name,hook) -> sets an internal IPython hook.
784
784
785 IPython exposes some of its internal API as user-modifiable hooks. By
785 IPython exposes some of its internal API as user-modifiable hooks. By
786 adding your function to one of these hooks, you can modify IPython's
786 adding your function to one of these hooks, you can modify IPython's
787 behavior to call at runtime your own routines."""
787 behavior to call at runtime your own routines."""
788
788
789 # At some point in the future, this should validate the hook before it
789 # At some point in the future, this should validate the hook before it
790 # accepts it. Probably at least check that the hook takes the number
790 # accepts it. Probably at least check that the hook takes the number
791 # of args it's supposed to.
791 # of args it's supposed to.
792
792
793 f = types.MethodType(hook,self)
793 f = types.MethodType(hook,self)
794
794
795 # check if the hook is for strdispatcher first
795 # check if the hook is for strdispatcher first
796 if str_key is not None:
796 if str_key is not None:
797 sdp = self.strdispatchers.get(name, StrDispatch())
797 sdp = self.strdispatchers.get(name, StrDispatch())
798 sdp.add_s(str_key, f, priority )
798 sdp.add_s(str_key, f, priority )
799 self.strdispatchers[name] = sdp
799 self.strdispatchers[name] = sdp
800 return
800 return
801 if re_key is not None:
801 if re_key is not None:
802 sdp = self.strdispatchers.get(name, StrDispatch())
802 sdp = self.strdispatchers.get(name, StrDispatch())
803 sdp.add_re(re.compile(re_key), f, priority )
803 sdp.add_re(re.compile(re_key), f, priority )
804 self.strdispatchers[name] = sdp
804 self.strdispatchers[name] = sdp
805 return
805 return
806
806
807 dp = getattr(self.hooks, name, None)
807 dp = getattr(self.hooks, name, None)
808 if name not in IPython.core.hooks.__all__:
808 if name not in IPython.core.hooks.__all__:
809 print("Warning! Hook '%s' is not one of %s" % \
809 print("Warning! Hook '%s' is not one of %s" % \
810 (name, IPython.core.hooks.__all__ ))
810 (name, IPython.core.hooks.__all__ ))
811 if not dp:
811 if not dp:
812 dp = IPython.core.hooks.CommandChainDispatcher()
812 dp = IPython.core.hooks.CommandChainDispatcher()
813
813
814 try:
814 try:
815 dp.add(f,priority)
815 dp.add(f,priority)
816 except AttributeError:
816 except AttributeError:
817 # it was not commandchain, plain old func - replace
817 # it was not commandchain, plain old func - replace
818 dp = f
818 dp = f
819
819
820 setattr(self.hooks,name, dp)
820 setattr(self.hooks,name, dp)
821
821
822 def register_post_execute(self, func):
822 def register_post_execute(self, func):
823 """Register a function for calling after code execution.
823 """Register a function for calling after code execution.
824 """
824 """
825 if not callable(func):
825 if not callable(func):
826 raise ValueError('argument %s must be callable' % func)
826 raise ValueError('argument %s must be callable' % func)
827 self._post_execute[func] = True
827 self._post_execute[func] = True
828
828
829 #-------------------------------------------------------------------------
829 #-------------------------------------------------------------------------
830 # Things related to the "main" module
830 # Things related to the "main" module
831 #-------------------------------------------------------------------------
831 #-------------------------------------------------------------------------
832
832
833 def new_main_mod(self, filename, modname):
833 def new_main_mod(self, filename, modname):
834 """Return a new 'main' module object for user code execution.
834 """Return a new 'main' module object for user code execution.
835
835
836 ``filename`` should be the path of the script which will be run in the
836 ``filename`` should be the path of the script which will be run in the
837 module. Requests with the same filename will get the same module, with
837 module. Requests with the same filename will get the same module, with
838 its namespace cleared.
838 its namespace cleared.
839
839
840 ``modname`` should be the module name - normally either '__main__' or
840 ``modname`` should be the module name - normally either '__main__' or
841 the basename of the file without the extension.
841 the basename of the file without the extension.
842
842
843 When scripts are executed via %run, we must keep a reference to their
843 When scripts are executed via %run, we must keep a reference to their
844 __main__ module around so that Python doesn't
844 __main__ module around so that Python doesn't
845 clear it, rendering references to module globals useless.
845 clear it, rendering references to module globals useless.
846
846
847 This method keeps said reference in a private dict, keyed by the
847 This method keeps said reference in a private dict, keyed by the
848 absolute path of the script. This way, for multiple executions of the
848 absolute path of the script. This way, for multiple executions of the
849 same script we only keep one copy of the namespace (the last one),
849 same script we only keep one copy of the namespace (the last one),
850 thus preventing memory leaks from old references while allowing the
850 thus preventing memory leaks from old references while allowing the
851 objects from the last execution to be accessible.
851 objects from the last execution to be accessible.
852 """
852 """
853 filename = os.path.abspath(filename)
853 filename = os.path.abspath(filename)
854 try:
854 try:
855 main_mod = self._main_mod_cache[filename]
855 main_mod = self._main_mod_cache[filename]
856 except KeyError:
856 except KeyError:
857 main_mod = self._main_mod_cache[filename] = types.ModuleType(modname,
857 main_mod = self._main_mod_cache[filename] = types.ModuleType(modname,
858 doc="Module created for script run in IPython")
858 doc="Module created for script run in IPython")
859 else:
859 else:
860 main_mod.__dict__.clear()
860 main_mod.__dict__.clear()
861 main_mod.__name__ = modname
861 main_mod.__name__ = modname
862
862
863 main_mod.__file__ = filename
863 main_mod.__file__ = filename
864 # It seems pydoc (and perhaps others) needs any module instance to
864 # It seems pydoc (and perhaps others) needs any module instance to
865 # implement a __nonzero__ method
865 # implement a __nonzero__ method
866 main_mod.__nonzero__ = lambda : True
866 main_mod.__nonzero__ = lambda : True
867
867
868 return main_mod
868 return main_mod
869
869
870 def clear_main_mod_cache(self):
870 def clear_main_mod_cache(self):
871 """Clear the cache of main modules.
871 """Clear the cache of main modules.
872
872
873 Mainly for use by utilities like %reset.
873 Mainly for use by utilities like %reset.
874
874
875 Examples
875 Examples
876 --------
876 --------
877
877
878 In [15]: import IPython
878 In [15]: import IPython
879
879
880 In [16]: m = _ip.new_main_mod(IPython.__file__, 'IPython')
880 In [16]: m = _ip.new_main_mod(IPython.__file__, 'IPython')
881
881
882 In [17]: len(_ip._main_mod_cache) > 0
882 In [17]: len(_ip._main_mod_cache) > 0
883 Out[17]: True
883 Out[17]: True
884
884
885 In [18]: _ip.clear_main_mod_cache()
885 In [18]: _ip.clear_main_mod_cache()
886
886
887 In [19]: len(_ip._main_mod_cache) == 0
887 In [19]: len(_ip._main_mod_cache) == 0
888 Out[19]: True
888 Out[19]: True
889 """
889 """
890 self._main_mod_cache.clear()
890 self._main_mod_cache.clear()
891
891
892 #-------------------------------------------------------------------------
892 #-------------------------------------------------------------------------
893 # Things related to debugging
893 # Things related to debugging
894 #-------------------------------------------------------------------------
894 #-------------------------------------------------------------------------
895
895
896 def init_pdb(self):
896 def init_pdb(self):
897 # Set calling of pdb on exceptions
897 # Set calling of pdb on exceptions
898 # self.call_pdb is a property
898 # self.call_pdb is a property
899 self.call_pdb = self.pdb
899 self.call_pdb = self.pdb
900
900
901 def _get_call_pdb(self):
901 def _get_call_pdb(self):
902 return self._call_pdb
902 return self._call_pdb
903
903
904 def _set_call_pdb(self,val):
904 def _set_call_pdb(self,val):
905
905
906 if val not in (0,1,False,True):
906 if val not in (0,1,False,True):
907 raise ValueError('new call_pdb value must be boolean')
907 raise ValueError('new call_pdb value must be boolean')
908
908
909 # store value in instance
909 # store value in instance
910 self._call_pdb = val
910 self._call_pdb = val
911
911
912 # notify the actual exception handlers
912 # notify the actual exception handlers
913 self.InteractiveTB.call_pdb = val
913 self.InteractiveTB.call_pdb = val
914
914
915 call_pdb = property(_get_call_pdb,_set_call_pdb,None,
915 call_pdb = property(_get_call_pdb,_set_call_pdb,None,
916 'Control auto-activation of pdb at exceptions')
916 'Control auto-activation of pdb at exceptions')
917
917
918 def debugger(self,force=False):
918 def debugger(self,force=False):
919 """Call the pydb/pdb debugger.
919 """Call the pydb/pdb debugger.
920
920
921 Keywords:
921 Keywords:
922
922
923 - force(False): by default, this routine checks the instance call_pdb
923 - force(False): by default, this routine checks the instance call_pdb
924 flag and does not actually invoke the debugger if the flag is false.
924 flag and does not actually invoke the debugger if the flag is false.
925 The 'force' option forces the debugger to activate even if the flag
925 The 'force' option forces the debugger to activate even if the flag
926 is false.
926 is false.
927 """
927 """
928
928
929 if not (force or self.call_pdb):
929 if not (force or self.call_pdb):
930 return
930 return
931
931
932 if not hasattr(sys,'last_traceback'):
932 if not hasattr(sys,'last_traceback'):
933 error('No traceback has been produced, nothing to debug.')
933 error('No traceback has been produced, nothing to debug.')
934 return
934 return
935
935
936 # use pydb if available
936 # use pydb if available
937 if debugger.has_pydb:
937 if debugger.has_pydb:
938 from pydb import pm
938 from pydb import pm
939 else:
939 else:
940 # fallback to our internal debugger
940 # fallback to our internal debugger
941 pm = lambda : self.InteractiveTB.debugger(force=True)
941 pm = lambda : self.InteractiveTB.debugger(force=True)
942
942
943 with self.readline_no_record:
943 with self.readline_no_record:
944 pm()
944 pm()
945
945
946 #-------------------------------------------------------------------------
946 #-------------------------------------------------------------------------
947 # Things related to IPython's various namespaces
947 # Things related to IPython's various namespaces
948 #-------------------------------------------------------------------------
948 #-------------------------------------------------------------------------
949 default_user_namespaces = True
949 default_user_namespaces = True
950
950
951 def init_create_namespaces(self, user_module=None, user_ns=None):
951 def init_create_namespaces(self, user_module=None, user_ns=None):
952 # Create the namespace where the user will operate. user_ns is
952 # Create the namespace where the user will operate. user_ns is
953 # normally the only one used, and it is passed to the exec calls as
953 # normally the only one used, and it is passed to the exec calls as
954 # the locals argument. But we do carry a user_global_ns namespace
954 # the locals argument. But we do carry a user_global_ns namespace
955 # given as the exec 'globals' argument, This is useful in embedding
955 # given as the exec 'globals' argument, This is useful in embedding
956 # situations where the ipython shell opens in a context where the
956 # situations where the ipython shell opens in a context where the
957 # distinction between locals and globals is meaningful. For
957 # distinction between locals and globals is meaningful. For
958 # non-embedded contexts, it is just the same object as the user_ns dict.
958 # non-embedded contexts, it is just the same object as the user_ns dict.
959
959
960 # FIXME. For some strange reason, __builtins__ is showing up at user
960 # FIXME. For some strange reason, __builtins__ is showing up at user
961 # level as a dict instead of a module. This is a manual fix, but I
961 # level as a dict instead of a module. This is a manual fix, but I
962 # should really track down where the problem is coming from. Alex
962 # should really track down where the problem is coming from. Alex
963 # Schmolck reported this problem first.
963 # Schmolck reported this problem first.
964
964
965 # A useful post by Alex Martelli on this topic:
965 # A useful post by Alex Martelli on this topic:
966 # Re: inconsistent value from __builtins__
966 # Re: inconsistent value from __builtins__
967 # Von: Alex Martelli <aleaxit@yahoo.com>
967 # Von: Alex Martelli <aleaxit@yahoo.com>
968 # Datum: Freitag 01 Oktober 2004 04:45:34 nachmittags/abends
968 # Datum: Freitag 01 Oktober 2004 04:45:34 nachmittags/abends
969 # Gruppen: comp.lang.python
969 # Gruppen: comp.lang.python
970
970
971 # Michael Hohn <hohn@hooknose.lbl.gov> wrote:
971 # Michael Hohn <hohn@hooknose.lbl.gov> wrote:
972 # > >>> print type(builtin_check.get_global_binding('__builtins__'))
972 # > >>> print type(builtin_check.get_global_binding('__builtins__'))
973 # > <type 'dict'>
973 # > <type 'dict'>
974 # > >>> print type(__builtins__)
974 # > >>> print type(__builtins__)
975 # > <type 'module'>
975 # > <type 'module'>
976 # > Is this difference in return value intentional?
976 # > Is this difference in return value intentional?
977
977
978 # Well, it's documented that '__builtins__' can be either a dictionary
978 # Well, it's documented that '__builtins__' can be either a dictionary
979 # or a module, and it's been that way for a long time. Whether it's
979 # or a module, and it's been that way for a long time. Whether it's
980 # intentional (or sensible), I don't know. In any case, the idea is
980 # intentional (or sensible), I don't know. In any case, the idea is
981 # that if you need to access the built-in namespace directly, you
981 # that if you need to access the built-in namespace directly, you
982 # should start with "import __builtin__" (note, no 's') which will
982 # should start with "import __builtin__" (note, no 's') which will
983 # definitely give you a module. Yeah, it's somewhat confusing:-(.
983 # definitely give you a module. Yeah, it's somewhat confusing:-(.
984
984
985 # These routines return a properly built module and dict as needed by
985 # These routines return a properly built module and dict as needed by
986 # the rest of the code, and can also be used by extension writers to
986 # the rest of the code, and can also be used by extension writers to
987 # generate properly initialized namespaces.
987 # generate properly initialized namespaces.
988 if (user_ns is not None) or (user_module is not None):
988 if (user_ns is not None) or (user_module is not None):
989 self.default_user_namespaces = False
989 self.default_user_namespaces = False
990 self.user_module, self.user_ns = self.prepare_user_module(user_module, user_ns)
990 self.user_module, self.user_ns = self.prepare_user_module(user_module, user_ns)
991
991
992 # A record of hidden variables we have added to the user namespace, so
992 # A record of hidden variables we have added to the user namespace, so
993 # we can list later only variables defined in actual interactive use.
993 # we can list later only variables defined in actual interactive use.
994 self.user_ns_hidden = {}
994 self.user_ns_hidden = {}
995
995
996 # Now that FakeModule produces a real module, we've run into a nasty
996 # Now that FakeModule produces a real module, we've run into a nasty
997 # problem: after script execution (via %run), the module where the user
997 # problem: after script execution (via %run), the module where the user
998 # code ran is deleted. Now that this object is a true module (needed
998 # code ran is deleted. Now that this object is a true module (needed
999 # so docetst and other tools work correctly), the Python module
999 # so docetst and other tools work correctly), the Python module
1000 # teardown mechanism runs over it, and sets to None every variable
1000 # teardown mechanism runs over it, and sets to None every variable
1001 # present in that module. Top-level references to objects from the
1001 # present in that module. Top-level references to objects from the
1002 # script survive, because the user_ns is updated with them. However,
1002 # script survive, because the user_ns is updated with them. However,
1003 # calling functions defined in the script that use other things from
1003 # calling functions defined in the script that use other things from
1004 # the script will fail, because the function's closure had references
1004 # the script will fail, because the function's closure had references
1005 # to the original objects, which are now all None. So we must protect
1005 # to the original objects, which are now all None. So we must protect
1006 # these modules from deletion by keeping a cache.
1006 # these modules from deletion by keeping a cache.
1007 #
1007 #
1008 # To avoid keeping stale modules around (we only need the one from the
1008 # To avoid keeping stale modules around (we only need the one from the
1009 # last run), we use a dict keyed with the full path to the script, so
1009 # last run), we use a dict keyed with the full path to the script, so
1010 # only the last version of the module is held in the cache. Note,
1010 # only the last version of the module is held in the cache. Note,
1011 # however, that we must cache the module *namespace contents* (their
1011 # however, that we must cache the module *namespace contents* (their
1012 # __dict__). Because if we try to cache the actual modules, old ones
1012 # __dict__). Because if we try to cache the actual modules, old ones
1013 # (uncached) could be destroyed while still holding references (such as
1013 # (uncached) could be destroyed while still holding references (such as
1014 # those held by GUI objects that tend to be long-lived)>
1014 # those held by GUI objects that tend to be long-lived)>
1015 #
1015 #
1016 # The %reset command will flush this cache. See the cache_main_mod()
1016 # The %reset command will flush this cache. See the cache_main_mod()
1017 # and clear_main_mod_cache() methods for details on use.
1017 # and clear_main_mod_cache() methods for details on use.
1018
1018
1019 # This is the cache used for 'main' namespaces
1019 # This is the cache used for 'main' namespaces
1020 self._main_mod_cache = {}
1020 self._main_mod_cache = {}
1021
1021
1022 # A table holding all the namespaces IPython deals with, so that
1022 # A table holding all the namespaces IPython deals with, so that
1023 # introspection facilities can search easily.
1023 # introspection facilities can search easily.
1024 self.ns_table = {'user_global':self.user_module.__dict__,
1024 self.ns_table = {'user_global':self.user_module.__dict__,
1025 'user_local':self.user_ns,
1025 'user_local':self.user_ns,
1026 'builtin':builtin_mod.__dict__
1026 'builtin':builtin_mod.__dict__
1027 }
1027 }
1028
1028
1029 @property
1029 @property
1030 def user_global_ns(self):
1030 def user_global_ns(self):
1031 return self.user_module.__dict__
1031 return self.user_module.__dict__
1032
1032
1033 def prepare_user_module(self, user_module=None, user_ns=None):
1033 def prepare_user_module(self, user_module=None, user_ns=None):
1034 """Prepare the module and namespace in which user code will be run.
1034 """Prepare the module and namespace in which user code will be run.
1035
1035
1036 When IPython is started normally, both parameters are None: a new module
1036 When IPython is started normally, both parameters are None: a new module
1037 is created automatically, and its __dict__ used as the namespace.
1037 is created automatically, and its __dict__ used as the namespace.
1038
1038
1039 If only user_module is provided, its __dict__ is used as the namespace.
1039 If only user_module is provided, its __dict__ is used as the namespace.
1040 If only user_ns is provided, a dummy module is created, and user_ns
1040 If only user_ns is provided, a dummy module is created, and user_ns
1041 becomes the global namespace. If both are provided (as they may be
1041 becomes the global namespace. If both are provided (as they may be
1042 when embedding), user_ns is the local namespace, and user_module
1042 when embedding), user_ns is the local namespace, and user_module
1043 provides the global namespace.
1043 provides the global namespace.
1044
1044
1045 Parameters
1045 Parameters
1046 ----------
1046 ----------
1047 user_module : module, optional
1047 user_module : module, optional
1048 The current user module in which IPython is being run. If None,
1048 The current user module in which IPython is being run. If None,
1049 a clean module will be created.
1049 a clean module will be created.
1050 user_ns : dict, optional
1050 user_ns : dict, optional
1051 A namespace in which to run interactive commands.
1051 A namespace in which to run interactive commands.
1052
1052
1053 Returns
1053 Returns
1054 -------
1054 -------
1055 A tuple of user_module and user_ns, each properly initialised.
1055 A tuple of user_module and user_ns, each properly initialised.
1056 """
1056 """
1057 if user_module is None and user_ns is not None:
1057 if user_module is None and user_ns is not None:
1058 user_ns.setdefault("__name__", "__main__")
1058 user_ns.setdefault("__name__", "__main__")
1059 user_module = DummyMod()
1059 user_module = DummyMod()
1060 user_module.__dict__ = user_ns
1060 user_module.__dict__ = user_ns
1061
1061
1062 if user_module is None:
1062 if user_module is None:
1063 user_module = types.ModuleType("__main__",
1063 user_module = types.ModuleType("__main__",
1064 doc="Automatically created module for IPython interactive environment")
1064 doc="Automatically created module for IPython interactive environment")
1065
1065
1066 # We must ensure that __builtin__ (without the final 's') is always
1066 # We must ensure that __builtin__ (without the final 's') is always
1067 # available and pointing to the __builtin__ *module*. For more details:
1067 # available and pointing to the __builtin__ *module*. For more details:
1068 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1068 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1069 user_module.__dict__.setdefault('__builtin__', builtin_mod)
1069 user_module.__dict__.setdefault('__builtin__', builtin_mod)
1070 user_module.__dict__.setdefault('__builtins__', builtin_mod)
1070 user_module.__dict__.setdefault('__builtins__', builtin_mod)
1071
1071
1072 if user_ns is None:
1072 if user_ns is None:
1073 user_ns = user_module.__dict__
1073 user_ns = user_module.__dict__
1074
1074
1075 return user_module, user_ns
1075 return user_module, user_ns
1076
1076
1077 def init_sys_modules(self):
1077 def init_sys_modules(self):
1078 # We need to insert into sys.modules something that looks like a
1078 # We need to insert into sys.modules something that looks like a
1079 # module but which accesses the IPython namespace, for shelve and
1079 # module but which accesses the IPython namespace, for shelve and
1080 # pickle to work interactively. Normally they rely on getting
1080 # pickle to work interactively. Normally they rely on getting
1081 # everything out of __main__, but for embedding purposes each IPython
1081 # everything out of __main__, but for embedding purposes each IPython
1082 # instance has its own private namespace, so we can't go shoving
1082 # instance has its own private namespace, so we can't go shoving
1083 # everything into __main__.
1083 # everything into __main__.
1084
1084
1085 # note, however, that we should only do this for non-embedded
1085 # note, however, that we should only do this for non-embedded
1086 # ipythons, which really mimic the __main__.__dict__ with their own
1086 # ipythons, which really mimic the __main__.__dict__ with their own
1087 # namespace. Embedded instances, on the other hand, should not do
1087 # namespace. Embedded instances, on the other hand, should not do
1088 # this because they need to manage the user local/global namespaces
1088 # this because they need to manage the user local/global namespaces
1089 # only, but they live within a 'normal' __main__ (meaning, they
1089 # only, but they live within a 'normal' __main__ (meaning, they
1090 # shouldn't overtake the execution environment of the script they're
1090 # shouldn't overtake the execution environment of the script they're
1091 # embedded in).
1091 # embedded in).
1092
1092
1093 # This is overridden in the InteractiveShellEmbed subclass to a no-op.
1093 # This is overridden in the InteractiveShellEmbed subclass to a no-op.
1094 main_name = self.user_module.__name__
1094 main_name = self.user_module.__name__
1095 sys.modules[main_name] = self.user_module
1095 sys.modules[main_name] = self.user_module
1096
1096
1097 def init_user_ns(self):
1097 def init_user_ns(self):
1098 """Initialize all user-visible namespaces to their minimum defaults.
1098 """Initialize all user-visible namespaces to their minimum defaults.
1099
1099
1100 Certain history lists are also initialized here, as they effectively
1100 Certain history lists are also initialized here, as they effectively
1101 act as user namespaces.
1101 act as user namespaces.
1102
1102
1103 Notes
1103 Notes
1104 -----
1104 -----
1105 All data structures here are only filled in, they are NOT reset by this
1105 All data structures here are only filled in, they are NOT reset by this
1106 method. If they were not empty before, data will simply be added to
1106 method. If they were not empty before, data will simply be added to
1107 therm.
1107 therm.
1108 """
1108 """
1109 # This function works in two parts: first we put a few things in
1109 # This function works in two parts: first we put a few things in
1110 # user_ns, and we sync that contents into user_ns_hidden so that these
1110 # user_ns, and we sync that contents into user_ns_hidden so that these
1111 # initial variables aren't shown by %who. After the sync, we add the
1111 # initial variables aren't shown by %who. After the sync, we add the
1112 # rest of what we *do* want the user to see with %who even on a new
1112 # rest of what we *do* want the user to see with %who even on a new
1113 # session (probably nothing, so theye really only see their own stuff)
1113 # session (probably nothing, so theye really only see their own stuff)
1114
1114
1115 # The user dict must *always* have a __builtin__ reference to the
1115 # The user dict must *always* have a __builtin__ reference to the
1116 # Python standard __builtin__ namespace, which must be imported.
1116 # Python standard __builtin__ namespace, which must be imported.
1117 # This is so that certain operations in prompt evaluation can be
1117 # This is so that certain operations in prompt evaluation can be
1118 # reliably executed with builtins. Note that we can NOT use
1118 # reliably executed with builtins. Note that we can NOT use
1119 # __builtins__ (note the 's'), because that can either be a dict or a
1119 # __builtins__ (note the 's'), because that can either be a dict or a
1120 # module, and can even mutate at runtime, depending on the context
1120 # module, and can even mutate at runtime, depending on the context
1121 # (Python makes no guarantees on it). In contrast, __builtin__ is
1121 # (Python makes no guarantees on it). In contrast, __builtin__ is
1122 # always a module object, though it must be explicitly imported.
1122 # always a module object, though it must be explicitly imported.
1123
1123
1124 # For more details:
1124 # For more details:
1125 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1125 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1126 ns = dict()
1126 ns = dict()
1127
1127
1128 # Put 'help' in the user namespace
1128 # Put 'help' in the user namespace
1129 try:
1129 try:
1130 from site import _Helper
1130 from site import _Helper
1131 ns['help'] = _Helper()
1131 ns['help'] = _Helper()
1132 except ImportError:
1132 except ImportError:
1133 warn('help() not available - check site.py')
1133 warn('help() not available - check site.py')
1134
1134
1135 # make global variables for user access to the histories
1135 # make global variables for user access to the histories
1136 ns['_ih'] = self.history_manager.input_hist_parsed
1136 ns['_ih'] = self.history_manager.input_hist_parsed
1137 ns['_oh'] = self.history_manager.output_hist
1137 ns['_oh'] = self.history_manager.output_hist
1138 ns['_dh'] = self.history_manager.dir_hist
1138 ns['_dh'] = self.history_manager.dir_hist
1139
1139
1140 ns['_sh'] = shadowns
1140 ns['_sh'] = shadowns
1141
1141
1142 # user aliases to input and output histories. These shouldn't show up
1142 # user aliases to input and output histories. These shouldn't show up
1143 # in %who, as they can have very large reprs.
1143 # in %who, as they can have very large reprs.
1144 ns['In'] = self.history_manager.input_hist_parsed
1144 ns['In'] = self.history_manager.input_hist_parsed
1145 ns['Out'] = self.history_manager.output_hist
1145 ns['Out'] = self.history_manager.output_hist
1146
1146
1147 # Store myself as the public api!!!
1147 # Store myself as the public api!!!
1148 ns['get_ipython'] = self.get_ipython
1148 ns['get_ipython'] = self.get_ipython
1149
1149
1150 ns['exit'] = self.exiter
1150 ns['exit'] = self.exiter
1151 ns['quit'] = self.exiter
1151 ns['quit'] = self.exiter
1152
1152
1153 # Sync what we've added so far to user_ns_hidden so these aren't seen
1153 # Sync what we've added so far to user_ns_hidden so these aren't seen
1154 # by %who
1154 # by %who
1155 self.user_ns_hidden.update(ns)
1155 self.user_ns_hidden.update(ns)
1156
1156
1157 # Anything put into ns now would show up in %who. Think twice before
1157 # Anything put into ns now would show up in %who. Think twice before
1158 # putting anything here, as we really want %who to show the user their
1158 # putting anything here, as we really want %who to show the user their
1159 # stuff, not our variables.
1159 # stuff, not our variables.
1160
1160
1161 # Finally, update the real user's namespace
1161 # Finally, update the real user's namespace
1162 self.user_ns.update(ns)
1162 self.user_ns.update(ns)
1163
1163
1164 @property
1164 @property
1165 def all_ns_refs(self):
1165 def all_ns_refs(self):
1166 """Get a list of references to all the namespace dictionaries in which
1166 """Get a list of references to all the namespace dictionaries in which
1167 IPython might store a user-created object.
1167 IPython might store a user-created object.
1168
1168
1169 Note that this does not include the displayhook, which also caches
1169 Note that this does not include the displayhook, which also caches
1170 objects from the output."""
1170 objects from the output."""
1171 return [self.user_ns, self.user_global_ns, self.user_ns_hidden] + \
1171 return [self.user_ns, self.user_global_ns, self.user_ns_hidden] + \
1172 [m.__dict__ for m in self._main_mod_cache.values()]
1172 [m.__dict__ for m in self._main_mod_cache.values()]
1173
1173
1174 def reset(self, new_session=True):
1174 def reset(self, new_session=True):
1175 """Clear all internal namespaces, and attempt to release references to
1175 """Clear all internal namespaces, and attempt to release references to
1176 user objects.
1176 user objects.
1177
1177
1178 If new_session is True, a new history session will be opened.
1178 If new_session is True, a new history session will be opened.
1179 """
1179 """
1180 # Clear histories
1180 # Clear histories
1181 self.history_manager.reset(new_session)
1181 self.history_manager.reset(new_session)
1182 # Reset counter used to index all histories
1182 # Reset counter used to index all histories
1183 if new_session:
1183 if new_session:
1184 self.execution_count = 1
1184 self.execution_count = 1
1185
1185
1186 # Flush cached output items
1186 # Flush cached output items
1187 if self.displayhook.do_full_cache:
1187 if self.displayhook.do_full_cache:
1188 self.displayhook.flush()
1188 self.displayhook.flush()
1189
1189
1190 # The main execution namespaces must be cleared very carefully,
1190 # The main execution namespaces must be cleared very carefully,
1191 # skipping the deletion of the builtin-related keys, because doing so
1191 # skipping the deletion of the builtin-related keys, because doing so
1192 # would cause errors in many object's __del__ methods.
1192 # would cause errors in many object's __del__ methods.
1193 if self.user_ns is not self.user_global_ns:
1193 if self.user_ns is not self.user_global_ns:
1194 self.user_ns.clear()
1194 self.user_ns.clear()
1195 ns = self.user_global_ns
1195 ns = self.user_global_ns
1196 drop_keys = set(ns.keys())
1196 drop_keys = set(ns.keys())
1197 drop_keys.discard('__builtin__')
1197 drop_keys.discard('__builtin__')
1198 drop_keys.discard('__builtins__')
1198 drop_keys.discard('__builtins__')
1199 drop_keys.discard('__name__')
1199 drop_keys.discard('__name__')
1200 for k in drop_keys:
1200 for k in drop_keys:
1201 del ns[k]
1201 del ns[k]
1202
1202
1203 self.user_ns_hidden.clear()
1203 self.user_ns_hidden.clear()
1204
1204
1205 # Restore the user namespaces to minimal usability
1205 # Restore the user namespaces to minimal usability
1206 self.init_user_ns()
1206 self.init_user_ns()
1207
1207
1208 # Restore the default and user aliases
1208 # Restore the default and user aliases
1209 self.alias_manager.clear_aliases()
1209 self.alias_manager.clear_aliases()
1210 self.alias_manager.init_aliases()
1210 self.alias_manager.init_aliases()
1211
1211
1212 # Flush the private list of module references kept for script
1212 # Flush the private list of module references kept for script
1213 # execution protection
1213 # execution protection
1214 self.clear_main_mod_cache()
1214 self.clear_main_mod_cache()
1215
1215
1216 def del_var(self, varname, by_name=False):
1216 def del_var(self, varname, by_name=False):
1217 """Delete a variable from the various namespaces, so that, as
1217 """Delete a variable from the various namespaces, so that, as
1218 far as possible, we're not keeping any hidden references to it.
1218 far as possible, we're not keeping any hidden references to it.
1219
1219
1220 Parameters
1220 Parameters
1221 ----------
1221 ----------
1222 varname : str
1222 varname : str
1223 The name of the variable to delete.
1223 The name of the variable to delete.
1224 by_name : bool
1224 by_name : bool
1225 If True, delete variables with the given name in each
1225 If True, delete variables with the given name in each
1226 namespace. If False (default), find the variable in the user
1226 namespace. If False (default), find the variable in the user
1227 namespace, and delete references to it.
1227 namespace, and delete references to it.
1228 """
1228 """
1229 if varname in ('__builtin__', '__builtins__'):
1229 if varname in ('__builtin__', '__builtins__'):
1230 raise ValueError("Refusing to delete %s" % varname)
1230 raise ValueError("Refusing to delete %s" % varname)
1231
1231
1232 ns_refs = self.all_ns_refs
1232 ns_refs = self.all_ns_refs
1233
1233
1234 if by_name: # Delete by name
1234 if by_name: # Delete by name
1235 for ns in ns_refs:
1235 for ns in ns_refs:
1236 try:
1236 try:
1237 del ns[varname]
1237 del ns[varname]
1238 except KeyError:
1238 except KeyError:
1239 pass
1239 pass
1240 else: # Delete by object
1240 else: # Delete by object
1241 try:
1241 try:
1242 obj = self.user_ns[varname]
1242 obj = self.user_ns[varname]
1243 except KeyError:
1243 except KeyError:
1244 raise NameError("name '%s' is not defined" % varname)
1244 raise NameError("name '%s' is not defined" % varname)
1245 # Also check in output history
1245 # Also check in output history
1246 ns_refs.append(self.history_manager.output_hist)
1246 ns_refs.append(self.history_manager.output_hist)
1247 for ns in ns_refs:
1247 for ns in ns_refs:
1248 to_delete = [n for n, o in iteritems(ns) if o is obj]
1248 to_delete = [n for n, o in iteritems(ns) if o is obj]
1249 for name in to_delete:
1249 for name in to_delete:
1250 del ns[name]
1250 del ns[name]
1251
1251
1252 # displayhook keeps extra references, but not in a dictionary
1252 # displayhook keeps extra references, but not in a dictionary
1253 for name in ('_', '__', '___'):
1253 for name in ('_', '__', '___'):
1254 if getattr(self.displayhook, name) is obj:
1254 if getattr(self.displayhook, name) is obj:
1255 setattr(self.displayhook, name, None)
1255 setattr(self.displayhook, name, None)
1256
1256
1257 def reset_selective(self, regex=None):
1257 def reset_selective(self, regex=None):
1258 """Clear selective variables from internal namespaces based on a
1258 """Clear selective variables from internal namespaces based on a
1259 specified regular expression.
1259 specified regular expression.
1260
1260
1261 Parameters
1261 Parameters
1262 ----------
1262 ----------
1263 regex : string or compiled pattern, optional
1263 regex : string or compiled pattern, optional
1264 A regular expression pattern that will be used in searching
1264 A regular expression pattern that will be used in searching
1265 variable names in the users namespaces.
1265 variable names in the users namespaces.
1266 """
1266 """
1267 if regex is not None:
1267 if regex is not None:
1268 try:
1268 try:
1269 m = re.compile(regex)
1269 m = re.compile(regex)
1270 except TypeError:
1270 except TypeError:
1271 raise TypeError('regex must be a string or compiled pattern')
1271 raise TypeError('regex must be a string or compiled pattern')
1272 # Search for keys in each namespace that match the given regex
1272 # Search for keys in each namespace that match the given regex
1273 # If a match is found, delete the key/value pair.
1273 # If a match is found, delete the key/value pair.
1274 for ns in self.all_ns_refs:
1274 for ns in self.all_ns_refs:
1275 for var in ns:
1275 for var in ns:
1276 if m.search(var):
1276 if m.search(var):
1277 del ns[var]
1277 del ns[var]
1278
1278
1279 def push(self, variables, interactive=True):
1279 def push(self, variables, interactive=True):
1280 """Inject a group of variables into the IPython user namespace.
1280 """Inject a group of variables into the IPython user namespace.
1281
1281
1282 Parameters
1282 Parameters
1283 ----------
1283 ----------
1284 variables : dict, str or list/tuple of str
1284 variables : dict, str or list/tuple of str
1285 The variables to inject into the user's namespace. If a dict, a
1285 The variables to inject into the user's namespace. If a dict, a
1286 simple update is done. If a str, the string is assumed to have
1286 simple update is done. If a str, the string is assumed to have
1287 variable names separated by spaces. A list/tuple of str can also
1287 variable names separated by spaces. A list/tuple of str can also
1288 be used to give the variable names. If just the variable names are
1288 be used to give the variable names. If just the variable names are
1289 give (list/tuple/str) then the variable values looked up in the
1289 give (list/tuple/str) then the variable values looked up in the
1290 callers frame.
1290 callers frame.
1291 interactive : bool
1291 interactive : bool
1292 If True (default), the variables will be listed with the ``who``
1292 If True (default), the variables will be listed with the ``who``
1293 magic.
1293 magic.
1294 """
1294 """
1295 vdict = None
1295 vdict = None
1296
1296
1297 # We need a dict of name/value pairs to do namespace updates.
1297 # We need a dict of name/value pairs to do namespace updates.
1298 if isinstance(variables, dict):
1298 if isinstance(variables, dict):
1299 vdict = variables
1299 vdict = variables
1300 elif isinstance(variables, string_types+(list, tuple)):
1300 elif isinstance(variables, string_types+(list, tuple)):
1301 if isinstance(variables, string_types):
1301 if isinstance(variables, string_types):
1302 vlist = variables.split()
1302 vlist = variables.split()
1303 else:
1303 else:
1304 vlist = variables
1304 vlist = variables
1305 vdict = {}
1305 vdict = {}
1306 cf = sys._getframe(1)
1306 cf = sys._getframe(1)
1307 for name in vlist:
1307 for name in vlist:
1308 try:
1308 try:
1309 vdict[name] = eval(name, cf.f_globals, cf.f_locals)
1309 vdict[name] = eval(name, cf.f_globals, cf.f_locals)
1310 except:
1310 except:
1311 print('Could not get variable %s from %s' %
1311 print('Could not get variable %s from %s' %
1312 (name,cf.f_code.co_name))
1312 (name,cf.f_code.co_name))
1313 else:
1313 else:
1314 raise ValueError('variables must be a dict/str/list/tuple')
1314 raise ValueError('variables must be a dict/str/list/tuple')
1315
1315
1316 # Propagate variables to user namespace
1316 # Propagate variables to user namespace
1317 self.user_ns.update(vdict)
1317 self.user_ns.update(vdict)
1318
1318
1319 # And configure interactive visibility
1319 # And configure interactive visibility
1320 user_ns_hidden = self.user_ns_hidden
1320 user_ns_hidden = self.user_ns_hidden
1321 if interactive:
1321 if interactive:
1322 for name in vdict:
1322 for name in vdict:
1323 user_ns_hidden.pop(name, None)
1323 user_ns_hidden.pop(name, None)
1324 else:
1324 else:
1325 user_ns_hidden.update(vdict)
1325 user_ns_hidden.update(vdict)
1326
1326
1327 def drop_by_id(self, variables):
1327 def drop_by_id(self, variables):
1328 """Remove a dict of variables from the user namespace, if they are the
1328 """Remove a dict of variables from the user namespace, if they are the
1329 same as the values in the dictionary.
1329 same as the values in the dictionary.
1330
1330
1331 This is intended for use by extensions: variables that they've added can
1331 This is intended for use by extensions: variables that they've added can
1332 be taken back out if they are unloaded, without removing any that the
1332 be taken back out if they are unloaded, without removing any that the
1333 user has overwritten.
1333 user has overwritten.
1334
1334
1335 Parameters
1335 Parameters
1336 ----------
1336 ----------
1337 variables : dict
1337 variables : dict
1338 A dictionary mapping object names (as strings) to the objects.
1338 A dictionary mapping object names (as strings) to the objects.
1339 """
1339 """
1340 for name, obj in iteritems(variables):
1340 for name, obj in iteritems(variables):
1341 if name in self.user_ns and self.user_ns[name] is obj:
1341 if name in self.user_ns and self.user_ns[name] is obj:
1342 del self.user_ns[name]
1342 del self.user_ns[name]
1343 self.user_ns_hidden.pop(name, None)
1343 self.user_ns_hidden.pop(name, None)
1344
1344
1345 #-------------------------------------------------------------------------
1345 #-------------------------------------------------------------------------
1346 # Things related to object introspection
1346 # Things related to object introspection
1347 #-------------------------------------------------------------------------
1347 #-------------------------------------------------------------------------
1348
1348
1349 def _ofind(self, oname, namespaces=None):
1349 def _ofind(self, oname, namespaces=None):
1350 """Find an object in the available namespaces.
1350 """Find an object in the available namespaces.
1351
1351
1352 self._ofind(oname) -> dict with keys: found,obj,ospace,ismagic
1352 self._ofind(oname) -> dict with keys: found,obj,ospace,ismagic
1353
1353
1354 Has special code to detect magic functions.
1354 Has special code to detect magic functions.
1355 """
1355 """
1356 oname = oname.strip()
1356 oname = oname.strip()
1357 #print '1- oname: <%r>' % oname # dbg
1357 #print '1- oname: <%r>' % oname # dbg
1358 if not oname.startswith(ESC_MAGIC) and \
1358 if not oname.startswith(ESC_MAGIC) and \
1359 not oname.startswith(ESC_MAGIC2) and \
1359 not oname.startswith(ESC_MAGIC2) and \
1360 not py3compat.isidentifier(oname, dotted=True):
1360 not py3compat.isidentifier(oname, dotted=True):
1361 return dict(found=False)
1361 return dict(found=False)
1362
1362
1363 alias_ns = None
1363 alias_ns = None
1364 if namespaces is None:
1364 if namespaces is None:
1365 # Namespaces to search in:
1365 # Namespaces to search in:
1366 # Put them in a list. The order is important so that we
1366 # Put them in a list. The order is important so that we
1367 # find things in the same order that Python finds them.
1367 # find things in the same order that Python finds them.
1368 namespaces = [ ('Interactive', self.user_ns),
1368 namespaces = [ ('Interactive', self.user_ns),
1369 ('Interactive (global)', self.user_global_ns),
1369 ('Interactive (global)', self.user_global_ns),
1370 ('Python builtin', builtin_mod.__dict__),
1370 ('Python builtin', builtin_mod.__dict__),
1371 ]
1371 ]
1372
1372
1373 # initialize results to 'null'
1373 # initialize results to 'null'
1374 found = False; obj = None; ospace = None; ds = None;
1374 found = False; obj = None; ospace = None; ds = None;
1375 ismagic = False; isalias = False; parent = None
1375 ismagic = False; isalias = False; parent = None
1376
1376
1377 # We need to special-case 'print', which as of python2.6 registers as a
1377 # We need to special-case 'print', which as of python2.6 registers as a
1378 # function but should only be treated as one if print_function was
1378 # function but should only be treated as one if print_function was
1379 # loaded with a future import. In this case, just bail.
1379 # loaded with a future import. In this case, just bail.
1380 if (oname == 'print' and not py3compat.PY3 and not \
1380 if (oname == 'print' and not py3compat.PY3 and not \
1381 (self.compile.compiler_flags & __future__.CO_FUTURE_PRINT_FUNCTION)):
1381 (self.compile.compiler_flags & __future__.CO_FUTURE_PRINT_FUNCTION)):
1382 return {'found':found, 'obj':obj, 'namespace':ospace,
1382 return {'found':found, 'obj':obj, 'namespace':ospace,
1383 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1383 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1384
1384
1385 # Look for the given name by splitting it in parts. If the head is
1385 # Look for the given name by splitting it in parts. If the head is
1386 # found, then we look for all the remaining parts as members, and only
1386 # found, then we look for all the remaining parts as members, and only
1387 # declare success if we can find them all.
1387 # declare success if we can find them all.
1388 oname_parts = oname.split('.')
1388 oname_parts = oname.split('.')
1389 oname_head, oname_rest = oname_parts[0],oname_parts[1:]
1389 oname_head, oname_rest = oname_parts[0],oname_parts[1:]
1390 for nsname,ns in namespaces:
1390 for nsname,ns in namespaces:
1391 try:
1391 try:
1392 obj = ns[oname_head]
1392 obj = ns[oname_head]
1393 except KeyError:
1393 except KeyError:
1394 continue
1394 continue
1395 else:
1395 else:
1396 #print 'oname_rest:', oname_rest # dbg
1396 #print 'oname_rest:', oname_rest # dbg
1397 for part in oname_rest:
1397 for part in oname_rest:
1398 try:
1398 try:
1399 parent = obj
1399 parent = obj
1400 obj = getattr(obj,part)
1400 obj = getattr(obj,part)
1401 except:
1401 except:
1402 # Blanket except b/c some badly implemented objects
1402 # Blanket except b/c some badly implemented objects
1403 # allow __getattr__ to raise exceptions other than
1403 # allow __getattr__ to raise exceptions other than
1404 # AttributeError, which then crashes IPython.
1404 # AttributeError, which then crashes IPython.
1405 break
1405 break
1406 else:
1406 else:
1407 # If we finish the for loop (no break), we got all members
1407 # If we finish the for loop (no break), we got all members
1408 found = True
1408 found = True
1409 ospace = nsname
1409 ospace = nsname
1410 break # namespace loop
1410 break # namespace loop
1411
1411
1412 # Try to see if it's magic
1412 # Try to see if it's magic
1413 if not found:
1413 if not found:
1414 obj = None
1414 obj = None
1415 if oname.startswith(ESC_MAGIC2):
1415 if oname.startswith(ESC_MAGIC2):
1416 oname = oname.lstrip(ESC_MAGIC2)
1416 oname = oname.lstrip(ESC_MAGIC2)
1417 obj = self.find_cell_magic(oname)
1417 obj = self.find_cell_magic(oname)
1418 elif oname.startswith(ESC_MAGIC):
1418 elif oname.startswith(ESC_MAGIC):
1419 oname = oname.lstrip(ESC_MAGIC)
1419 oname = oname.lstrip(ESC_MAGIC)
1420 obj = self.find_line_magic(oname)
1420 obj = self.find_line_magic(oname)
1421 else:
1421 else:
1422 # search without prefix, so run? will find %run?
1422 # search without prefix, so run? will find %run?
1423 obj = self.find_line_magic(oname)
1423 obj = self.find_line_magic(oname)
1424 if obj is None:
1424 if obj is None:
1425 obj = self.find_cell_magic(oname)
1425 obj = self.find_cell_magic(oname)
1426 if obj is not None:
1426 if obj is not None:
1427 found = True
1427 found = True
1428 ospace = 'IPython internal'
1428 ospace = 'IPython internal'
1429 ismagic = True
1429 ismagic = True
1430
1430
1431 # Last try: special-case some literals like '', [], {}, etc:
1431 # Last try: special-case some literals like '', [], {}, etc:
1432 if not found and oname_head in ["''",'""','[]','{}','()']:
1432 if not found and oname_head in ["''",'""','[]','{}','()']:
1433 obj = eval(oname_head)
1433 obj = eval(oname_head)
1434 found = True
1434 found = True
1435 ospace = 'Interactive'
1435 ospace = 'Interactive'
1436
1436
1437 return {'found':found, 'obj':obj, 'namespace':ospace,
1437 return {'found':found, 'obj':obj, 'namespace':ospace,
1438 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1438 'ismagic':ismagic, 'isalias':isalias, 'parent':parent}
1439
1439
1440 def _ofind_property(self, oname, info):
1440 def _ofind_property(self, oname, info):
1441 """Second part of object finding, to look for property details."""
1441 """Second part of object finding, to look for property details."""
1442 if info.found:
1442 if info.found:
1443 # Get the docstring of the class property if it exists.
1443 # Get the docstring of the class property if it exists.
1444 path = oname.split('.')
1444 path = oname.split('.')
1445 root = '.'.join(path[:-1])
1445 root = '.'.join(path[:-1])
1446 if info.parent is not None:
1446 if info.parent is not None:
1447 try:
1447 try:
1448 target = getattr(info.parent, '__class__')
1448 target = getattr(info.parent, '__class__')
1449 # The object belongs to a class instance.
1449 # The object belongs to a class instance.
1450 try:
1450 try:
1451 target = getattr(target, path[-1])
1451 target = getattr(target, path[-1])
1452 # The class defines the object.
1452 # The class defines the object.
1453 if isinstance(target, property):
1453 if isinstance(target, property):
1454 oname = root + '.__class__.' + path[-1]
1454 oname = root + '.__class__.' + path[-1]
1455 info = Struct(self._ofind(oname))
1455 info = Struct(self._ofind(oname))
1456 except AttributeError: pass
1456 except AttributeError: pass
1457 except AttributeError: pass
1457 except AttributeError: pass
1458
1458
1459 # We return either the new info or the unmodified input if the object
1459 # We return either the new info or the unmodified input if the object
1460 # hadn't been found
1460 # hadn't been found
1461 return info
1461 return info
1462
1462
1463 def _object_find(self, oname, namespaces=None):
1463 def _object_find(self, oname, namespaces=None):
1464 """Find an object and return a struct with info about it."""
1464 """Find an object and return a struct with info about it."""
1465 inf = Struct(self._ofind(oname, namespaces))
1465 inf = Struct(self._ofind(oname, namespaces))
1466 return Struct(self._ofind_property(oname, inf))
1466 return Struct(self._ofind_property(oname, inf))
1467
1467
1468 def _inspect(self, meth, oname, namespaces=None, **kw):
1468 def _inspect(self, meth, oname, namespaces=None, **kw):
1469 """Generic interface to the inspector system.
1469 """Generic interface to the inspector system.
1470
1470
1471 This function is meant to be called by pdef, pdoc & friends."""
1471 This function is meant to be called by pdef, pdoc & friends."""
1472 info = self._object_find(oname, namespaces)
1472 info = self._object_find(oname, namespaces)
1473 if info.found:
1473 if info.found:
1474 pmethod = getattr(self.inspector, meth)
1474 pmethod = getattr(self.inspector, meth)
1475 formatter = format_screen if info.ismagic else None
1475 formatter = format_screen if info.ismagic else None
1476 if meth == 'pdoc':
1476 if meth == 'pdoc':
1477 pmethod(info.obj, oname, formatter)
1477 pmethod(info.obj, oname, formatter)
1478 elif meth == 'pinfo':
1478 elif meth == 'pinfo':
1479 pmethod(info.obj, oname, formatter, info, **kw)
1479 pmethod(info.obj, oname, formatter, info, **kw)
1480 else:
1480 else:
1481 pmethod(info.obj, oname)
1481 pmethod(info.obj, oname)
1482 else:
1482 else:
1483 print('Object `%s` not found.' % oname)
1483 print('Object `%s` not found.' % oname)
1484 return 'not found' # so callers can take other action
1484 return 'not found' # so callers can take other action
1485
1485
1486 def object_inspect(self, oname, detail_level=0):
1486 def object_inspect(self, oname, detail_level=0):
1487 with self.builtin_trap:
1487 with self.builtin_trap:
1488 info = self._object_find(oname)
1488 info = self._object_find(oname)
1489 if info.found:
1489 if info.found:
1490 return self.inspector.info(info.obj, oname, info=info,
1490 return self.inspector.info(info.obj, oname, info=info,
1491 detail_level=detail_level
1491 detail_level=detail_level
1492 )
1492 )
1493 else:
1493 else:
1494 return oinspect.object_info(name=oname, found=False)
1494 return oinspect.object_info(name=oname, found=False)
1495
1495
1496 #-------------------------------------------------------------------------
1496 #-------------------------------------------------------------------------
1497 # Things related to history management
1497 # Things related to history management
1498 #-------------------------------------------------------------------------
1498 #-------------------------------------------------------------------------
1499
1499
1500 def init_history(self):
1500 def init_history(self):
1501 """Sets up the command history, and starts regular autosaves."""
1501 """Sets up the command history, and starts regular autosaves."""
1502 self.history_manager = HistoryManager(shell=self, parent=self)
1502 self.history_manager = HistoryManager(shell=self, parent=self)
1503 self.configurables.append(self.history_manager)
1503 self.configurables.append(self.history_manager)
1504
1504
1505 #-------------------------------------------------------------------------
1505 #-------------------------------------------------------------------------
1506 # Things related to exception handling and tracebacks (not debugging)
1506 # Things related to exception handling and tracebacks (not debugging)
1507 #-------------------------------------------------------------------------
1507 #-------------------------------------------------------------------------
1508
1508
1509 def init_traceback_handlers(self, custom_exceptions):
1509 def init_traceback_handlers(self, custom_exceptions):
1510 # Syntax error handler.
1510 # Syntax error handler.
1511 self.SyntaxTB = ultratb.SyntaxTB(color_scheme='NoColor')
1511 self.SyntaxTB = ultratb.SyntaxTB(color_scheme='NoColor')
1512
1512
1513 # The interactive one is initialized with an offset, meaning we always
1513 # The interactive one is initialized with an offset, meaning we always
1514 # want to remove the topmost item in the traceback, which is our own
1514 # want to remove the topmost item in the traceback, which is our own
1515 # internal code. Valid modes: ['Plain','Context','Verbose']
1515 # internal code. Valid modes: ['Plain','Context','Verbose']
1516 self.InteractiveTB = ultratb.AutoFormattedTB(mode = 'Plain',
1516 self.InteractiveTB = ultratb.AutoFormattedTB(mode = 'Plain',
1517 color_scheme='NoColor',
1517 color_scheme='NoColor',
1518 tb_offset = 1,
1518 tb_offset = 1,
1519 check_cache=check_linecache_ipython)
1519 check_cache=check_linecache_ipython)
1520
1520
1521 # The instance will store a pointer to the system-wide exception hook,
1521 # The instance will store a pointer to the system-wide exception hook,
1522 # so that runtime code (such as magics) can access it. This is because
1522 # so that runtime code (such as magics) can access it. This is because
1523 # during the read-eval loop, it may get temporarily overwritten.
1523 # during the read-eval loop, it may get temporarily overwritten.
1524 self.sys_excepthook = sys.excepthook
1524 self.sys_excepthook = sys.excepthook
1525
1525
1526 # and add any custom exception handlers the user may have specified
1526 # and add any custom exception handlers the user may have specified
1527 self.set_custom_exc(*custom_exceptions)
1527 self.set_custom_exc(*custom_exceptions)
1528
1528
1529 # Set the exception mode
1529 # Set the exception mode
1530 self.InteractiveTB.set_mode(mode=self.xmode)
1530 self.InteractiveTB.set_mode(mode=self.xmode)
1531
1531
1532 def set_custom_exc(self, exc_tuple, handler):
1532 def set_custom_exc(self, exc_tuple, handler):
1533 """set_custom_exc(exc_tuple,handler)
1533 """set_custom_exc(exc_tuple,handler)
1534
1534
1535 Set a custom exception handler, which will be called if any of the
1535 Set a custom exception handler, which will be called if any of the
1536 exceptions in exc_tuple occur in the mainloop (specifically, in the
1536 exceptions in exc_tuple occur in the mainloop (specifically, in the
1537 run_code() method).
1537 run_code() method).
1538
1538
1539 Parameters
1539 Parameters
1540 ----------
1540 ----------
1541
1541
1542 exc_tuple : tuple of exception classes
1542 exc_tuple : tuple of exception classes
1543 A *tuple* of exception classes, for which to call the defined
1543 A *tuple* of exception classes, for which to call the defined
1544 handler. It is very important that you use a tuple, and NOT A
1544 handler. It is very important that you use a tuple, and NOT A
1545 LIST here, because of the way Python's except statement works. If
1545 LIST here, because of the way Python's except statement works. If
1546 you only want to trap a single exception, use a singleton tuple::
1546 you only want to trap a single exception, use a singleton tuple::
1547
1547
1548 exc_tuple == (MyCustomException,)
1548 exc_tuple == (MyCustomException,)
1549
1549
1550 handler : callable
1550 handler : callable
1551 handler must have the following signature::
1551 handler must have the following signature::
1552
1552
1553 def my_handler(self, etype, value, tb, tb_offset=None):
1553 def my_handler(self, etype, value, tb, tb_offset=None):
1554 ...
1554 ...
1555 return structured_traceback
1555 return structured_traceback
1556
1556
1557 Your handler must return a structured traceback (a list of strings),
1557 Your handler must return a structured traceback (a list of strings),
1558 or None.
1558 or None.
1559
1559
1560 This will be made into an instance method (via types.MethodType)
1560 This will be made into an instance method (via types.MethodType)
1561 of IPython itself, and it will be called if any of the exceptions
1561 of IPython itself, and it will be called if any of the exceptions
1562 listed in the exc_tuple are caught. If the handler is None, an
1562 listed in the exc_tuple are caught. If the handler is None, an
1563 internal basic one is used, which just prints basic info.
1563 internal basic one is used, which just prints basic info.
1564
1564
1565 To protect IPython from crashes, if your handler ever raises an
1565 To protect IPython from crashes, if your handler ever raises an
1566 exception or returns an invalid result, it will be immediately
1566 exception or returns an invalid result, it will be immediately
1567 disabled.
1567 disabled.
1568
1568
1569 WARNING: by putting in your own exception handler into IPython's main
1569 WARNING: by putting in your own exception handler into IPython's main
1570 execution loop, you run a very good chance of nasty crashes. This
1570 execution loop, you run a very good chance of nasty crashes. This
1571 facility should only be used if you really know what you are doing."""
1571 facility should only be used if you really know what you are doing."""
1572
1572
1573 assert type(exc_tuple)==type(()) , \
1573 assert type(exc_tuple)==type(()) , \
1574 "The custom exceptions must be given AS A TUPLE."
1574 "The custom exceptions must be given AS A TUPLE."
1575
1575
1576 def dummy_handler(self,etype,value,tb,tb_offset=None):
1576 def dummy_handler(self,etype,value,tb,tb_offset=None):
1577 print('*** Simple custom exception handler ***')
1577 print('*** Simple custom exception handler ***')
1578 print('Exception type :',etype)
1578 print('Exception type :',etype)
1579 print('Exception value:',value)
1579 print('Exception value:',value)
1580 print('Traceback :',tb)
1580 print('Traceback :',tb)
1581 #print 'Source code :','\n'.join(self.buffer)
1581 #print 'Source code :','\n'.join(self.buffer)
1582
1582
1583 def validate_stb(stb):
1583 def validate_stb(stb):
1584 """validate structured traceback return type
1584 """validate structured traceback return type
1585
1585
1586 return type of CustomTB *should* be a list of strings, but allow
1586 return type of CustomTB *should* be a list of strings, but allow
1587 single strings or None, which are harmless.
1587 single strings or None, which are harmless.
1588
1588
1589 This function will *always* return a list of strings,
1589 This function will *always* return a list of strings,
1590 and will raise a TypeError if stb is inappropriate.
1590 and will raise a TypeError if stb is inappropriate.
1591 """
1591 """
1592 msg = "CustomTB must return list of strings, not %r" % stb
1592 msg = "CustomTB must return list of strings, not %r" % stb
1593 if stb is None:
1593 if stb is None:
1594 return []
1594 return []
1595 elif isinstance(stb, string_types):
1595 elif isinstance(stb, string_types):
1596 return [stb]
1596 return [stb]
1597 elif not isinstance(stb, list):
1597 elif not isinstance(stb, list):
1598 raise TypeError(msg)
1598 raise TypeError(msg)
1599 # it's a list
1599 # it's a list
1600 for line in stb:
1600 for line in stb:
1601 # check every element
1601 # check every element
1602 if not isinstance(line, string_types):
1602 if not isinstance(line, string_types):
1603 raise TypeError(msg)
1603 raise TypeError(msg)
1604 return stb
1604 return stb
1605
1605
1606 if handler is None:
1606 if handler is None:
1607 wrapped = dummy_handler
1607 wrapped = dummy_handler
1608 else:
1608 else:
1609 def wrapped(self,etype,value,tb,tb_offset=None):
1609 def wrapped(self,etype,value,tb,tb_offset=None):
1610 """wrap CustomTB handler, to protect IPython from user code
1610 """wrap CustomTB handler, to protect IPython from user code
1611
1611
1612 This makes it harder (but not impossible) for custom exception
1612 This makes it harder (but not impossible) for custom exception
1613 handlers to crash IPython.
1613 handlers to crash IPython.
1614 """
1614 """
1615 try:
1615 try:
1616 stb = handler(self,etype,value,tb,tb_offset=tb_offset)
1616 stb = handler(self,etype,value,tb,tb_offset=tb_offset)
1617 return validate_stb(stb)
1617 return validate_stb(stb)
1618 except:
1618 except:
1619 # clear custom handler immediately
1619 # clear custom handler immediately
1620 self.set_custom_exc((), None)
1620 self.set_custom_exc((), None)
1621 print("Custom TB Handler failed, unregistering", file=io.stderr)
1621 print("Custom TB Handler failed, unregistering", file=io.stderr)
1622 # show the exception in handler first
1622 # show the exception in handler first
1623 stb = self.InteractiveTB.structured_traceback(*sys.exc_info())
1623 stb = self.InteractiveTB.structured_traceback(*sys.exc_info())
1624 print(self.InteractiveTB.stb2text(stb), file=io.stdout)
1624 print(self.InteractiveTB.stb2text(stb), file=io.stdout)
1625 print("The original exception:", file=io.stdout)
1625 print("The original exception:", file=io.stdout)
1626 stb = self.InteractiveTB.structured_traceback(
1626 stb = self.InteractiveTB.structured_traceback(
1627 (etype,value,tb), tb_offset=tb_offset
1627 (etype,value,tb), tb_offset=tb_offset
1628 )
1628 )
1629 return stb
1629 return stb
1630
1630
1631 self.CustomTB = types.MethodType(wrapped,self)
1631 self.CustomTB = types.MethodType(wrapped,self)
1632 self.custom_exceptions = exc_tuple
1632 self.custom_exceptions = exc_tuple
1633
1633
1634 def excepthook(self, etype, value, tb):
1634 def excepthook(self, etype, value, tb):
1635 """One more defense for GUI apps that call sys.excepthook.
1635 """One more defense for GUI apps that call sys.excepthook.
1636
1636
1637 GUI frameworks like wxPython trap exceptions and call
1637 GUI frameworks like wxPython trap exceptions and call
1638 sys.excepthook themselves. I guess this is a feature that
1638 sys.excepthook themselves. I guess this is a feature that
1639 enables them to keep running after exceptions that would
1639 enables them to keep running after exceptions that would
1640 otherwise kill their mainloop. This is a bother for IPython
1640 otherwise kill their mainloop. This is a bother for IPython
1641 which excepts to catch all of the program exceptions with a try:
1641 which excepts to catch all of the program exceptions with a try:
1642 except: statement.
1642 except: statement.
1643
1643
1644 Normally, IPython sets sys.excepthook to a CrashHandler instance, so if
1644 Normally, IPython sets sys.excepthook to a CrashHandler instance, so if
1645 any app directly invokes sys.excepthook, it will look to the user like
1645 any app directly invokes sys.excepthook, it will look to the user like
1646 IPython crashed. In order to work around this, we can disable the
1646 IPython crashed. In order to work around this, we can disable the
1647 CrashHandler and replace it with this excepthook instead, which prints a
1647 CrashHandler and replace it with this excepthook instead, which prints a
1648 regular traceback using our InteractiveTB. In this fashion, apps which
1648 regular traceback using our InteractiveTB. In this fashion, apps which
1649 call sys.excepthook will generate a regular-looking exception from
1649 call sys.excepthook will generate a regular-looking exception from
1650 IPython, and the CrashHandler will only be triggered by real IPython
1650 IPython, and the CrashHandler will only be triggered by real IPython
1651 crashes.
1651 crashes.
1652
1652
1653 This hook should be used sparingly, only in places which are not likely
1653 This hook should be used sparingly, only in places which are not likely
1654 to be true IPython errors.
1654 to be true IPython errors.
1655 """
1655 """
1656 self.showtraceback((etype,value,tb),tb_offset=0)
1656 self.showtraceback((etype,value,tb),tb_offset=0)
1657
1657
1658 def _get_exc_info(self, exc_tuple=None):
1658 def _get_exc_info(self, exc_tuple=None):
1659 """get exc_info from a given tuple, sys.exc_info() or sys.last_type etc.
1659 """get exc_info from a given tuple, sys.exc_info() or sys.last_type etc.
1660
1660
1661 Ensures sys.last_type,value,traceback hold the exc_info we found,
1661 Ensures sys.last_type,value,traceback hold the exc_info we found,
1662 from whichever source.
1662 from whichever source.
1663
1663
1664 raises ValueError if none of these contain any information
1664 raises ValueError if none of these contain any information
1665 """
1665 """
1666 if exc_tuple is None:
1666 if exc_tuple is None:
1667 etype, value, tb = sys.exc_info()
1667 etype, value, tb = sys.exc_info()
1668 else:
1668 else:
1669 etype, value, tb = exc_tuple
1669 etype, value, tb = exc_tuple
1670
1670
1671 if etype is None:
1671 if etype is None:
1672 if hasattr(sys, 'last_type'):
1672 if hasattr(sys, 'last_type'):
1673 etype, value, tb = sys.last_type, sys.last_value, \
1673 etype, value, tb = sys.last_type, sys.last_value, \
1674 sys.last_traceback
1674 sys.last_traceback
1675
1675
1676 if etype is None:
1676 if etype is None:
1677 raise ValueError("No exception to find")
1677 raise ValueError("No exception to find")
1678
1678
1679 # Now store the exception info in sys.last_type etc.
1679 # Now store the exception info in sys.last_type etc.
1680 # WARNING: these variables are somewhat deprecated and not
1680 # WARNING: these variables are somewhat deprecated and not
1681 # necessarily safe to use in a threaded environment, but tools
1681 # necessarily safe to use in a threaded environment, but tools
1682 # like pdb depend on their existence, so let's set them. If we
1682 # like pdb depend on their existence, so let's set them. If we
1683 # find problems in the field, we'll need to revisit their use.
1683 # find problems in the field, we'll need to revisit their use.
1684 sys.last_type = etype
1684 sys.last_type = etype
1685 sys.last_value = value
1685 sys.last_value = value
1686 sys.last_traceback = tb
1686 sys.last_traceback = tb
1687
1687
1688 return etype, value, tb
1688 return etype, value, tb
1689
1689
1690 def show_usage_error(self, exc):
1690 def show_usage_error(self, exc):
1691 """Show a short message for UsageErrors
1691 """Show a short message for UsageErrors
1692
1692
1693 These are special exceptions that shouldn't show a traceback.
1693 These are special exceptions that shouldn't show a traceback.
1694 """
1694 """
1695 self.write_err("UsageError: %s" % exc)
1695 self.write_err("UsageError: %s" % exc)
1696
1696
1697 def showtraceback(self,exc_tuple = None,filename=None,tb_offset=None,
1697 def showtraceback(self,exc_tuple = None,filename=None,tb_offset=None,
1698 exception_only=False):
1698 exception_only=False):
1699 """Display the exception that just occurred.
1699 """Display the exception that just occurred.
1700
1700
1701 If nothing is known about the exception, this is the method which
1701 If nothing is known about the exception, this is the method which
1702 should be used throughout the code for presenting user tracebacks,
1702 should be used throughout the code for presenting user tracebacks,
1703 rather than directly invoking the InteractiveTB object.
1703 rather than directly invoking the InteractiveTB object.
1704
1704
1705 A specific showsyntaxerror() also exists, but this method can take
1705 A specific showsyntaxerror() also exists, but this method can take
1706 care of calling it if needed, so unless you are explicitly catching a
1706 care of calling it if needed, so unless you are explicitly catching a
1707 SyntaxError exception, don't try to analyze the stack manually and
1707 SyntaxError exception, don't try to analyze the stack manually and
1708 simply call this method."""
1708 simply call this method."""
1709
1709
1710 try:
1710 try:
1711 try:
1711 try:
1712 etype, value, tb = self._get_exc_info(exc_tuple)
1712 etype, value, tb = self._get_exc_info(exc_tuple)
1713 except ValueError:
1713 except ValueError:
1714 self.write_err('No traceback available to show.\n')
1714 self.write_err('No traceback available to show.\n')
1715 return
1715 return
1716
1716
1717 if issubclass(etype, SyntaxError):
1717 if issubclass(etype, SyntaxError):
1718 # Though this won't be called by syntax errors in the input
1718 # Though this won't be called by syntax errors in the input
1719 # line, there may be SyntaxError cases with imported code.
1719 # line, there may be SyntaxError cases with imported code.
1720 self.showsyntaxerror(filename)
1720 self.showsyntaxerror(filename)
1721 elif etype is UsageError:
1721 elif etype is UsageError:
1722 self.show_usage_error(value)
1722 self.show_usage_error(value)
1723 else:
1723 else:
1724 if exception_only:
1724 if exception_only:
1725 stb = ['An exception has occurred, use %tb to see '
1725 stb = ['An exception has occurred, use %tb to see '
1726 'the full traceback.\n']
1726 'the full traceback.\n']
1727 stb.extend(self.InteractiveTB.get_exception_only(etype,
1727 stb.extend(self.InteractiveTB.get_exception_only(etype,
1728 value))
1728 value))
1729 else:
1729 else:
1730 try:
1730 try:
1731 # Exception classes can customise their traceback - we
1731 # Exception classes can customise their traceback - we
1732 # use this in IPython.parallel for exceptions occurring
1732 # use this in IPython.parallel for exceptions occurring
1733 # in the engines. This should return a list of strings.
1733 # in the engines. This should return a list of strings.
1734 stb = value._render_traceback_()
1734 stb = value._render_traceback_()
1735 except Exception:
1735 except Exception:
1736 stb = self.InteractiveTB.structured_traceback(etype,
1736 stb = self.InteractiveTB.structured_traceback(etype,
1737 value, tb, tb_offset=tb_offset)
1737 value, tb, tb_offset=tb_offset)
1738
1738
1739 self._showtraceback(etype, value, stb)
1739 self._showtraceback(etype, value, stb)
1740 if self.call_pdb:
1740 if self.call_pdb:
1741 # drop into debugger
1741 # drop into debugger
1742 self.debugger(force=True)
1742 self.debugger(force=True)
1743 return
1743 return
1744
1744
1745 # Actually show the traceback
1745 # Actually show the traceback
1746 self._showtraceback(etype, value, stb)
1746 self._showtraceback(etype, value, stb)
1747
1747
1748 except KeyboardInterrupt:
1748 except KeyboardInterrupt:
1749 self.write_err("\nKeyboardInterrupt\n")
1749 self.write_err("\nKeyboardInterrupt\n")
1750
1750
1751 def _showtraceback(self, etype, evalue, stb):
1751 def _showtraceback(self, etype, evalue, stb):
1752 """Actually show a traceback.
1752 """Actually show a traceback.
1753
1753
1754 Subclasses may override this method to put the traceback on a different
1754 Subclasses may override this method to put the traceback on a different
1755 place, like a side channel.
1755 place, like a side channel.
1756 """
1756 """
1757 print(self.InteractiveTB.stb2text(stb), file=io.stdout)
1757 print(self.InteractiveTB.stb2text(stb), file=io.stdout)
1758
1758
1759 def showsyntaxerror(self, filename=None):
1759 def showsyntaxerror(self, filename=None):
1760 """Display the syntax error that just occurred.
1760 """Display the syntax error that just occurred.
1761
1761
1762 This doesn't display a stack trace because there isn't one.
1762 This doesn't display a stack trace because there isn't one.
1763
1763
1764 If a filename is given, it is stuffed in the exception instead
1764 If a filename is given, it is stuffed in the exception instead
1765 of what was there before (because Python's parser always uses
1765 of what was there before (because Python's parser always uses
1766 "<string>" when reading from a string).
1766 "<string>" when reading from a string).
1767 """
1767 """
1768 etype, value, last_traceback = self._get_exc_info()
1768 etype, value, last_traceback = self._get_exc_info()
1769
1769
1770 if filename and issubclass(etype, SyntaxError):
1770 if filename and issubclass(etype, SyntaxError):
1771 try:
1771 try:
1772 value.filename = filename
1772 value.filename = filename
1773 except:
1773 except:
1774 # Not the format we expect; leave it alone
1774 # Not the format we expect; leave it alone
1775 pass
1775 pass
1776
1776
1777 stb = self.SyntaxTB.structured_traceback(etype, value, [])
1777 stb = self.SyntaxTB.structured_traceback(etype, value, [])
1778 self._showtraceback(etype, value, stb)
1778 self._showtraceback(etype, value, stb)
1779
1779
1780 # This is overridden in TerminalInteractiveShell to show a message about
1780 # This is overridden in TerminalInteractiveShell to show a message about
1781 # the %paste magic.
1781 # the %paste magic.
1782 def showindentationerror(self):
1782 def showindentationerror(self):
1783 """Called by run_cell when there's an IndentationError in code entered
1783 """Called by run_cell when there's an IndentationError in code entered
1784 at the prompt.
1784 at the prompt.
1785
1785
1786 This is overridden in TerminalInteractiveShell to show a message about
1786 This is overridden in TerminalInteractiveShell to show a message about
1787 the %paste magic."""
1787 the %paste magic."""
1788 self.showsyntaxerror()
1788 self.showsyntaxerror()
1789
1789
1790 #-------------------------------------------------------------------------
1790 #-------------------------------------------------------------------------
1791 # Things related to readline
1791 # Things related to readline
1792 #-------------------------------------------------------------------------
1792 #-------------------------------------------------------------------------
1793
1793
1794 def init_readline(self):
1794 def init_readline(self):
1795 """Command history completion/saving/reloading."""
1795 """Command history completion/saving/reloading."""
1796
1796
1797 if self.readline_use:
1797 if self.readline_use:
1798 import IPython.utils.rlineimpl as readline
1798 import IPython.utils.rlineimpl as readline
1799
1799
1800 self.rl_next_input = None
1800 self.rl_next_input = None
1801 self.rl_do_indent = False
1801 self.rl_do_indent = False
1802
1802
1803 if not self.readline_use or not readline.have_readline:
1803 if not self.readline_use or not readline.have_readline:
1804 self.has_readline = False
1804 self.has_readline = False
1805 self.readline = None
1805 self.readline = None
1806 # Set a number of methods that depend on readline to be no-op
1806 # Set a number of methods that depend on readline to be no-op
1807 self.readline_no_record = no_op_context
1807 self.readline_no_record = no_op_context
1808 self.set_readline_completer = no_op
1808 self.set_readline_completer = no_op
1809 self.set_custom_completer = no_op
1809 self.set_custom_completer = no_op
1810 if self.readline_use:
1810 if self.readline_use:
1811 warn('Readline services not available or not loaded.')
1811 warn('Readline services not available or not loaded.')
1812 else:
1812 else:
1813 self.has_readline = True
1813 self.has_readline = True
1814 self.readline = readline
1814 self.readline = readline
1815 sys.modules['readline'] = readline
1815 sys.modules['readline'] = readline
1816
1816
1817 # Platform-specific configuration
1817 # Platform-specific configuration
1818 if os.name == 'nt':
1818 if os.name == 'nt':
1819 # FIXME - check with Frederick to see if we can harmonize
1819 # FIXME - check with Frederick to see if we can harmonize
1820 # naming conventions with pyreadline to avoid this
1820 # naming conventions with pyreadline to avoid this
1821 # platform-dependent check
1821 # platform-dependent check
1822 self.readline_startup_hook = readline.set_pre_input_hook
1822 self.readline_startup_hook = readline.set_pre_input_hook
1823 else:
1823 else:
1824 self.readline_startup_hook = readline.set_startup_hook
1824 self.readline_startup_hook = readline.set_startup_hook
1825
1825
1826 # Load user's initrc file (readline config)
1826 # Load user's initrc file (readline config)
1827 # Or if libedit is used, load editrc.
1827 # Or if libedit is used, load editrc.
1828 inputrc_name = os.environ.get('INPUTRC')
1828 inputrc_name = os.environ.get('INPUTRC')
1829 if inputrc_name is None:
1829 if inputrc_name is None:
1830 inputrc_name = '.inputrc'
1830 inputrc_name = '.inputrc'
1831 if readline.uses_libedit:
1831 if readline.uses_libedit:
1832 inputrc_name = '.editrc'
1832 inputrc_name = '.editrc'
1833 inputrc_name = os.path.join(self.home_dir, inputrc_name)
1833 inputrc_name = os.path.join(self.home_dir, inputrc_name)
1834 if os.path.isfile(inputrc_name):
1834 if os.path.isfile(inputrc_name):
1835 try:
1835 try:
1836 readline.read_init_file(inputrc_name)
1836 readline.read_init_file(inputrc_name)
1837 except:
1837 except:
1838 warn('Problems reading readline initialization file <%s>'
1838 warn('Problems reading readline initialization file <%s>'
1839 % inputrc_name)
1839 % inputrc_name)
1840
1840
1841 # Configure readline according to user's prefs
1841 # Configure readline according to user's prefs
1842 # This is only done if GNU readline is being used. If libedit
1842 # This is only done if GNU readline is being used. If libedit
1843 # is being used (as on Leopard) the readline config is
1843 # is being used (as on Leopard) the readline config is
1844 # not run as the syntax for libedit is different.
1844 # not run as the syntax for libedit is different.
1845 if not readline.uses_libedit:
1845 if not readline.uses_libedit:
1846 for rlcommand in self.readline_parse_and_bind:
1846 for rlcommand in self.readline_parse_and_bind:
1847 #print "loading rl:",rlcommand # dbg
1847 #print "loading rl:",rlcommand # dbg
1848 readline.parse_and_bind(rlcommand)
1848 readline.parse_and_bind(rlcommand)
1849
1849
1850 # Remove some chars from the delimiters list. If we encounter
1850 # Remove some chars from the delimiters list. If we encounter
1851 # unicode chars, discard them.
1851 # unicode chars, discard them.
1852 delims = readline.get_completer_delims()
1852 delims = readline.get_completer_delims()
1853 if not py3compat.PY3:
1853 if not py3compat.PY3:
1854 delims = delims.encode("ascii", "ignore")
1854 delims = delims.encode("ascii", "ignore")
1855 for d in self.readline_remove_delims:
1855 for d in self.readline_remove_delims:
1856 delims = delims.replace(d, "")
1856 delims = delims.replace(d, "")
1857 delims = delims.replace(ESC_MAGIC, '')
1857 delims = delims.replace(ESC_MAGIC, '')
1858 readline.set_completer_delims(delims)
1858 readline.set_completer_delims(delims)
1859 # Store these so we can restore them if something like rpy2 modifies
1859 # Store these so we can restore them if something like rpy2 modifies
1860 # them.
1860 # them.
1861 self.readline_delims = delims
1861 self.readline_delims = delims
1862 # otherwise we end up with a monster history after a while:
1862 # otherwise we end up with a monster history after a while:
1863 readline.set_history_length(self.history_length)
1863 readline.set_history_length(self.history_length)
1864
1864
1865 self.refill_readline_hist()
1865 self.refill_readline_hist()
1866 self.readline_no_record = ReadlineNoRecord(self)
1866 self.readline_no_record = ReadlineNoRecord(self)
1867
1867
1868 # Configure auto-indent for all platforms
1868 # Configure auto-indent for all platforms
1869 self.set_autoindent(self.autoindent)
1869 self.set_autoindent(self.autoindent)
1870
1870
1871 def refill_readline_hist(self):
1871 def refill_readline_hist(self):
1872 # Load the last 1000 lines from history
1872 # Load the last 1000 lines from history
1873 self.readline.clear_history()
1873 self.readline.clear_history()
1874 stdin_encoding = sys.stdin.encoding or "utf-8"
1874 stdin_encoding = sys.stdin.encoding or "utf-8"
1875 last_cell = u""
1875 last_cell = u""
1876 for _, _, cell in self.history_manager.get_tail(1000,
1876 for _, _, cell in self.history_manager.get_tail(1000,
1877 include_latest=True):
1877 include_latest=True):
1878 # Ignore blank lines and consecutive duplicates
1878 # Ignore blank lines and consecutive duplicates
1879 cell = cell.rstrip()
1879 cell = cell.rstrip()
1880 if cell and (cell != last_cell):
1880 if cell and (cell != last_cell):
1881 try:
1881 try:
1882 if self.multiline_history:
1882 if self.multiline_history:
1883 self.readline.add_history(py3compat.unicode_to_str(cell,
1883 self.readline.add_history(py3compat.unicode_to_str(cell,
1884 stdin_encoding))
1884 stdin_encoding))
1885 else:
1885 else:
1886 for line in cell.splitlines():
1886 for line in cell.splitlines():
1887 self.readline.add_history(py3compat.unicode_to_str(line,
1887 self.readline.add_history(py3compat.unicode_to_str(line,
1888 stdin_encoding))
1888 stdin_encoding))
1889 last_cell = cell
1889 last_cell = cell
1890
1890
1891 except TypeError:
1891 except TypeError:
1892 # The history DB can get corrupted so it returns strings
1892 # The history DB can get corrupted so it returns strings
1893 # containing null bytes, which readline objects to.
1893 # containing null bytes, which readline objects to.
1894 continue
1894 continue
1895
1895
1896 @skip_doctest
1896 @skip_doctest
1897 def set_next_input(self, s):
1897 def set_next_input(self, s):
1898 """ Sets the 'default' input string for the next command line.
1898 """ Sets the 'default' input string for the next command line.
1899
1899
1900 Requires readline.
1900 Requires readline.
1901
1901
1902 Example::
1902 Example::
1903
1903
1904 In [1]: _ip.set_next_input("Hello Word")
1904 In [1]: _ip.set_next_input("Hello Word")
1905 In [2]: Hello Word_ # cursor is here
1905 In [2]: Hello Word_ # cursor is here
1906 """
1906 """
1907 self.rl_next_input = py3compat.cast_bytes_py2(s)
1907 self.rl_next_input = py3compat.cast_bytes_py2(s)
1908
1908
1909 # Maybe move this to the terminal subclass?
1909 # Maybe move this to the terminal subclass?
1910 def pre_readline(self):
1910 def pre_readline(self):
1911 """readline hook to be used at the start of each line.
1911 """readline hook to be used at the start of each line.
1912
1912
1913 Currently it handles auto-indent only."""
1913 Currently it handles auto-indent only."""
1914
1914
1915 if self.rl_do_indent:
1915 if self.rl_do_indent:
1916 self.readline.insert_text(self._indent_current_str())
1916 self.readline.insert_text(self._indent_current_str())
1917 if self.rl_next_input is not None:
1917 if self.rl_next_input is not None:
1918 self.readline.insert_text(self.rl_next_input)
1918 self.readline.insert_text(self.rl_next_input)
1919 self.rl_next_input = None
1919 self.rl_next_input = None
1920
1920
1921 def _indent_current_str(self):
1921 def _indent_current_str(self):
1922 """return the current level of indentation as a string"""
1922 """return the current level of indentation as a string"""
1923 return self.input_splitter.indent_spaces * ' '
1923 return self.input_splitter.indent_spaces * ' '
1924
1924
1925 #-------------------------------------------------------------------------
1925 #-------------------------------------------------------------------------
1926 # Things related to text completion
1926 # Things related to text completion
1927 #-------------------------------------------------------------------------
1927 #-------------------------------------------------------------------------
1928
1928
1929 def init_completer(self):
1929 def init_completer(self):
1930 """Initialize the completion machinery.
1930 """Initialize the completion machinery.
1931
1931
1932 This creates completion machinery that can be used by client code,
1932 This creates completion machinery that can be used by client code,
1933 either interactively in-process (typically triggered by the readline
1933 either interactively in-process (typically triggered by the readline
1934 library), programatically (such as in test suites) or out-of-prcess
1934 library), programatically (such as in test suites) or out-of-prcess
1935 (typically over the network by remote frontends).
1935 (typically over the network by remote frontends).
1936 """
1936 """
1937 from IPython.core.completer import IPCompleter
1937 from IPython.core.completer import IPCompleter
1938 from IPython.core.completerlib import (module_completer,
1938 from IPython.core.completerlib import (module_completer,
1939 magic_run_completer, cd_completer, reset_completer)
1939 magic_run_completer, cd_completer, reset_completer)
1940
1940
1941 self.Completer = IPCompleter(shell=self,
1941 self.Completer = IPCompleter(shell=self,
1942 namespace=self.user_ns,
1942 namespace=self.user_ns,
1943 global_namespace=self.user_global_ns,
1943 global_namespace=self.user_global_ns,
1944 use_readline=self.has_readline,
1944 use_readline=self.has_readline,
1945 parent=self,
1945 parent=self,
1946 )
1946 )
1947 self.configurables.append(self.Completer)
1947 self.configurables.append(self.Completer)
1948
1948
1949 # Add custom completers to the basic ones built into IPCompleter
1949 # Add custom completers to the basic ones built into IPCompleter
1950 sdisp = self.strdispatchers.get('complete_command', StrDispatch())
1950 sdisp = self.strdispatchers.get('complete_command', StrDispatch())
1951 self.strdispatchers['complete_command'] = sdisp
1951 self.strdispatchers['complete_command'] = sdisp
1952 self.Completer.custom_completers = sdisp
1952 self.Completer.custom_completers = sdisp
1953
1953
1954 self.set_hook('complete_command', module_completer, str_key = 'import')
1954 self.set_hook('complete_command', module_completer, str_key = 'import')
1955 self.set_hook('complete_command', module_completer, str_key = 'from')
1955 self.set_hook('complete_command', module_completer, str_key = 'from')
1956 self.set_hook('complete_command', magic_run_completer, str_key = '%run')
1956 self.set_hook('complete_command', magic_run_completer, str_key = '%run')
1957 self.set_hook('complete_command', cd_completer, str_key = '%cd')
1957 self.set_hook('complete_command', cd_completer, str_key = '%cd')
1958 self.set_hook('complete_command', reset_completer, str_key = '%reset')
1958 self.set_hook('complete_command', reset_completer, str_key = '%reset')
1959
1959
1960 # Only configure readline if we truly are using readline. IPython can
1960 # Only configure readline if we truly are using readline. IPython can
1961 # do tab-completion over the network, in GUIs, etc, where readline
1961 # do tab-completion over the network, in GUIs, etc, where readline
1962 # itself may be absent
1962 # itself may be absent
1963 if self.has_readline:
1963 if self.has_readline:
1964 self.set_readline_completer()
1964 self.set_readline_completer()
1965
1965
1966 def complete(self, text, line=None, cursor_pos=None):
1966 def complete(self, text, line=None, cursor_pos=None):
1967 """Return the completed text and a list of completions.
1967 """Return the completed text and a list of completions.
1968
1968
1969 Parameters
1969 Parameters
1970 ----------
1970 ----------
1971
1971
1972 text : string
1972 text : string
1973 A string of text to be completed on. It can be given as empty and
1973 A string of text to be completed on. It can be given as empty and
1974 instead a line/position pair are given. In this case, the
1974 instead a line/position pair are given. In this case, the
1975 completer itself will split the line like readline does.
1975 completer itself will split the line like readline does.
1976
1976
1977 line : string, optional
1977 line : string, optional
1978 The complete line that text is part of.
1978 The complete line that text is part of.
1979
1979
1980 cursor_pos : int, optional
1980 cursor_pos : int, optional
1981 The position of the cursor on the input line.
1981 The position of the cursor on the input line.
1982
1982
1983 Returns
1983 Returns
1984 -------
1984 -------
1985 text : string
1985 text : string
1986 The actual text that was completed.
1986 The actual text that was completed.
1987
1987
1988 matches : list
1988 matches : list
1989 A sorted list with all possible completions.
1989 A sorted list with all possible completions.
1990
1990
1991 The optional arguments allow the completion to take more context into
1991 The optional arguments allow the completion to take more context into
1992 account, and are part of the low-level completion API.
1992 account, and are part of the low-level completion API.
1993
1993
1994 This is a wrapper around the completion mechanism, similar to what
1994 This is a wrapper around the completion mechanism, similar to what
1995 readline does at the command line when the TAB key is hit. By
1995 readline does at the command line when the TAB key is hit. By
1996 exposing it as a method, it can be used by other non-readline
1996 exposing it as a method, it can be used by other non-readline
1997 environments (such as GUIs) for text completion.
1997 environments (such as GUIs) for text completion.
1998
1998
1999 Simple usage example:
1999 Simple usage example:
2000
2000
2001 In [1]: x = 'hello'
2001 In [1]: x = 'hello'
2002
2002
2003 In [2]: _ip.complete('x.l')
2003 In [2]: _ip.complete('x.l')
2004 Out[2]: ('x.l', ['x.ljust', 'x.lower', 'x.lstrip'])
2004 Out[2]: ('x.l', ['x.ljust', 'x.lower', 'x.lstrip'])
2005 """
2005 """
2006
2006
2007 # Inject names into __builtin__ so we can complete on the added names.
2007 # Inject names into __builtin__ so we can complete on the added names.
2008 with self.builtin_trap:
2008 with self.builtin_trap:
2009 return self.Completer.complete(text, line, cursor_pos)
2009 return self.Completer.complete(text, line, cursor_pos)
2010
2010
2011 def set_custom_completer(self, completer, pos=0):
2011 def set_custom_completer(self, completer, pos=0):
2012 """Adds a new custom completer function.
2012 """Adds a new custom completer function.
2013
2013
2014 The position argument (defaults to 0) is the index in the completers
2014 The position argument (defaults to 0) is the index in the completers
2015 list where you want the completer to be inserted."""
2015 list where you want the completer to be inserted."""
2016
2016
2017 newcomp = types.MethodType(completer,self.Completer)
2017 newcomp = types.MethodType(completer,self.Completer)
2018 self.Completer.matchers.insert(pos,newcomp)
2018 self.Completer.matchers.insert(pos,newcomp)
2019
2019
2020 def set_readline_completer(self):
2020 def set_readline_completer(self):
2021 """Reset readline's completer to be our own."""
2021 """Reset readline's completer to be our own."""
2022 self.readline.set_completer(self.Completer.rlcomplete)
2022 self.readline.set_completer(self.Completer.rlcomplete)
2023
2023
2024 def set_completer_frame(self, frame=None):
2024 def set_completer_frame(self, frame=None):
2025 """Set the frame of the completer."""
2025 """Set the frame of the completer."""
2026 if frame:
2026 if frame:
2027 self.Completer.namespace = frame.f_locals
2027 self.Completer.namespace = frame.f_locals
2028 self.Completer.global_namespace = frame.f_globals
2028 self.Completer.global_namespace = frame.f_globals
2029 else:
2029 else:
2030 self.Completer.namespace = self.user_ns
2030 self.Completer.namespace = self.user_ns
2031 self.Completer.global_namespace = self.user_global_ns
2031 self.Completer.global_namespace = self.user_global_ns
2032
2032
2033 #-------------------------------------------------------------------------
2033 #-------------------------------------------------------------------------
2034 # Things related to magics
2034 # Things related to magics
2035 #-------------------------------------------------------------------------
2035 #-------------------------------------------------------------------------
2036
2036
2037 def init_magics(self):
2037 def init_magics(self):
2038 from IPython.core import magics as m
2038 from IPython.core import magics as m
2039 self.magics_manager = magic.MagicsManager(shell=self,
2039 self.magics_manager = magic.MagicsManager(shell=self,
2040 parent=self,
2040 parent=self,
2041 user_magics=m.UserMagics(self))
2041 user_magics=m.UserMagics(self))
2042 self.configurables.append(self.magics_manager)
2042 self.configurables.append(self.magics_manager)
2043
2043
2044 # Expose as public API from the magics manager
2044 # Expose as public API from the magics manager
2045 self.register_magics = self.magics_manager.register
2045 self.register_magics = self.magics_manager.register
2046 self.define_magic = self.magics_manager.define_magic
2046 self.define_magic = self.magics_manager.define_magic
2047
2047
2048 self.register_magics(m.AutoMagics, m.BasicMagics, m.CodeMagics,
2048 self.register_magics(m.AutoMagics, m.BasicMagics, m.CodeMagics,
2049 m.ConfigMagics, m.DeprecatedMagics, m.DisplayMagics, m.ExecutionMagics,
2049 m.ConfigMagics, m.DeprecatedMagics, m.DisplayMagics, m.ExecutionMagics,
2050 m.ExtensionMagics, m.HistoryMagics, m.LoggingMagics,
2050 m.ExtensionMagics, m.HistoryMagics, m.LoggingMagics,
2051 m.NamespaceMagics, m.OSMagics, m.PylabMagics, m.ScriptMagics,
2051 m.NamespaceMagics, m.OSMagics, m.PylabMagics, m.ScriptMagics,
2052 )
2052 )
2053
2053
2054 # Register Magic Aliases
2054 # Register Magic Aliases
2055 mman = self.magics_manager
2055 mman = self.magics_manager
2056 # FIXME: magic aliases should be defined by the Magics classes
2056 # FIXME: magic aliases should be defined by the Magics classes
2057 # or in MagicsManager, not here
2057 # or in MagicsManager, not here
2058 mman.register_alias('ed', 'edit')
2058 mman.register_alias('ed', 'edit')
2059 mman.register_alias('hist', 'history')
2059 mman.register_alias('hist', 'history')
2060 mman.register_alias('rep', 'recall')
2060 mman.register_alias('rep', 'recall')
2061 mman.register_alias('SVG', 'svg', 'cell')
2061 mman.register_alias('SVG', 'svg', 'cell')
2062 mman.register_alias('HTML', 'html', 'cell')
2062 mman.register_alias('HTML', 'html', 'cell')
2063 mman.register_alias('file', 'writefile', 'cell')
2063 mman.register_alias('file', 'writefile', 'cell')
2064
2064
2065 # FIXME: Move the color initialization to the DisplayHook, which
2065 # FIXME: Move the color initialization to the DisplayHook, which
2066 # should be split into a prompt manager and displayhook. We probably
2066 # should be split into a prompt manager and displayhook. We probably
2067 # even need a centralize colors management object.
2067 # even need a centralize colors management object.
2068 self.magic('colors %s' % self.colors)
2068 self.magic('colors %s' % self.colors)
2069
2069
2070 # Defined here so that it's included in the documentation
2070 # Defined here so that it's included in the documentation
2071 @functools.wraps(magic.MagicsManager.register_function)
2071 @functools.wraps(magic.MagicsManager.register_function)
2072 def register_magic_function(self, func, magic_kind='line', magic_name=None):
2072 def register_magic_function(self, func, magic_kind='line', magic_name=None):
2073 self.magics_manager.register_function(func,
2073 self.magics_manager.register_function(func,
2074 magic_kind=magic_kind, magic_name=magic_name)
2074 magic_kind=magic_kind, magic_name=magic_name)
2075
2075
2076 def run_line_magic(self, magic_name, line):
2076 def run_line_magic(self, magic_name, line):
2077 """Execute the given line magic.
2077 """Execute the given line magic.
2078
2078
2079 Parameters
2079 Parameters
2080 ----------
2080 ----------
2081 magic_name : str
2081 magic_name : str
2082 Name of the desired magic function, without '%' prefix.
2082 Name of the desired magic function, without '%' prefix.
2083
2083
2084 line : str
2084 line : str
2085 The rest of the input line as a single string.
2085 The rest of the input line as a single string.
2086 """
2086 """
2087 fn = self.find_line_magic(magic_name)
2087 fn = self.find_line_magic(magic_name)
2088 if fn is None:
2088 if fn is None:
2089 cm = self.find_cell_magic(magic_name)
2089 cm = self.find_cell_magic(magic_name)
2090 etpl = "Line magic function `%%%s` not found%s."
2090 etpl = "Line magic function `%%%s` not found%s."
2091 extra = '' if cm is None else (' (But cell magic `%%%%%s` exists, '
2091 extra = '' if cm is None else (' (But cell magic `%%%%%s` exists, '
2092 'did you mean that instead?)' % magic_name )
2092 'did you mean that instead?)' % magic_name )
2093 error(etpl % (magic_name, extra))
2093 error(etpl % (magic_name, extra))
2094 else:
2094 else:
2095 # Note: this is the distance in the stack to the user's frame.
2095 # Note: this is the distance in the stack to the user's frame.
2096 # This will need to be updated if the internal calling logic gets
2096 # This will need to be updated if the internal calling logic gets
2097 # refactored, or else we'll be expanding the wrong variables.
2097 # refactored, or else we'll be expanding the wrong variables.
2098 stack_depth = 2
2098 stack_depth = 2
2099 magic_arg_s = self.var_expand(line, stack_depth)
2099 magic_arg_s = self.var_expand(line, stack_depth)
2100 # Put magic args in a list so we can call with f(*a) syntax
2100 # Put magic args in a list so we can call with f(*a) syntax
2101 args = [magic_arg_s]
2101 args = [magic_arg_s]
2102 kwargs = {}
2102 kwargs = {}
2103 # Grab local namespace if we need it:
2103 # Grab local namespace if we need it:
2104 if getattr(fn, "needs_local_scope", False):
2104 if getattr(fn, "needs_local_scope", False):
2105 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2105 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2106 with self.builtin_trap:
2106 with self.builtin_trap:
2107 result = fn(*args,**kwargs)
2107 result = fn(*args,**kwargs)
2108 return result
2108 return result
2109
2109
2110 def run_cell_magic(self, magic_name, line, cell):
2110 def run_cell_magic(self, magic_name, line, cell):
2111 """Execute the given cell magic.
2111 """Execute the given cell magic.
2112
2112
2113 Parameters
2113 Parameters
2114 ----------
2114 ----------
2115 magic_name : str
2115 magic_name : str
2116 Name of the desired magic function, without '%' prefix.
2116 Name of the desired magic function, without '%' prefix.
2117
2117
2118 line : str
2118 line : str
2119 The rest of the first input line as a single string.
2119 The rest of the first input line as a single string.
2120
2120
2121 cell : str
2121 cell : str
2122 The body of the cell as a (possibly multiline) string.
2122 The body of the cell as a (possibly multiline) string.
2123 """
2123 """
2124 fn = self.find_cell_magic(magic_name)
2124 fn = self.find_cell_magic(magic_name)
2125 if fn is None:
2125 if fn is None:
2126 lm = self.find_line_magic(magic_name)
2126 lm = self.find_line_magic(magic_name)
2127 etpl = "Cell magic `%%{0}` not found{1}."
2127 etpl = "Cell magic `%%{0}` not found{1}."
2128 extra = '' if lm is None else (' (But line magic `%{0}` exists, '
2128 extra = '' if lm is None else (' (But line magic `%{0}` exists, '
2129 'did you mean that instead?)'.format(magic_name))
2129 'did you mean that instead?)'.format(magic_name))
2130 error(etpl.format(magic_name, extra))
2130 error(etpl.format(magic_name, extra))
2131 elif cell == '':
2131 elif cell == '':
2132 message = '%%{0} is a cell magic, but the cell body is empty.'.format(magic_name)
2132 message = '%%{0} is a cell magic, but the cell body is empty.'.format(magic_name)
2133 if self.find_line_magic(magic_name) is not None:
2133 if self.find_line_magic(magic_name) is not None:
2134 message += ' Did you mean the line magic %{0} (single %)?'.format(magic_name)
2134 message += ' Did you mean the line magic %{0} (single %)?'.format(magic_name)
2135 raise UsageError(message)
2135 raise UsageError(message)
2136 else:
2136 else:
2137 # Note: this is the distance in the stack to the user's frame.
2137 # Note: this is the distance in the stack to the user's frame.
2138 # This will need to be updated if the internal calling logic gets
2138 # This will need to be updated if the internal calling logic gets
2139 # refactored, or else we'll be expanding the wrong variables.
2139 # refactored, or else we'll be expanding the wrong variables.
2140 stack_depth = 2
2140 stack_depth = 2
2141 magic_arg_s = self.var_expand(line, stack_depth)
2141 magic_arg_s = self.var_expand(line, stack_depth)
2142 with self.builtin_trap:
2142 with self.builtin_trap:
2143 result = fn(magic_arg_s, cell)
2143 result = fn(magic_arg_s, cell)
2144 return result
2144 return result
2145
2145
2146 def find_line_magic(self, magic_name):
2146 def find_line_magic(self, magic_name):
2147 """Find and return a line magic by name.
2147 """Find and return a line magic by name.
2148
2148
2149 Returns None if the magic isn't found."""
2149 Returns None if the magic isn't found."""
2150 return self.magics_manager.magics['line'].get(magic_name)
2150 return self.magics_manager.magics['line'].get(magic_name)
2151
2151
2152 def find_cell_magic(self, magic_name):
2152 def find_cell_magic(self, magic_name):
2153 """Find and return a cell magic by name.
2153 """Find and return a cell magic by name.
2154
2154
2155 Returns None if the magic isn't found."""
2155 Returns None if the magic isn't found."""
2156 return self.magics_manager.magics['cell'].get(magic_name)
2156 return self.magics_manager.magics['cell'].get(magic_name)
2157
2157
2158 def find_magic(self, magic_name, magic_kind='line'):
2158 def find_magic(self, magic_name, magic_kind='line'):
2159 """Find and return a magic of the given type by name.
2159 """Find and return a magic of the given type by name.
2160
2160
2161 Returns None if the magic isn't found."""
2161 Returns None if the magic isn't found."""
2162 return self.magics_manager.magics[magic_kind].get(magic_name)
2162 return self.magics_manager.magics[magic_kind].get(magic_name)
2163
2163
2164 def magic(self, arg_s):
2164 def magic(self, arg_s):
2165 """DEPRECATED. Use run_line_magic() instead.
2165 """DEPRECATED. Use run_line_magic() instead.
2166
2166
2167 Call a magic function by name.
2167 Call a magic function by name.
2168
2168
2169 Input: a string containing the name of the magic function to call and
2169 Input: a string containing the name of the magic function to call and
2170 any additional arguments to be passed to the magic.
2170 any additional arguments to be passed to the magic.
2171
2171
2172 magic('name -opt foo bar') is equivalent to typing at the ipython
2172 magic('name -opt foo bar') is equivalent to typing at the ipython
2173 prompt:
2173 prompt:
2174
2174
2175 In[1]: %name -opt foo bar
2175 In[1]: %name -opt foo bar
2176
2176
2177 To call a magic without arguments, simply use magic('name').
2177 To call a magic without arguments, simply use magic('name').
2178
2178
2179 This provides a proper Python function to call IPython's magics in any
2179 This provides a proper Python function to call IPython's magics in any
2180 valid Python code you can type at the interpreter, including loops and
2180 valid Python code you can type at the interpreter, including loops and
2181 compound statements.
2181 compound statements.
2182 """
2182 """
2183 # TODO: should we issue a loud deprecation warning here?
2183 # TODO: should we issue a loud deprecation warning here?
2184 magic_name, _, magic_arg_s = arg_s.partition(' ')
2184 magic_name, _, magic_arg_s = arg_s.partition(' ')
2185 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
2185 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
2186 return self.run_line_magic(magic_name, magic_arg_s)
2186 return self.run_line_magic(magic_name, magic_arg_s)
2187
2187
2188 #-------------------------------------------------------------------------
2188 #-------------------------------------------------------------------------
2189 # Things related to macros
2189 # Things related to macros
2190 #-------------------------------------------------------------------------
2190 #-------------------------------------------------------------------------
2191
2191
2192 def define_macro(self, name, themacro):
2192 def define_macro(self, name, themacro):
2193 """Define a new macro
2193 """Define a new macro
2194
2194
2195 Parameters
2195 Parameters
2196 ----------
2196 ----------
2197 name : str
2197 name : str
2198 The name of the macro.
2198 The name of the macro.
2199 themacro : str or Macro
2199 themacro : str or Macro
2200 The action to do upon invoking the macro. If a string, a new
2200 The action to do upon invoking the macro. If a string, a new
2201 Macro object is created by passing the string to it.
2201 Macro object is created by passing the string to it.
2202 """
2202 """
2203
2203
2204 from IPython.core import macro
2204 from IPython.core import macro
2205
2205
2206 if isinstance(themacro, string_types):
2206 if isinstance(themacro, string_types):
2207 themacro = macro.Macro(themacro)
2207 themacro = macro.Macro(themacro)
2208 if not isinstance(themacro, macro.Macro):
2208 if not isinstance(themacro, macro.Macro):
2209 raise ValueError('A macro must be a string or a Macro instance.')
2209 raise ValueError('A macro must be a string or a Macro instance.')
2210 self.user_ns[name] = themacro
2210 self.user_ns[name] = themacro
2211
2211
2212 #-------------------------------------------------------------------------
2212 #-------------------------------------------------------------------------
2213 # Things related to the running of system commands
2213 # Things related to the running of system commands
2214 #-------------------------------------------------------------------------
2214 #-------------------------------------------------------------------------
2215
2215
2216 def system_piped(self, cmd):
2216 def system_piped(self, cmd):
2217 """Call the given cmd in a subprocess, piping stdout/err
2217 """Call the given cmd in a subprocess, piping stdout/err
2218
2218
2219 Parameters
2219 Parameters
2220 ----------
2220 ----------
2221 cmd : str
2221 cmd : str
2222 Command to execute (can not end in '&', as background processes are
2222 Command to execute (can not end in '&', as background processes are
2223 not supported. Should not be a command that expects input
2223 not supported. Should not be a command that expects input
2224 other than simple text.
2224 other than simple text.
2225 """
2225 """
2226 if cmd.rstrip().endswith('&'):
2226 if cmd.rstrip().endswith('&'):
2227 # this is *far* from a rigorous test
2227 # this is *far* from a rigorous test
2228 # We do not support backgrounding processes because we either use
2228 # We do not support backgrounding processes because we either use
2229 # pexpect or pipes to read from. Users can always just call
2229 # pexpect or pipes to read from. Users can always just call
2230 # os.system() or use ip.system=ip.system_raw
2230 # os.system() or use ip.system=ip.system_raw
2231 # if they really want a background process.
2231 # if they really want a background process.
2232 raise OSError("Background processes not supported.")
2232 raise OSError("Background processes not supported.")
2233
2233
2234 # we explicitly do NOT return the subprocess status code, because
2234 # we explicitly do NOT return the subprocess status code, because
2235 # a non-None value would trigger :func:`sys.displayhook` calls.
2235 # a non-None value would trigger :func:`sys.displayhook` calls.
2236 # Instead, we store the exit_code in user_ns.
2236 # Instead, we store the exit_code in user_ns.
2237 self.user_ns['_exit_code'] = system(self.var_expand(cmd, depth=1))
2237 self.user_ns['_exit_code'] = system(self.var_expand(cmd, depth=1))
2238
2238
2239 def system_raw(self, cmd):
2239 def system_raw(self, cmd):
2240 """Call the given cmd in a subprocess using os.system on Windows or
2240 """Call the given cmd in a subprocess using os.system on Windows or
2241 subprocess.call using the system shell on other platforms.
2241 subprocess.call using the system shell on other platforms.
2242
2242
2243 Parameters
2243 Parameters
2244 ----------
2244 ----------
2245 cmd : str
2245 cmd : str
2246 Command to execute.
2246 Command to execute.
2247 """
2247 """
2248 cmd = self.var_expand(cmd, depth=1)
2248 cmd = self.var_expand(cmd, depth=1)
2249 # protect os.system from UNC paths on Windows, which it can't handle:
2249 # protect os.system from UNC paths on Windows, which it can't handle:
2250 if sys.platform == 'win32':
2250 if sys.platform == 'win32':
2251 from IPython.utils._process_win32 import AvoidUNCPath
2251 from IPython.utils._process_win32 import AvoidUNCPath
2252 with AvoidUNCPath() as path:
2252 with AvoidUNCPath() as path:
2253 if path is not None:
2253 if path is not None:
2254 cmd = '"pushd %s &&"%s' % (path, cmd)
2254 cmd = '"pushd %s &&"%s' % (path, cmd)
2255 cmd = py3compat.unicode_to_str(cmd)
2255 cmd = py3compat.unicode_to_str(cmd)
2256 ec = os.system(cmd)
2256 ec = os.system(cmd)
2257 else:
2257 else:
2258 cmd = py3compat.unicode_to_str(cmd)
2258 cmd = py3compat.unicode_to_str(cmd)
2259 # Call the cmd using the OS shell, instead of the default /bin/sh, if set.
2259 # Call the cmd using the OS shell, instead of the default /bin/sh, if set.
2260 ec = subprocess.call(cmd, shell=True, executable=os.environ.get('SHELL', None))
2260 ec = subprocess.call(cmd, shell=True, executable=os.environ.get('SHELL', None))
2261 # exit code is positive for program failure, or negative for
2261 # exit code is positive for program failure, or negative for
2262 # terminating signal number.
2262 # terminating signal number.
2263
2263
2264 # We explicitly do NOT return the subprocess status code, because
2264 # We explicitly do NOT return the subprocess status code, because
2265 # a non-None value would trigger :func:`sys.displayhook` calls.
2265 # a non-None value would trigger :func:`sys.displayhook` calls.
2266 # Instead, we store the exit_code in user_ns.
2266 # Instead, we store the exit_code in user_ns.
2267 self.user_ns['_exit_code'] = ec
2267 self.user_ns['_exit_code'] = ec
2268
2268
2269 # use piped system by default, because it is better behaved
2269 # use piped system by default, because it is better behaved
2270 system = system_piped
2270 system = system_piped
2271
2271
2272 def getoutput(self, cmd, split=True, depth=0):
2272 def getoutput(self, cmd, split=True, depth=0):
2273 """Get output (possibly including stderr) from a subprocess.
2273 """Get output (possibly including stderr) from a subprocess.
2274
2274
2275 Parameters
2275 Parameters
2276 ----------
2276 ----------
2277 cmd : str
2277 cmd : str
2278 Command to execute (can not end in '&', as background processes are
2278 Command to execute (can not end in '&', as background processes are
2279 not supported.
2279 not supported.
2280 split : bool, optional
2280 split : bool, optional
2281 If True, split the output into an IPython SList. Otherwise, an
2281 If True, split the output into an IPython SList. Otherwise, an
2282 IPython LSString is returned. These are objects similar to normal
2282 IPython LSString is returned. These are objects similar to normal
2283 lists and strings, with a few convenience attributes for easier
2283 lists and strings, with a few convenience attributes for easier
2284 manipulation of line-based output. You can use '?' on them for
2284 manipulation of line-based output. You can use '?' on them for
2285 details.
2285 details.
2286 depth : int, optional
2286 depth : int, optional
2287 How many frames above the caller are the local variables which should
2287 How many frames above the caller are the local variables which should
2288 be expanded in the command string? The default (0) assumes that the
2288 be expanded in the command string? The default (0) assumes that the
2289 expansion variables are in the stack frame calling this function.
2289 expansion variables are in the stack frame calling this function.
2290 """
2290 """
2291 if cmd.rstrip().endswith('&'):
2291 if cmd.rstrip().endswith('&'):
2292 # this is *far* from a rigorous test
2292 # this is *far* from a rigorous test
2293 raise OSError("Background processes not supported.")
2293 raise OSError("Background processes not supported.")
2294 out = getoutput(self.var_expand(cmd, depth=depth+1))
2294 out = getoutput(self.var_expand(cmd, depth=depth+1))
2295 if split:
2295 if split:
2296 out = SList(out.splitlines())
2296 out = SList(out.splitlines())
2297 else:
2297 else:
2298 out = LSString(out)
2298 out = LSString(out)
2299 return out
2299 return out
2300
2300
2301 #-------------------------------------------------------------------------
2301 #-------------------------------------------------------------------------
2302 # Things related to aliases
2302 # Things related to aliases
2303 #-------------------------------------------------------------------------
2303 #-------------------------------------------------------------------------
2304
2304
2305 def init_alias(self):
2305 def init_alias(self):
2306 self.alias_manager = AliasManager(shell=self, parent=self)
2306 self.alias_manager = AliasManager(shell=self, parent=self)
2307 self.configurables.append(self.alias_manager)
2307 self.configurables.append(self.alias_manager)
2308
2308
2309 #-------------------------------------------------------------------------
2309 #-------------------------------------------------------------------------
2310 # Things related to extensions
2310 # Things related to extensions
2311 #-------------------------------------------------------------------------
2311 #-------------------------------------------------------------------------
2312
2312
2313 def init_extension_manager(self):
2313 def init_extension_manager(self):
2314 self.extension_manager = ExtensionManager(shell=self, parent=self)
2314 self.extension_manager = ExtensionManager(shell=self, parent=self)
2315 self.configurables.append(self.extension_manager)
2315 self.configurables.append(self.extension_manager)
2316
2316
2317 #-------------------------------------------------------------------------
2317 #-------------------------------------------------------------------------
2318 # Things related to payloads
2318 # Things related to payloads
2319 #-------------------------------------------------------------------------
2319 #-------------------------------------------------------------------------
2320
2320
2321 def init_payload(self):
2321 def init_payload(self):
2322 self.payload_manager = PayloadManager(parent=self)
2322 self.payload_manager = PayloadManager(parent=self)
2323 self.configurables.append(self.payload_manager)
2323 self.configurables.append(self.payload_manager)
2324
2324
2325 #-------------------------------------------------------------------------
2325 #-------------------------------------------------------------------------
2326 # Things related to widgets
2326 # Things related to widgets
2327 #-------------------------------------------------------------------------
2327 #-------------------------------------------------------------------------
2328
2328
2329 def init_comms(self):
2329 def init_comms(self):
2330 # not implemented in the base class
2330 # not implemented in the base class
2331 pass
2331 pass
2332
2332
2333 #-------------------------------------------------------------------------
2333 #-------------------------------------------------------------------------
2334 # Things related to the prefilter
2334 # Things related to the prefilter
2335 #-------------------------------------------------------------------------
2335 #-------------------------------------------------------------------------
2336
2336
2337 def init_prefilter(self):
2337 def init_prefilter(self):
2338 self.prefilter_manager = PrefilterManager(shell=self, parent=self)
2338 self.prefilter_manager = PrefilterManager(shell=self, parent=self)
2339 self.configurables.append(self.prefilter_manager)
2339 self.configurables.append(self.prefilter_manager)
2340 # Ultimately this will be refactored in the new interpreter code, but
2340 # Ultimately this will be refactored in the new interpreter code, but
2341 # for now, we should expose the main prefilter method (there's legacy
2341 # for now, we should expose the main prefilter method (there's legacy
2342 # code out there that may rely on this).
2342 # code out there that may rely on this).
2343 self.prefilter = self.prefilter_manager.prefilter_lines
2343 self.prefilter = self.prefilter_manager.prefilter_lines
2344
2344
2345 def auto_rewrite_input(self, cmd):
2345 def auto_rewrite_input(self, cmd):
2346 """Print to the screen the rewritten form of the user's command.
2346 """Print to the screen the rewritten form of the user's command.
2347
2347
2348 This shows visual feedback by rewriting input lines that cause
2348 This shows visual feedback by rewriting input lines that cause
2349 automatic calling to kick in, like::
2349 automatic calling to kick in, like::
2350
2350
2351 /f x
2351 /f x
2352
2352
2353 into::
2353 into::
2354
2354
2355 ------> f(x)
2355 ------> f(x)
2356
2356
2357 after the user's input prompt. This helps the user understand that the
2357 after the user's input prompt. This helps the user understand that the
2358 input line was transformed automatically by IPython.
2358 input line was transformed automatically by IPython.
2359 """
2359 """
2360 if not self.show_rewritten_input:
2360 if not self.show_rewritten_input:
2361 return
2361 return
2362
2362
2363 rw = self.prompt_manager.render('rewrite') + cmd
2363 rw = self.prompt_manager.render('rewrite') + cmd
2364
2364
2365 try:
2365 try:
2366 # plain ascii works better w/ pyreadline, on some machines, so
2366 # plain ascii works better w/ pyreadline, on some machines, so
2367 # we use it and only print uncolored rewrite if we have unicode
2367 # we use it and only print uncolored rewrite if we have unicode
2368 rw = str(rw)
2368 rw = str(rw)
2369 print(rw, file=io.stdout)
2369 print(rw, file=io.stdout)
2370 except UnicodeEncodeError:
2370 except UnicodeEncodeError:
2371 print("------> " + cmd)
2371 print("------> " + cmd)
2372
2372
2373 #-------------------------------------------------------------------------
2373 #-------------------------------------------------------------------------
2374 # Things related to extracting values/expressions from kernel and user_ns
2374 # Things related to extracting values/expressions from kernel and user_ns
2375 #-------------------------------------------------------------------------
2375 #-------------------------------------------------------------------------
2376
2376
2377 def _user_obj_error(self):
2377 def _user_obj_error(self):
2378 """return simple exception dict
2378 """return simple exception dict
2379
2379
2380 for use in user_variables / expressions
2380 for use in user_variables / expressions
2381 """
2381 """
2382
2382
2383 etype, evalue, tb = self._get_exc_info()
2383 etype, evalue, tb = self._get_exc_info()
2384 stb = self.InteractiveTB.get_exception_only(etype, evalue)
2384 stb = self.InteractiveTB.get_exception_only(etype, evalue)
2385
2385
2386 exc_info = {
2386 exc_info = {
2387 u'status' : 'error',
2387 u'status' : 'error',
2388 u'traceback' : stb,
2388 u'traceback' : stb,
2389 u'ename' : unicode_type(etype.__name__),
2389 u'ename' : unicode_type(etype.__name__),
2390 u'evalue' : py3compat.safe_unicode(evalue),
2390 u'evalue' : py3compat.safe_unicode(evalue),
2391 }
2391 }
2392
2392
2393 return exc_info
2393 return exc_info
2394
2394
2395 def _format_user_obj(self, obj):
2395 def _format_user_obj(self, obj):
2396 """format a user object to display dict
2396 """format a user object to display dict
2397
2397
2398 for use in user_expressions / variables
2398 for use in user_expressions / variables
2399 """
2399 """
2400
2400
2401 data, md = self.display_formatter.format(obj)
2401 data, md = self.display_formatter.format(obj)
2402 value = {
2402 value = {
2403 'status' : 'ok',
2403 'status' : 'ok',
2404 'data' : data,
2404 'data' : data,
2405 'metadata' : md,
2405 'metadata' : md,
2406 }
2406 }
2407 return value
2407 return value
2408
2408
2409 def user_variables(self, names):
2409 def user_variables(self, names):
2410 """Get a list of variable names from the user's namespace.
2410 """Get a list of variable names from the user's namespace.
2411
2411
2412 Parameters
2412 Parameters
2413 ----------
2413 ----------
2414 names : list of strings
2414 names : list of strings
2415 A list of names of variables to be read from the user namespace.
2415 A list of names of variables to be read from the user namespace.
2416
2416
2417 Returns
2417 Returns
2418 -------
2418 -------
2419 A dict, keyed by the input names and with the rich mime-type repr(s) of each value.
2419 A dict, keyed by the input names and with the rich mime-type repr(s) of each value.
2420 Each element will be a sub-dict of the same form as a display_data message.
2420 Each element will be a sub-dict of the same form as a display_data message.
2421 """
2421 """
2422 out = {}
2422 out = {}
2423 user_ns = self.user_ns
2423 user_ns = self.user_ns
2424
2424
2425 for varname in names:
2425 for varname in names:
2426 try:
2426 try:
2427 value = self._format_user_obj(user_ns[varname])
2427 value = self._format_user_obj(user_ns[varname])
2428 except:
2428 except:
2429 value = self._user_obj_error()
2429 value = self._user_obj_error()
2430 out[varname] = value
2430 out[varname] = value
2431 return out
2431 return out
2432
2432
2433 def user_expressions(self, expressions):
2433 def user_expressions(self, expressions):
2434 """Evaluate a dict of expressions in the user's namespace.
2434 """Evaluate a dict of expressions in the user's namespace.
2435
2435
2436 Parameters
2436 Parameters
2437 ----------
2437 ----------
2438 expressions : dict
2438 expressions : dict
2439 A dict with string keys and string values. The expression values
2439 A dict with string keys and string values. The expression values
2440 should be valid Python expressions, each of which will be evaluated
2440 should be valid Python expressions, each of which will be evaluated
2441 in the user namespace.
2441 in the user namespace.
2442
2442
2443 Returns
2443 Returns
2444 -------
2444 -------
2445 A dict, keyed like the input expressions dict, with the rich mime-typed
2445 A dict, keyed like the input expressions dict, with the rich mime-typed
2446 display_data of each value.
2446 display_data of each value.
2447 """
2447 """
2448 out = {}
2448 out = {}
2449 user_ns = self.user_ns
2449 user_ns = self.user_ns
2450 global_ns = self.user_global_ns
2450 global_ns = self.user_global_ns
2451
2451
2452 for key, expr in iteritems(expressions):
2452 for key, expr in iteritems(expressions):
2453 try:
2453 try:
2454 value = self._format_user_obj(eval(expr, global_ns, user_ns))
2454 value = self._format_user_obj(eval(expr, global_ns, user_ns))
2455 except:
2455 except:
2456 value = self._user_obj_error()
2456 value = self._user_obj_error()
2457 out[key] = value
2457 out[key] = value
2458 return out
2458 return out
2459
2459
2460 #-------------------------------------------------------------------------
2460 #-------------------------------------------------------------------------
2461 # Things related to the running of code
2461 # Things related to the running of code
2462 #-------------------------------------------------------------------------
2462 #-------------------------------------------------------------------------
2463
2463
2464 def ex(self, cmd):
2464 def ex(self, cmd):
2465 """Execute a normal python statement in user namespace."""
2465 """Execute a normal python statement in user namespace."""
2466 with self.builtin_trap:
2466 with self.builtin_trap:
2467 exec(cmd, self.user_global_ns, self.user_ns)
2467 exec(cmd, self.user_global_ns, self.user_ns)
2468
2468
2469 def ev(self, expr):
2469 def ev(self, expr):
2470 """Evaluate python expression expr in user namespace.
2470 """Evaluate python expression expr in user namespace.
2471
2471
2472 Returns the result of evaluation
2472 Returns the result of evaluation
2473 """
2473 """
2474 with self.builtin_trap:
2474 with self.builtin_trap:
2475 return eval(expr, self.user_global_ns, self.user_ns)
2475 return eval(expr, self.user_global_ns, self.user_ns)
2476
2476
2477 def safe_execfile(self, fname, *where, **kw):
2477 def safe_execfile(self, fname, *where, **kw):
2478 """A safe version of the builtin execfile().
2478 """A safe version of the builtin execfile().
2479
2479
2480 This version will never throw an exception, but instead print
2480 This version will never throw an exception, but instead print
2481 helpful error messages to the screen. This only works on pure
2481 helpful error messages to the screen. This only works on pure
2482 Python files with the .py extension.
2482 Python files with the .py extension.
2483
2483
2484 Parameters
2484 Parameters
2485 ----------
2485 ----------
2486 fname : string
2486 fname : string
2487 The name of the file to be executed.
2487 The name of the file to be executed.
2488 where : tuple
2488 where : tuple
2489 One or two namespaces, passed to execfile() as (globals,locals).
2489 One or two namespaces, passed to execfile() as (globals,locals).
2490 If only one is given, it is passed as both.
2490 If only one is given, it is passed as both.
2491 exit_ignore : bool (False)
2491 exit_ignore : bool (False)
2492 If True, then silence SystemExit for non-zero status (it is always
2492 If True, then silence SystemExit for non-zero status (it is always
2493 silenced for zero status, as it is so common).
2493 silenced for zero status, as it is so common).
2494 raise_exceptions : bool (False)
2494 raise_exceptions : bool (False)
2495 If True raise exceptions everywhere. Meant for testing.
2495 If True raise exceptions everywhere. Meant for testing.
2496
2496
2497 """
2497 """
2498 kw.setdefault('exit_ignore', False)
2498 kw.setdefault('exit_ignore', False)
2499 kw.setdefault('raise_exceptions', False)
2499 kw.setdefault('raise_exceptions', False)
2500
2500
2501 fname = os.path.abspath(os.path.expanduser(fname))
2501 fname = os.path.abspath(os.path.expanduser(fname))
2502
2502
2503 # Make sure we can open the file
2503 # Make sure we can open the file
2504 try:
2504 try:
2505 with open(fname) as thefile:
2505 with open(fname) as thefile:
2506 pass
2506 pass
2507 except:
2507 except:
2508 warn('Could not open file <%s> for safe execution.' % fname)
2508 warn('Could not open file <%s> for safe execution.' % fname)
2509 return
2509 return
2510
2510
2511 # Find things also in current directory. This is needed to mimic the
2511 # Find things also in current directory. This is needed to mimic the
2512 # behavior of running a script from the system command line, where
2512 # behavior of running a script from the system command line, where
2513 # Python inserts the script's directory into sys.path
2513 # Python inserts the script's directory into sys.path
2514 dname = os.path.dirname(fname)
2514 dname = os.path.dirname(fname)
2515
2515
2516 with prepended_to_syspath(dname):
2516 with prepended_to_syspath(dname):
2517 try:
2517 try:
2518 py3compat.execfile(fname,*where)
2518 py3compat.execfile(fname,*where)
2519 except SystemExit as status:
2519 except SystemExit as status:
2520 # If the call was made with 0 or None exit status (sys.exit(0)
2520 # If the call was made with 0 or None exit status (sys.exit(0)
2521 # or sys.exit() ), don't bother showing a traceback, as both of
2521 # or sys.exit() ), don't bother showing a traceback, as both of
2522 # these are considered normal by the OS:
2522 # these are considered normal by the OS:
2523 # > python -c'import sys;sys.exit(0)'; echo $?
2523 # > python -c'import sys;sys.exit(0)'; echo $?
2524 # 0
2524 # 0
2525 # > python -c'import sys;sys.exit()'; echo $?
2525 # > python -c'import sys;sys.exit()'; echo $?
2526 # 0
2526 # 0
2527 # For other exit status, we show the exception unless
2527 # For other exit status, we show the exception unless
2528 # explicitly silenced, but only in short form.
2528 # explicitly silenced, but only in short form.
2529 if kw['raise_exceptions']:
2529 if kw['raise_exceptions']:
2530 raise
2530 raise
2531 if status.code and not kw['exit_ignore']:
2531 if status.code and not kw['exit_ignore']:
2532 self.showtraceback(exception_only=True)
2532 self.showtraceback(exception_only=True)
2533 except:
2533 except:
2534 if kw['raise_exceptions']:
2534 if kw['raise_exceptions']:
2535 raise
2535 raise
2536 self.showtraceback()
2536 self.showtraceback()
2537
2537
2538 def safe_execfile_ipy(self, fname):
2538 def safe_execfile_ipy(self, fname):
2539 """Like safe_execfile, but for .ipy files with IPython syntax.
2539 """Like safe_execfile, but for .ipy files with IPython syntax.
2540
2540
2541 Parameters
2541 Parameters
2542 ----------
2542 ----------
2543 fname : str
2543 fname : str
2544 The name of the file to execute. The filename must have a
2544 The name of the file to execute. The filename must have a
2545 .ipy extension.
2545 .ipy extension.
2546 """
2546 """
2547 fname = os.path.abspath(os.path.expanduser(fname))
2547 fname = os.path.abspath(os.path.expanduser(fname))
2548
2548
2549 # Make sure we can open the file
2549 # Make sure we can open the file
2550 try:
2550 try:
2551 with open(fname) as thefile:
2551 with open(fname) as thefile:
2552 pass
2552 pass
2553 except:
2553 except:
2554 warn('Could not open file <%s> for safe execution.' % fname)
2554 warn('Could not open file <%s> for safe execution.' % fname)
2555 return
2555 return
2556
2556
2557 # Find things also in current directory. This is needed to mimic the
2557 # Find things also in current directory. This is needed to mimic the
2558 # behavior of running a script from the system command line, where
2558 # behavior of running a script from the system command line, where
2559 # Python inserts the script's directory into sys.path
2559 # Python inserts the script's directory into sys.path
2560 dname = os.path.dirname(fname)
2560 dname = os.path.dirname(fname)
2561
2561
2562 with prepended_to_syspath(dname):
2562 with prepended_to_syspath(dname):
2563 try:
2563 try:
2564 with open(fname) as thefile:
2564 with open(fname) as thefile:
2565 # self.run_cell currently captures all exceptions
2565 # self.run_cell currently captures all exceptions
2566 # raised in user code. It would be nice if there were
2566 # raised in user code. It would be nice if there were
2567 # versions of runlines, execfile that did raise, so
2567 # versions of runlines, execfile that did raise, so
2568 # we could catch the errors.
2568 # we could catch the errors.
2569 self.run_cell(thefile.read(), store_history=False, shell_futures=False)
2569 self.run_cell(thefile.read(), store_history=False, shell_futures=False)
2570 except:
2570 except:
2571 self.showtraceback()
2571 self.showtraceback()
2572 warn('Unknown failure executing file: <%s>' % fname)
2572 warn('Unknown failure executing file: <%s>' % fname)
2573
2573
2574 def safe_run_module(self, mod_name, where):
2574 def safe_run_module(self, mod_name, where):
2575 """A safe version of runpy.run_module().
2575 """A safe version of runpy.run_module().
2576
2576
2577 This version will never throw an exception, but instead print
2577 This version will never throw an exception, but instead print
2578 helpful error messages to the screen.
2578 helpful error messages to the screen.
2579
2579
2580 `SystemExit` exceptions with status code 0 or None are ignored.
2580 `SystemExit` exceptions with status code 0 or None are ignored.
2581
2581
2582 Parameters
2582 Parameters
2583 ----------
2583 ----------
2584 mod_name : string
2584 mod_name : string
2585 The name of the module to be executed.
2585 The name of the module to be executed.
2586 where : dict
2586 where : dict
2587 The globals namespace.
2587 The globals namespace.
2588 """
2588 """
2589 try:
2589 try:
2590 try:
2590 try:
2591 where.update(
2591 where.update(
2592 runpy.run_module(str(mod_name), run_name="__main__",
2592 runpy.run_module(str(mod_name), run_name="__main__",
2593 alter_sys=True)
2593 alter_sys=True)
2594 )
2594 )
2595 except SystemExit as status:
2595 except SystemExit as status:
2596 if status.code:
2596 if status.code:
2597 raise
2597 raise
2598 except:
2598 except:
2599 self.showtraceback()
2599 self.showtraceback()
2600 warn('Unknown failure executing module: <%s>' % mod_name)
2600 warn('Unknown failure executing module: <%s>' % mod_name)
2601
2601
2602 def _run_cached_cell_magic(self, magic_name, line):
2602 def _run_cached_cell_magic(self, magic_name, line):
2603 """Special method to call a cell magic with the data stored in self.
2603 """Special method to call a cell magic with the data stored in self.
2604 """
2604 """
2605 cell = self._current_cell_magic_body
2605 cell = self._current_cell_magic_body
2606 self._current_cell_magic_body = None
2606 self._current_cell_magic_body = None
2607 return self.run_cell_magic(magic_name, line, cell)
2607 return self.run_cell_magic(magic_name, line, cell)
2608
2608
2609 def run_cell(self, raw_cell, store_history=False, silent=False, shell_futures=True):
2609 def run_cell(self, raw_cell, store_history=False, silent=False, shell_futures=True):
2610 """Run a complete IPython cell.
2610 """Run a complete IPython cell.
2611
2611
2612 Parameters
2612 Parameters
2613 ----------
2613 ----------
2614 raw_cell : str
2614 raw_cell : str
2615 The code (including IPython code such as %magic functions) to run.
2615 The code (including IPython code such as %magic functions) to run.
2616 store_history : bool
2616 store_history : bool
2617 If True, the raw and translated cell will be stored in IPython's
2617 If True, the raw and translated cell will be stored in IPython's
2618 history. For user code calling back into IPython's machinery, this
2618 history. For user code calling back into IPython's machinery, this
2619 should be set to False.
2619 should be set to False.
2620 silent : bool
2620 silent : bool
2621 If True, avoid side-effects, such as implicit displayhooks and
2621 If True, avoid side-effects, such as implicit displayhooks and
2622 and logging. silent=True forces store_history=False.
2622 and logging. silent=True forces store_history=False.
2623 shell_futures : bool
2623 shell_futures : bool
2624 If True, the code will share future statements with the interactive
2624 If True, the code will share future statements with the interactive
2625 shell. It will both be affected by previous __future__ imports, and
2625 shell. It will both be affected by previous __future__ imports, and
2626 any __future__ imports in the code will affect the shell. If False,
2626 any __future__ imports in the code will affect the shell. If False,
2627 __future__ imports are not shared in either direction.
2627 __future__ imports are not shared in either direction.
2628 """
2628 """
2629 if (not raw_cell) or raw_cell.isspace():
2629 if (not raw_cell) or raw_cell.isspace():
2630 return
2630 return
2631
2631
2632 if silent:
2632 if silent:
2633 store_history = False
2633 store_history = False
2634
2634
2635 self.input_transformer_manager.push(raw_cell)
2635 self.input_transformer_manager.push(raw_cell)
2636 cell = self.input_transformer_manager.source_reset()
2636 cell = self.input_transformer_manager.source_reset()
2637
2637
2638 # Our own compiler remembers the __future__ environment. If we want to
2638 # Our own compiler remembers the __future__ environment. If we want to
2639 # run code with a separate __future__ environment, use the default
2639 # run code with a separate __future__ environment, use the default
2640 # compiler
2640 # compiler
2641 compiler = self.compile if shell_futures else CachingCompiler()
2641 compiler = self.compile if shell_futures else CachingCompiler()
2642
2642
2643 with self.builtin_trap:
2643 with self.builtin_trap:
2644 prefilter_failed = False
2644 prefilter_failed = False
2645 if len(cell.splitlines()) == 1:
2645 if len(cell.splitlines()) == 1:
2646 try:
2646 try:
2647 # use prefilter_lines to handle trailing newlines
2647 # use prefilter_lines to handle trailing newlines
2648 # restore trailing newline for ast.parse
2648 # restore trailing newline for ast.parse
2649 cell = self.prefilter_manager.prefilter_lines(cell) + '\n'
2649 cell = self.prefilter_manager.prefilter_lines(cell) + '\n'
2650 except AliasError as e:
2650 except AliasError as e:
2651 error(e)
2651 error(e)
2652 prefilter_failed = True
2652 prefilter_failed = True
2653 except Exception:
2653 except Exception:
2654 # don't allow prefilter errors to crash IPython
2654 # don't allow prefilter errors to crash IPython
2655 self.showtraceback()
2655 self.showtraceback()
2656 prefilter_failed = True
2656 prefilter_failed = True
2657
2657
2658 # Store raw and processed history
2658 # Store raw and processed history
2659 if store_history:
2659 if store_history:
2660 self.history_manager.store_inputs(self.execution_count,
2660 self.history_manager.store_inputs(self.execution_count,
2661 cell, raw_cell)
2661 cell, raw_cell)
2662 if not silent:
2662 if not silent:
2663 self.logger.log(cell, raw_cell)
2663 self.logger.log(cell, raw_cell)
2664
2664
2665 if not prefilter_failed:
2665 if not prefilter_failed:
2666 # don't run if prefilter failed
2666 # don't run if prefilter failed
2667 cell_name = self.compile.cache(cell, self.execution_count)
2667 cell_name = self.compile.cache(cell, self.execution_count)
2668
2668
2669 with self.display_trap:
2669 with self.display_trap:
2670 try:
2670 try:
2671 code_ast = compiler.ast_parse(cell, filename=cell_name)
2671 code_ast = compiler.ast_parse(cell, filename=cell_name)
2672 except IndentationError:
2672 except IndentationError:
2673 self.showindentationerror()
2673 self.showindentationerror()
2674 if store_history:
2674 if store_history:
2675 self.execution_count += 1
2675 self.execution_count += 1
2676 return None
2676 return None
2677 except (OverflowError, SyntaxError, ValueError, TypeError,
2677 except (OverflowError, SyntaxError, ValueError, TypeError,
2678 MemoryError):
2678 MemoryError):
2679 self.showsyntaxerror()
2679 self.showsyntaxerror()
2680 if store_history:
2680 if store_history:
2681 self.execution_count += 1
2681 self.execution_count += 1
2682 return None
2682 return None
2683
2683
2684 code_ast = self.transform_ast(code_ast)
2684 code_ast = self.transform_ast(code_ast)
2685
2685
2686 interactivity = "none" if silent else self.ast_node_interactivity
2686 interactivity = "none" if silent else self.ast_node_interactivity
2687 self.run_ast_nodes(code_ast.body, cell_name,
2687 self.run_ast_nodes(code_ast.body, cell_name,
2688 interactivity=interactivity, compiler=compiler)
2688 interactivity=interactivity, compiler=compiler)
2689
2689
2690 # Execute any registered post-execution functions.
2690 # Execute any registered post-execution functions.
2691 # unless we are silent
2691 # unless we are silent
2692 post_exec = [] if silent else iteritems(self._post_execute)
2692 post_exec = [] if silent else iteritems(self._post_execute)
2693
2693
2694 for func, status in post_exec:
2694 for func, status in post_exec:
2695 if self.disable_failing_post_execute and not status:
2695 if self.disable_failing_post_execute and not status:
2696 continue
2696 continue
2697 try:
2697 try:
2698 func()
2698 func()
2699 except KeyboardInterrupt:
2699 except KeyboardInterrupt:
2700 print("\nKeyboardInterrupt", file=io.stderr)
2700 print("\nKeyboardInterrupt", file=io.stderr)
2701 except Exception:
2701 except Exception:
2702 # register as failing:
2702 # register as failing:
2703 self._post_execute[func] = False
2703 self._post_execute[func] = False
2704 self.showtraceback()
2704 self.showtraceback()
2705 print('\n'.join([
2705 print('\n'.join([
2706 "post-execution function %r produced an error." % func,
2706 "post-execution function %r produced an error." % func,
2707 "If this problem persists, you can disable failing post-exec functions with:",
2707 "If this problem persists, you can disable failing post-exec functions with:",
2708 "",
2708 "",
2709 " get_ipython().disable_failing_post_execute = True"
2709 " get_ipython().disable_failing_post_execute = True"
2710 ]), file=io.stderr)
2710 ]), file=io.stderr)
2711
2711
2712 if store_history:
2712 if store_history:
2713 # Write output to the database. Does nothing unless
2713 # Write output to the database. Does nothing unless
2714 # history output logging is enabled.
2714 # history output logging is enabled.
2715 self.history_manager.store_output(self.execution_count)
2715 self.history_manager.store_output(self.execution_count)
2716 # Each cell is a *single* input, regardless of how many lines it has
2716 # Each cell is a *single* input, regardless of how many lines it has
2717 self.execution_count += 1
2717 self.execution_count += 1
2718
2718
2719 def transform_ast(self, node):
2719 def transform_ast(self, node):
2720 """Apply the AST transformations from self.ast_transformers
2720 """Apply the AST transformations from self.ast_transformers
2721
2721
2722 Parameters
2722 Parameters
2723 ----------
2723 ----------
2724 node : ast.Node
2724 node : ast.Node
2725 The root node to be transformed. Typically called with the ast.Module
2725 The root node to be transformed. Typically called with the ast.Module
2726 produced by parsing user input.
2726 produced by parsing user input.
2727
2727
2728 Returns
2728 Returns
2729 -------
2729 -------
2730 An ast.Node corresponding to the node it was called with. Note that it
2730 An ast.Node corresponding to the node it was called with. Note that it
2731 may also modify the passed object, so don't rely on references to the
2731 may also modify the passed object, so don't rely on references to the
2732 original AST.
2732 original AST.
2733 """
2733 """
2734 for transformer in self.ast_transformers:
2734 for transformer in self.ast_transformers:
2735 try:
2735 try:
2736 node = transformer.visit(node)
2736 node = transformer.visit(node)
2737 except Exception:
2737 except Exception:
2738 warn("AST transformer %r threw an error. It will be unregistered." % transformer)
2738 warn("AST transformer %r threw an error. It will be unregistered." % transformer)
2739 self.ast_transformers.remove(transformer)
2739 self.ast_transformers.remove(transformer)
2740
2740
2741 if self.ast_transformers:
2741 if self.ast_transformers:
2742 ast.fix_missing_locations(node)
2742 ast.fix_missing_locations(node)
2743 return node
2743 return node
2744
2744
2745
2745
2746 def run_ast_nodes(self, nodelist, cell_name, interactivity='last_expr',
2746 def run_ast_nodes(self, nodelist, cell_name, interactivity='last_expr',
2747 compiler=compile):
2747 compiler=compile):
2748 """Run a sequence of AST nodes. The execution mode depends on the
2748 """Run a sequence of AST nodes. The execution mode depends on the
2749 interactivity parameter.
2749 interactivity parameter.
2750
2750
2751 Parameters
2751 Parameters
2752 ----------
2752 ----------
2753 nodelist : list
2753 nodelist : list
2754 A sequence of AST nodes to run.
2754 A sequence of AST nodes to run.
2755 cell_name : str
2755 cell_name : str
2756 Will be passed to the compiler as the filename of the cell. Typically
2756 Will be passed to the compiler as the filename of the cell. Typically
2757 the value returned by ip.compile.cache(cell).
2757 the value returned by ip.compile.cache(cell).
2758 interactivity : str
2758 interactivity : str
2759 'all', 'last', 'last_expr' or 'none', specifying which nodes should be
2759 'all', 'last', 'last_expr' or 'none', specifying which nodes should be
2760 run interactively (displaying output from expressions). 'last_expr'
2760 run interactively (displaying output from expressions). 'last_expr'
2761 will run the last node interactively only if it is an expression (i.e.
2761 will run the last node interactively only if it is an expression (i.e.
2762 expressions in loops or other blocks are not displayed. Other values
2762 expressions in loops or other blocks are not displayed. Other values
2763 for this parameter will raise a ValueError.
2763 for this parameter will raise a ValueError.
2764 compiler : callable
2764 compiler : callable
2765 A function with the same interface as the built-in compile(), to turn
2765 A function with the same interface as the built-in compile(), to turn
2766 the AST nodes into code objects. Default is the built-in compile().
2766 the AST nodes into code objects. Default is the built-in compile().
2767 """
2767 """
2768 if not nodelist:
2768 if not nodelist:
2769 return
2769 return
2770
2770
2771 if interactivity == 'last_expr':
2771 if interactivity == 'last_expr':
2772 if isinstance(nodelist[-1], ast.Expr):
2772 if isinstance(nodelist[-1], ast.Expr):
2773 interactivity = "last"
2773 interactivity = "last"
2774 else:
2774 else:
2775 interactivity = "none"
2775 interactivity = "none"
2776
2776
2777 if interactivity == 'none':
2777 if interactivity == 'none':
2778 to_run_exec, to_run_interactive = nodelist, []
2778 to_run_exec, to_run_interactive = nodelist, []
2779 elif interactivity == 'last':
2779 elif interactivity == 'last':
2780 to_run_exec, to_run_interactive = nodelist[:-1], nodelist[-1:]
2780 to_run_exec, to_run_interactive = nodelist[:-1], nodelist[-1:]
2781 elif interactivity == 'all':
2781 elif interactivity == 'all':
2782 to_run_exec, to_run_interactive = [], nodelist
2782 to_run_exec, to_run_interactive = [], nodelist
2783 else:
2783 else:
2784 raise ValueError("Interactivity was %r" % interactivity)
2784 raise ValueError("Interactivity was %r" % interactivity)
2785
2785
2786 exec_count = self.execution_count
2786 exec_count = self.execution_count
2787
2787
2788 try:
2788 try:
2789 for i, node in enumerate(to_run_exec):
2789 for i, node in enumerate(to_run_exec):
2790 mod = ast.Module([node])
2790 mod = ast.Module([node])
2791 code = compiler(mod, cell_name, "exec")
2791 code = compiler(mod, cell_name, "exec")
2792 if self.run_code(code):
2792 if self.run_code(code):
2793 return True
2793 return True
2794
2794
2795 for i, node in enumerate(to_run_interactive):
2795 for i, node in enumerate(to_run_interactive):
2796 mod = ast.Interactive([node])
2796 mod = ast.Interactive([node])
2797 code = compiler(mod, cell_name, "single")
2797 code = compiler(mod, cell_name, "single")
2798 if self.run_code(code):
2798 if self.run_code(code):
2799 return True
2799 return True
2800
2800
2801 # Flush softspace
2801 # Flush softspace
2802 if softspace(sys.stdout, 0):
2802 if softspace(sys.stdout, 0):
2803 print()
2803 print()
2804
2804
2805 except:
2805 except:
2806 # It's possible to have exceptions raised here, typically by
2806 # It's possible to have exceptions raised here, typically by
2807 # compilation of odd code (such as a naked 'return' outside a
2807 # compilation of odd code (such as a naked 'return' outside a
2808 # function) that did parse but isn't valid. Typically the exception
2808 # function) that did parse but isn't valid. Typically the exception
2809 # is a SyntaxError, but it's safest just to catch anything and show
2809 # is a SyntaxError, but it's safest just to catch anything and show
2810 # the user a traceback.
2810 # the user a traceback.
2811
2811
2812 # We do only one try/except outside the loop to minimize the impact
2812 # We do only one try/except outside the loop to minimize the impact
2813 # on runtime, and also because if any node in the node list is
2813 # on runtime, and also because if any node in the node list is
2814 # broken, we should stop execution completely.
2814 # broken, we should stop execution completely.
2815 self.showtraceback()
2815 self.showtraceback()
2816
2816
2817 return False
2817 return False
2818
2818
2819 def run_code(self, code_obj):
2819 def run_code(self, code_obj):
2820 """Execute a code object.
2820 """Execute a code object.
2821
2821
2822 When an exception occurs, self.showtraceback() is called to display a
2822 When an exception occurs, self.showtraceback() is called to display a
2823 traceback.
2823 traceback.
2824
2824
2825 Parameters
2825 Parameters
2826 ----------
2826 ----------
2827 code_obj : code object
2827 code_obj : code object
2828 A compiled code object, to be executed
2828 A compiled code object, to be executed
2829
2829
2830 Returns
2830 Returns
2831 -------
2831 -------
2832 False : successful execution.
2832 False : successful execution.
2833 True : an error occurred.
2833 True : an error occurred.
2834 """
2834 """
2835
2835
2836 # Set our own excepthook in case the user code tries to call it
2836 # Set our own excepthook in case the user code tries to call it
2837 # directly, so that the IPython crash handler doesn't get triggered
2837 # directly, so that the IPython crash handler doesn't get triggered
2838 old_excepthook,sys.excepthook = sys.excepthook, self.excepthook
2838 old_excepthook,sys.excepthook = sys.excepthook, self.excepthook
2839
2839
2840 # we save the original sys.excepthook in the instance, in case config
2840 # we save the original sys.excepthook in the instance, in case config
2841 # code (such as magics) needs access to it.
2841 # code (such as magics) needs access to it.
2842 self.sys_excepthook = old_excepthook
2842 self.sys_excepthook = old_excepthook
2843 outflag = 1 # happens in more places, so it's easier as default
2843 outflag = 1 # happens in more places, so it's easier as default
2844 try:
2844 try:
2845 try:
2845 try:
2846 self.hooks.pre_run_code_hook()
2846 self.hooks.pre_run_code_hook()
2847 #rprint('Running code', repr(code_obj)) # dbg
2847 #rprint('Running code', repr(code_obj)) # dbg
2848 exec(code_obj, self.user_global_ns, self.user_ns)
2848 exec(code_obj, self.user_global_ns, self.user_ns)
2849 finally:
2849 finally:
2850 # Reset our crash handler in place
2850 # Reset our crash handler in place
2851 sys.excepthook = old_excepthook
2851 sys.excepthook = old_excepthook
2852 except SystemExit:
2852 except SystemExit:
2853 self.showtraceback(exception_only=True)
2853 self.showtraceback(exception_only=True)
2854 warn("To exit: use 'exit', 'quit', or Ctrl-D.", level=1)
2854 warn("To exit: use 'exit', 'quit', or Ctrl-D.", level=1)
2855 except self.custom_exceptions:
2855 except self.custom_exceptions:
2856 etype,value,tb = sys.exc_info()
2856 etype,value,tb = sys.exc_info()
2857 self.CustomTB(etype,value,tb)
2857 self.CustomTB(etype,value,tb)
2858 except:
2858 except:
2859 self.showtraceback()
2859 self.showtraceback()
2860 else:
2860 else:
2861 outflag = 0
2861 outflag = 0
2862 return outflag
2862 return outflag
2863
2863
2864 # For backwards compatibility
2864 # For backwards compatibility
2865 runcode = run_code
2865 runcode = run_code
2866
2866
2867 #-------------------------------------------------------------------------
2867 #-------------------------------------------------------------------------
2868 # Things related to GUI support and pylab
2868 # Things related to GUI support and pylab
2869 #-------------------------------------------------------------------------
2869 #-------------------------------------------------------------------------
2870
2870
2871 def enable_gui(self, gui=None):
2871 def enable_gui(self, gui=None):
2872 raise NotImplementedError('Implement enable_gui in a subclass')
2872 raise NotImplementedError('Implement enable_gui in a subclass')
2873
2873
2874 def enable_matplotlib(self, gui=None):
2874 def enable_matplotlib(self, gui=None):
2875 """Enable interactive matplotlib and inline figure support.
2875 """Enable interactive matplotlib and inline figure support.
2876
2876
2877 This takes the following steps:
2877 This takes the following steps:
2878
2878
2879 1. select the appropriate eventloop and matplotlib backend
2879 1. select the appropriate eventloop and matplotlib backend
2880 2. set up matplotlib for interactive use with that backend
2880 2. set up matplotlib for interactive use with that backend
2881 3. configure formatters for inline figure display
2881 3. configure formatters for inline figure display
2882 4. enable the selected gui eventloop
2882 4. enable the selected gui eventloop
2883
2883
2884 Parameters
2884 Parameters
2885 ----------
2885 ----------
2886 gui : optional, string
2886 gui : optional, string
2887 If given, dictates the choice of matplotlib GUI backend to use
2887 If given, dictates the choice of matplotlib GUI backend to use
2888 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
2888 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
2889 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
2889 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
2890 matplotlib (as dictated by the matplotlib build-time options plus the
2890 matplotlib (as dictated by the matplotlib build-time options plus the
2891 user's matplotlibrc configuration file). Note that not all backends
2891 user's matplotlibrc configuration file). Note that not all backends
2892 make sense in all contexts, for example a terminal ipython can't
2892 make sense in all contexts, for example a terminal ipython can't
2893 display figures inline.
2893 display figures inline.
2894 """
2894 """
2895 from IPython.core import pylabtools as pt
2895 from IPython.core import pylabtools as pt
2896 gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
2896 gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
2897
2897
2898 if gui != 'inline':
2898 if gui != 'inline':
2899 # If we have our first gui selection, store it
2899 # If we have our first gui selection, store it
2900 if self.pylab_gui_select is None:
2900 if self.pylab_gui_select is None:
2901 self.pylab_gui_select = gui
2901 self.pylab_gui_select = gui
2902 # Otherwise if they are different
2902 # Otherwise if they are different
2903 elif gui != self.pylab_gui_select:
2903 elif gui != self.pylab_gui_select:
2904 print ('Warning: Cannot change to a different GUI toolkit: %s.'
2904 print ('Warning: Cannot change to a different GUI toolkit: %s.'
2905 ' Using %s instead.' % (gui, self.pylab_gui_select))
2905 ' Using %s instead.' % (gui, self.pylab_gui_select))
2906 gui, backend = pt.find_gui_and_backend(self.pylab_gui_select)
2906 gui, backend = pt.find_gui_and_backend(self.pylab_gui_select)
2907
2907
2908 pt.activate_matplotlib(backend)
2908 pt.activate_matplotlib(backend)
2909 pt.configure_inline_support(self, backend)
2909 pt.configure_inline_support(self, backend)
2910
2910
2911 # Now we must activate the gui pylab wants to use, and fix %run to take
2911 # Now we must activate the gui pylab wants to use, and fix %run to take
2912 # plot updates into account
2912 # plot updates into account
2913 self.enable_gui(gui)
2913 self.enable_gui(gui)
2914 self.magics_manager.registry['ExecutionMagics'].default_runner = \
2914 self.magics_manager.registry['ExecutionMagics'].default_runner = \
2915 pt.mpl_runner(self.safe_execfile)
2915 pt.mpl_runner(self.safe_execfile)
2916
2916
2917 return gui, backend
2917 return gui, backend
2918
2918
2919 def enable_pylab(self, gui=None, import_all=True, welcome_message=False):
2919 def enable_pylab(self, gui=None, import_all=True, welcome_message=False):
2920 """Activate pylab support at runtime.
2920 """Activate pylab support at runtime.
2921
2921
2922 This turns on support for matplotlib, preloads into the interactive
2922 This turns on support for matplotlib, preloads into the interactive
2923 namespace all of numpy and pylab, and configures IPython to correctly
2923 namespace all of numpy and pylab, and configures IPython to correctly
2924 interact with the GUI event loop. The GUI backend to be used can be
2924 interact with the GUI event loop. The GUI backend to be used can be
2925 optionally selected with the optional ``gui`` argument.
2925 optionally selected with the optional ``gui`` argument.
2926
2926
2927 This method only adds preloading the namespace to InteractiveShell.enable_matplotlib.
2927 This method only adds preloading the namespace to InteractiveShell.enable_matplotlib.
2928
2928
2929 Parameters
2929 Parameters
2930 ----------
2930 ----------
2931 gui : optional, string
2931 gui : optional, string
2932 If given, dictates the choice of matplotlib GUI backend to use
2932 If given, dictates the choice of matplotlib GUI backend to use
2933 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
2933 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
2934 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
2934 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
2935 matplotlib (as dictated by the matplotlib build-time options plus the
2935 matplotlib (as dictated by the matplotlib build-time options plus the
2936 user's matplotlibrc configuration file). Note that not all backends
2936 user's matplotlibrc configuration file). Note that not all backends
2937 make sense in all contexts, for example a terminal ipython can't
2937 make sense in all contexts, for example a terminal ipython can't
2938 display figures inline.
2938 display figures inline.
2939 import_all : optional, bool, default: True
2939 import_all : optional, bool, default: True
2940 Whether to do `from numpy import *` and `from pylab import *`
2940 Whether to do `from numpy import *` and `from pylab import *`
2941 in addition to module imports.
2941 in addition to module imports.
2942 welcome_message : deprecated
2942 welcome_message : deprecated
2943 This argument is ignored, no welcome message will be displayed.
2943 This argument is ignored, no welcome message will be displayed.
2944 """
2944 """
2945 from IPython.core.pylabtools import import_pylab
2945 from IPython.core.pylabtools import import_pylab
2946
2946
2947 gui, backend = self.enable_matplotlib(gui)
2947 gui, backend = self.enable_matplotlib(gui)
2948
2948
2949 # We want to prevent the loading of pylab to pollute the user's
2949 # We want to prevent the loading of pylab to pollute the user's
2950 # namespace as shown by the %who* magics, so we execute the activation
2950 # namespace as shown by the %who* magics, so we execute the activation
2951 # code in an empty namespace, and we update *both* user_ns and
2951 # code in an empty namespace, and we update *both* user_ns and
2952 # user_ns_hidden with this information.
2952 # user_ns_hidden with this information.
2953 ns = {}
2953 ns = {}
2954 import_pylab(ns, import_all)
2954 import_pylab(ns, import_all)
2955 # warn about clobbered names
2955 # warn about clobbered names
2956 ignored = set(["__builtins__"])
2956 ignored = set(["__builtins__"])
2957 both = set(ns).intersection(self.user_ns).difference(ignored)
2957 both = set(ns).intersection(self.user_ns).difference(ignored)
2958 clobbered = [ name for name in both if self.user_ns[name] is not ns[name] ]
2958 clobbered = [ name for name in both if self.user_ns[name] is not ns[name] ]
2959 self.user_ns.update(ns)
2959 self.user_ns.update(ns)
2960 self.user_ns_hidden.update(ns)
2960 self.user_ns_hidden.update(ns)
2961 return gui, backend, clobbered
2961 return gui, backend, clobbered
2962
2962
2963 #-------------------------------------------------------------------------
2963 #-------------------------------------------------------------------------
2964 # Utilities
2964 # Utilities
2965 #-------------------------------------------------------------------------
2965 #-------------------------------------------------------------------------
2966
2966
2967 def var_expand(self, cmd, depth=0, formatter=DollarFormatter()):
2967 def var_expand(self, cmd, depth=0, formatter=DollarFormatter()):
2968 """Expand python variables in a string.
2968 """Expand python variables in a string.
2969
2969
2970 The depth argument indicates how many frames above the caller should
2970 The depth argument indicates how many frames above the caller should
2971 be walked to look for the local namespace where to expand variables.
2971 be walked to look for the local namespace where to expand variables.
2972
2972
2973 The global namespace for expansion is always the user's interactive
2973 The global namespace for expansion is always the user's interactive
2974 namespace.
2974 namespace.
2975 """
2975 """
2976 ns = self.user_ns.copy()
2976 ns = self.user_ns.copy()
2977 ns.update(sys._getframe(depth+1).f_locals)
2977 ns.update(sys._getframe(depth+1).f_locals)
2978 try:
2978 try:
2979 # We have to use .vformat() here, because 'self' is a valid and common
2979 # We have to use .vformat() here, because 'self' is a valid and common
2980 # name, and expanding **ns for .format() would make it collide with
2980 # name, and expanding **ns for .format() would make it collide with
2981 # the 'self' argument of the method.
2981 # the 'self' argument of the method.
2982 cmd = formatter.vformat(cmd, args=[], kwargs=ns)
2982 cmd = formatter.vformat(cmd, args=[], kwargs=ns)
2983 except Exception:
2983 except Exception:
2984 # if formatter couldn't format, just let it go untransformed
2984 # if formatter couldn't format, just let it go untransformed
2985 pass
2985 pass
2986 return cmd
2986 return cmd
2987
2987
2988 def mktempfile(self, data=None, prefix='ipython_edit_'):
2988 def mktempfile(self, data=None, prefix='ipython_edit_'):
2989 """Make a new tempfile and return its filename.
2989 """Make a new tempfile and return its filename.
2990
2990
2991 This makes a call to tempfile.mktemp, but it registers the created
2991 This makes a call to tempfile.mktemp, but it registers the created
2992 filename internally so ipython cleans it up at exit time.
2992 filename internally so ipython cleans it up at exit time.
2993
2993
2994 Optional inputs:
2994 Optional inputs:
2995
2995
2996 - data(None): if data is given, it gets written out to the temp file
2996 - data(None): if data is given, it gets written out to the temp file
2997 immediately, and the file is closed again."""
2997 immediately, and the file is closed again."""
2998
2998
2999 filename = tempfile.mktemp('.py', prefix)
2999 filename = tempfile.mktemp('.py', prefix)
3000 self.tempfiles.append(filename)
3000 self.tempfiles.append(filename)
3001
3001
3002 if data:
3002 if data:
3003 tmp_file = open(filename,'w')
3003 tmp_file = open(filename,'w')
3004 tmp_file.write(data)
3004 tmp_file.write(data)
3005 tmp_file.close()
3005 tmp_file.close()
3006 return filename
3006 return filename
3007
3007
3008 # TODO: This should be removed when Term is refactored.
3008 # TODO: This should be removed when Term is refactored.
3009 def write(self,data):
3009 def write(self,data):
3010 """Write a string to the default output"""
3010 """Write a string to the default output"""
3011 io.stdout.write(data)
3011 io.stdout.write(data)
3012
3012
3013 # TODO: This should be removed when Term is refactored.
3013 # TODO: This should be removed when Term is refactored.
3014 def write_err(self,data):
3014 def write_err(self,data):
3015 """Write a string to the default error output"""
3015 """Write a string to the default error output"""
3016 io.stderr.write(data)
3016 io.stderr.write(data)
3017
3017
3018 def ask_yes_no(self, prompt, default=None):
3018 def ask_yes_no(self, prompt, default=None):
3019 if self.quiet:
3019 if self.quiet:
3020 return True
3020 return True
3021 return ask_yes_no(prompt,default)
3021 return ask_yes_no(prompt,default)
3022
3022
3023 def show_usage(self):
3023 def show_usage(self):
3024 """Show a usage message"""
3024 """Show a usage message"""
3025 page.page(IPython.core.usage.interactive_usage)
3025 page.page(IPython.core.usage.interactive_usage)
3026
3026
3027 def extract_input_lines(self, range_str, raw=False):
3027 def extract_input_lines(self, range_str, raw=False):
3028 """Return as a string a set of input history slices.
3028 """Return as a string a set of input history slices.
3029
3029
3030 Parameters
3030 Parameters
3031 ----------
3031 ----------
3032 range_str : string
3032 range_str : string
3033 The set of slices is given as a string, like "~5/6-~4/2 4:8 9",
3033 The set of slices is given as a string, like "~5/6-~4/2 4:8 9",
3034 since this function is for use by magic functions which get their
3034 since this function is for use by magic functions which get their
3035 arguments as strings. The number before the / is the session
3035 arguments as strings. The number before the / is the session
3036 number: ~n goes n back from the current session.
3036 number: ~n goes n back from the current session.
3037
3037
3038 raw : bool, optional
3038 raw : bool, optional
3039 By default, the processed input is used. If this is true, the raw
3039 By default, the processed input is used. If this is true, the raw
3040 input history is used instead.
3040 input history is used instead.
3041
3041
3042 Notes
3042 Notes
3043 -----
3043 -----
3044
3044
3045 Slices can be described with two notations:
3045 Slices can be described with two notations:
3046
3046
3047 * ``N:M`` -> standard python form, means including items N...(M-1).
3047 * ``N:M`` -> standard python form, means including items N...(M-1).
3048 * ``N-M`` -> include items N..M (closed endpoint).
3048 * ``N-M`` -> include items N..M (closed endpoint).
3049 """
3049 """
3050 lines = self.history_manager.get_range_by_str(range_str, raw=raw)
3050 lines = self.history_manager.get_range_by_str(range_str, raw=raw)
3051 return "\n".join(x for _, _, x in lines)
3051 return "\n".join(x for _, _, x in lines)
3052
3052
3053 def find_user_code(self, target, raw=True, py_only=False, skip_encoding_cookie=True):
3053 def find_user_code(self, target, raw=True, py_only=False, skip_encoding_cookie=True):
3054 """Get a code string from history, file, url, or a string or macro.
3054 """Get a code string from history, file, url, or a string or macro.
3055
3055
3056 This is mainly used by magic functions.
3056 This is mainly used by magic functions.
3057
3057
3058 Parameters
3058 Parameters
3059 ----------
3059 ----------
3060
3060
3061 target : str
3061 target : str
3062
3062
3063 A string specifying code to retrieve. This will be tried respectively
3063 A string specifying code to retrieve. This will be tried respectively
3064 as: ranges of input history (see %history for syntax), url,
3064 as: ranges of input history (see %history for syntax), url,
3065 correspnding .py file, filename, or an expression evaluating to a
3065 correspnding .py file, filename, or an expression evaluating to a
3066 string or Macro in the user namespace.
3066 string or Macro in the user namespace.
3067
3067
3068 raw : bool
3068 raw : bool
3069 If true (default), retrieve raw history. Has no effect on the other
3069 If true (default), retrieve raw history. Has no effect on the other
3070 retrieval mechanisms.
3070 retrieval mechanisms.
3071
3071
3072 py_only : bool (default False)
3072 py_only : bool (default False)
3073 Only try to fetch python code, do not try alternative methods to decode file
3073 Only try to fetch python code, do not try alternative methods to decode file
3074 if unicode fails.
3074 if unicode fails.
3075
3075
3076 Returns
3076 Returns
3077 -------
3077 -------
3078 A string of code.
3078 A string of code.
3079
3079
3080 ValueError is raised if nothing is found, and TypeError if it evaluates
3080 ValueError is raised if nothing is found, and TypeError if it evaluates
3081 to an object of another type. In each case, .args[0] is a printable
3081 to an object of another type. In each case, .args[0] is a printable
3082 message.
3082 message.
3083 """
3083 """
3084 code = self.extract_input_lines(target, raw=raw) # Grab history
3084 code = self.extract_input_lines(target, raw=raw) # Grab history
3085 if code:
3085 if code:
3086 return code
3086 return code
3087 utarget = unquote_filename(target)
3087 utarget = unquote_filename(target)
3088 try:
3088 try:
3089 if utarget.startswith(('http://', 'https://')):
3089 if utarget.startswith(('http://', 'https://')):
3090 return openpy.read_py_url(utarget, skip_encoding_cookie=skip_encoding_cookie)
3090 return openpy.read_py_url(utarget, skip_encoding_cookie=skip_encoding_cookie)
3091 except UnicodeDecodeError:
3091 except UnicodeDecodeError:
3092 if not py_only :
3092 if not py_only :
3093 from urllib import urlopen # Deferred import
3093 from urllib import urlopen # Deferred import
3094 response = urlopen(target)
3094 response = urlopen(target)
3095 return response.read().decode('latin1')
3095 return response.read().decode('latin1')
3096 raise ValueError(("'%s' seem to be unreadable.") % utarget)
3096 raise ValueError(("'%s' seem to be unreadable.") % utarget)
3097
3097
3098 potential_target = [target]
3098 potential_target = [target]
3099 try :
3099 try :
3100 potential_target.insert(0,get_py_filename(target))
3100 potential_target.insert(0,get_py_filename(target))
3101 except IOError:
3101 except IOError:
3102 pass
3102 pass
3103
3103
3104 for tgt in potential_target :
3104 for tgt in potential_target :
3105 if os.path.isfile(tgt): # Read file
3105 if os.path.isfile(tgt): # Read file
3106 try :
3106 try :
3107 return openpy.read_py_file(tgt, skip_encoding_cookie=skip_encoding_cookie)
3107 return openpy.read_py_file(tgt, skip_encoding_cookie=skip_encoding_cookie)
3108 except UnicodeDecodeError :
3108 except UnicodeDecodeError :
3109 if not py_only :
3109 if not py_only :
3110 with io_open(tgt,'r', encoding='latin1') as f :
3110 with io_open(tgt,'r', encoding='latin1') as f :
3111 return f.read()
3111 return f.read()
3112 raise ValueError(("'%s' seem to be unreadable.") % target)
3112 raise ValueError(("'%s' seem to be unreadable.") % target)
3113 elif os.path.isdir(os.path.expanduser(tgt)):
3113 elif os.path.isdir(os.path.expanduser(tgt)):
3114 raise ValueError("'%s' is a directory, not a regular file." % target)
3114 raise ValueError("'%s' is a directory, not a regular file." % target)
3115
3115
3116 try: # User namespace
3116 try: # User namespace
3117 codeobj = eval(target, self.user_ns)
3117 codeobj = eval(target, self.user_ns)
3118 except Exception:
3118 except Exception:
3119 raise ValueError(("'%s' was not found in history, as a file, url, "
3119 raise ValueError(("'%s' was not found in history, as a file, url, "
3120 "nor in the user namespace.") % target)
3120 "nor in the user namespace.") % target)
3121 if isinstance(codeobj, string_types):
3121 if isinstance(codeobj, string_types):
3122 return codeobj
3122 return codeobj
3123 elif isinstance(codeobj, Macro):
3123 elif isinstance(codeobj, Macro):
3124 return codeobj.value
3124 return codeobj.value
3125
3125
3126 raise TypeError("%s is neither a string nor a macro." % target,
3126 raise TypeError("%s is neither a string nor a macro." % target,
3127 codeobj)
3127 codeobj)
3128
3128
3129 #-------------------------------------------------------------------------
3129 #-------------------------------------------------------------------------
3130 # Things related to IPython exiting
3130 # Things related to IPython exiting
3131 #-------------------------------------------------------------------------
3131 #-------------------------------------------------------------------------
3132 def atexit_operations(self):
3132 def atexit_operations(self):
3133 """This will be executed at the time of exit.
3133 """This will be executed at the time of exit.
3134
3134
3135 Cleanup operations and saving of persistent data that is done
3135 Cleanup operations and saving of persistent data that is done
3136 unconditionally by IPython should be performed here.
3136 unconditionally by IPython should be performed here.
3137
3137
3138 For things that may depend on startup flags or platform specifics (such
3138 For things that may depend on startup flags or platform specifics (such
3139 as having readline or not), register a separate atexit function in the
3139 as having readline or not), register a separate atexit function in the
3140 code that has the appropriate information, rather than trying to
3140 code that has the appropriate information, rather than trying to
3141 clutter
3141 clutter
3142 """
3142 """
3143 # Close the history session (this stores the end time and line count)
3143 # Close the history session (this stores the end time and line count)
3144 # this must be *before* the tempfile cleanup, in case of temporary
3144 # this must be *before* the tempfile cleanup, in case of temporary
3145 # history db
3145 # history db
3146 self.history_manager.end_session()
3146 self.history_manager.end_session()
3147
3147
3148 # Cleanup all tempfiles left around
3148 # Cleanup all tempfiles left around
3149 for tfile in self.tempfiles:
3149 for tfile in self.tempfiles:
3150 try:
3150 try:
3151 os.unlink(tfile)
3151 os.unlink(tfile)
3152 except OSError:
3152 except OSError:
3153 pass
3153 pass
3154
3154
3155 # Clear all user namespaces to release all references cleanly.
3155 # Clear all user namespaces to release all references cleanly.
3156 self.reset(new_session=False)
3156 self.reset(new_session=False)
3157
3157
3158 # Run user hooks
3158 # Run user hooks
3159 self.hooks.shutdown_hook()
3159 self.hooks.shutdown_hook()
3160
3160
3161 def cleanup(self):
3161 def cleanup(self):
3162 self.restore_sys_module_state()
3162 self.restore_sys_module_state()
3163
3163
3164
3164
3165 class InteractiveShellABC(with_metaclass(abc.ABCMeta, object)):
3165 class InteractiveShellABC(with_metaclass(abc.ABCMeta, object)):
3166 """An abstract base class for InteractiveShell."""
3166 """An abstract base class for InteractiveShell."""
3167
3167
3168 InteractiveShellABC.register(InteractiveShell)
3168 InteractiveShellABC.register(InteractiveShell)
@@ -1,1267 +1,1269 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """
2 """
3 ultratb.py -- Spice up your tracebacks!
3 ultratb.py -- Spice up your tracebacks!
4
4
5 * ColorTB
5 **ColorTB**
6
6 I've always found it a bit hard to visually parse tracebacks in Python. The
7 I've always found it a bit hard to visually parse tracebacks in Python. The
7 ColorTB class is a solution to that problem. It colors the different parts of a
8 ColorTB class is a solution to that problem. It colors the different parts of a
8 traceback in a manner similar to what you would expect from a syntax-highlighting
9 traceback in a manner similar to what you would expect from a syntax-highlighting
9 text editor.
10 text editor.
10
11
11 Installation instructions for ColorTB::
12 Installation instructions for ColorTB::
12
13
13 import sys,ultratb
14 import sys,ultratb
14 sys.excepthook = ultratb.ColorTB()
15 sys.excepthook = ultratb.ColorTB()
15
16
16 * VerboseTB
17 **VerboseTB**
18
17 I've also included a port of Ka-Ping Yee's "cgitb.py" that produces all kinds
19 I've also included a port of Ka-Ping Yee's "cgitb.py" that produces all kinds
18 of useful info when a traceback occurs. Ping originally had it spit out HTML
20 of useful info when a traceback occurs. Ping originally had it spit out HTML
19 and intended it for CGI programmers, but why should they have all the fun? I
21 and intended it for CGI programmers, but why should they have all the fun? I
20 altered it to spit out colored text to the terminal. It's a bit overwhelming,
22 altered it to spit out colored text to the terminal. It's a bit overwhelming,
21 but kind of neat, and maybe useful for long-running programs that you believe
23 but kind of neat, and maybe useful for long-running programs that you believe
22 are bug-free. If a crash *does* occur in that type of program you want details.
24 are bug-free. If a crash *does* occur in that type of program you want details.
23 Give it a shot--you'll love it or you'll hate it.
25 Give it a shot--you'll love it or you'll hate it.
24
26
25 .. note::
27 .. note::
26
28
27 The Verbose mode prints the variables currently visible where the exception
29 The Verbose mode prints the variables currently visible where the exception
28 happened (shortening their strings if too long). This can potentially be
30 happened (shortening their strings if too long). This can potentially be
29 very slow, if you happen to have a huge data structure whose string
31 very slow, if you happen to have a huge data structure whose string
30 representation is complex to compute. Your computer may appear to freeze for
32 representation is complex to compute. Your computer may appear to freeze for
31 a while with cpu usage at 100%. If this occurs, you can cancel the traceback
33 a while with cpu usage at 100%. If this occurs, you can cancel the traceback
32 with Ctrl-C (maybe hitting it more than once).
34 with Ctrl-C (maybe hitting it more than once).
33
35
34 If you encounter this kind of situation often, you may want to use the
36 If you encounter this kind of situation often, you may want to use the
35 Verbose_novars mode instead of the regular Verbose, which avoids formatting
37 Verbose_novars mode instead of the regular Verbose, which avoids formatting
36 variables (but otherwise includes the information and context given by
38 variables (but otherwise includes the information and context given by
37 Verbose).
39 Verbose).
38
40
39
41
40 Installation instructions for ColorTB::
42 Installation instructions for ColorTB::
41
43
42 import sys,ultratb
44 import sys,ultratb
43 sys.excepthook = ultratb.VerboseTB()
45 sys.excepthook = ultratb.VerboseTB()
44
46
45 Note: Much of the code in this module was lifted verbatim from the standard
47 Note: Much of the code in this module was lifted verbatim from the standard
46 library module 'traceback.py' and Ka-Ping Yee's 'cgitb.py'.
48 library module 'traceback.py' and Ka-Ping Yee's 'cgitb.py'.
47
49
48 Color schemes
50 Color schemes
49 -------------
51 -------------
50
52
51 The colors are defined in the class TBTools through the use of the
53 The colors are defined in the class TBTools through the use of the
52 ColorSchemeTable class. Currently the following exist:
54 ColorSchemeTable class. Currently the following exist:
53
55
54 - NoColor: allows all of this module to be used in any terminal (the color
56 - NoColor: allows all of this module to be used in any terminal (the color
55 escapes are just dummy blank strings).
57 escapes are just dummy blank strings).
56
58
57 - Linux: is meant to look good in a terminal like the Linux console (black
59 - Linux: is meant to look good in a terminal like the Linux console (black
58 or very dark background).
60 or very dark background).
59
61
60 - LightBG: similar to Linux but swaps dark/light colors to be more readable
62 - LightBG: similar to Linux but swaps dark/light colors to be more readable
61 in light background terminals.
63 in light background terminals.
62
64
63 You can implement other color schemes easily, the syntax is fairly
65 You can implement other color schemes easily, the syntax is fairly
64 self-explanatory. Please send back new schemes you develop to the author for
66 self-explanatory. Please send back new schemes you develop to the author for
65 possible inclusion in future releases.
67 possible inclusion in future releases.
66
68
67 Inheritance diagram:
69 Inheritance diagram:
68
70
69 .. inheritance-diagram:: IPython.core.ultratb
71 .. inheritance-diagram:: IPython.core.ultratb
70 :parts: 3
72 :parts: 3
71 """
73 """
72
74
73 #*****************************************************************************
75 #*****************************************************************************
74 # Copyright (C) 2001 Nathaniel Gray <n8gray@caltech.edu>
76 # Copyright (C) 2001 Nathaniel Gray <n8gray@caltech.edu>
75 # Copyright (C) 2001-2004 Fernando Perez <fperez@colorado.edu>
77 # Copyright (C) 2001-2004 Fernando Perez <fperez@colorado.edu>
76 #
78 #
77 # Distributed under the terms of the BSD License. The full license is in
79 # Distributed under the terms of the BSD License. The full license is in
78 # the file COPYING, distributed as part of this software.
80 # the file COPYING, distributed as part of this software.
79 #*****************************************************************************
81 #*****************************************************************************
80
82
81 from __future__ import unicode_literals
83 from __future__ import unicode_literals
82 from __future__ import print_function
84 from __future__ import print_function
83
85
84 import inspect
86 import inspect
85 import keyword
87 import keyword
86 import linecache
88 import linecache
87 import os
89 import os
88 import pydoc
90 import pydoc
89 import re
91 import re
90 import sys
92 import sys
91 import time
93 import time
92 import tokenize
94 import tokenize
93 import traceback
95 import traceback
94 import types
96 import types
95
97
96 try: # Python 2
98 try: # Python 2
97 generate_tokens = tokenize.generate_tokens
99 generate_tokens = tokenize.generate_tokens
98 except AttributeError: # Python 3
100 except AttributeError: # Python 3
99 generate_tokens = tokenize.tokenize
101 generate_tokens = tokenize.tokenize
100
102
101 # For purposes of monkeypatching inspect to fix a bug in it.
103 # For purposes of monkeypatching inspect to fix a bug in it.
102 from inspect import getsourcefile, getfile, getmodule,\
104 from inspect import getsourcefile, getfile, getmodule,\
103 ismodule, isclass, ismethod, isfunction, istraceback, isframe, iscode
105 ismodule, isclass, ismethod, isfunction, istraceback, isframe, iscode
104
106
105 # IPython's own modules
107 # IPython's own modules
106 # Modified pdb which doesn't damage IPython's readline handling
108 # Modified pdb which doesn't damage IPython's readline handling
107 from IPython import get_ipython
109 from IPython import get_ipython
108 from IPython.core import debugger
110 from IPython.core import debugger
109 from IPython.core.display_trap import DisplayTrap
111 from IPython.core.display_trap import DisplayTrap
110 from IPython.core.excolors import exception_colors
112 from IPython.core.excolors import exception_colors
111 from IPython.utils import PyColorize
113 from IPython.utils import PyColorize
112 from IPython.utils import io
114 from IPython.utils import io
113 from IPython.utils import openpy
115 from IPython.utils import openpy
114 from IPython.utils import path as util_path
116 from IPython.utils import path as util_path
115 from IPython.utils import py3compat
117 from IPython.utils import py3compat
116 from IPython.utils import ulinecache
118 from IPython.utils import ulinecache
117 from IPython.utils.data import uniq_stable
119 from IPython.utils.data import uniq_stable
118 from IPython.utils.warn import info, error
120 from IPython.utils.warn import info, error
119
121
120 # Globals
122 # Globals
121 # amount of space to put line numbers before verbose tracebacks
123 # amount of space to put line numbers before verbose tracebacks
122 INDENT_SIZE = 8
124 INDENT_SIZE = 8
123
125
124 # Default color scheme. This is used, for example, by the traceback
126 # Default color scheme. This is used, for example, by the traceback
125 # formatter. When running in an actual IPython instance, the user's rc.colors
127 # formatter. When running in an actual IPython instance, the user's rc.colors
126 # value is used, but havinga module global makes this functionality available
128 # value is used, but havinga module global makes this functionality available
127 # to users of ultratb who are NOT running inside ipython.
129 # to users of ultratb who are NOT running inside ipython.
128 DEFAULT_SCHEME = 'NoColor'
130 DEFAULT_SCHEME = 'NoColor'
129
131
130 #---------------------------------------------------------------------------
132 #---------------------------------------------------------------------------
131 # Code begins
133 # Code begins
132
134
133 # Utility functions
135 # Utility functions
134 def inspect_error():
136 def inspect_error():
135 """Print a message about internal inspect errors.
137 """Print a message about internal inspect errors.
136
138
137 These are unfortunately quite common."""
139 These are unfortunately quite common."""
138
140
139 error('Internal Python error in the inspect module.\n'
141 error('Internal Python error in the inspect module.\n'
140 'Below is the traceback from this internal error.\n')
142 'Below is the traceback from this internal error.\n')
141
143
142 # This function is a monkeypatch we apply to the Python inspect module. We have
144 # This function is a monkeypatch we apply to the Python inspect module. We have
143 # now found when it's needed (see discussion on issue gh-1456), and we have a
145 # now found when it's needed (see discussion on issue gh-1456), and we have a
144 # test case (IPython.core.tests.test_ultratb.ChangedPyFileTest) that fails if
146 # test case (IPython.core.tests.test_ultratb.ChangedPyFileTest) that fails if
145 # the monkeypatch is not applied. TK, Aug 2012.
147 # the monkeypatch is not applied. TK, Aug 2012.
146 def findsource(object):
148 def findsource(object):
147 """Return the entire source file and starting line number for an object.
149 """Return the entire source file and starting line number for an object.
148
150
149 The argument may be a module, class, method, function, traceback, frame,
151 The argument may be a module, class, method, function, traceback, frame,
150 or code object. The source code is returned as a list of all the lines
152 or code object. The source code is returned as a list of all the lines
151 in the file and the line number indexes a line in that list. An IOError
153 in the file and the line number indexes a line in that list. An IOError
152 is raised if the source code cannot be retrieved.
154 is raised if the source code cannot be retrieved.
153
155
154 FIXED version with which we monkeypatch the stdlib to work around a bug."""
156 FIXED version with which we monkeypatch the stdlib to work around a bug."""
155
157
156 file = getsourcefile(object) or getfile(object)
158 file = getsourcefile(object) or getfile(object)
157 # If the object is a frame, then trying to get the globals dict from its
159 # If the object is a frame, then trying to get the globals dict from its
158 # module won't work. Instead, the frame object itself has the globals
160 # module won't work. Instead, the frame object itself has the globals
159 # dictionary.
161 # dictionary.
160 globals_dict = None
162 globals_dict = None
161 if inspect.isframe(object):
163 if inspect.isframe(object):
162 # XXX: can this ever be false?
164 # XXX: can this ever be false?
163 globals_dict = object.f_globals
165 globals_dict = object.f_globals
164 else:
166 else:
165 module = getmodule(object, file)
167 module = getmodule(object, file)
166 if module:
168 if module:
167 globals_dict = module.__dict__
169 globals_dict = module.__dict__
168 lines = linecache.getlines(file, globals_dict)
170 lines = linecache.getlines(file, globals_dict)
169 if not lines:
171 if not lines:
170 raise IOError('could not get source code')
172 raise IOError('could not get source code')
171
173
172 if ismodule(object):
174 if ismodule(object):
173 return lines, 0
175 return lines, 0
174
176
175 if isclass(object):
177 if isclass(object):
176 name = object.__name__
178 name = object.__name__
177 pat = re.compile(r'^(\s*)class\s*' + name + r'\b')
179 pat = re.compile(r'^(\s*)class\s*' + name + r'\b')
178 # make some effort to find the best matching class definition:
180 # make some effort to find the best matching class definition:
179 # use the one with the least indentation, which is the one
181 # use the one with the least indentation, which is the one
180 # that's most probably not inside a function definition.
182 # that's most probably not inside a function definition.
181 candidates = []
183 candidates = []
182 for i in range(len(lines)):
184 for i in range(len(lines)):
183 match = pat.match(lines[i])
185 match = pat.match(lines[i])
184 if match:
186 if match:
185 # if it's at toplevel, it's already the best one
187 # if it's at toplevel, it's already the best one
186 if lines[i][0] == 'c':
188 if lines[i][0] == 'c':
187 return lines, i
189 return lines, i
188 # else add whitespace to candidate list
190 # else add whitespace to candidate list
189 candidates.append((match.group(1), i))
191 candidates.append((match.group(1), i))
190 if candidates:
192 if candidates:
191 # this will sort by whitespace, and by line number,
193 # this will sort by whitespace, and by line number,
192 # less whitespace first
194 # less whitespace first
193 candidates.sort()
195 candidates.sort()
194 return lines, candidates[0][1]
196 return lines, candidates[0][1]
195 else:
197 else:
196 raise IOError('could not find class definition')
198 raise IOError('could not find class definition')
197
199
198 if ismethod(object):
200 if ismethod(object):
199 object = object.__func__
201 object = object.__func__
200 if isfunction(object):
202 if isfunction(object):
201 object = object.__code__
203 object = object.__code__
202 if istraceback(object):
204 if istraceback(object):
203 object = object.tb_frame
205 object = object.tb_frame
204 if isframe(object):
206 if isframe(object):
205 object = object.f_code
207 object = object.f_code
206 if iscode(object):
208 if iscode(object):
207 if not hasattr(object, 'co_firstlineno'):
209 if not hasattr(object, 'co_firstlineno'):
208 raise IOError('could not find function definition')
210 raise IOError('could not find function definition')
209 pat = re.compile(r'^(\s*def\s)|(.*(?<!\w)lambda(:|\s))|^(\s*@)')
211 pat = re.compile(r'^(\s*def\s)|(.*(?<!\w)lambda(:|\s))|^(\s*@)')
210 pmatch = pat.match
212 pmatch = pat.match
211 # fperez - fix: sometimes, co_firstlineno can give a number larger than
213 # fperez - fix: sometimes, co_firstlineno can give a number larger than
212 # the length of lines, which causes an error. Safeguard against that.
214 # the length of lines, which causes an error. Safeguard against that.
213 lnum = min(object.co_firstlineno,len(lines))-1
215 lnum = min(object.co_firstlineno,len(lines))-1
214 while lnum > 0:
216 while lnum > 0:
215 if pmatch(lines[lnum]): break
217 if pmatch(lines[lnum]): break
216 lnum -= 1
218 lnum -= 1
217
219
218 return lines, lnum
220 return lines, lnum
219 raise IOError('could not find code object')
221 raise IOError('could not find code object')
220
222
221 # Monkeypatch inspect to apply our bugfix. This code only works with Python >= 2.5
223 # Monkeypatch inspect to apply our bugfix. This code only works with Python >= 2.5
222 inspect.findsource = findsource
224 inspect.findsource = findsource
223
225
224 def fix_frame_records_filenames(records):
226 def fix_frame_records_filenames(records):
225 """Try to fix the filenames in each record from inspect.getinnerframes().
227 """Try to fix the filenames in each record from inspect.getinnerframes().
226
228
227 Particularly, modules loaded from within zip files have useless filenames
229 Particularly, modules loaded from within zip files have useless filenames
228 attached to their code object, and inspect.getinnerframes() just uses it.
230 attached to their code object, and inspect.getinnerframes() just uses it.
229 """
231 """
230 fixed_records = []
232 fixed_records = []
231 for frame, filename, line_no, func_name, lines, index in records:
233 for frame, filename, line_no, func_name, lines, index in records:
232 # Look inside the frame's globals dictionary for __file__, which should
234 # Look inside the frame's globals dictionary for __file__, which should
233 # be better.
235 # be better.
234 better_fn = frame.f_globals.get('__file__', None)
236 better_fn = frame.f_globals.get('__file__', None)
235 if isinstance(better_fn, str):
237 if isinstance(better_fn, str):
236 # Check the type just in case someone did something weird with
238 # Check the type just in case someone did something weird with
237 # __file__. It might also be None if the error occurred during
239 # __file__. It might also be None if the error occurred during
238 # import.
240 # import.
239 filename = better_fn
241 filename = better_fn
240 fixed_records.append((frame, filename, line_no, func_name, lines, index))
242 fixed_records.append((frame, filename, line_no, func_name, lines, index))
241 return fixed_records
243 return fixed_records
242
244
243
245
244 def _fixed_getinnerframes(etb, context=1,tb_offset=0):
246 def _fixed_getinnerframes(etb, context=1,tb_offset=0):
245 LNUM_POS, LINES_POS, INDEX_POS = 2, 4, 5
247 LNUM_POS, LINES_POS, INDEX_POS = 2, 4, 5
246
248
247 records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))
249 records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))
248
250
249 # If the error is at the console, don't build any context, since it would
251 # If the error is at the console, don't build any context, since it would
250 # otherwise produce 5 blank lines printed out (there is no file at the
252 # otherwise produce 5 blank lines printed out (there is no file at the
251 # console)
253 # console)
252 rec_check = records[tb_offset:]
254 rec_check = records[tb_offset:]
253 try:
255 try:
254 rname = rec_check[0][1]
256 rname = rec_check[0][1]
255 if rname == '<ipython console>' or rname.endswith('<string>'):
257 if rname == '<ipython console>' or rname.endswith('<string>'):
256 return rec_check
258 return rec_check
257 except IndexError:
259 except IndexError:
258 pass
260 pass
259
261
260 aux = traceback.extract_tb(etb)
262 aux = traceback.extract_tb(etb)
261 assert len(records) == len(aux)
263 assert len(records) == len(aux)
262 for i, (file, lnum, _, _) in zip(range(len(records)), aux):
264 for i, (file, lnum, _, _) in zip(range(len(records)), aux):
263 maybeStart = lnum-1 - context//2
265 maybeStart = lnum-1 - context//2
264 start = max(maybeStart, 0)
266 start = max(maybeStart, 0)
265 end = start + context
267 end = start + context
266 lines = ulinecache.getlines(file)[start:end]
268 lines = ulinecache.getlines(file)[start:end]
267 buf = list(records[i])
269 buf = list(records[i])
268 buf[LNUM_POS] = lnum
270 buf[LNUM_POS] = lnum
269 buf[INDEX_POS] = lnum - 1 - start
271 buf[INDEX_POS] = lnum - 1 - start
270 buf[LINES_POS] = lines
272 buf[LINES_POS] = lines
271 records[i] = tuple(buf)
273 records[i] = tuple(buf)
272 return records[tb_offset:]
274 return records[tb_offset:]
273
275
274 # Helper function -- largely belongs to VerboseTB, but we need the same
276 # Helper function -- largely belongs to VerboseTB, but we need the same
275 # functionality to produce a pseudo verbose TB for SyntaxErrors, so that they
277 # functionality to produce a pseudo verbose TB for SyntaxErrors, so that they
276 # can be recognized properly by ipython.el's py-traceback-line-re
278 # can be recognized properly by ipython.el's py-traceback-line-re
277 # (SyntaxErrors have to be treated specially because they have no traceback)
279 # (SyntaxErrors have to be treated specially because they have no traceback)
278
280
279 _parser = PyColorize.Parser()
281 _parser = PyColorize.Parser()
280
282
281 def _format_traceback_lines(lnum, index, lines, Colors, lvals=None,scheme=None):
283 def _format_traceback_lines(lnum, index, lines, Colors, lvals=None,scheme=None):
282 numbers_width = INDENT_SIZE - 1
284 numbers_width = INDENT_SIZE - 1
283 res = []
285 res = []
284 i = lnum - index
286 i = lnum - index
285
287
286 # This lets us get fully syntax-highlighted tracebacks.
288 # This lets us get fully syntax-highlighted tracebacks.
287 if scheme is None:
289 if scheme is None:
288 ipinst = get_ipython()
290 ipinst = get_ipython()
289 if ipinst is not None:
291 if ipinst is not None:
290 scheme = ipinst.colors
292 scheme = ipinst.colors
291 else:
293 else:
292 scheme = DEFAULT_SCHEME
294 scheme = DEFAULT_SCHEME
293
295
294 _line_format = _parser.format2
296 _line_format = _parser.format2
295
297
296 for line in lines:
298 for line in lines:
297 line = py3compat.cast_unicode(line)
299 line = py3compat.cast_unicode(line)
298
300
299 new_line, err = _line_format(line, 'str', scheme)
301 new_line, err = _line_format(line, 'str', scheme)
300 if not err: line = new_line
302 if not err: line = new_line
301
303
302 if i == lnum:
304 if i == lnum:
303 # This is the line with the error
305 # This is the line with the error
304 pad = numbers_width - len(str(i))
306 pad = numbers_width - len(str(i))
305 if pad >= 3:
307 if pad >= 3:
306 marker = '-'*(pad-3) + '-> '
308 marker = '-'*(pad-3) + '-> '
307 elif pad == 2:
309 elif pad == 2:
308 marker = '> '
310 marker = '> '
309 elif pad == 1:
311 elif pad == 1:
310 marker = '>'
312 marker = '>'
311 else:
313 else:
312 marker = ''
314 marker = ''
313 num = marker + str(i)
315 num = marker + str(i)
314 line = '%s%s%s %s%s' %(Colors.linenoEm, num,
316 line = '%s%s%s %s%s' %(Colors.linenoEm, num,
315 Colors.line, line, Colors.Normal)
317 Colors.line, line, Colors.Normal)
316 else:
318 else:
317 num = '%*s' % (numbers_width,i)
319 num = '%*s' % (numbers_width,i)
318 line = '%s%s%s %s' %(Colors.lineno, num,
320 line = '%s%s%s %s' %(Colors.lineno, num,
319 Colors.Normal, line)
321 Colors.Normal, line)
320
322
321 res.append(line)
323 res.append(line)
322 if lvals and i == lnum:
324 if lvals and i == lnum:
323 res.append(lvals + '\n')
325 res.append(lvals + '\n')
324 i = i + 1
326 i = i + 1
325 return res
327 return res
326
328
327
329
328 #---------------------------------------------------------------------------
330 #---------------------------------------------------------------------------
329 # Module classes
331 # Module classes
330 class TBTools(object):
332 class TBTools(object):
331 """Basic tools used by all traceback printer classes."""
333 """Basic tools used by all traceback printer classes."""
332
334
333 # Number of frames to skip when reporting tracebacks
335 # Number of frames to skip when reporting tracebacks
334 tb_offset = 0
336 tb_offset = 0
335
337
336 def __init__(self, color_scheme='NoColor', call_pdb=False, ostream=None):
338 def __init__(self, color_scheme='NoColor', call_pdb=False, ostream=None):
337 # Whether to call the interactive pdb debugger after printing
339 # Whether to call the interactive pdb debugger after printing
338 # tracebacks or not
340 # tracebacks or not
339 self.call_pdb = call_pdb
341 self.call_pdb = call_pdb
340
342
341 # Output stream to write to. Note that we store the original value in
343 # Output stream to write to. Note that we store the original value in
342 # a private attribute and then make the public ostream a property, so
344 # a private attribute and then make the public ostream a property, so
343 # that we can delay accessing io.stdout until runtime. The way
345 # that we can delay accessing io.stdout until runtime. The way
344 # things are written now, the io.stdout object is dynamically managed
346 # things are written now, the io.stdout object is dynamically managed
345 # so a reference to it should NEVER be stored statically. This
347 # so a reference to it should NEVER be stored statically. This
346 # property approach confines this detail to a single location, and all
348 # property approach confines this detail to a single location, and all
347 # subclasses can simply access self.ostream for writing.
349 # subclasses can simply access self.ostream for writing.
348 self._ostream = ostream
350 self._ostream = ostream
349
351
350 # Create color table
352 # Create color table
351 self.color_scheme_table = exception_colors()
353 self.color_scheme_table = exception_colors()
352
354
353 self.set_colors(color_scheme)
355 self.set_colors(color_scheme)
354 self.old_scheme = color_scheme # save initial value for toggles
356 self.old_scheme = color_scheme # save initial value for toggles
355
357
356 if call_pdb:
358 if call_pdb:
357 self.pdb = debugger.Pdb(self.color_scheme_table.active_scheme_name)
359 self.pdb = debugger.Pdb(self.color_scheme_table.active_scheme_name)
358 else:
360 else:
359 self.pdb = None
361 self.pdb = None
360
362
361 def _get_ostream(self):
363 def _get_ostream(self):
362 """Output stream that exceptions are written to.
364 """Output stream that exceptions are written to.
363
365
364 Valid values are:
366 Valid values are:
365
367
366 - None: the default, which means that IPython will dynamically resolve
368 - None: the default, which means that IPython will dynamically resolve
367 to io.stdout. This ensures compatibility with most tools, including
369 to io.stdout. This ensures compatibility with most tools, including
368 Windows (where plain stdout doesn't recognize ANSI escapes).
370 Windows (where plain stdout doesn't recognize ANSI escapes).
369
371
370 - Any object with 'write' and 'flush' attributes.
372 - Any object with 'write' and 'flush' attributes.
371 """
373 """
372 return io.stdout if self._ostream is None else self._ostream
374 return io.stdout if self._ostream is None else self._ostream
373
375
374 def _set_ostream(self, val):
376 def _set_ostream(self, val):
375 assert val is None or (hasattr(val, 'write') and hasattr(val, 'flush'))
377 assert val is None or (hasattr(val, 'write') and hasattr(val, 'flush'))
376 self._ostream = val
378 self._ostream = val
377
379
378 ostream = property(_get_ostream, _set_ostream)
380 ostream = property(_get_ostream, _set_ostream)
379
381
380 def set_colors(self,*args,**kw):
382 def set_colors(self,*args,**kw):
381 """Shorthand access to the color table scheme selector method."""
383 """Shorthand access to the color table scheme selector method."""
382
384
383 # Set own color table
385 # Set own color table
384 self.color_scheme_table.set_active_scheme(*args,**kw)
386 self.color_scheme_table.set_active_scheme(*args,**kw)
385 # for convenience, set Colors to the active scheme
387 # for convenience, set Colors to the active scheme
386 self.Colors = self.color_scheme_table.active_colors
388 self.Colors = self.color_scheme_table.active_colors
387 # Also set colors of debugger
389 # Also set colors of debugger
388 if hasattr(self,'pdb') and self.pdb is not None:
390 if hasattr(self,'pdb') and self.pdb is not None:
389 self.pdb.set_colors(*args,**kw)
391 self.pdb.set_colors(*args,**kw)
390
392
391 def color_toggle(self):
393 def color_toggle(self):
392 """Toggle between the currently active color scheme and NoColor."""
394 """Toggle between the currently active color scheme and NoColor."""
393
395
394 if self.color_scheme_table.active_scheme_name == 'NoColor':
396 if self.color_scheme_table.active_scheme_name == 'NoColor':
395 self.color_scheme_table.set_active_scheme(self.old_scheme)
397 self.color_scheme_table.set_active_scheme(self.old_scheme)
396 self.Colors = self.color_scheme_table.active_colors
398 self.Colors = self.color_scheme_table.active_colors
397 else:
399 else:
398 self.old_scheme = self.color_scheme_table.active_scheme_name
400 self.old_scheme = self.color_scheme_table.active_scheme_name
399 self.color_scheme_table.set_active_scheme('NoColor')
401 self.color_scheme_table.set_active_scheme('NoColor')
400 self.Colors = self.color_scheme_table.active_colors
402 self.Colors = self.color_scheme_table.active_colors
401
403
402 def stb2text(self, stb):
404 def stb2text(self, stb):
403 """Convert a structured traceback (a list) to a string."""
405 """Convert a structured traceback (a list) to a string."""
404 return '\n'.join(stb)
406 return '\n'.join(stb)
405
407
406 def text(self, etype, value, tb, tb_offset=None, context=5):
408 def text(self, etype, value, tb, tb_offset=None, context=5):
407 """Return formatted traceback.
409 """Return formatted traceback.
408
410
409 Subclasses may override this if they add extra arguments.
411 Subclasses may override this if they add extra arguments.
410 """
412 """
411 tb_list = self.structured_traceback(etype, value, tb,
413 tb_list = self.structured_traceback(etype, value, tb,
412 tb_offset, context)
414 tb_offset, context)
413 return self.stb2text(tb_list)
415 return self.stb2text(tb_list)
414
416
415 def structured_traceback(self, etype, evalue, tb, tb_offset=None,
417 def structured_traceback(self, etype, evalue, tb, tb_offset=None,
416 context=5, mode=None):
418 context=5, mode=None):
417 """Return a list of traceback frames.
419 """Return a list of traceback frames.
418
420
419 Must be implemented by each class.
421 Must be implemented by each class.
420 """
422 """
421 raise NotImplementedError()
423 raise NotImplementedError()
422
424
423
425
424 #---------------------------------------------------------------------------
426 #---------------------------------------------------------------------------
425 class ListTB(TBTools):
427 class ListTB(TBTools):
426 """Print traceback information from a traceback list, with optional color.
428 """Print traceback information from a traceback list, with optional color.
427
429
428 Calling requires 3 arguments: (etype, evalue, elist)
430 Calling requires 3 arguments: (etype, evalue, elist)
429 as would be obtained by::
431 as would be obtained by::
430
432
431 etype, evalue, tb = sys.exc_info()
433 etype, evalue, tb = sys.exc_info()
432 if tb:
434 if tb:
433 elist = traceback.extract_tb(tb)
435 elist = traceback.extract_tb(tb)
434 else:
436 else:
435 elist = None
437 elist = None
436
438
437 It can thus be used by programs which need to process the traceback before
439 It can thus be used by programs which need to process the traceback before
438 printing (such as console replacements based on the code module from the
440 printing (such as console replacements based on the code module from the
439 standard library).
441 standard library).
440
442
441 Because they are meant to be called without a full traceback (only a
443 Because they are meant to be called without a full traceback (only a
442 list), instances of this class can't call the interactive pdb debugger."""
444 list), instances of this class can't call the interactive pdb debugger."""
443
445
444 def __init__(self,color_scheme = 'NoColor', call_pdb=False, ostream=None):
446 def __init__(self,color_scheme = 'NoColor', call_pdb=False, ostream=None):
445 TBTools.__init__(self, color_scheme=color_scheme, call_pdb=call_pdb,
447 TBTools.__init__(self, color_scheme=color_scheme, call_pdb=call_pdb,
446 ostream=ostream)
448 ostream=ostream)
447
449
448 def __call__(self, etype, value, elist):
450 def __call__(self, etype, value, elist):
449 self.ostream.flush()
451 self.ostream.flush()
450 self.ostream.write(self.text(etype, value, elist))
452 self.ostream.write(self.text(etype, value, elist))
451 self.ostream.write('\n')
453 self.ostream.write('\n')
452
454
453 def structured_traceback(self, etype, value, elist, tb_offset=None,
455 def structured_traceback(self, etype, value, elist, tb_offset=None,
454 context=5):
456 context=5):
455 """Return a color formatted string with the traceback info.
457 """Return a color formatted string with the traceback info.
456
458
457 Parameters
459 Parameters
458 ----------
460 ----------
459 etype : exception type
461 etype : exception type
460 Type of the exception raised.
462 Type of the exception raised.
461
463
462 value : object
464 value : object
463 Data stored in the exception
465 Data stored in the exception
464
466
465 elist : list
467 elist : list
466 List of frames, see class docstring for details.
468 List of frames, see class docstring for details.
467
469
468 tb_offset : int, optional
470 tb_offset : int, optional
469 Number of frames in the traceback to skip. If not given, the
471 Number of frames in the traceback to skip. If not given, the
470 instance value is used (set in constructor).
472 instance value is used (set in constructor).
471
473
472 context : int, optional
474 context : int, optional
473 Number of lines of context information to print.
475 Number of lines of context information to print.
474
476
475 Returns
477 Returns
476 -------
478 -------
477 String with formatted exception.
479 String with formatted exception.
478 """
480 """
479 tb_offset = self.tb_offset if tb_offset is None else tb_offset
481 tb_offset = self.tb_offset if tb_offset is None else tb_offset
480 Colors = self.Colors
482 Colors = self.Colors
481 out_list = []
483 out_list = []
482 if elist:
484 if elist:
483
485
484 if tb_offset and len(elist) > tb_offset:
486 if tb_offset and len(elist) > tb_offset:
485 elist = elist[tb_offset:]
487 elist = elist[tb_offset:]
486
488
487 out_list.append('Traceback %s(most recent call last)%s:' %
489 out_list.append('Traceback %s(most recent call last)%s:' %
488 (Colors.normalEm, Colors.Normal) + '\n')
490 (Colors.normalEm, Colors.Normal) + '\n')
489 out_list.extend(self._format_list(elist))
491 out_list.extend(self._format_list(elist))
490 # The exception info should be a single entry in the list.
492 # The exception info should be a single entry in the list.
491 lines = ''.join(self._format_exception_only(etype, value))
493 lines = ''.join(self._format_exception_only(etype, value))
492 out_list.append(lines)
494 out_list.append(lines)
493
495
494 # Note: this code originally read:
496 # Note: this code originally read:
495
497
496 ## for line in lines[:-1]:
498 ## for line in lines[:-1]:
497 ## out_list.append(" "+line)
499 ## out_list.append(" "+line)
498 ## out_list.append(lines[-1])
500 ## out_list.append(lines[-1])
499
501
500 # This means it was indenting everything but the last line by a little
502 # This means it was indenting everything but the last line by a little
501 # bit. I've disabled this for now, but if we see ugliness somewhre we
503 # bit. I've disabled this for now, but if we see ugliness somewhre we
502 # can restore it.
504 # can restore it.
503
505
504 return out_list
506 return out_list
505
507
506 def _format_list(self, extracted_list):
508 def _format_list(self, extracted_list):
507 """Format a list of traceback entry tuples for printing.
509 """Format a list of traceback entry tuples for printing.
508
510
509 Given a list of tuples as returned by extract_tb() or
511 Given a list of tuples as returned by extract_tb() or
510 extract_stack(), return a list of strings ready for printing.
512 extract_stack(), return a list of strings ready for printing.
511 Each string in the resulting list corresponds to the item with the
513 Each string in the resulting list corresponds to the item with the
512 same index in the argument list. Each string ends in a newline;
514 same index in the argument list. Each string ends in a newline;
513 the strings may contain internal newlines as well, for those items
515 the strings may contain internal newlines as well, for those items
514 whose source text line is not None.
516 whose source text line is not None.
515
517
516 Lifted almost verbatim from traceback.py
518 Lifted almost verbatim from traceback.py
517 """
519 """
518
520
519 Colors = self.Colors
521 Colors = self.Colors
520 list = []
522 list = []
521 for filename, lineno, name, line in extracted_list[:-1]:
523 for filename, lineno, name, line in extracted_list[:-1]:
522 item = ' File %s"%s"%s, line %s%d%s, in %s%s%s\n' % \
524 item = ' File %s"%s"%s, line %s%d%s, in %s%s%s\n' % \
523 (Colors.filename, filename, Colors.Normal,
525 (Colors.filename, filename, Colors.Normal,
524 Colors.lineno, lineno, Colors.Normal,
526 Colors.lineno, lineno, Colors.Normal,
525 Colors.name, name, Colors.Normal)
527 Colors.name, name, Colors.Normal)
526 if line:
528 if line:
527 item += ' %s\n' % line.strip()
529 item += ' %s\n' % line.strip()
528 list.append(item)
530 list.append(item)
529 # Emphasize the last entry
531 # Emphasize the last entry
530 filename, lineno, name, line = extracted_list[-1]
532 filename, lineno, name, line = extracted_list[-1]
531 item = '%s File %s"%s"%s, line %s%d%s, in %s%s%s%s\n' % \
533 item = '%s File %s"%s"%s, line %s%d%s, in %s%s%s%s\n' % \
532 (Colors.normalEm,
534 (Colors.normalEm,
533 Colors.filenameEm, filename, Colors.normalEm,
535 Colors.filenameEm, filename, Colors.normalEm,
534 Colors.linenoEm, lineno, Colors.normalEm,
536 Colors.linenoEm, lineno, Colors.normalEm,
535 Colors.nameEm, name, Colors.normalEm,
537 Colors.nameEm, name, Colors.normalEm,
536 Colors.Normal)
538 Colors.Normal)
537 if line:
539 if line:
538 item += '%s %s%s\n' % (Colors.line, line.strip(),
540 item += '%s %s%s\n' % (Colors.line, line.strip(),
539 Colors.Normal)
541 Colors.Normal)
540 list.append(item)
542 list.append(item)
541 #from pprint import pformat; print 'LISTTB', pformat(list) # dbg
543 #from pprint import pformat; print 'LISTTB', pformat(list) # dbg
542 return list
544 return list
543
545
544 def _format_exception_only(self, etype, value):
546 def _format_exception_only(self, etype, value):
545 """Format the exception part of a traceback.
547 """Format the exception part of a traceback.
546
548
547 The arguments are the exception type and value such as given by
549 The arguments are the exception type and value such as given by
548 sys.exc_info()[:2]. The return value is a list of strings, each ending
550 sys.exc_info()[:2]. The return value is a list of strings, each ending
549 in a newline. Normally, the list contains a single string; however,
551 in a newline. Normally, the list contains a single string; however,
550 for SyntaxError exceptions, it contains several lines that (when
552 for SyntaxError exceptions, it contains several lines that (when
551 printed) display detailed information about where the syntax error
553 printed) display detailed information about where the syntax error
552 occurred. The message indicating which exception occurred is the
554 occurred. The message indicating which exception occurred is the
553 always last string in the list.
555 always last string in the list.
554
556
555 Also lifted nearly verbatim from traceback.py
557 Also lifted nearly verbatim from traceback.py
556 """
558 """
557 have_filedata = False
559 have_filedata = False
558 Colors = self.Colors
560 Colors = self.Colors
559 list = []
561 list = []
560 stype = Colors.excName + etype.__name__ + Colors.Normal
562 stype = Colors.excName + etype.__name__ + Colors.Normal
561 if value is None:
563 if value is None:
562 # Not sure if this can still happen in Python 2.6 and above
564 # Not sure if this can still happen in Python 2.6 and above
563 list.append( py3compat.cast_unicode(stype) + '\n')
565 list.append( py3compat.cast_unicode(stype) + '\n')
564 else:
566 else:
565 if issubclass(etype, SyntaxError):
567 if issubclass(etype, SyntaxError):
566 have_filedata = True
568 have_filedata = True
567 #print 'filename is',filename # dbg
569 #print 'filename is',filename # dbg
568 if not value.filename: value.filename = "<string>"
570 if not value.filename: value.filename = "<string>"
569 if value.lineno:
571 if value.lineno:
570 lineno = value.lineno
572 lineno = value.lineno
571 textline = ulinecache.getline(value.filename, value.lineno)
573 textline = ulinecache.getline(value.filename, value.lineno)
572 else:
574 else:
573 lineno = 'unknown'
575 lineno = 'unknown'
574 textline = ''
576 textline = ''
575 list.append('%s File %s"%s"%s, line %s%s%s\n' % \
577 list.append('%s File %s"%s"%s, line %s%s%s\n' % \
576 (Colors.normalEm,
578 (Colors.normalEm,
577 Colors.filenameEm, py3compat.cast_unicode(value.filename), Colors.normalEm,
579 Colors.filenameEm, py3compat.cast_unicode(value.filename), Colors.normalEm,
578 Colors.linenoEm, lineno, Colors.Normal ))
580 Colors.linenoEm, lineno, Colors.Normal ))
579 if textline == '':
581 if textline == '':
580 textline = py3compat.cast_unicode(value.text, "utf-8")
582 textline = py3compat.cast_unicode(value.text, "utf-8")
581
583
582 if textline is not None:
584 if textline is not None:
583 i = 0
585 i = 0
584 while i < len(textline) and textline[i].isspace():
586 while i < len(textline) and textline[i].isspace():
585 i += 1
587 i += 1
586 list.append('%s %s%s\n' % (Colors.line,
588 list.append('%s %s%s\n' % (Colors.line,
587 textline.strip(),
589 textline.strip(),
588 Colors.Normal))
590 Colors.Normal))
589 if value.offset is not None:
591 if value.offset is not None:
590 s = ' '
592 s = ' '
591 for c in textline[i:value.offset-1]:
593 for c in textline[i:value.offset-1]:
592 if c.isspace():
594 if c.isspace():
593 s += c
595 s += c
594 else:
596 else:
595 s += ' '
597 s += ' '
596 list.append('%s%s^%s\n' % (Colors.caret, s,
598 list.append('%s%s^%s\n' % (Colors.caret, s,
597 Colors.Normal) )
599 Colors.Normal) )
598
600
599 try:
601 try:
600 s = value.msg
602 s = value.msg
601 except Exception:
603 except Exception:
602 s = self._some_str(value)
604 s = self._some_str(value)
603 if s:
605 if s:
604 list.append('%s%s:%s %s\n' % (str(stype), Colors.excName,
606 list.append('%s%s:%s %s\n' % (str(stype), Colors.excName,
605 Colors.Normal, s))
607 Colors.Normal, s))
606 else:
608 else:
607 list.append('%s\n' % str(stype))
609 list.append('%s\n' % str(stype))
608
610
609 # sync with user hooks
611 # sync with user hooks
610 if have_filedata:
612 if have_filedata:
611 ipinst = get_ipython()
613 ipinst = get_ipython()
612 if ipinst is not None:
614 if ipinst is not None:
613 ipinst.hooks.synchronize_with_editor(value.filename, value.lineno, 0)
615 ipinst.hooks.synchronize_with_editor(value.filename, value.lineno, 0)
614
616
615 return list
617 return list
616
618
617 def get_exception_only(self, etype, value):
619 def get_exception_only(self, etype, value):
618 """Only print the exception type and message, without a traceback.
620 """Only print the exception type and message, without a traceback.
619
621
620 Parameters
622 Parameters
621 ----------
623 ----------
622 etype : exception type
624 etype : exception type
623 value : exception value
625 value : exception value
624 """
626 """
625 return ListTB.structured_traceback(self, etype, value, [])
627 return ListTB.structured_traceback(self, etype, value, [])
626
628
627
629
628 def show_exception_only(self, etype, evalue):
630 def show_exception_only(self, etype, evalue):
629 """Only print the exception type and message, without a traceback.
631 """Only print the exception type and message, without a traceback.
630
632
631 Parameters
633 Parameters
632 ----------
634 ----------
633 etype : exception type
635 etype : exception type
634 value : exception value
636 value : exception value
635 """
637 """
636 # This method needs to use __call__ from *this* class, not the one from
638 # This method needs to use __call__ from *this* class, not the one from
637 # a subclass whose signature or behavior may be different
639 # a subclass whose signature or behavior may be different
638 ostream = self.ostream
640 ostream = self.ostream
639 ostream.flush()
641 ostream.flush()
640 ostream.write('\n'.join(self.get_exception_only(etype, evalue)))
642 ostream.write('\n'.join(self.get_exception_only(etype, evalue)))
641 ostream.flush()
643 ostream.flush()
642
644
643 def _some_str(self, value):
645 def _some_str(self, value):
644 # Lifted from traceback.py
646 # Lifted from traceback.py
645 try:
647 try:
646 return str(value)
648 return str(value)
647 except:
649 except:
648 return '<unprintable %s object>' % type(value).__name__
650 return '<unprintable %s object>' % type(value).__name__
649
651
650 #----------------------------------------------------------------------------
652 #----------------------------------------------------------------------------
651 class VerboseTB(TBTools):
653 class VerboseTB(TBTools):
652 """A port of Ka-Ping Yee's cgitb.py module that outputs color text instead
654 """A port of Ka-Ping Yee's cgitb.py module that outputs color text instead
653 of HTML. Requires inspect and pydoc. Crazy, man.
655 of HTML. Requires inspect and pydoc. Crazy, man.
654
656
655 Modified version which optionally strips the topmost entries from the
657 Modified version which optionally strips the topmost entries from the
656 traceback, to be used with alternate interpreters (because their own code
658 traceback, to be used with alternate interpreters (because their own code
657 would appear in the traceback)."""
659 would appear in the traceback)."""
658
660
659 def __init__(self,color_scheme = 'Linux', call_pdb=False, ostream=None,
661 def __init__(self,color_scheme = 'Linux', call_pdb=False, ostream=None,
660 tb_offset=0, long_header=False, include_vars=True,
662 tb_offset=0, long_header=False, include_vars=True,
661 check_cache=None):
663 check_cache=None):
662 """Specify traceback offset, headers and color scheme.
664 """Specify traceback offset, headers and color scheme.
663
665
664 Define how many frames to drop from the tracebacks. Calling it with
666 Define how many frames to drop from the tracebacks. Calling it with
665 tb_offset=1 allows use of this handler in interpreters which will have
667 tb_offset=1 allows use of this handler in interpreters which will have
666 their own code at the top of the traceback (VerboseTB will first
668 their own code at the top of the traceback (VerboseTB will first
667 remove that frame before printing the traceback info)."""
669 remove that frame before printing the traceback info)."""
668 TBTools.__init__(self, color_scheme=color_scheme, call_pdb=call_pdb,
670 TBTools.__init__(self, color_scheme=color_scheme, call_pdb=call_pdb,
669 ostream=ostream)
671 ostream=ostream)
670 self.tb_offset = tb_offset
672 self.tb_offset = tb_offset
671 self.long_header = long_header
673 self.long_header = long_header
672 self.include_vars = include_vars
674 self.include_vars = include_vars
673 # By default we use linecache.checkcache, but the user can provide a
675 # By default we use linecache.checkcache, but the user can provide a
674 # different check_cache implementation. This is used by the IPython
676 # different check_cache implementation. This is used by the IPython
675 # kernel to provide tracebacks for interactive code that is cached,
677 # kernel to provide tracebacks for interactive code that is cached,
676 # by a compiler instance that flushes the linecache but preserves its
678 # by a compiler instance that flushes the linecache but preserves its
677 # own code cache.
679 # own code cache.
678 if check_cache is None:
680 if check_cache is None:
679 check_cache = linecache.checkcache
681 check_cache = linecache.checkcache
680 self.check_cache = check_cache
682 self.check_cache = check_cache
681
683
682 def structured_traceback(self, etype, evalue, etb, tb_offset=None,
684 def structured_traceback(self, etype, evalue, etb, tb_offset=None,
683 context=5):
685 context=5):
684 """Return a nice text document describing the traceback."""
686 """Return a nice text document describing the traceback."""
685
687
686 tb_offset = self.tb_offset if tb_offset is None else tb_offset
688 tb_offset = self.tb_offset if tb_offset is None else tb_offset
687
689
688 # some locals
690 # some locals
689 try:
691 try:
690 etype = etype.__name__
692 etype = etype.__name__
691 except AttributeError:
693 except AttributeError:
692 pass
694 pass
693 Colors = self.Colors # just a shorthand + quicker name lookup
695 Colors = self.Colors # just a shorthand + quicker name lookup
694 ColorsNormal = Colors.Normal # used a lot
696 ColorsNormal = Colors.Normal # used a lot
695 col_scheme = self.color_scheme_table.active_scheme_name
697 col_scheme = self.color_scheme_table.active_scheme_name
696 indent = ' '*INDENT_SIZE
698 indent = ' '*INDENT_SIZE
697 em_normal = '%s\n%s%s' % (Colors.valEm, indent,ColorsNormal)
699 em_normal = '%s\n%s%s' % (Colors.valEm, indent,ColorsNormal)
698 undefined = '%sundefined%s' % (Colors.em, ColorsNormal)
700 undefined = '%sundefined%s' % (Colors.em, ColorsNormal)
699 exc = '%s%s%s' % (Colors.excName,etype,ColorsNormal)
701 exc = '%s%s%s' % (Colors.excName,etype,ColorsNormal)
700
702
701 # some internal-use functions
703 # some internal-use functions
702 def text_repr(value):
704 def text_repr(value):
703 """Hopefully pretty robust repr equivalent."""
705 """Hopefully pretty robust repr equivalent."""
704 # this is pretty horrible but should always return *something*
706 # this is pretty horrible but should always return *something*
705 try:
707 try:
706 return pydoc.text.repr(value)
708 return pydoc.text.repr(value)
707 except KeyboardInterrupt:
709 except KeyboardInterrupt:
708 raise
710 raise
709 except:
711 except:
710 try:
712 try:
711 return repr(value)
713 return repr(value)
712 except KeyboardInterrupt:
714 except KeyboardInterrupt:
713 raise
715 raise
714 except:
716 except:
715 try:
717 try:
716 # all still in an except block so we catch
718 # all still in an except block so we catch
717 # getattr raising
719 # getattr raising
718 name = getattr(value, '__name__', None)
720 name = getattr(value, '__name__', None)
719 if name:
721 if name:
720 # ick, recursion
722 # ick, recursion
721 return text_repr(name)
723 return text_repr(name)
722 klass = getattr(value, '__class__', None)
724 klass = getattr(value, '__class__', None)
723 if klass:
725 if klass:
724 return '%s instance' % text_repr(klass)
726 return '%s instance' % text_repr(klass)
725 except KeyboardInterrupt:
727 except KeyboardInterrupt:
726 raise
728 raise
727 except:
729 except:
728 return 'UNRECOVERABLE REPR FAILURE'
730 return 'UNRECOVERABLE REPR FAILURE'
729 def eqrepr(value, repr=text_repr): return '=%s' % repr(value)
731 def eqrepr(value, repr=text_repr): return '=%s' % repr(value)
730 def nullrepr(value, repr=text_repr): return ''
732 def nullrepr(value, repr=text_repr): return ''
731
733
732 # meat of the code begins
734 # meat of the code begins
733 try:
735 try:
734 etype = etype.__name__
736 etype = etype.__name__
735 except AttributeError:
737 except AttributeError:
736 pass
738 pass
737
739
738 if self.long_header:
740 if self.long_header:
739 # Header with the exception type, python version, and date
741 # Header with the exception type, python version, and date
740 pyver = 'Python ' + sys.version.split()[0] + ': ' + sys.executable
742 pyver = 'Python ' + sys.version.split()[0] + ': ' + sys.executable
741 date = time.ctime(time.time())
743 date = time.ctime(time.time())
742
744
743 head = '%s%s%s\n%s%s%s\n%s' % (Colors.topline, '-'*75, ColorsNormal,
745 head = '%s%s%s\n%s%s%s\n%s' % (Colors.topline, '-'*75, ColorsNormal,
744 exc, ' '*(75-len(str(etype))-len(pyver)),
746 exc, ' '*(75-len(str(etype))-len(pyver)),
745 pyver, date.rjust(75) )
747 pyver, date.rjust(75) )
746 head += "\nA problem occured executing Python code. Here is the sequence of function"\
748 head += "\nA problem occured executing Python code. Here is the sequence of function"\
747 "\ncalls leading up to the error, with the most recent (innermost) call last."
749 "\ncalls leading up to the error, with the most recent (innermost) call last."
748 else:
750 else:
749 # Simplified header
751 # Simplified header
750 head = '%s%s%s\n%s%s' % (Colors.topline, '-'*75, ColorsNormal,exc,
752 head = '%s%s%s\n%s%s' % (Colors.topline, '-'*75, ColorsNormal,exc,
751 'Traceback (most recent call last)'.\
753 'Traceback (most recent call last)'.\
752 rjust(75 - len(str(etype)) ) )
754 rjust(75 - len(str(etype)) ) )
753 frames = []
755 frames = []
754 # Flush cache before calling inspect. This helps alleviate some of the
756 # Flush cache before calling inspect. This helps alleviate some of the
755 # problems with python 2.3's inspect.py.
757 # problems with python 2.3's inspect.py.
756 ##self.check_cache()
758 ##self.check_cache()
757 # Drop topmost frames if requested
759 # Drop topmost frames if requested
758 try:
760 try:
759 # Try the default getinnerframes and Alex's: Alex's fixes some
761 # Try the default getinnerframes and Alex's: Alex's fixes some
760 # problems, but it generates empty tracebacks for console errors
762 # problems, but it generates empty tracebacks for console errors
761 # (5 blanks lines) where none should be returned.
763 # (5 blanks lines) where none should be returned.
762 #records = inspect.getinnerframes(etb, context)[tb_offset:]
764 #records = inspect.getinnerframes(etb, context)[tb_offset:]
763 #print 'python records:', records # dbg
765 #print 'python records:', records # dbg
764 records = _fixed_getinnerframes(etb, context, tb_offset)
766 records = _fixed_getinnerframes(etb, context, tb_offset)
765 #print 'alex records:', records # dbg
767 #print 'alex records:', records # dbg
766 except:
768 except:
767
769
768 # FIXME: I've been getting many crash reports from python 2.3
770 # FIXME: I've been getting many crash reports from python 2.3
769 # users, traceable to inspect.py. If I can find a small test-case
771 # users, traceable to inspect.py. If I can find a small test-case
770 # to reproduce this, I should either write a better workaround or
772 # to reproduce this, I should either write a better workaround or
771 # file a bug report against inspect (if that's the real problem).
773 # file a bug report against inspect (if that's the real problem).
772 # So far, I haven't been able to find an isolated example to
774 # So far, I haven't been able to find an isolated example to
773 # reproduce the problem.
775 # reproduce the problem.
774 inspect_error()
776 inspect_error()
775 traceback.print_exc(file=self.ostream)
777 traceback.print_exc(file=self.ostream)
776 info('\nUnfortunately, your original traceback can not be constructed.\n')
778 info('\nUnfortunately, your original traceback can not be constructed.\n')
777 return ''
779 return ''
778
780
779 # build some color string templates outside these nested loops
781 # build some color string templates outside these nested loops
780 tpl_link = '%s%%s%s' % (Colors.filenameEm,ColorsNormal)
782 tpl_link = '%s%%s%s' % (Colors.filenameEm,ColorsNormal)
781 tpl_call = 'in %s%%s%s%%s%s' % (Colors.vName, Colors.valEm,
783 tpl_call = 'in %s%%s%s%%s%s' % (Colors.vName, Colors.valEm,
782 ColorsNormal)
784 ColorsNormal)
783 tpl_call_fail = 'in %s%%s%s(***failed resolving arguments***)%s' % \
785 tpl_call_fail = 'in %s%%s%s(***failed resolving arguments***)%s' % \
784 (Colors.vName, Colors.valEm, ColorsNormal)
786 (Colors.vName, Colors.valEm, ColorsNormal)
785 tpl_local_var = '%s%%s%s' % (Colors.vName, ColorsNormal)
787 tpl_local_var = '%s%%s%s' % (Colors.vName, ColorsNormal)
786 tpl_global_var = '%sglobal%s %s%%s%s' % (Colors.em, ColorsNormal,
788 tpl_global_var = '%sglobal%s %s%%s%s' % (Colors.em, ColorsNormal,
787 Colors.vName, ColorsNormal)
789 Colors.vName, ColorsNormal)
788 tpl_name_val = '%%s %s= %%s%s' % (Colors.valEm, ColorsNormal)
790 tpl_name_val = '%%s %s= %%s%s' % (Colors.valEm, ColorsNormal)
789 tpl_line = '%s%%s%s %%s' % (Colors.lineno, ColorsNormal)
791 tpl_line = '%s%%s%s %%s' % (Colors.lineno, ColorsNormal)
790 tpl_line_em = '%s%%s%s %%s%s' % (Colors.linenoEm,Colors.line,
792 tpl_line_em = '%s%%s%s %%s%s' % (Colors.linenoEm,Colors.line,
791 ColorsNormal)
793 ColorsNormal)
792
794
793 # now, loop over all records printing context and info
795 # now, loop over all records printing context and info
794 abspath = os.path.abspath
796 abspath = os.path.abspath
795 for frame, file, lnum, func, lines, index in records:
797 for frame, file, lnum, func, lines, index in records:
796 #print '*** record:',file,lnum,func,lines,index # dbg
798 #print '*** record:',file,lnum,func,lines,index # dbg
797 if not file:
799 if not file:
798 file = '?'
800 file = '?'
799 elif not(file.startswith(str("<")) and file.endswith(str(">"))):
801 elif not(file.startswith(str("<")) and file.endswith(str(">"))):
800 # Guess that filenames like <string> aren't real filenames, so
802 # Guess that filenames like <string> aren't real filenames, so
801 # don't call abspath on them.
803 # don't call abspath on them.
802 try:
804 try:
803 file = abspath(file)
805 file = abspath(file)
804 except OSError:
806 except OSError:
805 # Not sure if this can still happen: abspath now works with
807 # Not sure if this can still happen: abspath now works with
806 # file names like <string>
808 # file names like <string>
807 pass
809 pass
808 file = py3compat.cast_unicode(file, util_path.fs_encoding)
810 file = py3compat.cast_unicode(file, util_path.fs_encoding)
809 link = tpl_link % file
811 link = tpl_link % file
810 args, varargs, varkw, locals = inspect.getargvalues(frame)
812 args, varargs, varkw, locals = inspect.getargvalues(frame)
811
813
812 if func == '?':
814 if func == '?':
813 call = ''
815 call = ''
814 else:
816 else:
815 # Decide whether to include variable details or not
817 # Decide whether to include variable details or not
816 var_repr = self.include_vars and eqrepr or nullrepr
818 var_repr = self.include_vars and eqrepr or nullrepr
817 try:
819 try:
818 call = tpl_call % (func,inspect.formatargvalues(args,
820 call = tpl_call % (func,inspect.formatargvalues(args,
819 varargs, varkw,
821 varargs, varkw,
820 locals,formatvalue=var_repr))
822 locals,formatvalue=var_repr))
821 except KeyError:
823 except KeyError:
822 # This happens in situations like errors inside generator
824 # This happens in situations like errors inside generator
823 # expressions, where local variables are listed in the
825 # expressions, where local variables are listed in the
824 # line, but can't be extracted from the frame. I'm not
826 # line, but can't be extracted from the frame. I'm not
825 # 100% sure this isn't actually a bug in inspect itself,
827 # 100% sure this isn't actually a bug in inspect itself,
826 # but since there's no info for us to compute with, the
828 # but since there's no info for us to compute with, the
827 # best we can do is report the failure and move on. Here
829 # best we can do is report the failure and move on. Here
828 # we must *not* call any traceback construction again,
830 # we must *not* call any traceback construction again,
829 # because that would mess up use of %debug later on. So we
831 # because that would mess up use of %debug later on. So we
830 # simply report the failure and move on. The only
832 # simply report the failure and move on. The only
831 # limitation will be that this frame won't have locals
833 # limitation will be that this frame won't have locals
832 # listed in the call signature. Quite subtle problem...
834 # listed in the call signature. Quite subtle problem...
833 # I can't think of a good way to validate this in a unit
835 # I can't think of a good way to validate this in a unit
834 # test, but running a script consisting of:
836 # test, but running a script consisting of:
835 # dict( (k,v.strip()) for (k,v) in range(10) )
837 # dict( (k,v.strip()) for (k,v) in range(10) )
836 # will illustrate the error, if this exception catch is
838 # will illustrate the error, if this exception catch is
837 # disabled.
839 # disabled.
838 call = tpl_call_fail % func
840 call = tpl_call_fail % func
839
841
840 # Don't attempt to tokenize binary files.
842 # Don't attempt to tokenize binary files.
841 if file.endswith(('.so', '.pyd', '.dll')):
843 if file.endswith(('.so', '.pyd', '.dll')):
842 frames.append('%s %s\n' % (link,call))
844 frames.append('%s %s\n' % (link,call))
843 continue
845 continue
844 elif file.endswith(('.pyc','.pyo')):
846 elif file.endswith(('.pyc','.pyo')):
845 # Look up the corresponding source file.
847 # Look up the corresponding source file.
846 file = openpy.source_from_cache(file)
848 file = openpy.source_from_cache(file)
847
849
848 def linereader(file=file, lnum=[lnum], getline=ulinecache.getline):
850 def linereader(file=file, lnum=[lnum], getline=ulinecache.getline):
849 line = getline(file, lnum[0])
851 line = getline(file, lnum[0])
850 lnum[0] += 1
852 lnum[0] += 1
851 return line
853 return line
852
854
853 # Build the list of names on this line of code where the exception
855 # Build the list of names on this line of code where the exception
854 # occurred.
856 # occurred.
855 try:
857 try:
856 names = []
858 names = []
857 name_cont = False
859 name_cont = False
858
860
859 for token_type, token, start, end, line in generate_tokens(linereader):
861 for token_type, token, start, end, line in generate_tokens(linereader):
860 # build composite names
862 # build composite names
861 if token_type == tokenize.NAME and token not in keyword.kwlist:
863 if token_type == tokenize.NAME and token not in keyword.kwlist:
862 if name_cont:
864 if name_cont:
863 # Continuation of a dotted name
865 # Continuation of a dotted name
864 try:
866 try:
865 names[-1].append(token)
867 names[-1].append(token)
866 except IndexError:
868 except IndexError:
867 names.append([token])
869 names.append([token])
868 name_cont = False
870 name_cont = False
869 else:
871 else:
870 # Regular new names. We append everything, the caller
872 # Regular new names. We append everything, the caller
871 # will be responsible for pruning the list later. It's
873 # will be responsible for pruning the list later. It's
872 # very tricky to try to prune as we go, b/c composite
874 # very tricky to try to prune as we go, b/c composite
873 # names can fool us. The pruning at the end is easy
875 # names can fool us. The pruning at the end is easy
874 # to do (or the caller can print a list with repeated
876 # to do (or the caller can print a list with repeated
875 # names if so desired.
877 # names if so desired.
876 names.append([token])
878 names.append([token])
877 elif token == '.':
879 elif token == '.':
878 name_cont = True
880 name_cont = True
879 elif token_type == tokenize.NEWLINE:
881 elif token_type == tokenize.NEWLINE:
880 break
882 break
881
883
882 except (IndexError, UnicodeDecodeError):
884 except (IndexError, UnicodeDecodeError):
883 # signals exit of tokenizer
885 # signals exit of tokenizer
884 pass
886 pass
885 except tokenize.TokenError as msg:
887 except tokenize.TokenError as msg:
886 _m = ("An unexpected error occurred while tokenizing input\n"
888 _m = ("An unexpected error occurred while tokenizing input\n"
887 "The following traceback may be corrupted or invalid\n"
889 "The following traceback may be corrupted or invalid\n"
888 "The error message is: %s\n" % msg)
890 "The error message is: %s\n" % msg)
889 error(_m)
891 error(_m)
890
892
891 # Join composite names (e.g. "dict.fromkeys")
893 # Join composite names (e.g. "dict.fromkeys")
892 names = ['.'.join(n) for n in names]
894 names = ['.'.join(n) for n in names]
893 # prune names list of duplicates, but keep the right order
895 # prune names list of duplicates, but keep the right order
894 unique_names = uniq_stable(names)
896 unique_names = uniq_stable(names)
895
897
896 # Start loop over vars
898 # Start loop over vars
897 lvals = []
899 lvals = []
898 if self.include_vars:
900 if self.include_vars:
899 for name_full in unique_names:
901 for name_full in unique_names:
900 name_base = name_full.split('.',1)[0]
902 name_base = name_full.split('.',1)[0]
901 if name_base in frame.f_code.co_varnames:
903 if name_base in frame.f_code.co_varnames:
902 if name_base in locals:
904 if name_base in locals:
903 try:
905 try:
904 value = repr(eval(name_full,locals))
906 value = repr(eval(name_full,locals))
905 except:
907 except:
906 value = undefined
908 value = undefined
907 else:
909 else:
908 value = undefined
910 value = undefined
909 name = tpl_local_var % name_full
911 name = tpl_local_var % name_full
910 else:
912 else:
911 if name_base in frame.f_globals:
913 if name_base in frame.f_globals:
912 try:
914 try:
913 value = repr(eval(name_full,frame.f_globals))
915 value = repr(eval(name_full,frame.f_globals))
914 except:
916 except:
915 value = undefined
917 value = undefined
916 else:
918 else:
917 value = undefined
919 value = undefined
918 name = tpl_global_var % name_full
920 name = tpl_global_var % name_full
919 lvals.append(tpl_name_val % (name,value))
921 lvals.append(tpl_name_val % (name,value))
920 if lvals:
922 if lvals:
921 lvals = '%s%s' % (indent,em_normal.join(lvals))
923 lvals = '%s%s' % (indent,em_normal.join(lvals))
922 else:
924 else:
923 lvals = ''
925 lvals = ''
924
926
925 level = '%s %s\n' % (link,call)
927 level = '%s %s\n' % (link,call)
926
928
927 if index is None:
929 if index is None:
928 frames.append(level)
930 frames.append(level)
929 else:
931 else:
930 frames.append('%s%s' % (level,''.join(
932 frames.append('%s%s' % (level,''.join(
931 _format_traceback_lines(lnum,index,lines,Colors,lvals,
933 _format_traceback_lines(lnum,index,lines,Colors,lvals,
932 col_scheme))))
934 col_scheme))))
933
935
934 # Get (safely) a string form of the exception info
936 # Get (safely) a string form of the exception info
935 try:
937 try:
936 etype_str,evalue_str = map(str,(etype,evalue))
938 etype_str,evalue_str = map(str,(etype,evalue))
937 except:
939 except:
938 # User exception is improperly defined.
940 # User exception is improperly defined.
939 etype,evalue = str,sys.exc_info()[:2]
941 etype,evalue = str,sys.exc_info()[:2]
940 etype_str,evalue_str = map(str,(etype,evalue))
942 etype_str,evalue_str = map(str,(etype,evalue))
941 # ... and format it
943 # ... and format it
942 exception = ['%s%s%s: %s' % (Colors.excName, etype_str,
944 exception = ['%s%s%s: %s' % (Colors.excName, etype_str,
943 ColorsNormal, py3compat.cast_unicode(evalue_str))]
945 ColorsNormal, py3compat.cast_unicode(evalue_str))]
944 if (not py3compat.PY3) and type(evalue) is types.InstanceType:
946 if (not py3compat.PY3) and type(evalue) is types.InstanceType:
945 try:
947 try:
946 names = [w for w in dir(evalue) if isinstance(w, py3compat.string_types)]
948 names = [w for w in dir(evalue) if isinstance(w, py3compat.string_types)]
947 except:
949 except:
948 # Every now and then, an object with funny inernals blows up
950 # Every now and then, an object with funny inernals blows up
949 # when dir() is called on it. We do the best we can to report
951 # when dir() is called on it. We do the best we can to report
950 # the problem and continue
952 # the problem and continue
951 _m = '%sException reporting error (object with broken dir())%s:'
953 _m = '%sException reporting error (object with broken dir())%s:'
952 exception.append(_m % (Colors.excName,ColorsNormal))
954 exception.append(_m % (Colors.excName,ColorsNormal))
953 etype_str,evalue_str = map(str,sys.exc_info()[:2])
955 etype_str,evalue_str = map(str,sys.exc_info()[:2])
954 exception.append('%s%s%s: %s' % (Colors.excName,etype_str,
956 exception.append('%s%s%s: %s' % (Colors.excName,etype_str,
955 ColorsNormal, py3compat.cast_unicode(evalue_str)))
957 ColorsNormal, py3compat.cast_unicode(evalue_str)))
956 names = []
958 names = []
957 for name in names:
959 for name in names:
958 value = text_repr(getattr(evalue, name))
960 value = text_repr(getattr(evalue, name))
959 exception.append('\n%s%s = %s' % (indent, name, value))
961 exception.append('\n%s%s = %s' % (indent, name, value))
960
962
961 # vds: >>
963 # vds: >>
962 if records:
964 if records:
963 filepath, lnum = records[-1][1:3]
965 filepath, lnum = records[-1][1:3]
964 #print "file:", str(file), "linenb", str(lnum) # dbg
966 #print "file:", str(file), "linenb", str(lnum) # dbg
965 filepath = os.path.abspath(filepath)
967 filepath = os.path.abspath(filepath)
966 ipinst = get_ipython()
968 ipinst = get_ipython()
967 if ipinst is not None:
969 if ipinst is not None:
968 ipinst.hooks.synchronize_with_editor(filepath, lnum, 0)
970 ipinst.hooks.synchronize_with_editor(filepath, lnum, 0)
969 # vds: <<
971 # vds: <<
970
972
971 # return all our info assembled as a single string
973 # return all our info assembled as a single string
972 # return '%s\n\n%s\n%s' % (head,'\n'.join(frames),''.join(exception[0]) )
974 # return '%s\n\n%s\n%s' % (head,'\n'.join(frames),''.join(exception[0]) )
973 return [head] + frames + [''.join(exception[0])]
975 return [head] + frames + [''.join(exception[0])]
974
976
975 def debugger(self,force=False):
977 def debugger(self,force=False):
976 """Call up the pdb debugger if desired, always clean up the tb
978 """Call up the pdb debugger if desired, always clean up the tb
977 reference.
979 reference.
978
980
979 Keywords:
981 Keywords:
980
982
981 - force(False): by default, this routine checks the instance call_pdb
983 - force(False): by default, this routine checks the instance call_pdb
982 flag and does not actually invoke the debugger if the flag is false.
984 flag and does not actually invoke the debugger if the flag is false.
983 The 'force' option forces the debugger to activate even if the flag
985 The 'force' option forces the debugger to activate even if the flag
984 is false.
986 is false.
985
987
986 If the call_pdb flag is set, the pdb interactive debugger is
988 If the call_pdb flag is set, the pdb interactive debugger is
987 invoked. In all cases, the self.tb reference to the current traceback
989 invoked. In all cases, the self.tb reference to the current traceback
988 is deleted to prevent lingering references which hamper memory
990 is deleted to prevent lingering references which hamper memory
989 management.
991 management.
990
992
991 Note that each call to pdb() does an 'import readline', so if your app
993 Note that each call to pdb() does an 'import readline', so if your app
992 requires a special setup for the readline completers, you'll have to
994 requires a special setup for the readline completers, you'll have to
993 fix that by hand after invoking the exception handler."""
995 fix that by hand after invoking the exception handler."""
994
996
995 if force or self.call_pdb:
997 if force or self.call_pdb:
996 if self.pdb is None:
998 if self.pdb is None:
997 self.pdb = debugger.Pdb(
999 self.pdb = debugger.Pdb(
998 self.color_scheme_table.active_scheme_name)
1000 self.color_scheme_table.active_scheme_name)
999 # the system displayhook may have changed, restore the original
1001 # the system displayhook may have changed, restore the original
1000 # for pdb
1002 # for pdb
1001 display_trap = DisplayTrap(hook=sys.__displayhook__)
1003 display_trap = DisplayTrap(hook=sys.__displayhook__)
1002 with display_trap:
1004 with display_trap:
1003 self.pdb.reset()
1005 self.pdb.reset()
1004 # Find the right frame so we don't pop up inside ipython itself
1006 # Find the right frame so we don't pop up inside ipython itself
1005 if hasattr(self,'tb') and self.tb is not None:
1007 if hasattr(self,'tb') and self.tb is not None:
1006 etb = self.tb
1008 etb = self.tb
1007 else:
1009 else:
1008 etb = self.tb = sys.last_traceback
1010 etb = self.tb = sys.last_traceback
1009 while self.tb is not None and self.tb.tb_next is not None:
1011 while self.tb is not None and self.tb.tb_next is not None:
1010 self.tb = self.tb.tb_next
1012 self.tb = self.tb.tb_next
1011 if etb and etb.tb_next:
1013 if etb and etb.tb_next:
1012 etb = etb.tb_next
1014 etb = etb.tb_next
1013 self.pdb.botframe = etb.tb_frame
1015 self.pdb.botframe = etb.tb_frame
1014 self.pdb.interaction(self.tb.tb_frame, self.tb)
1016 self.pdb.interaction(self.tb.tb_frame, self.tb)
1015
1017
1016 if hasattr(self,'tb'):
1018 if hasattr(self,'tb'):
1017 del self.tb
1019 del self.tb
1018
1020
1019 def handler(self, info=None):
1021 def handler(self, info=None):
1020 (etype, evalue, etb) = info or sys.exc_info()
1022 (etype, evalue, etb) = info or sys.exc_info()
1021 self.tb = etb
1023 self.tb = etb
1022 ostream = self.ostream
1024 ostream = self.ostream
1023 ostream.flush()
1025 ostream.flush()
1024 ostream.write(self.text(etype, evalue, etb))
1026 ostream.write(self.text(etype, evalue, etb))
1025 ostream.write('\n')
1027 ostream.write('\n')
1026 ostream.flush()
1028 ostream.flush()
1027
1029
1028 # Changed so an instance can just be called as VerboseTB_inst() and print
1030 # Changed so an instance can just be called as VerboseTB_inst() and print
1029 # out the right info on its own.
1031 # out the right info on its own.
1030 def __call__(self, etype=None, evalue=None, etb=None):
1032 def __call__(self, etype=None, evalue=None, etb=None):
1031 """This hook can replace sys.excepthook (for Python 2.1 or higher)."""
1033 """This hook can replace sys.excepthook (for Python 2.1 or higher)."""
1032 if etb is None:
1034 if etb is None:
1033 self.handler()
1035 self.handler()
1034 else:
1036 else:
1035 self.handler((etype, evalue, etb))
1037 self.handler((etype, evalue, etb))
1036 try:
1038 try:
1037 self.debugger()
1039 self.debugger()
1038 except KeyboardInterrupt:
1040 except KeyboardInterrupt:
1039 print("\nKeyboardInterrupt")
1041 print("\nKeyboardInterrupt")
1040
1042
1041 #----------------------------------------------------------------------------
1043 #----------------------------------------------------------------------------
1042 class FormattedTB(VerboseTB, ListTB):
1044 class FormattedTB(VerboseTB, ListTB):
1043 """Subclass ListTB but allow calling with a traceback.
1045 """Subclass ListTB but allow calling with a traceback.
1044
1046
1045 It can thus be used as a sys.excepthook for Python > 2.1.
1047 It can thus be used as a sys.excepthook for Python > 2.1.
1046
1048
1047 Also adds 'Context' and 'Verbose' modes, not available in ListTB.
1049 Also adds 'Context' and 'Verbose' modes, not available in ListTB.
1048
1050
1049 Allows a tb_offset to be specified. This is useful for situations where
1051 Allows a tb_offset to be specified. This is useful for situations where
1050 one needs to remove a number of topmost frames from the traceback (such as
1052 one needs to remove a number of topmost frames from the traceback (such as
1051 occurs with python programs that themselves execute other python code,
1053 occurs with python programs that themselves execute other python code,
1052 like Python shells). """
1054 like Python shells). """
1053
1055
1054 def __init__(self, mode='Plain', color_scheme='Linux', call_pdb=False,
1056 def __init__(self, mode='Plain', color_scheme='Linux', call_pdb=False,
1055 ostream=None,
1057 ostream=None,
1056 tb_offset=0, long_header=False, include_vars=False,
1058 tb_offset=0, long_header=False, include_vars=False,
1057 check_cache=None):
1059 check_cache=None):
1058
1060
1059 # NEVER change the order of this list. Put new modes at the end:
1061 # NEVER change the order of this list. Put new modes at the end:
1060 self.valid_modes = ['Plain','Context','Verbose']
1062 self.valid_modes = ['Plain','Context','Verbose']
1061 self.verbose_modes = self.valid_modes[1:3]
1063 self.verbose_modes = self.valid_modes[1:3]
1062
1064
1063 VerboseTB.__init__(self, color_scheme=color_scheme, call_pdb=call_pdb,
1065 VerboseTB.__init__(self, color_scheme=color_scheme, call_pdb=call_pdb,
1064 ostream=ostream, tb_offset=tb_offset,
1066 ostream=ostream, tb_offset=tb_offset,
1065 long_header=long_header, include_vars=include_vars,
1067 long_header=long_header, include_vars=include_vars,
1066 check_cache=check_cache)
1068 check_cache=check_cache)
1067
1069
1068 # Different types of tracebacks are joined with different separators to
1070 # Different types of tracebacks are joined with different separators to
1069 # form a single string. They are taken from this dict
1071 # form a single string. They are taken from this dict
1070 self._join_chars = dict(Plain='', Context='\n', Verbose='\n')
1072 self._join_chars = dict(Plain='', Context='\n', Verbose='\n')
1071 # set_mode also sets the tb_join_char attribute
1073 # set_mode also sets the tb_join_char attribute
1072 self.set_mode(mode)
1074 self.set_mode(mode)
1073
1075
1074 def _extract_tb(self,tb):
1076 def _extract_tb(self,tb):
1075 if tb:
1077 if tb:
1076 return traceback.extract_tb(tb)
1078 return traceback.extract_tb(tb)
1077 else:
1079 else:
1078 return None
1080 return None
1079
1081
1080 def structured_traceback(self, etype, value, tb, tb_offset=None, context=5):
1082 def structured_traceback(self, etype, value, tb, tb_offset=None, context=5):
1081 tb_offset = self.tb_offset if tb_offset is None else tb_offset
1083 tb_offset = self.tb_offset if tb_offset is None else tb_offset
1082 mode = self.mode
1084 mode = self.mode
1083 if mode in self.verbose_modes:
1085 if mode in self.verbose_modes:
1084 # Verbose modes need a full traceback
1086 # Verbose modes need a full traceback
1085 return VerboseTB.structured_traceback(
1087 return VerboseTB.structured_traceback(
1086 self, etype, value, tb, tb_offset, context
1088 self, etype, value, tb, tb_offset, context
1087 )
1089 )
1088 else:
1090 else:
1089 # We must check the source cache because otherwise we can print
1091 # We must check the source cache because otherwise we can print
1090 # out-of-date source code.
1092 # out-of-date source code.
1091 self.check_cache()
1093 self.check_cache()
1092 # Now we can extract and format the exception
1094 # Now we can extract and format the exception
1093 elist = self._extract_tb(tb)
1095 elist = self._extract_tb(tb)
1094 return ListTB.structured_traceback(
1096 return ListTB.structured_traceback(
1095 self, etype, value, elist, tb_offset, context
1097 self, etype, value, elist, tb_offset, context
1096 )
1098 )
1097
1099
1098 def stb2text(self, stb):
1100 def stb2text(self, stb):
1099 """Convert a structured traceback (a list) to a string."""
1101 """Convert a structured traceback (a list) to a string."""
1100 return self.tb_join_char.join(stb)
1102 return self.tb_join_char.join(stb)
1101
1103
1102
1104
1103 def set_mode(self,mode=None):
1105 def set_mode(self,mode=None):
1104 """Switch to the desired mode.
1106 """Switch to the desired mode.
1105
1107
1106 If mode is not specified, cycles through the available modes."""
1108 If mode is not specified, cycles through the available modes."""
1107
1109
1108 if not mode:
1110 if not mode:
1109 new_idx = ( self.valid_modes.index(self.mode) + 1 ) % \
1111 new_idx = ( self.valid_modes.index(self.mode) + 1 ) % \
1110 len(self.valid_modes)
1112 len(self.valid_modes)
1111 self.mode = self.valid_modes[new_idx]
1113 self.mode = self.valid_modes[new_idx]
1112 elif mode not in self.valid_modes:
1114 elif mode not in self.valid_modes:
1113 raise ValueError('Unrecognized mode in FormattedTB: <'+mode+'>\n'
1115 raise ValueError('Unrecognized mode in FormattedTB: <'+mode+'>\n'
1114 'Valid modes: '+str(self.valid_modes))
1116 'Valid modes: '+str(self.valid_modes))
1115 else:
1117 else:
1116 self.mode = mode
1118 self.mode = mode
1117 # include variable details only in 'Verbose' mode
1119 # include variable details only in 'Verbose' mode
1118 self.include_vars = (self.mode == self.valid_modes[2])
1120 self.include_vars = (self.mode == self.valid_modes[2])
1119 # Set the join character for generating text tracebacks
1121 # Set the join character for generating text tracebacks
1120 self.tb_join_char = self._join_chars[self.mode]
1122 self.tb_join_char = self._join_chars[self.mode]
1121
1123
1122 # some convenient shorcuts
1124 # some convenient shorcuts
1123 def plain(self):
1125 def plain(self):
1124 self.set_mode(self.valid_modes[0])
1126 self.set_mode(self.valid_modes[0])
1125
1127
1126 def context(self):
1128 def context(self):
1127 self.set_mode(self.valid_modes[1])
1129 self.set_mode(self.valid_modes[1])
1128
1130
1129 def verbose(self):
1131 def verbose(self):
1130 self.set_mode(self.valid_modes[2])
1132 self.set_mode(self.valid_modes[2])
1131
1133
1132 #----------------------------------------------------------------------------
1134 #----------------------------------------------------------------------------
1133 class AutoFormattedTB(FormattedTB):
1135 class AutoFormattedTB(FormattedTB):
1134 """A traceback printer which can be called on the fly.
1136 """A traceback printer which can be called on the fly.
1135
1137
1136 It will find out about exceptions by itself.
1138 It will find out about exceptions by itself.
1137
1139
1138 A brief example::
1140 A brief example::
1139
1141
1140 AutoTB = AutoFormattedTB(mode = 'Verbose',color_scheme='Linux')
1142 AutoTB = AutoFormattedTB(mode = 'Verbose',color_scheme='Linux')
1141 try:
1143 try:
1142 ...
1144 ...
1143 except:
1145 except:
1144 AutoTB() # or AutoTB(out=logfile) where logfile is an open file object
1146 AutoTB() # or AutoTB(out=logfile) where logfile is an open file object
1145 """
1147 """
1146
1148
1147 def __call__(self,etype=None,evalue=None,etb=None,
1149 def __call__(self,etype=None,evalue=None,etb=None,
1148 out=None,tb_offset=None):
1150 out=None,tb_offset=None):
1149 """Print out a formatted exception traceback.
1151 """Print out a formatted exception traceback.
1150
1152
1151 Optional arguments:
1153 Optional arguments:
1152 - out: an open file-like object to direct output to.
1154 - out: an open file-like object to direct output to.
1153
1155
1154 - tb_offset: the number of frames to skip over in the stack, on a
1156 - tb_offset: the number of frames to skip over in the stack, on a
1155 per-call basis (this overrides temporarily the instance's tb_offset
1157 per-call basis (this overrides temporarily the instance's tb_offset
1156 given at initialization time. """
1158 given at initialization time. """
1157
1159
1158
1160
1159 if out is None:
1161 if out is None:
1160 out = self.ostream
1162 out = self.ostream
1161 out.flush()
1163 out.flush()
1162 out.write(self.text(etype, evalue, etb, tb_offset))
1164 out.write(self.text(etype, evalue, etb, tb_offset))
1163 out.write('\n')
1165 out.write('\n')
1164 out.flush()
1166 out.flush()
1165 # FIXME: we should remove the auto pdb behavior from here and leave
1167 # FIXME: we should remove the auto pdb behavior from here and leave
1166 # that to the clients.
1168 # that to the clients.
1167 try:
1169 try:
1168 self.debugger()
1170 self.debugger()
1169 except KeyboardInterrupt:
1171 except KeyboardInterrupt:
1170 print("\nKeyboardInterrupt")
1172 print("\nKeyboardInterrupt")
1171
1173
1172 def structured_traceback(self, etype=None, value=None, tb=None,
1174 def structured_traceback(self, etype=None, value=None, tb=None,
1173 tb_offset=None, context=5):
1175 tb_offset=None, context=5):
1174 if etype is None:
1176 if etype is None:
1175 etype,value,tb = sys.exc_info()
1177 etype,value,tb = sys.exc_info()
1176 self.tb = tb
1178 self.tb = tb
1177 return FormattedTB.structured_traceback(
1179 return FormattedTB.structured_traceback(
1178 self, etype, value, tb, tb_offset, context)
1180 self, etype, value, tb, tb_offset, context)
1179
1181
1180 #---------------------------------------------------------------------------
1182 #---------------------------------------------------------------------------
1181
1183
1182 # A simple class to preserve Nathan's original functionality.
1184 # A simple class to preserve Nathan's original functionality.
1183 class ColorTB(FormattedTB):
1185 class ColorTB(FormattedTB):
1184 """Shorthand to initialize a FormattedTB in Linux colors mode."""
1186 """Shorthand to initialize a FormattedTB in Linux colors mode."""
1185 def __init__(self,color_scheme='Linux',call_pdb=0):
1187 def __init__(self,color_scheme='Linux',call_pdb=0):
1186 FormattedTB.__init__(self,color_scheme=color_scheme,
1188 FormattedTB.__init__(self,color_scheme=color_scheme,
1187 call_pdb=call_pdb)
1189 call_pdb=call_pdb)
1188
1190
1189
1191
1190 class SyntaxTB(ListTB):
1192 class SyntaxTB(ListTB):
1191 """Extension which holds some state: the last exception value"""
1193 """Extension which holds some state: the last exception value"""
1192
1194
1193 def __init__(self,color_scheme = 'NoColor'):
1195 def __init__(self,color_scheme = 'NoColor'):
1194 ListTB.__init__(self,color_scheme)
1196 ListTB.__init__(self,color_scheme)
1195 self.last_syntax_error = None
1197 self.last_syntax_error = None
1196
1198
1197 def __call__(self, etype, value, elist):
1199 def __call__(self, etype, value, elist):
1198 self.last_syntax_error = value
1200 self.last_syntax_error = value
1199 ListTB.__call__(self,etype,value,elist)
1201 ListTB.__call__(self,etype,value,elist)
1200
1202
1201 def structured_traceback(self, etype, value, elist, tb_offset=None,
1203 def structured_traceback(self, etype, value, elist, tb_offset=None,
1202 context=5):
1204 context=5):
1203 # If the source file has been edited, the line in the syntax error can
1205 # If the source file has been edited, the line in the syntax error can
1204 # be wrong (retrieved from an outdated cache). This replaces it with
1206 # be wrong (retrieved from an outdated cache). This replaces it with
1205 # the current value.
1207 # the current value.
1206 if isinstance(value, SyntaxError) \
1208 if isinstance(value, SyntaxError) \
1207 and isinstance(value.filename, py3compat.string_types) \
1209 and isinstance(value.filename, py3compat.string_types) \
1208 and isinstance(value.lineno, int):
1210 and isinstance(value.lineno, int):
1209 linecache.checkcache(value.filename)
1211 linecache.checkcache(value.filename)
1210 newtext = ulinecache.getline(value.filename, value.lineno)
1212 newtext = ulinecache.getline(value.filename, value.lineno)
1211 if newtext:
1213 if newtext:
1212 value.text = newtext
1214 value.text = newtext
1213 return super(SyntaxTB, self).structured_traceback(etype, value, elist,
1215 return super(SyntaxTB, self).structured_traceback(etype, value, elist,
1214 tb_offset=tb_offset, context=context)
1216 tb_offset=tb_offset, context=context)
1215
1217
1216 def clear_err_state(self):
1218 def clear_err_state(self):
1217 """Return the current error state and clear it"""
1219 """Return the current error state and clear it"""
1218 e = self.last_syntax_error
1220 e = self.last_syntax_error
1219 self.last_syntax_error = None
1221 self.last_syntax_error = None
1220 return e
1222 return e
1221
1223
1222 def stb2text(self, stb):
1224 def stb2text(self, stb):
1223 """Convert a structured traceback (a list) to a string."""
1225 """Convert a structured traceback (a list) to a string."""
1224 return ''.join(stb)
1226 return ''.join(stb)
1225
1227
1226
1228
1227 #----------------------------------------------------------------------------
1229 #----------------------------------------------------------------------------
1228 # module testing (minimal)
1230 # module testing (minimal)
1229 if __name__ == "__main__":
1231 if __name__ == "__main__":
1230 def spam(c, d_e):
1232 def spam(c, d_e):
1231 (d, e) = d_e
1233 (d, e) = d_e
1232 x = c + d
1234 x = c + d
1233 y = c * d
1235 y = c * d
1234 foo(x, y)
1236 foo(x, y)
1235
1237
1236 def foo(a, b, bar=1):
1238 def foo(a, b, bar=1):
1237 eggs(a, b + bar)
1239 eggs(a, b + bar)
1238
1240
1239 def eggs(f, g, z=globals()):
1241 def eggs(f, g, z=globals()):
1240 h = f + g
1242 h = f + g
1241 i = f - g
1243 i = f - g
1242 return h / i
1244 return h / i
1243
1245
1244 print('')
1246 print('')
1245 print('*** Before ***')
1247 print('*** Before ***')
1246 try:
1248 try:
1247 print(spam(1, (2, 3)))
1249 print(spam(1, (2, 3)))
1248 except:
1250 except:
1249 traceback.print_exc()
1251 traceback.print_exc()
1250 print('')
1252 print('')
1251
1253
1252 handler = ColorTB()
1254 handler = ColorTB()
1253 print('*** ColorTB ***')
1255 print('*** ColorTB ***')
1254 try:
1256 try:
1255 print(spam(1, (2, 3)))
1257 print(spam(1, (2, 3)))
1256 except:
1258 except:
1257 handler(*sys.exc_info())
1259 handler(*sys.exc_info())
1258 print('')
1260 print('')
1259
1261
1260 handler = VerboseTB()
1262 handler = VerboseTB()
1261 print('*** VerboseTB ***')
1263 print('*** VerboseTB ***')
1262 try:
1264 try:
1263 print(spam(1, (2, 3)))
1265 print(spam(1, (2, 3)))
1264 except:
1266 except:
1265 handler(*sys.exc_info())
1267 handler(*sys.exc_info())
1266 print('')
1268 print('')
1267
1269
@@ -1,278 +1,281 b''
1 """Tornado handlers for the notebooks web service.
1 """Tornado handlers for the notebooks web service.
2
2
3 Authors:
3 Authors:
4
4
5 * Brian Granger
5 * Brian Granger
6 """
6 """
7
7
8 #-----------------------------------------------------------------------------
8 #-----------------------------------------------------------------------------
9 # Copyright (C) 2011 The IPython Development Team
9 # Copyright (C) 2011 The IPython Development Team
10 #
10 #
11 # Distributed under the terms of the BSD License. The full license is in
11 # Distributed under the terms of the BSD License. The full license is in
12 # the file COPYING, distributed as part of this software.
12 # the file COPYING, distributed as part of this software.
13 #-----------------------------------------------------------------------------
13 #-----------------------------------------------------------------------------
14
14
15 #-----------------------------------------------------------------------------
15 #-----------------------------------------------------------------------------
16 # Imports
16 # Imports
17 #-----------------------------------------------------------------------------
17 #-----------------------------------------------------------------------------
18
18
19 import json
19 import json
20
20
21 from tornado import web
21 from tornado import web
22
22
23 from IPython.html.utils import url_path_join, url_escape
23 from IPython.html.utils import url_path_join, url_escape
24 from IPython.utils.jsonutil import date_default
24 from IPython.utils.jsonutil import date_default
25
25
26 from IPython.html.base.handlers import IPythonHandler, json_errors
26 from IPython.html.base.handlers import IPythonHandler, json_errors
27
27
28 #-----------------------------------------------------------------------------
28 #-----------------------------------------------------------------------------
29 # Notebook web service handlers
29 # Notebook web service handlers
30 #-----------------------------------------------------------------------------
30 #-----------------------------------------------------------------------------
31
31
32
32
33 class NotebookHandler(IPythonHandler):
33 class NotebookHandler(IPythonHandler):
34
34
35 SUPPORTED_METHODS = (u'GET', u'PUT', u'PATCH', u'POST', u'DELETE')
35 SUPPORTED_METHODS = (u'GET', u'PUT', u'PATCH', u'POST', u'DELETE')
36
36
37 def notebook_location(self, name, path=''):
37 def notebook_location(self, name, path=''):
38 """Return the full URL location of a notebook based.
38 """Return the full URL location of a notebook based.
39
39
40 Parameters
40 Parameters
41 ----------
41 ----------
42 name : unicode
42 name : unicode
43 The base name of the notebook, such as "foo.ipynb".
43 The base name of the notebook, such as "foo.ipynb".
44 path : unicode
44 path : unicode
45 The URL path of the notebook.
45 The URL path of the notebook.
46 """
46 """
47 return url_escape(url_path_join(
47 return url_escape(url_path_join(
48 self.base_project_url, 'api', 'notebooks', path, name
48 self.base_project_url, 'api', 'notebooks', path, name
49 ))
49 ))
50
50
51 def _finish_model(self, model, location=True):
51 def _finish_model(self, model, location=True):
52 """Finish a JSON request with a model, setting relevant headers, etc."""
52 """Finish a JSON request with a model, setting relevant headers, etc."""
53 if location:
53 if location:
54 location = self.notebook_location(model['name'], model['path'])
54 location = self.notebook_location(model['name'], model['path'])
55 self.set_header('Location', location)
55 self.set_header('Location', location)
56 self.set_header('Last-Modified', model['last_modified'])
56 self.set_header('Last-Modified', model['last_modified'])
57 self.finish(json.dumps(model, default=date_default))
57 self.finish(json.dumps(model, default=date_default))
58
58
59 @web.authenticated
59 @web.authenticated
60 @json_errors
60 @json_errors
61 def get(self, path='', name=None):
61 def get(self, path='', name=None):
62 """Return a Notebook or list of notebooks.
62 """Return a Notebook or list of notebooks.
63
63
64 * GET with path and no notebook name lists notebooks in a directory
64 * GET with path and no notebook name lists notebooks in a directory
65 * GET with path and notebook name returns notebook JSON
65 * GET with path and notebook name returns notebook JSON
66 """
66 """
67 nbm = self.notebook_manager
67 nbm = self.notebook_manager
68 # Check to see if a notebook name was given
68 # Check to see if a notebook name was given
69 if name is None:
69 if name is None:
70 # List notebooks in 'path'
70 # List notebooks in 'path'
71 notebooks = nbm.list_notebooks(path)
71 notebooks = nbm.list_notebooks(path)
72 self.finish(json.dumps(notebooks, default=date_default))
72 self.finish(json.dumps(notebooks, default=date_default))
73 return
73 return
74 # get and return notebook representation
74 # get and return notebook representation
75 model = nbm.get_notebook_model(name, path)
75 model = nbm.get_notebook_model(name, path)
76 self._finish_model(model, location=False)
76 self._finish_model(model, location=False)
77
77
78 @web.authenticated
78 @web.authenticated
79 @json_errors
79 @json_errors
80 def patch(self, path='', name=None):
80 def patch(self, path='', name=None):
81 """PATCH renames a notebook without re-uploading content."""
81 """PATCH renames a notebook without re-uploading content."""
82 nbm = self.notebook_manager
82 nbm = self.notebook_manager
83 if name is None:
83 if name is None:
84 raise web.HTTPError(400, u'Notebook name missing')
84 raise web.HTTPError(400, u'Notebook name missing')
85 model = self.get_json_body()
85 model = self.get_json_body()
86 if model is None:
86 if model is None:
87 raise web.HTTPError(400, u'JSON body missing')
87 raise web.HTTPError(400, u'JSON body missing')
88 model = nbm.update_notebook_model(model, name, path)
88 model = nbm.update_notebook_model(model, name, path)
89 self._finish_model(model)
89 self._finish_model(model)
90
90
91 def _copy_notebook(self, copy_from, path, copy_to=None):
91 def _copy_notebook(self, copy_from, path, copy_to=None):
92 """Copy a notebook in path, optionally specifying the new name.
92 """Copy a notebook in path, optionally specifying the new name.
93
93
94 Only support copying within the same directory.
94 Only support copying within the same directory.
95 """
95 """
96 self.log.info(u"Copying notebook from %s/%s to %s/%s",
96 self.log.info(u"Copying notebook from %s/%s to %s/%s",
97 path, copy_from,
97 path, copy_from,
98 path, copy_to or '',
98 path, copy_to or '',
99 )
99 )
100 model = self.notebook_manager.copy_notebook(copy_from, copy_to, path)
100 model = self.notebook_manager.copy_notebook(copy_from, copy_to, path)
101 self.set_status(201)
101 self.set_status(201)
102 self._finish_model(model)
102 self._finish_model(model)
103
103
104 def _upload_notebook(self, model, path, name=None):
104 def _upload_notebook(self, model, path, name=None):
105 """Upload a notebook
105 """Upload a notebook
106
106
107 If name specified, create it in path/name.
107 If name specified, create it in path/name.
108 """
108 """
109 self.log.info(u"Uploading notebook to %s/%s", path, name or '')
109 self.log.info(u"Uploading notebook to %s/%s", path, name or '')
110 if name:
110 if name:
111 model['name'] = name
111 model['name'] = name
112
112
113 model = self.notebook_manager.create_notebook_model(model, path)
113 model = self.notebook_manager.create_notebook_model(model, path)
114 self.set_status(201)
114 self.set_status(201)
115 self._finish_model(model)
115 self._finish_model(model)
116
116
117 def _create_empty_notebook(self, path, name=None):
117 def _create_empty_notebook(self, path, name=None):
118 """Create an empty notebook in path
118 """Create an empty notebook in path
119
119
120 If name specified, create it in path/name.
120 If name specified, create it in path/name.
121 """
121 """
122 self.log.info(u"Creating new notebook in %s/%s", path, name or '')
122 self.log.info(u"Creating new notebook in %s/%s", path, name or '')
123 model = {}
123 model = {}
124 if name:
124 if name:
125 model['name'] = name
125 model['name'] = name
126 model = self.notebook_manager.create_notebook_model(model, path=path)
126 model = self.notebook_manager.create_notebook_model(model, path=path)
127 self.set_status(201)
127 self.set_status(201)
128 self._finish_model(model)
128 self._finish_model(model)
129
129
130 def _save_notebook(self, model, path, name):
130 def _save_notebook(self, model, path, name):
131 """Save an existing notebook."""
131 """Save an existing notebook."""
132 self.log.info(u"Saving notebook at %s/%s", path, name)
132 self.log.info(u"Saving notebook at %s/%s", path, name)
133 model = self.notebook_manager.save_notebook_model(model, name, path)
133 model = self.notebook_manager.save_notebook_model(model, name, path)
134 if model['path'] != path.strip('/') or model['name'] != name:
134 if model['path'] != path.strip('/') or model['name'] != name:
135 # a rename happened, set Location header
135 # a rename happened, set Location header
136 location = True
136 location = True
137 else:
137 else:
138 location = False
138 location = False
139 self._finish_model(model, location)
139 self._finish_model(model, location)
140
140
141 @web.authenticated
141 @web.authenticated
142 @json_errors
142 @json_errors
143 def post(self, path='', name=None):
143 def post(self, path='', name=None):
144 """Create a new notebook in the specified path.
144 """Create a new notebook in the specified path.
145
145
146 POST creates new notebooks. The server always decides on the notebook name.
146 POST creates new notebooks. The server always decides on the notebook name.
147
147
148 POST /api/notebooks/path : new untitled notebook in path
148 POST /api/notebooks/path
149 If content specified, upload a notebook, otherwise start empty.
149 New untitled notebook in path. If content specified, upload a
150 POST /api/notebooks/path?copy=OtherNotebook.ipynb : new copy of OtherNotebook in path
150 notebook, otherwise start empty.
151 POST /api/notebooks/path?copy=OtherNotebook.ipynb
152 New copy of OtherNotebook in path
151 """
153 """
152
154
153 if name is not None:
155 if name is not None:
154 raise web.HTTPError(400, "Only POST to directories. Use PUT for full names.")
156 raise web.HTTPError(400, "Only POST to directories. Use PUT for full names.")
155
157
156 model = self.get_json_body()
158 model = self.get_json_body()
157
159
158 if model is not None:
160 if model is not None:
159 copy_from = model.get('copy_from')
161 copy_from = model.get('copy_from')
160 if copy_from:
162 if copy_from:
161 if model.get('content'):
163 if model.get('content'):
162 raise web.HTTPError(400, "Can't upload and copy at the same time.")
164 raise web.HTTPError(400, "Can't upload and copy at the same time.")
163 self._copy_notebook(copy_from, path)
165 self._copy_notebook(copy_from, path)
164 else:
166 else:
165 self._upload_notebook(model, path)
167 self._upload_notebook(model, path)
166 else:
168 else:
167 self._create_empty_notebook(path)
169 self._create_empty_notebook(path)
168
170
169 @web.authenticated
171 @web.authenticated
170 @json_errors
172 @json_errors
171 def put(self, path='', name=None):
173 def put(self, path='', name=None):
172 """Saves the notebook in the location specified by name and path.
174 """Saves the notebook in the location specified by name and path.
173
175
174 PUT /api/notebooks/path/Name.ipynb : Save notebook at path/Name.ipynb
176 PUT is very similar to POST, but the requester specifies the name,
175 Notebook structure is specified in `content` key of JSON request body.
177 whereas with POST, the server picks the name.
176 If content is not specified, create a new empty notebook.
177 PUT /api/notebooks/path/Name.ipynb?copy=OtherNotebook.ipynb : copy OtherNotebook to Name
178
178
179 POST and PUT are basically the same. The only difference:
179 PUT /api/notebooks/path/Name.ipynb
180
180 Save notebook at ``path/Name.ipynb``. Notebook structure is specified
181 - with POST, server always picks the name, with PUT the requester does
181 in `content` key of JSON request body. If content is not specified,
182 create a new empty notebook.
183 PUT /api/notebooks/path/Name.ipynb?copy=OtherNotebook.ipynb
184 Copy OtherNotebook to Name
182 """
185 """
183 if name is None:
186 if name is None:
184 raise web.HTTPError(400, "Only PUT to full names. Use POST for directories.")
187 raise web.HTTPError(400, "Only PUT to full names. Use POST for directories.")
185
188
186 model = self.get_json_body()
189 model = self.get_json_body()
187 if model:
190 if model:
188 copy_from = model.get('copy_from')
191 copy_from = model.get('copy_from')
189 if copy_from:
192 if copy_from:
190 if model.get('content'):
193 if model.get('content'):
191 raise web.HTTPError(400, "Can't upload and copy at the same time.")
194 raise web.HTTPError(400, "Can't upload and copy at the same time.")
192 self._copy_notebook(copy_from, path, name)
195 self._copy_notebook(copy_from, path, name)
193 elif self.notebook_manager.notebook_exists(name, path):
196 elif self.notebook_manager.notebook_exists(name, path):
194 self._save_notebook(model, path, name)
197 self._save_notebook(model, path, name)
195 else:
198 else:
196 self._upload_notebook(model, path, name)
199 self._upload_notebook(model, path, name)
197 else:
200 else:
198 self._create_empty_notebook(path, name)
201 self._create_empty_notebook(path, name)
199
202
200 @web.authenticated
203 @web.authenticated
201 @json_errors
204 @json_errors
202 def delete(self, path='', name=None):
205 def delete(self, path='', name=None):
203 """delete the notebook in the given notebook path"""
206 """delete the notebook in the given notebook path"""
204 nbm = self.notebook_manager
207 nbm = self.notebook_manager
205 nbm.delete_notebook_model(name, path)
208 nbm.delete_notebook_model(name, path)
206 self.set_status(204)
209 self.set_status(204)
207 self.finish()
210 self.finish()
208
211
209
212
210 class NotebookCheckpointsHandler(IPythonHandler):
213 class NotebookCheckpointsHandler(IPythonHandler):
211
214
212 SUPPORTED_METHODS = ('GET', 'POST')
215 SUPPORTED_METHODS = ('GET', 'POST')
213
216
214 @web.authenticated
217 @web.authenticated
215 @json_errors
218 @json_errors
216 def get(self, path='', name=None):
219 def get(self, path='', name=None):
217 """get lists checkpoints for a notebook"""
220 """get lists checkpoints for a notebook"""
218 nbm = self.notebook_manager
221 nbm = self.notebook_manager
219 checkpoints = nbm.list_checkpoints(name, path)
222 checkpoints = nbm.list_checkpoints(name, path)
220 data = json.dumps(checkpoints, default=date_default)
223 data = json.dumps(checkpoints, default=date_default)
221 self.finish(data)
224 self.finish(data)
222
225
223 @web.authenticated
226 @web.authenticated
224 @json_errors
227 @json_errors
225 def post(self, path='', name=None):
228 def post(self, path='', name=None):
226 """post creates a new checkpoint"""
229 """post creates a new checkpoint"""
227 nbm = self.notebook_manager
230 nbm = self.notebook_manager
228 checkpoint = nbm.create_checkpoint(name, path)
231 checkpoint = nbm.create_checkpoint(name, path)
229 data = json.dumps(checkpoint, default=date_default)
232 data = json.dumps(checkpoint, default=date_default)
230 location = url_path_join(self.base_project_url, 'api/notebooks',
233 location = url_path_join(self.base_project_url, 'api/notebooks',
231 path, name, 'checkpoints', checkpoint['id'])
234 path, name, 'checkpoints', checkpoint['id'])
232 self.set_header('Location', url_escape(location))
235 self.set_header('Location', url_escape(location))
233 self.set_status(201)
236 self.set_status(201)
234 self.finish(data)
237 self.finish(data)
235
238
236
239
237 class ModifyNotebookCheckpointsHandler(IPythonHandler):
240 class ModifyNotebookCheckpointsHandler(IPythonHandler):
238
241
239 SUPPORTED_METHODS = ('POST', 'DELETE')
242 SUPPORTED_METHODS = ('POST', 'DELETE')
240
243
241 @web.authenticated
244 @web.authenticated
242 @json_errors
245 @json_errors
243 def post(self, path, name, checkpoint_id):
246 def post(self, path, name, checkpoint_id):
244 """post restores a notebook from a checkpoint"""
247 """post restores a notebook from a checkpoint"""
245 nbm = self.notebook_manager
248 nbm = self.notebook_manager
246 nbm.restore_checkpoint(checkpoint_id, name, path)
249 nbm.restore_checkpoint(checkpoint_id, name, path)
247 self.set_status(204)
250 self.set_status(204)
248 self.finish()
251 self.finish()
249
252
250 @web.authenticated
253 @web.authenticated
251 @json_errors
254 @json_errors
252 def delete(self, path, name, checkpoint_id):
255 def delete(self, path, name, checkpoint_id):
253 """delete clears a checkpoint for a given notebook"""
256 """delete clears a checkpoint for a given notebook"""
254 nbm = self.notebook_manager
257 nbm = self.notebook_manager
255 nbm.delete_checkpoint(checkpoint_id, name, path)
258 nbm.delete_checkpoint(checkpoint_id, name, path)
256 self.set_status(204)
259 self.set_status(204)
257 self.finish()
260 self.finish()
258
261
259 #-----------------------------------------------------------------------------
262 #-----------------------------------------------------------------------------
260 # URL to handler mappings
263 # URL to handler mappings
261 #-----------------------------------------------------------------------------
264 #-----------------------------------------------------------------------------
262
265
263
266
264 _path_regex = r"(?P<path>(?:/.*)*)"
267 _path_regex = r"(?P<path>(?:/.*)*)"
265 _checkpoint_id_regex = r"(?P<checkpoint_id>[\w-]+)"
268 _checkpoint_id_regex = r"(?P<checkpoint_id>[\w-]+)"
266 _notebook_name_regex = r"(?P<name>[^/]+\.ipynb)"
269 _notebook_name_regex = r"(?P<name>[^/]+\.ipynb)"
267 _notebook_path_regex = "%s/%s" % (_path_regex, _notebook_name_regex)
270 _notebook_path_regex = "%s/%s" % (_path_regex, _notebook_name_regex)
268
271
269 default_handlers = [
272 default_handlers = [
270 (r"/api/notebooks%s/checkpoints" % _notebook_path_regex, NotebookCheckpointsHandler),
273 (r"/api/notebooks%s/checkpoints" % _notebook_path_regex, NotebookCheckpointsHandler),
271 (r"/api/notebooks%s/checkpoints/%s" % (_notebook_path_regex, _checkpoint_id_regex),
274 (r"/api/notebooks%s/checkpoints/%s" % (_notebook_path_regex, _checkpoint_id_regex),
272 ModifyNotebookCheckpointsHandler),
275 ModifyNotebookCheckpointsHandler),
273 (r"/api/notebooks%s" % _notebook_path_regex, NotebookHandler),
276 (r"/api/notebooks%s" % _notebook_path_regex, NotebookHandler),
274 (r"/api/notebooks%s" % _path_regex, NotebookHandler),
277 (r"/api/notebooks%s" % _path_regex, NotebookHandler),
275 ]
278 ]
276
279
277
280
278
281
@@ -1,649 +1,649 b''
1 """Base classes to manage a Client's interaction with a running kernel
1 """Base classes to manage a Client's interaction with a running kernel
2 """
2 """
3
3
4 #-----------------------------------------------------------------------------
4 #-----------------------------------------------------------------------------
5 # Copyright (C) 2013 The IPython Development Team
5 # Copyright (C) 2013 The IPython Development Team
6 #
6 #
7 # Distributed under the terms of the BSD License. The full license is in
7 # Distributed under the terms of the BSD License. The full license is in
8 # the file COPYING, distributed as part of this software.
8 # the file COPYING, distributed as part of this software.
9 #-----------------------------------------------------------------------------
9 #-----------------------------------------------------------------------------
10
10
11 #-----------------------------------------------------------------------------
11 #-----------------------------------------------------------------------------
12 # Imports
12 # Imports
13 #-----------------------------------------------------------------------------
13 #-----------------------------------------------------------------------------
14
14
15 from __future__ import absolute_import
15 from __future__ import absolute_import
16
16
17 # Standard library imports
17 # Standard library imports
18 import atexit
18 import atexit
19 import errno
19 import errno
20 from threading import Thread
20 from threading import Thread
21 import time
21 import time
22
22
23 import zmq
23 import zmq
24 # import ZMQError in top-level namespace, to avoid ugly attribute-error messages
24 # import ZMQError in top-level namespace, to avoid ugly attribute-error messages
25 # during garbage collection of threads at exit:
25 # during garbage collection of threads at exit:
26 from zmq import ZMQError
26 from zmq import ZMQError
27 from zmq.eventloop import ioloop, zmqstream
27 from zmq.eventloop import ioloop, zmqstream
28
28
29 # Local imports
29 # Local imports
30 from .channelsabc import (
30 from .channelsabc import (
31 ShellChannelABC, IOPubChannelABC,
31 ShellChannelABC, IOPubChannelABC,
32 HBChannelABC, StdInChannelABC,
32 HBChannelABC, StdInChannelABC,
33 )
33 )
34 from IPython.utils.py3compat import string_types, iteritems
34 from IPython.utils.py3compat import string_types, iteritems
35
35
36 #-----------------------------------------------------------------------------
36 #-----------------------------------------------------------------------------
37 # Constants and exceptions
37 # Constants and exceptions
38 #-----------------------------------------------------------------------------
38 #-----------------------------------------------------------------------------
39
39
40 class InvalidPortNumber(Exception):
40 class InvalidPortNumber(Exception):
41 pass
41 pass
42
42
43 #-----------------------------------------------------------------------------
43 #-----------------------------------------------------------------------------
44 # Utility functions
44 # Utility functions
45 #-----------------------------------------------------------------------------
45 #-----------------------------------------------------------------------------
46
46
47 # some utilities to validate message structure, these might get moved elsewhere
47 # some utilities to validate message structure, these might get moved elsewhere
48 # if they prove to have more generic utility
48 # if they prove to have more generic utility
49
49
50 def validate_string_list(lst):
50 def validate_string_list(lst):
51 """Validate that the input is a list of strings.
51 """Validate that the input is a list of strings.
52
52
53 Raises ValueError if not."""
53 Raises ValueError if not."""
54 if not isinstance(lst, list):
54 if not isinstance(lst, list):
55 raise ValueError('input %r must be a list' % lst)
55 raise ValueError('input %r must be a list' % lst)
56 for x in lst:
56 for x in lst:
57 if not isinstance(x, string_types):
57 if not isinstance(x, string_types):
58 raise ValueError('element %r in list must be a string' % x)
58 raise ValueError('element %r in list must be a string' % x)
59
59
60
60
61 def validate_string_dict(dct):
61 def validate_string_dict(dct):
62 """Validate that the input is a dict with string keys and values.
62 """Validate that the input is a dict with string keys and values.
63
63
64 Raises ValueError if not."""
64 Raises ValueError if not."""
65 for k,v in iteritems(dct):
65 for k,v in iteritems(dct):
66 if not isinstance(k, string_types):
66 if not isinstance(k, string_types):
67 raise ValueError('key %r in dict must be a string' % k)
67 raise ValueError('key %r in dict must be a string' % k)
68 if not isinstance(v, string_types):
68 if not isinstance(v, string_types):
69 raise ValueError('value %r in dict must be a string' % v)
69 raise ValueError('value %r in dict must be a string' % v)
70
70
71
71
72 #-----------------------------------------------------------------------------
72 #-----------------------------------------------------------------------------
73 # ZMQ Socket Channel classes
73 # ZMQ Socket Channel classes
74 #-----------------------------------------------------------------------------
74 #-----------------------------------------------------------------------------
75
75
76 class ZMQSocketChannel(Thread):
76 class ZMQSocketChannel(Thread):
77 """The base class for the channels that use ZMQ sockets."""
77 """The base class for the channels that use ZMQ sockets."""
78 context = None
78 context = None
79 session = None
79 session = None
80 socket = None
80 socket = None
81 ioloop = None
81 ioloop = None
82 stream = None
82 stream = None
83 _address = None
83 _address = None
84 _exiting = False
84 _exiting = False
85 proxy_methods = []
85 proxy_methods = []
86
86
87 def __init__(self, context, session, address):
87 def __init__(self, context, session, address):
88 """Create a channel.
88 """Create a channel.
89
89
90 Parameters
90 Parameters
91 ----------
91 ----------
92 context : :class:`zmq.Context`
92 context : :class:`zmq.Context`
93 The ZMQ context to use.
93 The ZMQ context to use.
94 session : :class:`session.Session`
94 session : :class:`session.Session`
95 The session to use.
95 The session to use.
96 address : zmq url
96 address : zmq url
97 Standard (ip, port) tuple that the kernel is listening on.
97 Standard (ip, port) tuple that the kernel is listening on.
98 """
98 """
99 super(ZMQSocketChannel, self).__init__()
99 super(ZMQSocketChannel, self).__init__()
100 self.daemon = True
100 self.daemon = True
101
101
102 self.context = context
102 self.context = context
103 self.session = session
103 self.session = session
104 if isinstance(address, tuple):
104 if isinstance(address, tuple):
105 if address[1] == 0:
105 if address[1] == 0:
106 message = 'The port number for a channel cannot be 0.'
106 message = 'The port number for a channel cannot be 0.'
107 raise InvalidPortNumber(message)
107 raise InvalidPortNumber(message)
108 address = "tcp://%s:%i" % address
108 address = "tcp://%s:%i" % address
109 self._address = address
109 self._address = address
110 atexit.register(self._notice_exit)
110 atexit.register(self._notice_exit)
111
111
112 def _notice_exit(self):
112 def _notice_exit(self):
113 self._exiting = True
113 self._exiting = True
114
114
115 def _run_loop(self):
115 def _run_loop(self):
116 """Run my loop, ignoring EINTR events in the poller"""
116 """Run my loop, ignoring EINTR events in the poller"""
117 while True:
117 while True:
118 try:
118 try:
119 self.ioloop.start()
119 self.ioloop.start()
120 except ZMQError as e:
120 except ZMQError as e:
121 if e.errno == errno.EINTR:
121 if e.errno == errno.EINTR:
122 continue
122 continue
123 else:
123 else:
124 raise
124 raise
125 except Exception:
125 except Exception:
126 if self._exiting:
126 if self._exiting:
127 break
127 break
128 else:
128 else:
129 raise
129 raise
130 else:
130 else:
131 break
131 break
132
132
133 def stop(self):
133 def stop(self):
134 """Stop the channel's event loop and join its thread.
134 """Stop the channel's event loop and join its thread.
135
135
136 This calls :method:`Thread.join` and returns when the thread
136 This calls :meth:`~threading.Thread.join` and returns when the thread
137 terminates. :class:`RuntimeError` will be raised if
137 terminates. :class:`RuntimeError` will be raised if
138 :method:`self.start` is called again.
138 :meth:`~threading.Thread.start` is called again.
139 """
139 """
140 self.join()
140 self.join()
141
141
142 @property
142 @property
143 def address(self):
143 def address(self):
144 """Get the channel's address as a zmq url string.
144 """Get the channel's address as a zmq url string.
145
145
146 These URLS have the form: 'tcp://127.0.0.1:5555'.
146 These URLS have the form: 'tcp://127.0.0.1:5555'.
147 """
147 """
148 return self._address
148 return self._address
149
149
150 def _queue_send(self, msg):
150 def _queue_send(self, msg):
151 """Queue a message to be sent from the IOLoop's thread.
151 """Queue a message to be sent from the IOLoop's thread.
152
152
153 Parameters
153 Parameters
154 ----------
154 ----------
155 msg : message to send
155 msg : message to send
156
156
157 This is threadsafe, as it uses IOLoop.add_callback to give the loop's
157 This is threadsafe, as it uses IOLoop.add_callback to give the loop's
158 thread control of the action.
158 thread control of the action.
159 """
159 """
160 def thread_send():
160 def thread_send():
161 self.session.send(self.stream, msg)
161 self.session.send(self.stream, msg)
162 self.ioloop.add_callback(thread_send)
162 self.ioloop.add_callback(thread_send)
163
163
164 def _handle_recv(self, msg):
164 def _handle_recv(self, msg):
165 """Callback for stream.on_recv.
165 """Callback for stream.on_recv.
166
166
167 Unpacks message, and calls handlers with it.
167 Unpacks message, and calls handlers with it.
168 """
168 """
169 ident,smsg = self.session.feed_identities(msg)
169 ident,smsg = self.session.feed_identities(msg)
170 self.call_handlers(self.session.unserialize(smsg))
170 self.call_handlers(self.session.unserialize(smsg))
171
171
172
172
173
173
174 class ShellChannel(ZMQSocketChannel):
174 class ShellChannel(ZMQSocketChannel):
175 """The shell channel for issuing request/replies to the kernel."""
175 """The shell channel for issuing request/replies to the kernel."""
176
176
177 command_queue = None
177 command_queue = None
178 # flag for whether execute requests should be allowed to call raw_input:
178 # flag for whether execute requests should be allowed to call raw_input:
179 allow_stdin = True
179 allow_stdin = True
180 proxy_methods = [
180 proxy_methods = [
181 'execute',
181 'execute',
182 'complete',
182 'complete',
183 'object_info',
183 'object_info',
184 'history',
184 'history',
185 'kernel_info',
185 'kernel_info',
186 'shutdown',
186 'shutdown',
187 ]
187 ]
188
188
189 def __init__(self, context, session, address):
189 def __init__(self, context, session, address):
190 super(ShellChannel, self).__init__(context, session, address)
190 super(ShellChannel, self).__init__(context, session, address)
191 self.ioloop = ioloop.IOLoop()
191 self.ioloop = ioloop.IOLoop()
192
192
193 def run(self):
193 def run(self):
194 """The thread's main activity. Call start() instead."""
194 """The thread's main activity. Call start() instead."""
195 self.socket = self.context.socket(zmq.DEALER)
195 self.socket = self.context.socket(zmq.DEALER)
196 self.socket.setsockopt(zmq.IDENTITY, self.session.bsession)
196 self.socket.setsockopt(zmq.IDENTITY, self.session.bsession)
197 self.socket.connect(self.address)
197 self.socket.connect(self.address)
198 self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
198 self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
199 self.stream.on_recv(self._handle_recv)
199 self.stream.on_recv(self._handle_recv)
200 self._run_loop()
200 self._run_loop()
201 try:
201 try:
202 self.socket.close()
202 self.socket.close()
203 except:
203 except:
204 pass
204 pass
205
205
206 def stop(self):
206 def stop(self):
207 """Stop the channel's event loop and join its thread."""
207 """Stop the channel's event loop and join its thread."""
208 self.ioloop.stop()
208 self.ioloop.stop()
209 super(ShellChannel, self).stop()
209 super(ShellChannel, self).stop()
210
210
211 def call_handlers(self, msg):
211 def call_handlers(self, msg):
212 """This method is called in the ioloop thread when a message arrives.
212 """This method is called in the ioloop thread when a message arrives.
213
213
214 Subclasses should override this method to handle incoming messages.
214 Subclasses should override this method to handle incoming messages.
215 It is important to remember that this method is called in the thread
215 It is important to remember that this method is called in the thread
216 so that some logic must be done to ensure that the application level
216 so that some logic must be done to ensure that the application level
217 handlers are called in the application thread.
217 handlers are called in the application thread.
218 """
218 """
219 raise NotImplementedError('call_handlers must be defined in a subclass.')
219 raise NotImplementedError('call_handlers must be defined in a subclass.')
220
220
221 def execute(self, code, silent=False, store_history=True,
221 def execute(self, code, silent=False, store_history=True,
222 user_variables=None, user_expressions=None, allow_stdin=None):
222 user_variables=None, user_expressions=None, allow_stdin=None):
223 """Execute code in the kernel.
223 """Execute code in the kernel.
224
224
225 Parameters
225 Parameters
226 ----------
226 ----------
227 code : str
227 code : str
228 A string of Python code.
228 A string of Python code.
229
229
230 silent : bool, optional (default False)
230 silent : bool, optional (default False)
231 If set, the kernel will execute the code as quietly possible, and
231 If set, the kernel will execute the code as quietly possible, and
232 will force store_history to be False.
232 will force store_history to be False.
233
233
234 store_history : bool, optional (default True)
234 store_history : bool, optional (default True)
235 If set, the kernel will store command history. This is forced
235 If set, the kernel will store command history. This is forced
236 to be False if silent is True.
236 to be False if silent is True.
237
237
238 user_variables : list, optional
238 user_variables : list, optional
239 A list of variable names to pull from the user's namespace. They
239 A list of variable names to pull from the user's namespace. They
240 will come back as a dict with these names as keys and their
240 will come back as a dict with these names as keys and their
241 :func:`repr` as values.
241 :func:`repr` as values.
242
242
243 user_expressions : dict, optional
243 user_expressions : dict, optional
244 A dict mapping names to expressions to be evaluated in the user's
244 A dict mapping names to expressions to be evaluated in the user's
245 dict. The expression values are returned as strings formatted using
245 dict. The expression values are returned as strings formatted using
246 :func:`repr`.
246 :func:`repr`.
247
247
248 allow_stdin : bool, optional (default self.allow_stdin)
248 allow_stdin : bool, optional (default self.allow_stdin)
249 Flag for whether the kernel can send stdin requests to frontends.
249 Flag for whether the kernel can send stdin requests to frontends.
250
250
251 Some frontends (e.g. the Notebook) do not support stdin requests.
251 Some frontends (e.g. the Notebook) do not support stdin requests.
252 If raw_input is called from code executed from such a frontend, a
252 If raw_input is called from code executed from such a frontend, a
253 StdinNotImplementedError will be raised.
253 StdinNotImplementedError will be raised.
254
254
255 Returns
255 Returns
256 -------
256 -------
257 The msg_id of the message sent.
257 The msg_id of the message sent.
258 """
258 """
259 if user_variables is None:
259 if user_variables is None:
260 user_variables = []
260 user_variables = []
261 if user_expressions is None:
261 if user_expressions is None:
262 user_expressions = {}
262 user_expressions = {}
263 if allow_stdin is None:
263 if allow_stdin is None:
264 allow_stdin = self.allow_stdin
264 allow_stdin = self.allow_stdin
265
265
266
266
267 # Don't waste network traffic if inputs are invalid
267 # Don't waste network traffic if inputs are invalid
268 if not isinstance(code, string_types):
268 if not isinstance(code, string_types):
269 raise ValueError('code %r must be a string' % code)
269 raise ValueError('code %r must be a string' % code)
270 validate_string_list(user_variables)
270 validate_string_list(user_variables)
271 validate_string_dict(user_expressions)
271 validate_string_dict(user_expressions)
272
272
273 # Create class for content/msg creation. Related to, but possibly
273 # Create class for content/msg creation. Related to, but possibly
274 # not in Session.
274 # not in Session.
275 content = dict(code=code, silent=silent, store_history=store_history,
275 content = dict(code=code, silent=silent, store_history=store_history,
276 user_variables=user_variables,
276 user_variables=user_variables,
277 user_expressions=user_expressions,
277 user_expressions=user_expressions,
278 allow_stdin=allow_stdin,
278 allow_stdin=allow_stdin,
279 )
279 )
280 msg = self.session.msg('execute_request', content)
280 msg = self.session.msg('execute_request', content)
281 self._queue_send(msg)
281 self._queue_send(msg)
282 return msg['header']['msg_id']
282 return msg['header']['msg_id']
283
283
284 def complete(self, text, line, cursor_pos, block=None):
284 def complete(self, text, line, cursor_pos, block=None):
285 """Tab complete text in the kernel's namespace.
285 """Tab complete text in the kernel's namespace.
286
286
287 Parameters
287 Parameters
288 ----------
288 ----------
289 text : str
289 text : str
290 The text to complete.
290 The text to complete.
291 line : str
291 line : str
292 The full line of text that is the surrounding context for the
292 The full line of text that is the surrounding context for the
293 text to complete.
293 text to complete.
294 cursor_pos : int
294 cursor_pos : int
295 The position of the cursor in the line where the completion was
295 The position of the cursor in the line where the completion was
296 requested.
296 requested.
297 block : str, optional
297 block : str, optional
298 The full block of code in which the completion is being requested.
298 The full block of code in which the completion is being requested.
299
299
300 Returns
300 Returns
301 -------
301 -------
302 The msg_id of the message sent.
302 The msg_id of the message sent.
303 """
303 """
304 content = dict(text=text, line=line, block=block, cursor_pos=cursor_pos)
304 content = dict(text=text, line=line, block=block, cursor_pos=cursor_pos)
305 msg = self.session.msg('complete_request', content)
305 msg = self.session.msg('complete_request', content)
306 self._queue_send(msg)
306 self._queue_send(msg)
307 return msg['header']['msg_id']
307 return msg['header']['msg_id']
308
308
309 def object_info(self, oname, detail_level=0):
309 def object_info(self, oname, detail_level=0):
310 """Get metadata information about an object in the kernel's namespace.
310 """Get metadata information about an object in the kernel's namespace.
311
311
312 Parameters
312 Parameters
313 ----------
313 ----------
314 oname : str
314 oname : str
315 A string specifying the object name.
315 A string specifying the object name.
316 detail_level : int, optional
316 detail_level : int, optional
317 The level of detail for the introspection (0-2)
317 The level of detail for the introspection (0-2)
318
318
319 Returns
319 Returns
320 -------
320 -------
321 The msg_id of the message sent.
321 The msg_id of the message sent.
322 """
322 """
323 content = dict(oname=oname, detail_level=detail_level)
323 content = dict(oname=oname, detail_level=detail_level)
324 msg = self.session.msg('object_info_request', content)
324 msg = self.session.msg('object_info_request', content)
325 self._queue_send(msg)
325 self._queue_send(msg)
326 return msg['header']['msg_id']
326 return msg['header']['msg_id']
327
327
328 def history(self, raw=True, output=False, hist_access_type='range', **kwargs):
328 def history(self, raw=True, output=False, hist_access_type='range', **kwargs):
329 """Get entries from the kernel's history list.
329 """Get entries from the kernel's history list.
330
330
331 Parameters
331 Parameters
332 ----------
332 ----------
333 raw : bool
333 raw : bool
334 If True, return the raw input.
334 If True, return the raw input.
335 output : bool
335 output : bool
336 If True, then return the output as well.
336 If True, then return the output as well.
337 hist_access_type : str
337 hist_access_type : str
338 'range' (fill in session, start and stop params), 'tail' (fill in n)
338 'range' (fill in session, start and stop params), 'tail' (fill in n)
339 or 'search' (fill in pattern param).
339 or 'search' (fill in pattern param).
340
340
341 session : int
341 session : int
342 For a range request, the session from which to get lines. Session
342 For a range request, the session from which to get lines. Session
343 numbers are positive integers; negative ones count back from the
343 numbers are positive integers; negative ones count back from the
344 current session.
344 current session.
345 start : int
345 start : int
346 The first line number of a history range.
346 The first line number of a history range.
347 stop : int
347 stop : int
348 The final (excluded) line number of a history range.
348 The final (excluded) line number of a history range.
349
349
350 n : int
350 n : int
351 The number of lines of history to get for a tail request.
351 The number of lines of history to get for a tail request.
352
352
353 pattern : str
353 pattern : str
354 The glob-syntax pattern for a search request.
354 The glob-syntax pattern for a search request.
355
355
356 Returns
356 Returns
357 -------
357 -------
358 The msg_id of the message sent.
358 The msg_id of the message sent.
359 """
359 """
360 content = dict(raw=raw, output=output, hist_access_type=hist_access_type,
360 content = dict(raw=raw, output=output, hist_access_type=hist_access_type,
361 **kwargs)
361 **kwargs)
362 msg = self.session.msg('history_request', content)
362 msg = self.session.msg('history_request', content)
363 self._queue_send(msg)
363 self._queue_send(msg)
364 return msg['header']['msg_id']
364 return msg['header']['msg_id']
365
365
366 def kernel_info(self):
366 def kernel_info(self):
367 """Request kernel info."""
367 """Request kernel info."""
368 msg = self.session.msg('kernel_info_request')
368 msg = self.session.msg('kernel_info_request')
369 self._queue_send(msg)
369 self._queue_send(msg)
370 return msg['header']['msg_id']
370 return msg['header']['msg_id']
371
371
372 def shutdown(self, restart=False):
372 def shutdown(self, restart=False):
373 """Request an immediate kernel shutdown.
373 """Request an immediate kernel shutdown.
374
374
375 Upon receipt of the (empty) reply, client code can safely assume that
375 Upon receipt of the (empty) reply, client code can safely assume that
376 the kernel has shut down and it's safe to forcefully terminate it if
376 the kernel has shut down and it's safe to forcefully terminate it if
377 it's still alive.
377 it's still alive.
378
378
379 The kernel will send the reply via a function registered with Python's
379 The kernel will send the reply via a function registered with Python's
380 atexit module, ensuring it's truly done as the kernel is done with all
380 atexit module, ensuring it's truly done as the kernel is done with all
381 normal operation.
381 normal operation.
382 """
382 """
383 # Send quit message to kernel. Once we implement kernel-side setattr,
383 # Send quit message to kernel. Once we implement kernel-side setattr,
384 # this should probably be done that way, but for now this will do.
384 # this should probably be done that way, but for now this will do.
385 msg = self.session.msg('shutdown_request', {'restart':restart})
385 msg = self.session.msg('shutdown_request', {'restart':restart})
386 self._queue_send(msg)
386 self._queue_send(msg)
387 return msg['header']['msg_id']
387 return msg['header']['msg_id']
388
388
389
389
390
390
391 class IOPubChannel(ZMQSocketChannel):
391 class IOPubChannel(ZMQSocketChannel):
392 """The iopub channel which listens for messages that the kernel publishes.
392 """The iopub channel which listens for messages that the kernel publishes.
393
393
394 This channel is where all output is published to frontends.
394 This channel is where all output is published to frontends.
395 """
395 """
396
396
397 def __init__(self, context, session, address):
397 def __init__(self, context, session, address):
398 super(IOPubChannel, self).__init__(context, session, address)
398 super(IOPubChannel, self).__init__(context, session, address)
399 self.ioloop = ioloop.IOLoop()
399 self.ioloop = ioloop.IOLoop()
400
400
401 def run(self):
401 def run(self):
402 """The thread's main activity. Call start() instead."""
402 """The thread's main activity. Call start() instead."""
403 self.socket = self.context.socket(zmq.SUB)
403 self.socket = self.context.socket(zmq.SUB)
404 self.socket.setsockopt(zmq.SUBSCRIBE,b'')
404 self.socket.setsockopt(zmq.SUBSCRIBE,b'')
405 self.socket.setsockopt(zmq.IDENTITY, self.session.bsession)
405 self.socket.setsockopt(zmq.IDENTITY, self.session.bsession)
406 self.socket.connect(self.address)
406 self.socket.connect(self.address)
407 self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
407 self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
408 self.stream.on_recv(self._handle_recv)
408 self.stream.on_recv(self._handle_recv)
409 self._run_loop()
409 self._run_loop()
410 try:
410 try:
411 self.socket.close()
411 self.socket.close()
412 except:
412 except:
413 pass
413 pass
414
414
415 def stop(self):
415 def stop(self):
416 """Stop the channel's event loop and join its thread."""
416 """Stop the channel's event loop and join its thread."""
417 self.ioloop.stop()
417 self.ioloop.stop()
418 super(IOPubChannel, self).stop()
418 super(IOPubChannel, self).stop()
419
419
420 def call_handlers(self, msg):
420 def call_handlers(self, msg):
421 """This method is called in the ioloop thread when a message arrives.
421 """This method is called in the ioloop thread when a message arrives.
422
422
423 Subclasses should override this method to handle incoming messages.
423 Subclasses should override this method to handle incoming messages.
424 It is important to remember that this method is called in the thread
424 It is important to remember that this method is called in the thread
425 so that some logic must be done to ensure that the application leve
425 so that some logic must be done to ensure that the application leve
426 handlers are called in the application thread.
426 handlers are called in the application thread.
427 """
427 """
428 raise NotImplementedError('call_handlers must be defined in a subclass.')
428 raise NotImplementedError('call_handlers must be defined in a subclass.')
429
429
430 def flush(self, timeout=1.0):
430 def flush(self, timeout=1.0):
431 """Immediately processes all pending messages on the iopub channel.
431 """Immediately processes all pending messages on the iopub channel.
432
432
433 Callers should use this method to ensure that :method:`call_handlers`
433 Callers should use this method to ensure that :meth:`call_handlers`
434 has been called for all messages that have been received on the
434 has been called for all messages that have been received on the
435 0MQ SUB socket of this channel.
435 0MQ SUB socket of this channel.
436
436
437 This method is thread safe.
437 This method is thread safe.
438
438
439 Parameters
439 Parameters
440 ----------
440 ----------
441 timeout : float, optional
441 timeout : float, optional
442 The maximum amount of time to spend flushing, in seconds. The
442 The maximum amount of time to spend flushing, in seconds. The
443 default is one second.
443 default is one second.
444 """
444 """
445 # We do the IOLoop callback process twice to ensure that the IOLoop
445 # We do the IOLoop callback process twice to ensure that the IOLoop
446 # gets to perform at least one full poll.
446 # gets to perform at least one full poll.
447 stop_time = time.time() + timeout
447 stop_time = time.time() + timeout
448 for i in range(2):
448 for i in range(2):
449 self._flushed = False
449 self._flushed = False
450 self.ioloop.add_callback(self._flush)
450 self.ioloop.add_callback(self._flush)
451 while not self._flushed and time.time() < stop_time:
451 while not self._flushed and time.time() < stop_time:
452 time.sleep(0.01)
452 time.sleep(0.01)
453
453
454 def _flush(self):
454 def _flush(self):
455 """Callback for :method:`self.flush`."""
455 """Callback for :method:`self.flush`."""
456 self.stream.flush()
456 self.stream.flush()
457 self._flushed = True
457 self._flushed = True
458
458
459
459
460 class StdInChannel(ZMQSocketChannel):
460 class StdInChannel(ZMQSocketChannel):
461 """The stdin channel to handle raw_input requests that the kernel makes."""
461 """The stdin channel to handle raw_input requests that the kernel makes."""
462
462
463 msg_queue = None
463 msg_queue = None
464 proxy_methods = ['input']
464 proxy_methods = ['input']
465
465
466 def __init__(self, context, session, address):
466 def __init__(self, context, session, address):
467 super(StdInChannel, self).__init__(context, session, address)
467 super(StdInChannel, self).__init__(context, session, address)
468 self.ioloop = ioloop.IOLoop()
468 self.ioloop = ioloop.IOLoop()
469
469
470 def run(self):
470 def run(self):
471 """The thread's main activity. Call start() instead."""
471 """The thread's main activity. Call start() instead."""
472 self.socket = self.context.socket(zmq.DEALER)
472 self.socket = self.context.socket(zmq.DEALER)
473 self.socket.setsockopt(zmq.IDENTITY, self.session.bsession)
473 self.socket.setsockopt(zmq.IDENTITY, self.session.bsession)
474 self.socket.connect(self.address)
474 self.socket.connect(self.address)
475 self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
475 self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
476 self.stream.on_recv(self._handle_recv)
476 self.stream.on_recv(self._handle_recv)
477 self._run_loop()
477 self._run_loop()
478 try:
478 try:
479 self.socket.close()
479 self.socket.close()
480 except:
480 except:
481 pass
481 pass
482
482
483 def stop(self):
483 def stop(self):
484 """Stop the channel's event loop and join its thread."""
484 """Stop the channel's event loop and join its thread."""
485 self.ioloop.stop()
485 self.ioloop.stop()
486 super(StdInChannel, self).stop()
486 super(StdInChannel, self).stop()
487
487
488 def call_handlers(self, msg):
488 def call_handlers(self, msg):
489 """This method is called in the ioloop thread when a message arrives.
489 """This method is called in the ioloop thread when a message arrives.
490
490
491 Subclasses should override this method to handle incoming messages.
491 Subclasses should override this method to handle incoming messages.
492 It is important to remember that this method is called in the thread
492 It is important to remember that this method is called in the thread
493 so that some logic must be done to ensure that the application leve
493 so that some logic must be done to ensure that the application leve
494 handlers are called in the application thread.
494 handlers are called in the application thread.
495 """
495 """
496 raise NotImplementedError('call_handlers must be defined in a subclass.')
496 raise NotImplementedError('call_handlers must be defined in a subclass.')
497
497
498 def input(self, string):
498 def input(self, string):
499 """Send a string of raw input to the kernel."""
499 """Send a string of raw input to the kernel."""
500 content = dict(value=string)
500 content = dict(value=string)
501 msg = self.session.msg('input_reply', content)
501 msg = self.session.msg('input_reply', content)
502 self._queue_send(msg)
502 self._queue_send(msg)
503
503
504
504
505 class HBChannel(ZMQSocketChannel):
505 class HBChannel(ZMQSocketChannel):
506 """The heartbeat channel which monitors the kernel heartbeat.
506 """The heartbeat channel which monitors the kernel heartbeat.
507
507
508 Note that the heartbeat channel is paused by default. As long as you start
508 Note that the heartbeat channel is paused by default. As long as you start
509 this channel, the kernel manager will ensure that it is paused and un-paused
509 this channel, the kernel manager will ensure that it is paused and un-paused
510 as appropriate.
510 as appropriate.
511 """
511 """
512
512
513 time_to_dead = 3.0
513 time_to_dead = 3.0
514 socket = None
514 socket = None
515 poller = None
515 poller = None
516 _running = None
516 _running = None
517 _pause = None
517 _pause = None
518 _beating = None
518 _beating = None
519
519
520 def __init__(self, context, session, address):
520 def __init__(self, context, session, address):
521 super(HBChannel, self).__init__(context, session, address)
521 super(HBChannel, self).__init__(context, session, address)
522 self._running = False
522 self._running = False
523 self._pause =True
523 self._pause =True
524 self.poller = zmq.Poller()
524 self.poller = zmq.Poller()
525
525
526 def _create_socket(self):
526 def _create_socket(self):
527 if self.socket is not None:
527 if self.socket is not None:
528 # close previous socket, before opening a new one
528 # close previous socket, before opening a new one
529 self.poller.unregister(self.socket)
529 self.poller.unregister(self.socket)
530 self.socket.close()
530 self.socket.close()
531 self.socket = self.context.socket(zmq.REQ)
531 self.socket = self.context.socket(zmq.REQ)
532 self.socket.setsockopt(zmq.LINGER, 0)
532 self.socket.setsockopt(zmq.LINGER, 0)
533 self.socket.connect(self.address)
533 self.socket.connect(self.address)
534
534
535 self.poller.register(self.socket, zmq.POLLIN)
535 self.poller.register(self.socket, zmq.POLLIN)
536
536
537 def _poll(self, start_time):
537 def _poll(self, start_time):
538 """poll for heartbeat replies until we reach self.time_to_dead.
538 """poll for heartbeat replies until we reach self.time_to_dead.
539
539
540 Ignores interrupts, and returns the result of poll(), which
540 Ignores interrupts, and returns the result of poll(), which
541 will be an empty list if no messages arrived before the timeout,
541 will be an empty list if no messages arrived before the timeout,
542 or the event tuple if there is a message to receive.
542 or the event tuple if there is a message to receive.
543 """
543 """
544
544
545 until_dead = self.time_to_dead - (time.time() - start_time)
545 until_dead = self.time_to_dead - (time.time() - start_time)
546 # ensure poll at least once
546 # ensure poll at least once
547 until_dead = max(until_dead, 1e-3)
547 until_dead = max(until_dead, 1e-3)
548 events = []
548 events = []
549 while True:
549 while True:
550 try:
550 try:
551 events = self.poller.poll(1000 * until_dead)
551 events = self.poller.poll(1000 * until_dead)
552 except ZMQError as e:
552 except ZMQError as e:
553 if e.errno == errno.EINTR:
553 if e.errno == errno.EINTR:
554 # ignore interrupts during heartbeat
554 # ignore interrupts during heartbeat
555 # this may never actually happen
555 # this may never actually happen
556 until_dead = self.time_to_dead - (time.time() - start_time)
556 until_dead = self.time_to_dead - (time.time() - start_time)
557 until_dead = max(until_dead, 1e-3)
557 until_dead = max(until_dead, 1e-3)
558 pass
558 pass
559 else:
559 else:
560 raise
560 raise
561 except Exception:
561 except Exception:
562 if self._exiting:
562 if self._exiting:
563 break
563 break
564 else:
564 else:
565 raise
565 raise
566 else:
566 else:
567 break
567 break
568 return events
568 return events
569
569
570 def run(self):
570 def run(self):
571 """The thread's main activity. Call start() instead."""
571 """The thread's main activity. Call start() instead."""
572 self._create_socket()
572 self._create_socket()
573 self._running = True
573 self._running = True
574 self._beating = True
574 self._beating = True
575
575
576 while self._running:
576 while self._running:
577 if self._pause:
577 if self._pause:
578 # just sleep, and skip the rest of the loop
578 # just sleep, and skip the rest of the loop
579 time.sleep(self.time_to_dead)
579 time.sleep(self.time_to_dead)
580 continue
580 continue
581
581
582 since_last_heartbeat = 0.0
582 since_last_heartbeat = 0.0
583 # io.rprint('Ping from HB channel') # dbg
583 # io.rprint('Ping from HB channel') # dbg
584 # no need to catch EFSM here, because the previous event was
584 # no need to catch EFSM here, because the previous event was
585 # either a recv or connect, which cannot be followed by EFSM
585 # either a recv or connect, which cannot be followed by EFSM
586 self.socket.send(b'ping')
586 self.socket.send(b'ping')
587 request_time = time.time()
587 request_time = time.time()
588 ready = self._poll(request_time)
588 ready = self._poll(request_time)
589 if ready:
589 if ready:
590 self._beating = True
590 self._beating = True
591 # the poll above guarantees we have something to recv
591 # the poll above guarantees we have something to recv
592 self.socket.recv()
592 self.socket.recv()
593 # sleep the remainder of the cycle
593 # sleep the remainder of the cycle
594 remainder = self.time_to_dead - (time.time() - request_time)
594 remainder = self.time_to_dead - (time.time() - request_time)
595 if remainder > 0:
595 if remainder > 0:
596 time.sleep(remainder)
596 time.sleep(remainder)
597 continue
597 continue
598 else:
598 else:
599 # nothing was received within the time limit, signal heart failure
599 # nothing was received within the time limit, signal heart failure
600 self._beating = False
600 self._beating = False
601 since_last_heartbeat = time.time() - request_time
601 since_last_heartbeat = time.time() - request_time
602 self.call_handlers(since_last_heartbeat)
602 self.call_handlers(since_last_heartbeat)
603 # and close/reopen the socket, because the REQ/REP cycle has been broken
603 # and close/reopen the socket, because the REQ/REP cycle has been broken
604 self._create_socket()
604 self._create_socket()
605 continue
605 continue
606 try:
606 try:
607 self.socket.close()
607 self.socket.close()
608 except:
608 except:
609 pass
609 pass
610
610
611 def pause(self):
611 def pause(self):
612 """Pause the heartbeat."""
612 """Pause the heartbeat."""
613 self._pause = True
613 self._pause = True
614
614
615 def unpause(self):
615 def unpause(self):
616 """Unpause the heartbeat."""
616 """Unpause the heartbeat."""
617 self._pause = False
617 self._pause = False
618
618
619 def is_beating(self):
619 def is_beating(self):
620 """Is the heartbeat running and responsive (and not paused)."""
620 """Is the heartbeat running and responsive (and not paused)."""
621 if self.is_alive() and not self._pause and self._beating:
621 if self.is_alive() and not self._pause and self._beating:
622 return True
622 return True
623 else:
623 else:
624 return False
624 return False
625
625
626 def stop(self):
626 def stop(self):
627 """Stop the channel's event loop and join its thread."""
627 """Stop the channel's event loop and join its thread."""
628 self._running = False
628 self._running = False
629 super(HBChannel, self).stop()
629 super(HBChannel, self).stop()
630
630
631 def call_handlers(self, since_last_heartbeat):
631 def call_handlers(self, since_last_heartbeat):
632 """This method is called in the ioloop thread when a message arrives.
632 """This method is called in the ioloop thread when a message arrives.
633
633
634 Subclasses should override this method to handle incoming messages.
634 Subclasses should override this method to handle incoming messages.
635 It is important to remember that this method is called in the thread
635 It is important to remember that this method is called in the thread
636 so that some logic must be done to ensure that the application level
636 so that some logic must be done to ensure that the application level
637 handlers are called in the application thread.
637 handlers are called in the application thread.
638 """
638 """
639 raise NotImplementedError('call_handlers must be defined in a subclass.')
639 raise NotImplementedError('call_handlers must be defined in a subclass.')
640
640
641
641
642 #---------------------------------------------------------------------#-----------------------------------------------------------------------------
642 #---------------------------------------------------------------------#-----------------------------------------------------------------------------
643 # ABC Registration
643 # ABC Registration
644 #-----------------------------------------------------------------------------
644 #-----------------------------------------------------------------------------
645
645
646 ShellChannelABC.register(ShellChannel)
646 ShellChannelABC.register(ShellChannel)
647 IOPubChannelABC.register(IOPubChannel)
647 IOPubChannelABC.register(IOPubChannel)
648 HBChannelABC.register(HBChannel)
648 HBChannelABC.register(HBChannel)
649 StdInChannelABC.register(StdInChannel)
649 StdInChannelABC.register(StdInChannel)
@@ -1,118 +1,116 b''
1 """
1 """
2 Password generation for the IPython notebook.
2 Password generation for the IPython notebook.
3 """
3 """
4 #-----------------------------------------------------------------------------
4 #-----------------------------------------------------------------------------
5 # Imports
5 # Imports
6 #-----------------------------------------------------------------------------
6 #-----------------------------------------------------------------------------
7 # Stdlib
7 # Stdlib
8 import getpass
8 import getpass
9 import hashlib
9 import hashlib
10 import random
10 import random
11
11
12 # Our own
12 # Our own
13 from IPython.core.error import UsageError
13 from IPython.core.error import UsageError
14 from IPython.testing.skipdoctest import skip_doctest
14 from IPython.testing.skipdoctest import skip_doctest
15 from IPython.utils.py3compat import cast_bytes, str_to_bytes
15 from IPython.utils.py3compat import cast_bytes, str_to_bytes
16
16
17 #-----------------------------------------------------------------------------
17 #-----------------------------------------------------------------------------
18 # Globals
18 # Globals
19 #-----------------------------------------------------------------------------
19 #-----------------------------------------------------------------------------
20
20
21 # Length of the salt in nr of hex chars, which implies salt_len * 4
21 # Length of the salt in nr of hex chars, which implies salt_len * 4
22 # bits of randomness.
22 # bits of randomness.
23 salt_len = 12
23 salt_len = 12
24
24
25 #-----------------------------------------------------------------------------
25 #-----------------------------------------------------------------------------
26 # Functions
26 # Functions
27 #-----------------------------------------------------------------------------
27 #-----------------------------------------------------------------------------
28
28
29 @skip_doctest
29 @skip_doctest
30 def passwd(passphrase=None, algorithm='sha1'):
30 def passwd(passphrase=None, algorithm='sha1'):
31 """Generate hashed password and salt for use in notebook configuration.
31 """Generate hashed password and salt for use in notebook configuration.
32
32
33 In the notebook configuration, set `c.NotebookApp.password` to
33 In the notebook configuration, set `c.NotebookApp.password` to
34 the generated string.
34 the generated string.
35
35
36 Parameters
36 Parameters
37 ----------
37 ----------
38 passphrase : str
38 passphrase : str
39 Password to hash. If unspecified, the user is asked to input
39 Password to hash. If unspecified, the user is asked to input
40 and verify a password.
40 and verify a password.
41 algorithm : str
41 algorithm : str
42 Hashing algorithm to use (e.g, 'sha1' or any argument supported
42 Hashing algorithm to use (e.g, 'sha1' or any argument supported
43 by :func:`hashlib.new`).
43 by :func:`hashlib.new`).
44
44
45 Returns
45 Returns
46 -------
46 -------
47 hashed_passphrase : str
47 hashed_passphrase : str
48 Hashed password, in the format 'hash_algorithm:salt:passphrase_hash'.
48 Hashed password, in the format 'hash_algorithm:salt:passphrase_hash'.
49
49
50 Examples
50 Examples
51 --------
51 --------
52 In [1]: passwd('mypassword')
52 >>> passwd('mypassword')
53 Out[1]: 'sha1:7cf3:b7d6da294ea9592a9480c8f52e63cd42cfb9dd12'
53 'sha1:7cf3:b7d6da294ea9592a9480c8f52e63cd42cfb9dd12'
54
54
55 """
55 """
56 if passphrase is None:
56 if passphrase is None:
57 for i in range(3):
57 for i in range(3):
58 p0 = getpass.getpass('Enter password: ')
58 p0 = getpass.getpass('Enter password: ')
59 p1 = getpass.getpass('Verify password: ')
59 p1 = getpass.getpass('Verify password: ')
60 if p0 == p1:
60 if p0 == p1:
61 passphrase = p0
61 passphrase = p0
62 break
62 break
63 else:
63 else:
64 print('Passwords do not match.')
64 print('Passwords do not match.')
65 else:
65 else:
66 raise UsageError('No matching passwords found. Giving up.')
66 raise UsageError('No matching passwords found. Giving up.')
67
67
68 h = hashlib.new(algorithm)
68 h = hashlib.new(algorithm)
69 salt = ('%0' + str(salt_len) + 'x') % random.getrandbits(4 * salt_len)
69 salt = ('%0' + str(salt_len) + 'x') % random.getrandbits(4 * salt_len)
70 h.update(cast_bytes(passphrase, 'utf-8') + str_to_bytes(salt, 'ascii'))
70 h.update(cast_bytes(passphrase, 'utf-8') + str_to_bytes(salt, 'ascii'))
71
71
72 return ':'.join((algorithm, salt, h.hexdigest()))
72 return ':'.join((algorithm, salt, h.hexdigest()))
73
73
74
74
75 def passwd_check(hashed_passphrase, passphrase):
75 def passwd_check(hashed_passphrase, passphrase):
76 """Verify that a given passphrase matches its hashed version.
76 """Verify that a given passphrase matches its hashed version.
77
77
78 Parameters
78 Parameters
79 ----------
79 ----------
80 hashed_passphrase : str
80 hashed_passphrase : str
81 Hashed password, in the format returned by `passwd`.
81 Hashed password, in the format returned by `passwd`.
82 passphrase : str
82 passphrase : str
83 Passphrase to validate.
83 Passphrase to validate.
84
84
85 Returns
85 Returns
86 -------
86 -------
87 valid : bool
87 valid : bool
88 True if the passphrase matches the hash.
88 True if the passphrase matches the hash.
89
89
90 Examples
90 Examples
91 --------
91 --------
92 In [1]: from IPython.lib.security import passwd_check
92 >>> from IPython.lib.security import passwd_check
93
93 >>> passwd_check('sha1:0e112c3ddfce:a68df677475c2b47b6e86d0467eec97ac5f4b85a',
94 In [2]: passwd_check('sha1:0e112c3ddfce:a68df677475c2b47b6e86d0467eec97ac5f4b85a',
94 ... 'mypassword')
95 ...: 'mypassword')
95 True
96 Out[2]: True
96
97
97 >>> passwd_check('sha1:0e112c3ddfce:a68df677475c2b47b6e86d0467eec97ac5f4b85a',
98 In [3]: passwd_check('sha1:0e112c3ddfce:a68df677475c2b47b6e86d0467eec97ac5f4b85a',
98 ... 'anotherpassword')
99 ...: 'anotherpassword')
99 False
100 Out[3]: False
101
102 """
100 """
103 try:
101 try:
104 algorithm, salt, pw_digest = hashed_passphrase.split(':', 2)
102 algorithm, salt, pw_digest = hashed_passphrase.split(':', 2)
105 except (ValueError, TypeError):
103 except (ValueError, TypeError):
106 return False
104 return False
107
105
108 try:
106 try:
109 h = hashlib.new(algorithm)
107 h = hashlib.new(algorithm)
110 except ValueError:
108 except ValueError:
111 return False
109 return False
112
110
113 if len(pw_digest) == 0:
111 if len(pw_digest) == 0:
114 return False
112 return False
115
113
116 h.update(cast_bytes(passphrase, 'utf-8') + cast_bytes(salt, 'ascii'))
114 h.update(cast_bytes(passphrase, 'utf-8') + cast_bytes(salt, 'ascii'))
117
115
118 return h.hexdigest() == pw_digest
116 return h.hexdigest() == pw_digest
@@ -1,171 +1,179 b''
1 """
1 """
2 Module containing single call export functions.
2 Module containing single call export functions.
3 """
3 """
4 #-----------------------------------------------------------------------------
4 #-----------------------------------------------------------------------------
5 # Copyright (c) 2013, the IPython Development Team.
5 # Copyright (c) 2013, the IPython Development Team.
6 #
6 #
7 # Distributed under the terms of the Modified BSD License.
7 # Distributed under the terms of the Modified BSD License.
8 #
8 #
9 # The full license is in the file COPYING.txt, distributed with this software.
9 # The full license is in the file COPYING.txt, distributed with this software.
10 #-----------------------------------------------------------------------------
10 #-----------------------------------------------------------------------------
11
11
12 #-----------------------------------------------------------------------------
12 #-----------------------------------------------------------------------------
13 # Imports
13 # Imports
14 #-----------------------------------------------------------------------------
14 #-----------------------------------------------------------------------------
15
15
16 from functools import wraps
16 from functools import wraps
17
17
18 from IPython.nbformat.v3.nbbase import NotebookNode
18 from IPython.nbformat.v3.nbbase import NotebookNode
19 from IPython.config import Config
19 from IPython.utils.decorators import undoc
20 from IPython.utils.py3compat import string_types
20 from IPython.utils.py3compat import string_types
21
21
22 from .exporter import Exporter
22 from .exporter import Exporter
23 from .templateexporter import TemplateExporter
23 from .templateexporter import TemplateExporter
24 from .html import HTMLExporter
24 from .html import HTMLExporter
25 from .slides import SlidesExporter
25 from .slides import SlidesExporter
26 from .latex import LatexExporter
26 from .latex import LatexExporter
27 from .markdown import MarkdownExporter
27 from .markdown import MarkdownExporter
28 from .python import PythonExporter
28 from .python import PythonExporter
29 from .rst import RSTExporter
29 from .rst import RSTExporter
30
30
31 #-----------------------------------------------------------------------------
31 #-----------------------------------------------------------------------------
32 # Classes
32 # Classes
33 #-----------------------------------------------------------------------------
33 #-----------------------------------------------------------------------------
34
34
35 @undoc
35 def DocDecorator(f):
36 def DocDecorator(f):
36
37
37 #Set docstring of function
38 #Set docstring of function
38 f.__doc__ = f.__doc__ + """
39 f.__doc__ = f.__doc__ + """
39 nb : Notebook node
40 nb : :class:`~IPython.nbformat.v3.nbbase.NotebookNode`
41 The notebook to export.
40 config : config (optional, keyword arg)
42 config : config (optional, keyword arg)
41 User configuration instance.
43 User configuration instance.
42 resources : dict (optional, keyword arg)
44 resources : dict (optional, keyword arg)
43 Resources used in the conversion process.
45 Resources used in the conversion process.
44
46
45 Returns
47 Returns
46 ----------
48 -------
47 tuple- output, resources, exporter_instance
49 tuple- output, resources, exporter_instance
48 output : str
50 output : str
49 Jinja 2 output. This is the resulting converted notebook.
51 Jinja 2 output. This is the resulting converted notebook.
50 resources : dictionary
52 resources : dictionary
51 Dictionary of resources used prior to and during the conversion
53 Dictionary of resources used prior to and during the conversion
52 process.
54 process.
53 exporter_instance : Exporter
55 exporter_instance : Exporter
54 Instance of the Exporter class used to export the document. Useful
56 Instance of the Exporter class used to export the document. Useful
55 to caller because it provides a 'file_extension' property which
57 to caller because it provides a 'file_extension' property which
56 specifies what extension the output should be saved as.
58 specifies what extension the output should be saved as.
57
59
60 Notes
61 -----
58 WARNING: API WILL CHANGE IN FUTURE RELEASES OF NBCONVERT
62 WARNING: API WILL CHANGE IN FUTURE RELEASES OF NBCONVERT
59 """
63 """
60
64
61 @wraps(f)
65 @wraps(f)
62 def decorator(*args, **kwargs):
66 def decorator(*args, **kwargs):
63 return f(*args, **kwargs)
67 return f(*args, **kwargs)
64
68
65 return decorator
69 return decorator
66
70
67
71
68 #-----------------------------------------------------------------------------
72 #-----------------------------------------------------------------------------
69 # Functions
73 # Functions
70 #-----------------------------------------------------------------------------
74 #-----------------------------------------------------------------------------
71
75
72 __all__ = [
76 __all__ = [
73 'export',
77 'export',
74 'export_html',
78 'export_html',
75 'export_custom',
79 'export_custom',
76 'export_slides',
80 'export_slides',
77 'export_latex',
81 'export_latex',
78 'export_markdown',
82 'export_markdown',
79 'export_python',
83 'export_python',
80 'export_rst',
84 'export_rst',
81 'export_by_name',
85 'export_by_name',
82 'get_export_names',
86 'get_export_names',
83 'ExporterNameError'
87 'ExporterNameError'
84 ]
88 ]
85
89
86
90
87 class ExporterNameError(NameError):
91 class ExporterNameError(NameError):
88 pass
92 pass
89
93
90 @DocDecorator
94 @DocDecorator
91 def export(exporter, nb, **kw):
95 def export(exporter, nb, **kw):
92 """
96 """
93 Export a notebook object using specific exporter class.
97 Export a notebook object using specific exporter class.
94
98
95 exporter : Exporter class type or instance
99 Parameters
100 ----------
101 exporter : class:`~IPython.nbconvert.exporters.exporter.Exporter` class or instance
96 Class type or instance of the exporter that should be used. If the
102 Class type or instance of the exporter that should be used. If the
97 method initializes it's own instance of the class, it is ASSUMED that
103 method initializes it's own instance of the class, it is ASSUMED that
98 the class type provided exposes a constructor (__init__) with the same
104 the class type provided exposes a constructor (``__init__``) with the same
99 signature as the base Exporter class.
105 signature as the base Exporter class.
100 """
106 """
101
107
102 #Check arguments
108 #Check arguments
103 if exporter is None:
109 if exporter is None:
104 raise TypeError("Exporter is None")
110 raise TypeError("Exporter is None")
105 elif not isinstance(exporter, Exporter) and not issubclass(exporter, Exporter):
111 elif not isinstance(exporter, Exporter) and not issubclass(exporter, Exporter):
106 raise TypeError("exporter does not inherit from Exporter (base)")
112 raise TypeError("exporter does not inherit from Exporter (base)")
107 if nb is None:
113 if nb is None:
108 raise TypeError("nb is None")
114 raise TypeError("nb is None")
109
115
110 #Create the exporter
116 #Create the exporter
111 resources = kw.pop('resources', None)
117 resources = kw.pop('resources', None)
112 if isinstance(exporter, Exporter):
118 if isinstance(exporter, Exporter):
113 exporter_instance = exporter
119 exporter_instance = exporter
114 else:
120 else:
115 exporter_instance = exporter(**kw)
121 exporter_instance = exporter(**kw)
116
122
117 #Try to convert the notebook using the appropriate conversion function.
123 #Try to convert the notebook using the appropriate conversion function.
118 if isinstance(nb, NotebookNode):
124 if isinstance(nb, NotebookNode):
119 output, resources = exporter_instance.from_notebook_node(nb, resources)
125 output, resources = exporter_instance.from_notebook_node(nb, resources)
120 elif isinstance(nb, string_types):
126 elif isinstance(nb, string_types):
121 output, resources = exporter_instance.from_filename(nb, resources)
127 output, resources = exporter_instance.from_filename(nb, resources)
122 else:
128 else:
123 output, resources = exporter_instance.from_file(nb, resources)
129 output, resources = exporter_instance.from_file(nb, resources)
124 return output, resources
130 return output, resources
125
131
126 exporter_map = dict(
132 exporter_map = dict(
127 custom=TemplateExporter,
133 custom=TemplateExporter,
128 html=HTMLExporter,
134 html=HTMLExporter,
129 slides=SlidesExporter,
135 slides=SlidesExporter,
130 latex=LatexExporter,
136 latex=LatexExporter,
131 markdown=MarkdownExporter,
137 markdown=MarkdownExporter,
132 python=PythonExporter,
138 python=PythonExporter,
133 rst=RSTExporter,
139 rst=RSTExporter,
134 )
140 )
135
141
136 def _make_exporter(name, E):
142 def _make_exporter(name, E):
137 """make an export_foo function from a short key and Exporter class E"""
143 """make an export_foo function from a short key and Exporter class E"""
138 def _export(nb, **kw):
144 def _export(nb, **kw):
139 return export(E, nb, **kw)
145 return export(E, nb, **kw)
140 _export.__doc__ = """Export a notebook object to {0} format""".format(name)
146 _export.__doc__ = """Export a notebook object to {0} format""".format(name)
141 return _export
147 return _export
142
148
143 g = globals()
149 g = globals()
144
150
145 for name, E in exporter_map.items():
151 for name, E in exporter_map.items():
146 g['export_%s' % name] = DocDecorator(_make_exporter(name, E))
152 g['export_%s' % name] = DocDecorator(_make_exporter(name, E))
147
153
148 @DocDecorator
154 @DocDecorator
149 def export_by_name(format_name, nb, **kw):
155 def export_by_name(format_name, nb, **kw):
150 """
156 """
151 Export a notebook object to a template type by its name. Reflection
157 Export a notebook object to a template type by its name. Reflection
152 (Inspect) is used to find the template's corresponding explicit export
158 (Inspect) is used to find the template's corresponding explicit export
153 method defined in this module. That method is then called directly.
159 method defined in this module. That method is then called directly.
154
160
161 Parameters
162 ----------
155 format_name : str
163 format_name : str
156 Name of the template style to export to.
164 Name of the template style to export to.
157 """
165 """
158
166
159 function_name = "export_" + format_name.lower()
167 function_name = "export_" + format_name.lower()
160
168
161 if function_name in globals():
169 if function_name in globals():
162 return globals()[function_name](nb, **kw)
170 return globals()[function_name](nb, **kw)
163 else:
171 else:
164 raise ExporterNameError("template for `%s` not found" % function_name)
172 raise ExporterNameError("template for `%s` not found" % function_name)
165
173
166
174
167 def get_export_names():
175 def get_export_names():
168 """Return a list of the currently supported export targets
176 """Return a list of the currently supported export targets
169
177
170 WARNING: API WILL CHANGE IN FUTURE RELEASES OF NBCONVERT"""
178 WARNING: API WILL CHANGE IN FUTURE RELEASES OF NBCONVERT"""
171 return sorted(exporter_map.keys())
179 return sorted(exporter_map.keys())
@@ -1,323 +1,323 b''
1 #!/usr/bin/env python
1 #!/usr/bin/env python
2 """NBConvert is a utility for conversion of .ipynb files.
2 """NBConvert is a utility for conversion of .ipynb files.
3
3
4 Command-line interface for the NbConvert conversion utility.
4 Command-line interface for the NbConvert conversion utility.
5 """
5 """
6 #-----------------------------------------------------------------------------
6 #-----------------------------------------------------------------------------
7 #Copyright (c) 2013, the IPython Development Team.
7 #Copyright (c) 2013, the IPython Development Team.
8 #
8 #
9 #Distributed under the terms of the Modified BSD License.
9 #Distributed under the terms of the Modified BSD License.
10 #
10 #
11 #The full license is in the file COPYING.txt, distributed with this software.
11 #The full license is in the file COPYING.txt, distributed with this software.
12 #-----------------------------------------------------------------------------
12 #-----------------------------------------------------------------------------
13
13
14 #-----------------------------------------------------------------------------
14 #-----------------------------------------------------------------------------
15 #Imports
15 #Imports
16 #-----------------------------------------------------------------------------
16 #-----------------------------------------------------------------------------
17
17
18 # Stdlib imports
18 # Stdlib imports
19 from __future__ import print_function
19 from __future__ import print_function
20
20
21 import logging
21 import logging
22 import sys
22 import sys
23 import os
23 import os
24 import glob
24 import glob
25
25
26 # From IPython
26 # From IPython
27 from IPython.core.application import BaseIPythonApplication, base_aliases, base_flags
27 from IPython.core.application import BaseIPythonApplication, base_aliases, base_flags
28 from IPython.core.profiledir import ProfileDir
28 from IPython.core.profiledir import ProfileDir
29 from IPython.config import catch_config_error, Configurable
29 from IPython.config import catch_config_error, Configurable
30 from IPython.utils.traitlets import (
30 from IPython.utils.traitlets import (
31 Unicode, List, Instance, DottedObjectName, Type, CaselessStrEnum,
31 Unicode, List, Instance, DottedObjectName, Type, CaselessStrEnum,
32 )
32 )
33 from IPython.utils.importstring import import_item
33 from IPython.utils.importstring import import_item
34 from IPython.utils.text import dedent
34 from IPython.utils.text import dedent
35
35
36 from .exporters.export import get_export_names, exporter_map
36 from .exporters.export import get_export_names, exporter_map
37 from IPython.nbconvert import exporters, preprocessors, writers, postprocessors
37 from IPython.nbconvert import exporters, preprocessors, writers, postprocessors
38 from .utils.base import NbConvertBase
38 from .utils.base import NbConvertBase
39 from .utils.exceptions import ConversionException
39 from .utils.exceptions import ConversionException
40
40
41 #-----------------------------------------------------------------------------
41 #-----------------------------------------------------------------------------
42 #Classes and functions
42 #Classes and functions
43 #-----------------------------------------------------------------------------
43 #-----------------------------------------------------------------------------
44
44
45 class DottedOrNone(DottedObjectName):
45 class DottedOrNone(DottedObjectName):
46 """
46 """
47 A string holding a valid dotted object name in Python, such as A.b3._c
47 A string holding a valid dotted object name in Python, such as A.b3._c
48 Also allows for None type."""
48 Also allows for None type."""
49
49
50 default_value = u''
50 default_value = u''
51
51
52 def validate(self, obj, value):
52 def validate(self, obj, value):
53 if value is not None and len(value) > 0:
53 if value is not None and len(value) > 0:
54 return super(DottedOrNone, self).validate(obj, value)
54 return super(DottedOrNone, self).validate(obj, value)
55 else:
55 else:
56 return value
56 return value
57
57
58 nbconvert_aliases = {}
58 nbconvert_aliases = {}
59 nbconvert_aliases.update(base_aliases)
59 nbconvert_aliases.update(base_aliases)
60 nbconvert_aliases.update({
60 nbconvert_aliases.update({
61 'to' : 'NbConvertApp.export_format',
61 'to' : 'NbConvertApp.export_format',
62 'template' : 'TemplateExporter.template_file',
62 'template' : 'TemplateExporter.template_file',
63 'writer' : 'NbConvertApp.writer_class',
63 'writer' : 'NbConvertApp.writer_class',
64 'post': 'NbConvertApp.postprocessor_class',
64 'post': 'NbConvertApp.postprocessor_class',
65 'output': 'NbConvertApp.output_base',
65 'output': 'NbConvertApp.output_base',
66 'reveal-prefix': 'RevealHelpPreprocessor.url_prefix',
66 'reveal-prefix': 'RevealHelpPreprocessor.url_prefix',
67 })
67 })
68
68
69 nbconvert_flags = {}
69 nbconvert_flags = {}
70 nbconvert_flags.update(base_flags)
70 nbconvert_flags.update(base_flags)
71 nbconvert_flags.update({
71 nbconvert_flags.update({
72 'stdout' : (
72 'stdout' : (
73 {'NbConvertApp' : {'writer_class' : "StdoutWriter"}},
73 {'NbConvertApp' : {'writer_class' : "StdoutWriter"}},
74 "Write notebook output to stdout instead of files."
74 "Write notebook output to stdout instead of files."
75 )
75 )
76 })
76 })
77
77
78
78
79 class NbConvertApp(BaseIPythonApplication):
79 class NbConvertApp(BaseIPythonApplication):
80 """Application used to convert to and from notebook file type (*.ipynb)"""
80 """Application used to convert from notebook file type (``*.ipynb``)"""
81
81
82 name = 'ipython-nbconvert'
82 name = 'ipython-nbconvert'
83 aliases = nbconvert_aliases
83 aliases = nbconvert_aliases
84 flags = nbconvert_flags
84 flags = nbconvert_flags
85
85
86 def _log_level_default(self):
86 def _log_level_default(self):
87 return logging.INFO
87 return logging.INFO
88
88
89 def _classes_default(self):
89 def _classes_default(self):
90 classes = [NbConvertBase, ProfileDir]
90 classes = [NbConvertBase, ProfileDir]
91 for pkg in (exporters, preprocessors, writers, postprocessors):
91 for pkg in (exporters, preprocessors, writers, postprocessors):
92 for name in dir(pkg):
92 for name in dir(pkg):
93 cls = getattr(pkg, name)
93 cls = getattr(pkg, name)
94 if isinstance(cls, type) and issubclass(cls, Configurable):
94 if isinstance(cls, type) and issubclass(cls, Configurable):
95 classes.append(cls)
95 classes.append(cls)
96
96
97 return classes
97 return classes
98
98
99 description = Unicode(
99 description = Unicode(
100 u"""This application is used to convert notebook files (*.ipynb)
100 u"""This application is used to convert notebook files (*.ipynb)
101 to various other formats.
101 to various other formats.
102
102
103 WARNING: THE COMMANDLINE INTERFACE MAY CHANGE IN FUTURE RELEASES.""")
103 WARNING: THE COMMANDLINE INTERFACE MAY CHANGE IN FUTURE RELEASES.""")
104
104
105 output_base = Unicode('', config=True, help='''overwrite base name use for output files.
105 output_base = Unicode('', config=True, help='''overwrite base name use for output files.
106 can only be use when converting one notebook at a time.
106 can only be use when converting one notebook at a time.
107 ''')
107 ''')
108
108
109 examples = Unicode(u"""
109 examples = Unicode(u"""
110 The simplest way to use nbconvert is
110 The simplest way to use nbconvert is
111
111
112 > ipython nbconvert mynotebook.ipynb
112 > ipython nbconvert mynotebook.ipynb
113
113
114 which will convert mynotebook.ipynb to the default format (probably HTML).
114 which will convert mynotebook.ipynb to the default format (probably HTML).
115
115
116 You can specify the export format with `--to`.
116 You can specify the export format with `--to`.
117 Options include {0}
117 Options include {0}
118
118
119 > ipython nbconvert --to latex mynotebook.ipnynb
119 > ipython nbconvert --to latex mynotebook.ipnynb
120
120
121 Both HTML and LaTeX support multiple output templates. LaTeX includes
121 Both HTML and LaTeX support multiple output templates. LaTeX includes
122 'basic', 'book', and 'article'. HTML includes 'basic' and 'full'. You
122 'basic', 'book', and 'article'. HTML includes 'basic' and 'full'. You
123 can specify the flavor of the format used.
123 can specify the flavor of the format used.
124
124
125 > ipython nbconvert --to html --template basic mynotebook.ipynb
125 > ipython nbconvert --to html --template basic mynotebook.ipynb
126
126
127 You can also pipe the output to stdout, rather than a file
127 You can also pipe the output to stdout, rather than a file
128
128
129 > ipython nbconvert mynotebook.ipynb --stdout
129 > ipython nbconvert mynotebook.ipynb --stdout
130
130
131 A post-processor can be used to compile a PDF
131 A post-processor can be used to compile a PDF
132
132
133 > ipython nbconvert mynotebook.ipynb --to latex --post PDF
133 > ipython nbconvert mynotebook.ipynb --to latex --post PDF
134
134
135 You can get (and serve) a Reveal.js-powered slideshow
135 You can get (and serve) a Reveal.js-powered slideshow
136
136
137 > ipython nbconvert myslides.ipynb --to slides --post serve
137 > ipython nbconvert myslides.ipynb --to slides --post serve
138
138
139 Multiple notebooks can be given at the command line in a couple of
139 Multiple notebooks can be given at the command line in a couple of
140 different ways:
140 different ways:
141
141
142 > ipython nbconvert notebook*.ipynb
142 > ipython nbconvert notebook*.ipynb
143 > ipython nbconvert notebook1.ipynb notebook2.ipynb
143 > ipython nbconvert notebook1.ipynb notebook2.ipynb
144
144
145 or you can specify the notebooks list in a config file, containing::
145 or you can specify the notebooks list in a config file, containing::
146
146
147 c.NbConvertApp.notebooks = ["my_notebook.ipynb"]
147 c.NbConvertApp.notebooks = ["my_notebook.ipynb"]
148
148
149 > ipython nbconvert --config mycfg.py
149 > ipython nbconvert --config mycfg.py
150 """.format(get_export_names()))
150 """.format(get_export_names()))
151
151
152 # Writer specific variables
152 # Writer specific variables
153 writer = Instance('IPython.nbconvert.writers.base.WriterBase',
153 writer = Instance('IPython.nbconvert.writers.base.WriterBase',
154 help="""Instance of the writer class used to write the
154 help="""Instance of the writer class used to write the
155 results of the conversion.""")
155 results of the conversion.""")
156 writer_class = DottedObjectName('FilesWriter', config=True,
156 writer_class = DottedObjectName('FilesWriter', config=True,
157 help="""Writer class used to write the
157 help="""Writer class used to write the
158 results of the conversion""")
158 results of the conversion""")
159 writer_aliases = {'fileswriter': 'IPython.nbconvert.writers.files.FilesWriter',
159 writer_aliases = {'fileswriter': 'IPython.nbconvert.writers.files.FilesWriter',
160 'debugwriter': 'IPython.nbconvert.writers.debug.DebugWriter',
160 'debugwriter': 'IPython.nbconvert.writers.debug.DebugWriter',
161 'stdoutwriter': 'IPython.nbconvert.writers.stdout.StdoutWriter'}
161 'stdoutwriter': 'IPython.nbconvert.writers.stdout.StdoutWriter'}
162 writer_factory = Type()
162 writer_factory = Type()
163
163
164 def _writer_class_changed(self, name, old, new):
164 def _writer_class_changed(self, name, old, new):
165 if new.lower() in self.writer_aliases:
165 if new.lower() in self.writer_aliases:
166 new = self.writer_aliases[new.lower()]
166 new = self.writer_aliases[new.lower()]
167 self.writer_factory = import_item(new)
167 self.writer_factory = import_item(new)
168
168
169 # Post-processor specific variables
169 # Post-processor specific variables
170 postprocessor = Instance('IPython.nbconvert.postprocessors.base.PostProcessorBase',
170 postprocessor = Instance('IPython.nbconvert.postprocessors.base.PostProcessorBase',
171 help="""Instance of the PostProcessor class used to write the
171 help="""Instance of the PostProcessor class used to write the
172 results of the conversion.""")
172 results of the conversion.""")
173
173
174 postprocessor_class = DottedOrNone(config=True,
174 postprocessor_class = DottedOrNone(config=True,
175 help="""PostProcessor class used to write the
175 help="""PostProcessor class used to write the
176 results of the conversion""")
176 results of the conversion""")
177 postprocessor_aliases = {'pdf': 'IPython.nbconvert.postprocessors.pdf.PDFPostProcessor',
177 postprocessor_aliases = {'pdf': 'IPython.nbconvert.postprocessors.pdf.PDFPostProcessor',
178 'serve': 'IPython.nbconvert.postprocessors.serve.ServePostProcessor'}
178 'serve': 'IPython.nbconvert.postprocessors.serve.ServePostProcessor'}
179 postprocessor_factory = Type()
179 postprocessor_factory = Type()
180
180
181 def _postprocessor_class_changed(self, name, old, new):
181 def _postprocessor_class_changed(self, name, old, new):
182 if new.lower() in self.postprocessor_aliases:
182 if new.lower() in self.postprocessor_aliases:
183 new = self.postprocessor_aliases[new.lower()]
183 new = self.postprocessor_aliases[new.lower()]
184 if new:
184 if new:
185 self.postprocessor_factory = import_item(new)
185 self.postprocessor_factory = import_item(new)
186
186
187
187
188 # Other configurable variables
188 # Other configurable variables
189 export_format = CaselessStrEnum(get_export_names(),
189 export_format = CaselessStrEnum(get_export_names(),
190 default_value="html",
190 default_value="html",
191 config=True,
191 config=True,
192 help="""The export format to be used."""
192 help="""The export format to be used."""
193 )
193 )
194
194
195 notebooks = List([], config=True, help="""List of notebooks to convert.
195 notebooks = List([], config=True, help="""List of notebooks to convert.
196 Wildcards are supported.
196 Wildcards are supported.
197 Filenames passed positionally will be added to the list.
197 Filenames passed positionally will be added to the list.
198 """)
198 """)
199
199
200 @catch_config_error
200 @catch_config_error
201 def initialize(self, argv=None):
201 def initialize(self, argv=None):
202 super(NbConvertApp, self).initialize(argv)
202 super(NbConvertApp, self).initialize(argv)
203 self.init_syspath()
203 self.init_syspath()
204 self.init_notebooks()
204 self.init_notebooks()
205 self.init_writer()
205 self.init_writer()
206 self.init_postprocessor()
206 self.init_postprocessor()
207
207
208
208
209
209
210 def init_syspath(self):
210 def init_syspath(self):
211 """
211 """
212 Add the cwd to the sys.path ($PYTHONPATH)
212 Add the cwd to the sys.path ($PYTHONPATH)
213 """
213 """
214 sys.path.insert(0, os.getcwd())
214 sys.path.insert(0, os.getcwd())
215
215
216
216
217 def init_notebooks(self):
217 def init_notebooks(self):
218 """Construct the list of notebooks.
218 """Construct the list of notebooks.
219 If notebooks are passed on the command-line,
219 If notebooks are passed on the command-line,
220 they override notebooks specified in config files.
220 they override notebooks specified in config files.
221 Glob each notebook to replace notebook patterns with filenames.
221 Glob each notebook to replace notebook patterns with filenames.
222 """
222 """
223
223
224 # Specifying notebooks on the command-line overrides (rather than adds)
224 # Specifying notebooks on the command-line overrides (rather than adds)
225 # the notebook list
225 # the notebook list
226 if self.extra_args:
226 if self.extra_args:
227 patterns = self.extra_args
227 patterns = self.extra_args
228 else:
228 else:
229 patterns = self.notebooks
229 patterns = self.notebooks
230
230
231 # Use glob to replace all the notebook patterns with filenames.
231 # Use glob to replace all the notebook patterns with filenames.
232 filenames = []
232 filenames = []
233 for pattern in patterns:
233 for pattern in patterns:
234
234
235 # Use glob to find matching filenames. Allow the user to convert
235 # Use glob to find matching filenames. Allow the user to convert
236 # notebooks without having to type the extension.
236 # notebooks without having to type the extension.
237 globbed_files = glob.glob(pattern)
237 globbed_files = glob.glob(pattern)
238 globbed_files.extend(glob.glob(pattern + '.ipynb'))
238 globbed_files.extend(glob.glob(pattern + '.ipynb'))
239 if not globbed_files:
239 if not globbed_files:
240 self.log.warn("pattern %r matched no files", pattern)
240 self.log.warn("pattern %r matched no files", pattern)
241
241
242 for filename in globbed_files:
242 for filename in globbed_files:
243 if not filename in filenames:
243 if not filename in filenames:
244 filenames.append(filename)
244 filenames.append(filename)
245 self.notebooks = filenames
245 self.notebooks = filenames
246
246
247 def init_writer(self):
247 def init_writer(self):
248 """
248 """
249 Initialize the writer (which is stateless)
249 Initialize the writer (which is stateless)
250 """
250 """
251 self._writer_class_changed(None, self.writer_class, self.writer_class)
251 self._writer_class_changed(None, self.writer_class, self.writer_class)
252 self.writer = self.writer_factory(parent=self)
252 self.writer = self.writer_factory(parent=self)
253
253
254 def init_postprocessor(self):
254 def init_postprocessor(self):
255 """
255 """
256 Initialize the postprocessor (which is stateless)
256 Initialize the postprocessor (which is stateless)
257 """
257 """
258 self._postprocessor_class_changed(None, self.postprocessor_class,
258 self._postprocessor_class_changed(None, self.postprocessor_class,
259 self.postprocessor_class)
259 self.postprocessor_class)
260 if self.postprocessor_factory:
260 if self.postprocessor_factory:
261 self.postprocessor = self.postprocessor_factory(parent=self)
261 self.postprocessor = self.postprocessor_factory(parent=self)
262
262
263 def start(self):
263 def start(self):
264 """
264 """
265 Ran after initialization completed
265 Ran after initialization completed
266 """
266 """
267 super(NbConvertApp, self).start()
267 super(NbConvertApp, self).start()
268 self.convert_notebooks()
268 self.convert_notebooks()
269
269
270 def convert_notebooks(self):
270 def convert_notebooks(self):
271 """
271 """
272 Convert the notebooks in the self.notebook traitlet
272 Convert the notebooks in the self.notebook traitlet
273 """
273 """
274 # Export each notebook
274 # Export each notebook
275 conversion_success = 0
275 conversion_success = 0
276
276
277 if self.output_base != '' and len(self.notebooks) > 1:
277 if self.output_base != '' and len(self.notebooks) > 1:
278 self.log.error(
278 self.log.error(
279 """UsageError: --output flag or `NbConvertApp.output_base` config option
279 """UsageError: --output flag or `NbConvertApp.output_base` config option
280 cannot be used when converting multiple notebooks.
280 cannot be used when converting multiple notebooks.
281 """)
281 """)
282 self.exit(1)
282 self.exit(1)
283
283
284 exporter = exporter_map[self.export_format](config=self.config)
284 exporter = exporter_map[self.export_format](config=self.config)
285
285
286 for notebook_filename in self.notebooks:
286 for notebook_filename in self.notebooks:
287 self.log.info("Converting notebook %s to %s", notebook_filename, self.export_format)
287 self.log.info("Converting notebook %s to %s", notebook_filename, self.export_format)
288
288
289 # Get a unique key for the notebook and set it in the resources object.
289 # Get a unique key for the notebook and set it in the resources object.
290 basename = os.path.basename(notebook_filename)
290 basename = os.path.basename(notebook_filename)
291 notebook_name = basename[:basename.rfind('.')]
291 notebook_name = basename[:basename.rfind('.')]
292 if self.output_base:
292 if self.output_base:
293 notebook_name = self.output_base
293 notebook_name = self.output_base
294 resources = {}
294 resources = {}
295 resources['unique_key'] = notebook_name
295 resources['unique_key'] = notebook_name
296 resources['output_files_dir'] = '%s_files' % notebook_name
296 resources['output_files_dir'] = '%s_files' % notebook_name
297 self.log.info("Support files will be in %s", os.path.join(resources['output_files_dir'], ''))
297 self.log.info("Support files will be in %s", os.path.join(resources['output_files_dir'], ''))
298
298
299 # Try to export
299 # Try to export
300 try:
300 try:
301 output, resources = exporter.from_filename(notebook_filename, resources=resources)
301 output, resources = exporter.from_filename(notebook_filename, resources=resources)
302 except ConversionException as e:
302 except ConversionException as e:
303 self.log.error("Error while converting '%s'", notebook_filename,
303 self.log.error("Error while converting '%s'", notebook_filename,
304 exc_info=True)
304 exc_info=True)
305 self.exit(1)
305 self.exit(1)
306 else:
306 else:
307 write_resultes = self.writer.write(output, resources, notebook_name=notebook_name)
307 write_resultes = self.writer.write(output, resources, notebook_name=notebook_name)
308
308
309 #Post-process if post processor has been defined.
309 #Post-process if post processor has been defined.
310 if hasattr(self, 'postprocessor') and self.postprocessor:
310 if hasattr(self, 'postprocessor') and self.postprocessor:
311 self.postprocessor(write_resultes)
311 self.postprocessor(write_resultes)
312 conversion_success += 1
312 conversion_success += 1
313
313
314 # If nothing was converted successfully, help the user.
314 # If nothing was converted successfully, help the user.
315 if conversion_success == 0:
315 if conversion_success == 0:
316 self.print_help()
316 self.print_help()
317 sys.exit(-1)
317 sys.exit(-1)
318
318
319 #-----------------------------------------------------------------------------
319 #-----------------------------------------------------------------------------
320 # Main entry point
320 # Main entry point
321 #-----------------------------------------------------------------------------
321 #-----------------------------------------------------------------------------
322
322
323 launch_new_instance = NbConvertApp.launch_instance
323 launch_new_instance = NbConvertApp.launch_instance
General Comments 0
You need to be logged in to leave comments. Login now