##// END OF EJS Templates
Small edits for clarity and local optimization.
Fernando Perez -
Show More
@@ -1,772 +1,773 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 from __future__ import print_function
63 from __future__ import print_function
64
64
65 #-----------------------------------------------------------------------------
65 #-----------------------------------------------------------------------------
66 # Imports
66 # Imports
67 #-----------------------------------------------------------------------------
67 #-----------------------------------------------------------------------------
68 # stdlib
68 # stdlib
69 import ast
69 import ast
70 import codeop
70 import codeop
71 import re
71 import re
72 import sys
72 import sys
73 import tokenize
73 import tokenize
74 from StringIO import StringIO
74 from StringIO import StringIO
75
75
76 # IPython modules
76 # IPython modules
77 from IPython.core.splitinput import split_user_input, LineInfo
77 from IPython.core.splitinput import split_user_input, LineInfo
78 from IPython.utils.text import make_quoted_expr
78 from IPython.utils.text import make_quoted_expr
79 from IPython.utils.py3compat import cast_unicode
79 from IPython.utils.py3compat import cast_unicode
80
80
81 #-----------------------------------------------------------------------------
81 #-----------------------------------------------------------------------------
82 # Globals
82 # Globals
83 #-----------------------------------------------------------------------------
83 #-----------------------------------------------------------------------------
84
84
85 # The escape sequences that define the syntax transformations IPython will
85 # The escape sequences that define the syntax transformations IPython will
86 # apply to user input. These can NOT be just changed here: many regular
86 # apply to user input. These can NOT be just changed here: many regular
87 # expressions and other parts of the code may use their hardcoded values, and
87 # expressions and other parts of the code may use their hardcoded values, and
88 # for all intents and purposes they constitute the 'IPython syntax', so they
88 # for all intents and purposes they constitute the 'IPython syntax', so they
89 # should be considered fixed.
89 # should be considered fixed.
90
90
91 ESC_SHELL = '!' # Send line to underlying system shell
91 ESC_SHELL = '!' # Send line to underlying system shell
92 ESC_SH_CAP = '!!' # Send line to system shell and capture output
92 ESC_SH_CAP = '!!' # Send line to system shell and capture output
93 ESC_HELP = '?' # Find information about object
93 ESC_HELP = '?' # Find information about object
94 ESC_HELP2 = '??' # Find extra-detailed information about object
94 ESC_HELP2 = '??' # Find extra-detailed information about object
95 ESC_MAGIC = '%' # Call magic function
95 ESC_MAGIC = '%' # Call magic function
96 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
96 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
97 ESC_QUOTE2 = ';' # Quote all args as a single string, call
97 ESC_QUOTE2 = ';' # Quote all args as a single string, call
98 ESC_PAREN = '/' # Call first argument with rest of line as arguments
98 ESC_PAREN = '/' # Call first argument with rest of line as arguments
99
99
100 #-----------------------------------------------------------------------------
100 #-----------------------------------------------------------------------------
101 # Utilities
101 # Utilities
102 #-----------------------------------------------------------------------------
102 #-----------------------------------------------------------------------------
103
103
104 # FIXME: These are general-purpose utilities that later can be moved to the
104 # FIXME: These are general-purpose utilities that later can be moved to the
105 # general ward. Kept here for now because we're being very strict about test
105 # general ward. Kept here for now because we're being very strict about test
106 # coverage with this code, and this lets us ensure that we keep 100% coverage
106 # coverage with this code, and this lets us ensure that we keep 100% coverage
107 # while developing.
107 # while developing.
108
108
109 # compiled regexps for autoindent management
109 # compiled regexps for autoindent management
110 dedent_re = re.compile('|'.join([
110 dedent_re = re.compile('|'.join([
111 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
111 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
112 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
112 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
113 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
113 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
114 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
114 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
115 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
115 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
116 ]))
116 ]))
117 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
117 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
118
118
119 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
119 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
120 # before pure comments
120 # before pure comments
121 comment_line_re = re.compile('^\s*\#')
121 comment_line_re = re.compile('^\s*\#')
122
122
123
123
124 def num_ini_spaces(s):
124 def num_ini_spaces(s):
125 """Return the number of initial spaces in a string.
125 """Return the number of initial spaces in a string.
126
126
127 Note that tabs are counted as a single space. For now, we do *not* support
127 Note that tabs are counted as a single space. For now, we do *not* support
128 mixing of tabs and spaces in the user's input.
128 mixing of tabs and spaces in the user's input.
129
129
130 Parameters
130 Parameters
131 ----------
131 ----------
132 s : string
132 s : string
133
133
134 Returns
134 Returns
135 -------
135 -------
136 n : int
136 n : int
137 """
137 """
138
138
139 ini_spaces = ini_spaces_re.match(s)
139 ini_spaces = ini_spaces_re.match(s)
140 if ini_spaces:
140 if ini_spaces:
141 return ini_spaces.end()
141 return ini_spaces.end()
142 else:
142 else:
143 return 0
143 return 0
144
144
145
145
146 def remove_comments(src):
146 def remove_comments(src):
147 """Remove all comments from input source.
147 """Remove all comments from input source.
148
148
149 Note: comments are NOT recognized inside of strings!
149 Note: comments are NOT recognized inside of strings!
150
150
151 Parameters
151 Parameters
152 ----------
152 ----------
153 src : string
153 src : string
154 A single or multiline input string.
154 A single or multiline input string.
155
155
156 Returns
156 Returns
157 -------
157 -------
158 String with all Python comments removed.
158 String with all Python comments removed.
159 """
159 """
160
160
161 return re.sub('#.*', '', src)
161 return re.sub('#.*', '', src)
162
162
163 def has_comment(src):
163 def has_comment(src):
164 """Indicate whether an input line has (i.e. ends in, or is) a comment.
164 """Indicate whether an input line has (i.e. ends in, or is) a comment.
165
165
166 This uses tokenize, so it can distinguish comments from # inside strings.
166 This uses tokenize, so it can distinguish comments from # inside strings.
167
167
168 Parameters
168 Parameters
169 ----------
169 ----------
170 src : string
170 src : string
171 A single line input string.
171 A single line input string.
172
172
173 Returns
173 Returns
174 -------
174 -------
175 Boolean: True if source has a comment.
175 Boolean: True if source has a comment.
176 """
176 """
177 readline = StringIO(src).readline
177 readline = StringIO(src).readline
178 toktypes = set()
178 toktypes = set()
179 try:
179 try:
180 for t in tokenize.generate_tokens(readline):
180 for t in tokenize.generate_tokens(readline):
181 toktypes.add(t[0])
181 toktypes.add(t[0])
182 except tokenize.TokenError:
182 except tokenize.TokenError:
183 pass
183 pass
184 return(tokenize.COMMENT in toktypes)
184 return(tokenize.COMMENT in toktypes)
185
185
186
186
187 def get_input_encoding():
187 def get_input_encoding():
188 """Return the default standard input encoding.
188 """Return the default standard input encoding.
189
189
190 If sys.stdin has no encoding, 'ascii' is returned."""
190 If sys.stdin has no encoding, 'ascii' is returned."""
191 # There are strange environments for which sys.stdin.encoding is None. We
191 # There are strange environments for which sys.stdin.encoding is None. We
192 # ensure that a valid encoding is returned.
192 # ensure that a valid encoding is returned.
193 encoding = getattr(sys.stdin, 'encoding', None)
193 encoding = getattr(sys.stdin, 'encoding', None)
194 if encoding is None:
194 if encoding is None:
195 encoding = 'ascii'
195 encoding = 'ascii'
196 return encoding
196 return encoding
197
197
198 #-----------------------------------------------------------------------------
198 #-----------------------------------------------------------------------------
199 # Classes and functions for normal Python syntax handling
199 # Classes and functions for normal Python syntax handling
200 #-----------------------------------------------------------------------------
200 #-----------------------------------------------------------------------------
201
201
202 class InputSplitter(object):
202 class InputSplitter(object):
203 """An object that can accumulate lines of Python source before execution.
203 """An object that can accumulate lines of Python source before execution.
204
204
205 This object is designed to be fed python source line-by-line, using
205 This object is designed to be fed python source line-by-line, using
206 :meth:`push`. It will return on each push whether the currently pushed
206 :meth:`push`. It will return on each push whether the currently pushed
207 code could be executed already. In addition, it provides a method called
207 code could be executed already. In addition, it provides a method called
208 :meth:`push_accepts_more` that can be used to query whether more input
208 :meth:`push_accepts_more` that can be used to query whether more input
209 can be pushed into a single interactive block.
209 can be pushed into a single interactive block.
210
210
211 This is a simple example of how an interactive terminal-based client can use
211 This is a simple example of how an interactive terminal-based client can use
212 this tool::
212 this tool::
213
213
214 isp = InputSplitter()
214 isp = InputSplitter()
215 while isp.push_accepts_more():
215 while isp.push_accepts_more():
216 indent = ' '*isp.indent_spaces
216 indent = ' '*isp.indent_spaces
217 prompt = '>>> ' + indent
217 prompt = '>>> ' + indent
218 line = indent + raw_input(prompt)
218 line = indent + raw_input(prompt)
219 isp.push(line)
219 isp.push(line)
220 print 'Input source was:\n', isp.source_reset(),
220 print 'Input source was:\n', isp.source_reset(),
221 """
221 """
222 # Number of spaces of indentation computed from input that has been pushed
222 # Number of spaces of indentation computed from input that has been pushed
223 # so far. This is the attributes callers should query to get the current
223 # so far. This is the attributes callers should query to get the current
224 # indentation level, in order to provide auto-indent facilities.
224 # indentation level, in order to provide auto-indent facilities.
225 indent_spaces = 0
225 indent_spaces = 0
226 # String, indicating the default input encoding. It is computed by default
226 # String, indicating the default input encoding. It is computed by default
227 # at initialization time via get_input_encoding(), but it can be reset by a
227 # at initialization time via get_input_encoding(), but it can be reset by a
228 # client with specific knowledge of the encoding.
228 # client with specific knowledge of the encoding.
229 encoding = ''
229 encoding = ''
230 # String where the current full source input is stored, properly encoded.
230 # String where the current full source input is stored, properly encoded.
231 # Reading this attribute is the normal way of querying the currently pushed
231 # Reading this attribute is the normal way of querying the currently pushed
232 # source code, that has been properly encoded.
232 # source code, that has been properly encoded.
233 source = ''
233 source = ''
234 # Code object corresponding to the current source. It is automatically
234 # Code object corresponding to the current source. It is automatically
235 # synced to the source, so it can be queried at any time to obtain the code
235 # synced to the source, so it can be queried at any time to obtain the code
236 # object; it will be None if the source doesn't compile to valid Python.
236 # object; it will be None if the source doesn't compile to valid Python.
237 code = None
237 code = None
238 # Input mode
238 # Input mode
239 input_mode = 'line'
239 input_mode = 'line'
240
240
241 # Private attributes
241 # Private attributes
242
242
243 # List with lines of input accumulated so far
243 # List with lines of input accumulated so far
244 _buffer = None
244 _buffer = None
245 # Command compiler
245 # Command compiler
246 _compile = None
246 _compile = None
247 # Mark when input has changed indentation all the way back to flush-left
247 # Mark when input has changed indentation all the way back to flush-left
248 _full_dedent = False
248 _full_dedent = False
249 # Boolean indicating whether the current block is complete
249 # Boolean indicating whether the current block is complete
250 _is_complete = None
250 _is_complete = None
251
251
252 def __init__(self, input_mode=None):
252 def __init__(self, input_mode=None):
253 """Create a new InputSplitter instance.
253 """Create a new InputSplitter instance.
254
254
255 Parameters
255 Parameters
256 ----------
256 ----------
257 input_mode : str
257 input_mode : str
258
258
259 One of ['line', 'cell']; default is 'line'.
259 One of ['line', 'cell']; default is 'line'.
260
260
261 The input_mode parameter controls how new inputs are used when fed via
261 The input_mode parameter controls how new inputs are used when fed via
262 the :meth:`push` method:
262 the :meth:`push` method:
263
263
264 - 'line': meant for line-oriented clients, inputs are appended one at a
264 - 'line': meant for line-oriented clients, inputs are appended one at a
265 time to the internal buffer and the whole buffer is compiled.
265 time to the internal buffer and the whole buffer is compiled.
266
266
267 - 'cell': meant for clients that can edit multi-line 'cells' of text at
267 - 'cell': meant for clients that can edit multi-line 'cells' of text at
268 a time. A cell can contain one or more blocks that can be compile in
268 a time. A cell can contain one or more blocks that can be compile in
269 'single' mode by Python. In this mode, each new input new input
269 'single' mode by Python. In this mode, each new input new input
270 completely replaces all prior inputs. Cell mode is thus equivalent
270 completely replaces all prior inputs. Cell mode is thus equivalent
271 to prepending a full reset() to every push() call.
271 to prepending a full reset() to every push() call.
272 """
272 """
273 self._buffer = []
273 self._buffer = []
274 self._compile = codeop.CommandCompiler()
274 self._compile = codeop.CommandCompiler()
275 self.encoding = get_input_encoding()
275 self.encoding = get_input_encoding()
276 self.input_mode = InputSplitter.input_mode if input_mode is None \
276 self.input_mode = InputSplitter.input_mode if input_mode is None \
277 else input_mode
277 else input_mode
278
278
279 def reset(self):
279 def reset(self):
280 """Reset the input buffer and associated state."""
280 """Reset the input buffer and associated state."""
281 self.indent_spaces = 0
281 self.indent_spaces = 0
282 self._buffer[:] = []
282 self._buffer[:] = []
283 self.source = ''
283 self.source = ''
284 self.code = None
284 self.code = None
285 self._is_complete = False
285 self._is_complete = False
286 self._full_dedent = False
286 self._full_dedent = False
287
287
288 def source_reset(self):
288 def source_reset(self):
289 """Return the input source and perform a full reset.
289 """Return the input source and perform a full reset.
290 """
290 """
291 out = self.source
291 out = self.source
292 self.reset()
292 self.reset()
293 return out
293 return out
294
294
295 def push(self, lines):
295 def push(self, lines):
296 """Push one or more lines of input.
296 """Push one or more lines of input.
297
297
298 This stores the given lines and returns a status code indicating
298 This stores the given lines and returns a status code indicating
299 whether the code forms a complete Python block or not.
299 whether the code forms a complete Python block or not.
300
300
301 Any exceptions generated in compilation are swallowed, but if an
301 Any exceptions generated in compilation are swallowed, but if an
302 exception was produced, the method returns True.
302 exception was produced, the method returns True.
303
303
304 Parameters
304 Parameters
305 ----------
305 ----------
306 lines : string
306 lines : string
307 One or more lines of Python input.
307 One or more lines of Python input.
308
308
309 Returns
309 Returns
310 -------
310 -------
311 is_complete : boolean
311 is_complete : boolean
312 True if the current input source (the result of the current input
312 True if the current input source (the result of the current input
313 plus prior inputs) forms a complete Python execution block. Note that
313 plus prior inputs) forms a complete Python execution block. Note that
314 this value is also stored as a private attribute (_is_complete), so it
314 this value is also stored as a private attribute (_is_complete), so it
315 can be queried at any time.
315 can be queried at any time.
316 """
316 """
317 if self.input_mode == 'cell':
317 if self.input_mode == 'cell':
318 self.reset()
318 self.reset()
319
319
320 self._store(lines)
320 self._store(lines)
321 source = self.source
321 source = self.source
322
322
323 # Before calling _compile(), reset the code object to None so that if an
323 # Before calling _compile(), reset the code object to None so that if an
324 # exception is raised in compilation, we don't mislead by having
324 # exception is raised in compilation, we don't mislead by having
325 # inconsistent code/source attributes.
325 # inconsistent code/source attributes.
326 self.code, self._is_complete = None, None
326 self.code, self._is_complete = None, None
327
327
328 # Honor termination lines properly
328 # Honor termination lines properly
329 if source.rstrip().endswith('\\'):
329 if source.rstrip().endswith('\\'):
330 return False
330 return False
331
331
332 self._update_indent(lines)
332 self._update_indent(lines)
333 try:
333 try:
334 self.code = self._compile(source, symbol="exec")
334 self.code = self._compile(source, symbol="exec")
335 # Invalid syntax can produce any of a number of different errors from
335 # Invalid syntax can produce any of a number of different errors from
336 # inside the compiler, so we have to catch them all. Syntax errors
336 # inside the compiler, so we have to catch them all. Syntax errors
337 # immediately produce a 'ready' block, so the invalid Python can be
337 # immediately produce a 'ready' block, so the invalid Python can be
338 # sent to the kernel for evaluation with possible ipython
338 # sent to the kernel for evaluation with possible ipython
339 # special-syntax conversion.
339 # special-syntax conversion.
340 except (SyntaxError, OverflowError, ValueError, TypeError,
340 except (SyntaxError, OverflowError, ValueError, TypeError,
341 MemoryError):
341 MemoryError):
342 self._is_complete = True
342 self._is_complete = True
343 else:
343 else:
344 # Compilation didn't produce any exceptions (though it may not have
344 # Compilation didn't produce any exceptions (though it may not have
345 # given a complete code object)
345 # given a complete code object)
346 self._is_complete = self.code is not None
346 self._is_complete = self.code is not None
347
347
348 return self._is_complete
348 return self._is_complete
349
349
350 def push_accepts_more(self):
350 def push_accepts_more(self):
351 """Return whether a block of interactive input can accept more input.
351 """Return whether a block of interactive input can accept more input.
352
352
353 This method is meant to be used by line-oriented frontends, who need to
353 This method is meant to be used by line-oriented frontends, who need to
354 guess whether a block is complete or not based solely on prior and
354 guess whether a block is complete or not based solely on prior and
355 current input lines. The InputSplitter considers it has a complete
355 current input lines. The InputSplitter considers it has a complete
356 interactive block and will not accept more input only when either a
356 interactive block and will not accept more input only when either a
357 SyntaxError is raised, or *all* of the following are true:
357 SyntaxError is raised, or *all* of the following are true:
358
358
359 1. The input compiles to a complete statement.
359 1. The input compiles to a complete statement.
360
360
361 2. The indentation level is flush-left (because if we are indented,
361 2. The indentation level is flush-left (because if we are indented,
362 like inside a function definition or for loop, we need to keep
362 like inside a function definition or for loop, we need to keep
363 reading new input).
363 reading new input).
364
364
365 3. There is one extra line consisting only of whitespace.
365 3. There is one extra line consisting only of whitespace.
366
366
367 Because of condition #3, this method should be used only by
367 Because of condition #3, this method should be used only by
368 *line-oriented* frontends, since it means that intermediate blank lines
368 *line-oriented* frontends, since it means that intermediate blank lines
369 are not allowed in function definitions (or any other indented block).
369 are not allowed in function definitions (or any other indented block).
370
370
371 If the current input produces a syntax error, this method immediately
371 If the current input produces a syntax error, this method immediately
372 returns False but does *not* raise the syntax error exception, as
372 returns False but does *not* raise the syntax error exception, as
373 typically clients will want to send invalid syntax to an execution
373 typically clients will want to send invalid syntax to an execution
374 backend which might convert the invalid syntax into valid Python via
374 backend which might convert the invalid syntax into valid Python via
375 one of the dynamic IPython mechanisms.
375 one of the dynamic IPython mechanisms.
376 """
376 """
377
377
378 # With incomplete input, unconditionally accept more
378 # With incomplete input, unconditionally accept more
379 if not self._is_complete:
379 if not self._is_complete:
380 return True
380 return True
381
381
382 # If we already have complete input and we're flush left, the answer
382 # If we already have complete input and we're flush left, the answer
383 # depends. In line mode, if there hasn't been any indentation,
383 # depends. In line mode, if there hasn't been any indentation,
384 # that's it. If we've come back from some indentation, we need
384 # that's it. If we've come back from some indentation, we need
385 # the blank final line to finish.
385 # the blank final line to finish.
386 # In cell mode, we need to check how many blocks the input so far
386 # In cell mode, we need to check how many blocks the input so far
387 # compiles into, because if there's already more than one full
387 # compiles into, because if there's already more than one full
388 # independent block of input, then the client has entered full
388 # independent block of input, then the client has entered full
389 # 'cell' mode and is feeding lines that each is complete. In this
389 # 'cell' mode and is feeding lines that each is complete. In this
390 # case we should then keep accepting. The Qt terminal-like console
390 # case we should then keep accepting. The Qt terminal-like console
391 # does precisely this, to provide the convenience of terminal-like
391 # does precisely this, to provide the convenience of terminal-like
392 # input of single expressions, but allowing the user (with a
392 # input of single expressions, but allowing the user (with a
393 # separate keystroke) to switch to 'cell' mode and type multiple
393 # separate keystroke) to switch to 'cell' mode and type multiple
394 # expressions in one shot.
394 # expressions in one shot.
395 if self.indent_spaces==0:
395 if self.indent_spaces==0:
396 if self.input_mode=='line':
396 if self.input_mode=='line':
397 if not self._full_dedent:
397 if not self._full_dedent:
398 return False
398 return False
399 else:
399 else:
400 try:
400 try:
401 code_ast = ast.parse(u''.join(self._buffer))
401 code_ast = ast.parse(u''.join(self._buffer))
402 except Exception:
402 except Exception:
403 return False
403 return False
404 else:
404 else:
405 if len(code_ast.body) == 1:
405 if len(code_ast.body) == 1:
406 return False
406 return False
407
407
408 # When input is complete, then termination is marked by an extra blank
408 # When input is complete, then termination is marked by an extra blank
409 # line at the end.
409 # line at the end.
410 last_line = self.source.splitlines()[-1]
410 last_line = self.source.splitlines()[-1]
411 return bool(last_line and not last_line.isspace())
411 return bool(last_line and not last_line.isspace())
412
412
413 #------------------------------------------------------------------------
413 #------------------------------------------------------------------------
414 # Private interface
414 # Private interface
415 #------------------------------------------------------------------------
415 #------------------------------------------------------------------------
416
416
417 def _find_indent(self, line):
417 def _find_indent(self, line):
418 """Compute the new indentation level for a single line.
418 """Compute the new indentation level for a single line.
419
419
420 Parameters
420 Parameters
421 ----------
421 ----------
422 line : str
422 line : str
423 A single new line of non-whitespace, non-comment Python input.
423 A single new line of non-whitespace, non-comment Python input.
424
424
425 Returns
425 Returns
426 -------
426 -------
427 indent_spaces : int
427 indent_spaces : int
428 New value for the indent level (it may be equal to self.indent_spaces
428 New value for the indent level (it may be equal to self.indent_spaces
429 if indentation doesn't change.
429 if indentation doesn't change.
430
430
431 full_dedent : boolean
431 full_dedent : boolean
432 Whether the new line causes a full flush-left dedent.
432 Whether the new line causes a full flush-left dedent.
433 """
433 """
434 indent_spaces = self.indent_spaces
434 indent_spaces = self.indent_spaces
435 full_dedent = self._full_dedent
435 full_dedent = self._full_dedent
436
436
437 inisp = num_ini_spaces(line)
437 inisp = num_ini_spaces(line)
438 if inisp < indent_spaces:
438 if inisp < indent_spaces:
439 indent_spaces = inisp
439 indent_spaces = inisp
440 if indent_spaces <= 0:
440 if indent_spaces <= 0:
441 #print 'Full dedent in text',self.source # dbg
441 #print 'Full dedent in text',self.source # dbg
442 full_dedent = True
442 full_dedent = True
443
443
444 if line.rstrip()[-1] == ':':
444 if line.rstrip()[-1] == ':':
445 indent_spaces += 4
445 indent_spaces += 4
446 elif dedent_re.match(line):
446 elif dedent_re.match(line):
447 indent_spaces -= 4
447 indent_spaces -= 4
448 if indent_spaces <= 0:
448 if indent_spaces <= 0:
449 full_dedent = True
449 full_dedent = True
450
450
451 # Safety
451 # Safety
452 if indent_spaces < 0:
452 if indent_spaces < 0:
453 indent_spaces = 0
453 indent_spaces = 0
454 #print 'safety' # dbg
454 #print 'safety' # dbg
455
455
456 return indent_spaces, full_dedent
456 return indent_spaces, full_dedent
457
457
458 def _update_indent(self, lines):
458 def _update_indent(self, lines):
459 for line in remove_comments(lines).splitlines():
459 for line in remove_comments(lines).splitlines():
460 if line and not line.isspace():
460 if line and not line.isspace():
461 self.indent_spaces, self._full_dedent = self._find_indent(line)
461 self.indent_spaces, self._full_dedent = self._find_indent(line)
462
462
463 def _store(self, lines, buffer=None, store='source'):
463 def _store(self, lines, buffer=None, store='source'):
464 """Store one or more lines of input.
464 """Store one or more lines of input.
465
465
466 If input lines are not newline-terminated, a newline is automatically
466 If input lines are not newline-terminated, a newline is automatically
467 appended."""
467 appended."""
468
468
469 if buffer is None:
469 if buffer is None:
470 buffer = self._buffer
470 buffer = self._buffer
471
471
472 if lines.endswith('\n'):
472 if lines.endswith('\n'):
473 buffer.append(lines)
473 buffer.append(lines)
474 else:
474 else:
475 buffer.append(lines+'\n')
475 buffer.append(lines+'\n')
476 setattr(self, store, self._set_source(buffer))
476 setattr(self, store, self._set_source(buffer))
477
477
478 def _set_source(self, buffer):
478 def _set_source(self, buffer):
479 return u''.join(buffer)
479 return u''.join(buffer)
480
480
481
481
482 #-----------------------------------------------------------------------------
482 #-----------------------------------------------------------------------------
483 # Functions and classes for IPython-specific syntactic support
483 # Functions and classes for IPython-specific syntactic support
484 #-----------------------------------------------------------------------------
484 #-----------------------------------------------------------------------------
485
485
486 # The escaped translators ALL receive a line where their own escape has been
486 # The escaped translators ALL receive a line where their own escape has been
487 # stripped. Only '?' is valid at the end of the line, all others can only be
487 # stripped. Only '?' is valid at the end of the line, all others can only be
488 # placed at the start.
488 # placed at the start.
489
489
490 # Transformations of the special syntaxes that don't rely on an explicit escape
490 # Transformations of the special syntaxes that don't rely on an explicit escape
491 # character but instead on patterns on the input line
491 # character but instead on patterns on the input line
492
492
493 # The core transformations are implemented as standalone functions that can be
493 # The core transformations are implemented as standalone functions that can be
494 # tested and validated in isolation. Each of these uses a regexp, we
494 # tested and validated in isolation. Each of these uses a regexp, we
495 # pre-compile these and keep them close to each function definition for clarity
495 # pre-compile these and keep them close to each function definition for clarity
496
496
497 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
497 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
498 r'\s*=\s*!\s*(?P<cmd>.*)')
498 r'\s*=\s*!\s*(?P<cmd>.*)')
499
499
500 def transform_assign_system(line):
500 def transform_assign_system(line):
501 """Handle the `files = !ls` syntax."""
501 """Handle the `files = !ls` syntax."""
502 m = _assign_system_re.match(line)
502 m = _assign_system_re.match(line)
503 if m is not None:
503 if m is not None:
504 cmd = m.group('cmd')
504 cmd = m.group('cmd')
505 lhs = m.group('lhs')
505 lhs = m.group('lhs')
506 expr = make_quoted_expr(cmd)
506 expr = make_quoted_expr(cmd)
507 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
507 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
508 return new_line
508 return new_line
509 return line
509 return line
510
510
511
511
512 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
512 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
513 r'\s*=\s*%\s*(?P<cmd>.*)')
513 r'\s*=\s*%\s*(?P<cmd>.*)')
514
514
515 def transform_assign_magic(line):
515 def transform_assign_magic(line):
516 """Handle the `a = %who` syntax."""
516 """Handle the `a = %who` syntax."""
517 m = _assign_magic_re.match(line)
517 m = _assign_magic_re.match(line)
518 if m is not None:
518 if m is not None:
519 cmd = m.group('cmd')
519 cmd = m.group('cmd')
520 lhs = m.group('lhs')
520 lhs = m.group('lhs')
521 expr = make_quoted_expr(cmd)
521 expr = make_quoted_expr(cmd)
522 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
522 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
523 return new_line
523 return new_line
524 return line
524 return line
525
525
526
526
527 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
527 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
528
528
529 def transform_classic_prompt(line):
529 def transform_classic_prompt(line):
530 """Handle inputs that start with '>>> ' syntax."""
530 """Handle inputs that start with '>>> ' syntax."""
531
531
532 if not line or line.isspace():
532 if not line or line.isspace():
533 return line
533 return line
534 m = _classic_prompt_re.match(line)
534 m = _classic_prompt_re.match(line)
535 if m:
535 if m:
536 return line[len(m.group(0)):]
536 return line[len(m.group(0)):]
537 else:
537 else:
538 return line
538 return line
539
539
540
540
541 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
541 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
542
542
543 def transform_ipy_prompt(line):
543 def transform_ipy_prompt(line):
544 """Handle inputs that start classic IPython prompt syntax."""
544 """Handle inputs that start classic IPython prompt syntax."""
545
545
546 if not line or line.isspace():
546 if not line or line.isspace():
547 return line
547 return line
548 #print 'LINE: %r' % line # dbg
548 #print 'LINE: %r' % line # dbg
549 m = _ipy_prompt_re.match(line)
549 m = _ipy_prompt_re.match(line)
550 if m:
550 if m:
551 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
551 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
552 return line[len(m.group(0)):]
552 return line[len(m.group(0)):]
553 else:
553 else:
554 return line
554 return line
555
555
556
556
557 def _make_help_call(target, esc, lspace, next_input=None):
557 def _make_help_call(target, esc, lspace, next_input=None):
558 """Prepares a pinfo(2)/psearch call from a target name and the escape
558 """Prepares a pinfo(2)/psearch call from a target name and the escape
559 (i.e. ? or ??)"""
559 (i.e. ? or ??)"""
560 method = 'pinfo2' if esc == '??' \
560 method = 'pinfo2' if esc == '??' \
561 else 'psearch' if '*' in target \
561 else 'psearch' if '*' in target \
562 else 'pinfo'
562 else 'pinfo'
563 arg = make_quoted_expr(" ".join([method, target]))
563 arg = make_quoted_expr(" ".join([method, target]))
564
564
565 if next_input:
565 if next_input:
566 tpl = '%sget_ipython().magic(%s, next_input=%s)'
566 tpl = '%sget_ipython().magic(%s, next_input=%s)'
567 return tpl % (lspace, arg, make_quoted_expr(next_input))
567 return tpl % (lspace, arg, make_quoted_expr(next_input))
568 else:
568 else:
569 return '%sget_ipython().magic(%s)' % (lspace, arg)
569 return '%sget_ipython().magic(%s)' % (lspace, arg)
570
570
571 _initial_space_re = re.compile(r'\s*')
571 _initial_space_re = re.compile(r'\s*')
572 _help_end_re = re.compile(r"""(%?
572 _help_end_re = re.compile(r"""(%?
573 [a-zA-Z_*][\w*]* # Variable name
573 [a-zA-Z_*][\w*]* # Variable name
574 (\.[a-zA-Z_*][\w*]*)* # .etc.etc
574 (\.[a-zA-Z_*][\w*]*)* # .etc.etc
575 )
575 )
576 (\?\??)$ # ? or ??""",
576 (\?\??)$ # ? or ??""",
577 re.VERBOSE)
577 re.VERBOSE)
578 def transform_help_end(line):
578 def transform_help_end(line):
579 """Translate lines with ?/?? at the end"""
579 """Translate lines with ?/?? at the end"""
580 m = _help_end_re.search(line)
580 m = _help_end_re.search(line)
581 if m is None or has_comment(line):
581 if m is None or has_comment(line):
582 return line
582 return line
583 target = m.group(1)
583 target = m.group(1)
584 esc = m.group(3)
584 esc = m.group(3)
585 lspace = _initial_space_re.match(line).group(0)
585 lspace = _initial_space_re.match(line).group(0)
586
586
587 # If we're mid-command, put it back on the next prompt for the user.
587 # If we're mid-command, put it back on the next prompt for the user.
588 next_input = line.rstrip('?') if line.strip() != m.group(0) else None
588 next_input = line.rstrip('?') if line.strip() != m.group(0) else None
589
589
590 return _make_help_call(target, esc, lspace, next_input)
590 return _make_help_call(target, esc, lspace, next_input)
591
591
592
592
593 class EscapedTransformer(object):
593 class EscapedTransformer(object):
594 """Class to transform lines that are explicitly escaped out."""
594 """Class to transform lines that are explicitly escaped out."""
595
595
596 def __init__(self):
596 def __init__(self):
597 tr = { ESC_SHELL : self._tr_system,
597 tr = { ESC_SHELL : self._tr_system,
598 ESC_SH_CAP : self._tr_system2,
598 ESC_SH_CAP : self._tr_system2,
599 ESC_HELP : self._tr_help,
599 ESC_HELP : self._tr_help,
600 ESC_HELP2 : self._tr_help,
600 ESC_HELP2 : self._tr_help,
601 ESC_MAGIC : self._tr_magic,
601 ESC_MAGIC : self._tr_magic,
602 ESC_QUOTE : self._tr_quote,
602 ESC_QUOTE : self._tr_quote,
603 ESC_QUOTE2 : self._tr_quote2,
603 ESC_QUOTE2 : self._tr_quote2,
604 ESC_PAREN : self._tr_paren }
604 ESC_PAREN : self._tr_paren }
605 self.tr = tr
605 self.tr = tr
606
606
607 # Support for syntax transformations that use explicit escapes typed by the
607 # Support for syntax transformations that use explicit escapes typed by the
608 # user at the beginning of a line
608 # user at the beginning of a line
609 @staticmethod
609 @staticmethod
610 def _tr_system(line_info):
610 def _tr_system(line_info):
611 "Translate lines escaped with: !"
611 "Translate lines escaped with: !"
612 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
612 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
613 return '%sget_ipython().system(%s)' % (line_info.pre,
613 return '%sget_ipython().system(%s)' % (line_info.pre,
614 make_quoted_expr(cmd))
614 make_quoted_expr(cmd))
615
615
616 @staticmethod
616 @staticmethod
617 def _tr_system2(line_info):
617 def _tr_system2(line_info):
618 "Translate lines escaped with: !!"
618 "Translate lines escaped with: !!"
619 cmd = line_info.line.lstrip()[2:]
619 cmd = line_info.line.lstrip()[2:]
620 return '%sget_ipython().getoutput(%s)' % (line_info.pre,
620 return '%sget_ipython().getoutput(%s)' % (line_info.pre,
621 make_quoted_expr(cmd))
621 make_quoted_expr(cmd))
622
622
623 @staticmethod
623 @staticmethod
624 def _tr_help(line_info):
624 def _tr_help(line_info):
625 "Translate lines escaped with: ?/??"
625 "Translate lines escaped with: ?/??"
626 # A naked help line should just fire the intro help screen
626 # A naked help line should just fire the intro help screen
627 if not line_info.line[1:]:
627 if not line_info.line[1:]:
628 return 'get_ipython().show_usage()'
628 return 'get_ipython().show_usage()'
629
629
630 return _make_help_call(line_info.ifun, line_info.esc, line_info.pre)
630 return _make_help_call(line_info.ifun, line_info.esc, line_info.pre)
631
631
632 @staticmethod
632 @staticmethod
633 def _tr_magic(line_info):
633 def _tr_magic(line_info):
634 "Translate lines escaped with: %"
634 "Translate lines escaped with: %"
635 tpl = '%sget_ipython().magic(%s)'
635 tpl = '%sget_ipython().magic(%s)'
636 cmd = make_quoted_expr(' '.join([line_info.ifun,
636 cmd = make_quoted_expr(' '.join([line_info.ifun,
637 line_info.the_rest]).strip())
637 line_info.the_rest]).strip())
638 return tpl % (line_info.pre, cmd)
638 return tpl % (line_info.pre, cmd)
639
639
640 @staticmethod
640 @staticmethod
641 def _tr_quote(line_info):
641 def _tr_quote(line_info):
642 "Translate lines escaped with: ,"
642 "Translate lines escaped with: ,"
643 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
643 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
644 '", "'.join(line_info.the_rest.split()) )
644 '", "'.join(line_info.the_rest.split()) )
645
645
646 @staticmethod
646 @staticmethod
647 def _tr_quote2(line_info):
647 def _tr_quote2(line_info):
648 "Translate lines escaped with: ;"
648 "Translate lines escaped with: ;"
649 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
649 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
650 line_info.the_rest)
650 line_info.the_rest)
651
651
652 @staticmethod
652 @staticmethod
653 def _tr_paren(line_info):
653 def _tr_paren(line_info):
654 "Translate lines escaped with: /"
654 "Translate lines escaped with: /"
655 return '%s%s(%s)' % (line_info.pre, line_info.ifun,
655 return '%s%s(%s)' % (line_info.pre, line_info.ifun,
656 ", ".join(line_info.the_rest.split()))
656 ", ".join(line_info.the_rest.split()))
657
657
658 def __call__(self, line):
658 def __call__(self, line):
659 """Class to transform lines that are explicitly escaped out.
659 """Class to transform lines that are explicitly escaped out.
660
660
661 This calls the above _tr_* static methods for the actual line
661 This calls the above _tr_* static methods for the actual line
662 translations."""
662 translations."""
663
663
664 # Empty lines just get returned unmodified
664 # Empty lines just get returned unmodified
665 if not line or line.isspace():
665 if not line or line.isspace():
666 return line
666 return line
667
667
668 # Get line endpoints, where the escapes can be
668 # Get line endpoints, where the escapes can be
669 line_info = LineInfo(line)
669 line_info = LineInfo(line)
670
670
671 if not line_info.esc in self.tr:
671 if not line_info.esc in self.tr:
672 # If we don't recognize the escape, don't modify the line
672 # If we don't recognize the escape, don't modify the line
673 return line
673 return line
674
674
675 return self.tr[line_info.esc](line_info)
675 return self.tr[line_info.esc](line_info)
676
676
677
677
678 # A function-looking object to be used by the rest of the code. The purpose of
678 # A function-looking object to be used by the rest of the code. The purpose of
679 # the class in this case is to organize related functionality, more than to
679 # the class in this case is to organize related functionality, more than to
680 # manage state.
680 # manage state.
681 transform_escaped = EscapedTransformer()
681 transform_escaped = EscapedTransformer()
682
682
683
683
684 class IPythonInputSplitter(InputSplitter):
684 class IPythonInputSplitter(InputSplitter):
685 """An input splitter that recognizes all of IPython's special syntax."""
685 """An input splitter that recognizes all of IPython's special syntax."""
686
686
687 # String with raw, untransformed input.
687 # String with raw, untransformed input.
688 source_raw = ''
688 source_raw = ''
689
689
690 # Private attributes
690 # Private attributes
691
691
692 # List with lines of raw input accumulated so far.
692 # List with lines of raw input accumulated so far.
693 _buffer_raw = None
693 _buffer_raw = None
694
694
695 def __init__(self, input_mode=None):
695 def __init__(self, input_mode=None):
696 InputSplitter.__init__(self, input_mode)
696 InputSplitter.__init__(self, input_mode)
697 self._buffer_raw = []
697 self._buffer_raw = []
698
698
699 def reset(self):
699 def reset(self):
700 """Reset the input buffer and associated state."""
700 """Reset the input buffer and associated state."""
701 InputSplitter.reset(self)
701 InputSplitter.reset(self)
702 self._buffer_raw[:] = []
702 self._buffer_raw[:] = []
703 self.source_raw = ''
703 self.source_raw = ''
704
704
705 def source_raw_reset(self):
705 def source_raw_reset(self):
706 """Return input and raw source and perform a full reset.
706 """Return input and raw source and perform a full reset.
707 """
707 """
708 out = self.source
708 out = self.source
709 out_r = self.source_raw
709 out_r = self.source_raw
710 self.reset()
710 self.reset()
711 return out, out_r
711 return out, out_r
712
712
713 def push(self, lines):
713 def push(self, lines):
714 """Push one or more lines of IPython input.
714 """Push one or more lines of IPython input.
715 """
715 """
716 if not lines:
716 if not lines:
717 return super(IPythonInputSplitter, self).push(lines)
717 return super(IPythonInputSplitter, self).push(lines)
718
718
719 # We must ensure all input is pure unicode
719 # We must ensure all input is pure unicode
720 lines = cast_unicode(lines, self.encoding)
720 lines = cast_unicode(lines, self.encoding)
721
721
722 lines_list = lines.splitlines()
722 lines_list = lines.splitlines()
723
723
724 transforms = [transform_ipy_prompt, transform_classic_prompt,
724 transforms = [transform_ipy_prompt, transform_classic_prompt,
725 transform_help_end, transform_escaped,
725 transform_help_end, transform_escaped,
726 transform_assign_system, transform_assign_magic]
726 transform_assign_system, transform_assign_magic]
727
727
728 # Transform logic
728 # Transform logic
729 #
729 #
730 # We only apply the line transformers to the input if we have either no
730 # We only apply the line transformers to the input if we have either no
731 # input yet, or complete input, or if the last line of the buffer ends
731 # input yet, or complete input, or if the last line of the buffer ends
732 # with ':' (opening an indented block). This prevents the accidental
732 # with ':' (opening an indented block). This prevents the accidental
733 # transformation of escapes inside multiline expressions like
733 # transformation of escapes inside multiline expressions like
734 # triple-quoted strings or parenthesized expressions.
734 # triple-quoted strings or parenthesized expressions.
735 #
735 #
736 # The last heuristic, while ugly, ensures that the first line of an
736 # The last heuristic, while ugly, ensures that the first line of an
737 # indented block is correctly transformed.
737 # indented block is correctly transformed.
738 #
738 #
739 # FIXME: try to find a cleaner approach for this last bit.
739 # FIXME: try to find a cleaner approach for this last bit.
740
740
741 # If we were in 'block' mode, since we're going to pump the parent
741 # If we were in 'block' mode, since we're going to pump the parent
742 # class by hand line by line, we need to temporarily switch out to
742 # class by hand line by line, we need to temporarily switch out to
743 # 'line' mode, do a single manual reset and then feed the lines one
743 # 'line' mode, do a single manual reset and then feed the lines one
744 # by one. Note that this only matters if the input has more than one
744 # by one. Note that this only matters if the input has more than one
745 # line.
745 # line.
746 changed_input_mode = False
746 changed_input_mode = False
747
747
748 if self.input_mode == 'cell':
748 if self.input_mode == 'cell':
749 self.reset()
749 self.reset()
750 changed_input_mode = True
750 changed_input_mode = True
751 saved_input_mode = 'cell'
751 saved_input_mode = 'cell'
752 self.input_mode = 'line'
752 self.input_mode = 'line'
753
753
754 # Store raw source before applying any transformations to it. Note
754 # Store raw source before applying any transformations to it. Note
755 # that this must be done *after* the reset() call that would otherwise
755 # that this must be done *after* the reset() call that would otherwise
756 # flush the buffer.
756 # flush the buffer.
757 self._store(lines, self._buffer_raw, 'source_raw')
757 self._store(lines, self._buffer_raw, 'source_raw')
758
758
759 try:
759 try:
760 push = super(IPythonInputSplitter, self).push
760 push = super(IPythonInputSplitter, self).push
761 buf = self._buffer
761 for line in lines_list:
762 for line in lines_list:
762 if self._is_complete or not self._buffer or \
763 if self._is_complete or not buf or \
763 (self._buffer and (self._buffer[-1].rstrip().endswith(':') or\
764 (buf and (buf[-1].rstrip().endswith(':') or
764 self._buffer[-1].rstrip().endswith(','))):
765 buf[-1].rstrip().endswith(',')) ):
765 for f in transforms:
766 for f in transforms:
766 line = f(line)
767 line = f(line)
767
768
768 out = push(line)
769 out = push(line)
769 finally:
770 finally:
770 if changed_input_mode:
771 if changed_input_mode:
771 self.input_mode = saved_input_mode
772 self.input_mode = saved_input_mode
772 return out
773 return out
General Comments 0
You need to be logged in to leave comments. Login now