##// END OF EJS Templates
Fix behaviour of dedent triggering. Closes gh-142.
David Warde-Farley -
Show More
@@ -1,859 +1,865 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 from __future__ import print_function
63 from __future__ import print_function
64
64
65 #-----------------------------------------------------------------------------
65 #-----------------------------------------------------------------------------
66 # Imports
66 # Imports
67 #-----------------------------------------------------------------------------
67 #-----------------------------------------------------------------------------
68 # stdlib
68 # stdlib
69 import ast
69 import ast
70 import codeop
70 import codeop
71 import re
71 import re
72 import sys
72 import sys
73
73
74 # IPython modules
74 # IPython modules
75 from IPython.utils.text import make_quoted_expr
75 from IPython.utils.text import make_quoted_expr
76
76
77 #-----------------------------------------------------------------------------
77 #-----------------------------------------------------------------------------
78 # Globals
78 # Globals
79 #-----------------------------------------------------------------------------
79 #-----------------------------------------------------------------------------
80
80
81 # The escape sequences that define the syntax transformations IPython will
81 # The escape sequences that define the syntax transformations IPython will
82 # apply to user input. These can NOT be just changed here: many regular
82 # apply to user input. These can NOT be just changed here: many regular
83 # expressions and other parts of the code may use their hardcoded values, and
83 # expressions and other parts of the code may use their hardcoded values, and
84 # for all intents and purposes they constitute the 'IPython syntax', so they
84 # for all intents and purposes they constitute the 'IPython syntax', so they
85 # should be considered fixed.
85 # should be considered fixed.
86
86
87 ESC_SHELL = '!' # Send line to underlying system shell
87 ESC_SHELL = '!' # Send line to underlying system shell
88 ESC_SH_CAP = '!!' # Send line to system shell and capture output
88 ESC_SH_CAP = '!!' # Send line to system shell and capture output
89 ESC_HELP = '?' # Find information about object
89 ESC_HELP = '?' # Find information about object
90 ESC_HELP2 = '??' # Find extra-detailed information about object
90 ESC_HELP2 = '??' # Find extra-detailed information about object
91 ESC_MAGIC = '%' # Call magic function
91 ESC_MAGIC = '%' # Call magic function
92 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
92 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
93 ESC_QUOTE2 = ';' # Quote all args as a single string, call
93 ESC_QUOTE2 = ';' # Quote all args as a single string, call
94 ESC_PAREN = '/' # Call first argument with rest of line as arguments
94 ESC_PAREN = '/' # Call first argument with rest of line as arguments
95
95
96 #-----------------------------------------------------------------------------
96 #-----------------------------------------------------------------------------
97 # Utilities
97 # Utilities
98 #-----------------------------------------------------------------------------
98 #-----------------------------------------------------------------------------
99
99
100 # FIXME: These are general-purpose utilities that later can be moved to the
100 # FIXME: These are general-purpose utilities that later can be moved to the
101 # general ward. Kept here for now because we're being very strict about test
101 # general ward. Kept here for now because we're being very strict about test
102 # coverage with this code, and this lets us ensure that we keep 100% coverage
102 # coverage with this code, and this lets us ensure that we keep 100% coverage
103 # while developing.
103 # while developing.
104
104
105 # compiled regexps for autoindent management
105 # compiled regexps for autoindent management
106 dedent_re = re.compile(r'^\s+raise|^\s+return|^\s+pass')
106 dedent_re = re.compile('|'.join([
107 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
108 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
109 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
110 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
111 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
112 ]))
107 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
113 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
108
114
109 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
115 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
110 # before pure comments
116 # before pure comments
111 comment_line_re = re.compile('^\s*\#')
117 comment_line_re = re.compile('^\s*\#')
112
118
113
119
114 def num_ini_spaces(s):
120 def num_ini_spaces(s):
115 """Return the number of initial spaces in a string.
121 """Return the number of initial spaces in a string.
116
122
117 Note that tabs are counted as a single space. For now, we do *not* support
123 Note that tabs are counted as a single space. For now, we do *not* support
118 mixing of tabs and spaces in the user's input.
124 mixing of tabs and spaces in the user's input.
119
125
120 Parameters
126 Parameters
121 ----------
127 ----------
122 s : string
128 s : string
123
129
124 Returns
130 Returns
125 -------
131 -------
126 n : int
132 n : int
127 """
133 """
128
134
129 ini_spaces = ini_spaces_re.match(s)
135 ini_spaces = ini_spaces_re.match(s)
130 if ini_spaces:
136 if ini_spaces:
131 return ini_spaces.end()
137 return ini_spaces.end()
132 else:
138 else:
133 return 0
139 return 0
134
140
135
141
136 def remove_comments(src):
142 def remove_comments(src):
137 """Remove all comments from input source.
143 """Remove all comments from input source.
138
144
139 Note: comments are NOT recognized inside of strings!
145 Note: comments are NOT recognized inside of strings!
140
146
141 Parameters
147 Parameters
142 ----------
148 ----------
143 src : string
149 src : string
144 A single or multiline input string.
150 A single or multiline input string.
145
151
146 Returns
152 Returns
147 -------
153 -------
148 String with all Python comments removed.
154 String with all Python comments removed.
149 """
155 """
150
156
151 return re.sub('#.*', '', src)
157 return re.sub('#.*', '', src)
152
158
153
159
154 def get_input_encoding():
160 def get_input_encoding():
155 """Return the default standard input encoding.
161 """Return the default standard input encoding.
156
162
157 If sys.stdin has no encoding, 'ascii' is returned."""
163 If sys.stdin has no encoding, 'ascii' is returned."""
158 # There are strange environments for which sys.stdin.encoding is None. We
164 # There are strange environments for which sys.stdin.encoding is None. We
159 # ensure that a valid encoding is returned.
165 # ensure that a valid encoding is returned.
160 encoding = getattr(sys.stdin, 'encoding', None)
166 encoding = getattr(sys.stdin, 'encoding', None)
161 if encoding is None:
167 if encoding is None:
162 encoding = 'ascii'
168 encoding = 'ascii'
163 return encoding
169 return encoding
164
170
165 #-----------------------------------------------------------------------------
171 #-----------------------------------------------------------------------------
166 # Classes and functions for normal Python syntax handling
172 # Classes and functions for normal Python syntax handling
167 #-----------------------------------------------------------------------------
173 #-----------------------------------------------------------------------------
168
174
169 class InputSplitter(object):
175 class InputSplitter(object):
170 """An object that can split Python source input in executable blocks.
176 """An object that can split Python source input in executable blocks.
171
177
172 This object is designed to be used in one of two basic modes:
178 This object is designed to be used in one of two basic modes:
173
179
174 1. By feeding it python source line-by-line, using :meth:`push`. In this
180 1. By feeding it python source line-by-line, using :meth:`push`. In this
175 mode, it will return on each push whether the currently pushed code
181 mode, it will return on each push whether the currently pushed code
176 could be executed already. In addition, it provides a method called
182 could be executed already. In addition, it provides a method called
177 :meth:`push_accepts_more` that can be used to query whether more input
183 :meth:`push_accepts_more` that can be used to query whether more input
178 can be pushed into a single interactive block.
184 can be pushed into a single interactive block.
179
185
180 2. By calling :meth:`split_blocks` with a single, multiline Python string,
186 2. By calling :meth:`split_blocks` with a single, multiline Python string,
181 that is then split into blocks each of which can be executed
187 that is then split into blocks each of which can be executed
182 interactively as a single statement.
188 interactively as a single statement.
183
189
184 This is a simple example of how an interactive terminal-based client can use
190 This is a simple example of how an interactive terminal-based client can use
185 this tool::
191 this tool::
186
192
187 isp = InputSplitter()
193 isp = InputSplitter()
188 while isp.push_accepts_more():
194 while isp.push_accepts_more():
189 indent = ' '*isp.indent_spaces
195 indent = ' '*isp.indent_spaces
190 prompt = '>>> ' + indent
196 prompt = '>>> ' + indent
191 line = indent + raw_input(prompt)
197 line = indent + raw_input(prompt)
192 isp.push(line)
198 isp.push(line)
193 print 'Input source was:\n', isp.source_reset(),
199 print 'Input source was:\n', isp.source_reset(),
194 """
200 """
195 # Number of spaces of indentation computed from input that has been pushed
201 # Number of spaces of indentation computed from input that has been pushed
196 # so far. This is the attributes callers should query to get the current
202 # so far. This is the attributes callers should query to get the current
197 # indentation level, in order to provide auto-indent facilities.
203 # indentation level, in order to provide auto-indent facilities.
198 indent_spaces = 0
204 indent_spaces = 0
199 # String, indicating the default input encoding. It is computed by default
205 # String, indicating the default input encoding. It is computed by default
200 # at initialization time via get_input_encoding(), but it can be reset by a
206 # at initialization time via get_input_encoding(), but it can be reset by a
201 # client with specific knowledge of the encoding.
207 # client with specific knowledge of the encoding.
202 encoding = ''
208 encoding = ''
203 # String where the current full source input is stored, properly encoded.
209 # String where the current full source input is stored, properly encoded.
204 # Reading this attribute is the normal way of querying the currently pushed
210 # Reading this attribute is the normal way of querying the currently pushed
205 # source code, that has been properly encoded.
211 # source code, that has been properly encoded.
206 source = ''
212 source = ''
207 # Code object corresponding to the current source. It is automatically
213 # Code object corresponding to the current source. It is automatically
208 # synced to the source, so it can be queried at any time to obtain the code
214 # synced to the source, so it can be queried at any time to obtain the code
209 # object; it will be None if the source doesn't compile to valid Python.
215 # object; it will be None if the source doesn't compile to valid Python.
210 code = None
216 code = None
211 # Input mode
217 # Input mode
212 input_mode = 'line'
218 input_mode = 'line'
213
219
214 # Private attributes
220 # Private attributes
215
221
216 # List with lines of input accumulated so far
222 # List with lines of input accumulated so far
217 _buffer = None
223 _buffer = None
218 # Command compiler
224 # Command compiler
219 _compile = None
225 _compile = None
220 # Mark when input has changed indentation all the way back to flush-left
226 # Mark when input has changed indentation all the way back to flush-left
221 _full_dedent = False
227 _full_dedent = False
222 # Boolean indicating whether the current block is complete
228 # Boolean indicating whether the current block is complete
223 _is_complete = None
229 _is_complete = None
224
230
225 def __init__(self, input_mode=None):
231 def __init__(self, input_mode=None):
226 """Create a new InputSplitter instance.
232 """Create a new InputSplitter instance.
227
233
228 Parameters
234 Parameters
229 ----------
235 ----------
230 input_mode : str
236 input_mode : str
231
237
232 One of ['line', 'cell']; default is 'line'.
238 One of ['line', 'cell']; default is 'line'.
233
239
234 The input_mode parameter controls how new inputs are used when fed via
240 The input_mode parameter controls how new inputs are used when fed via
235 the :meth:`push` method:
241 the :meth:`push` method:
236
242
237 - 'line': meant for line-oriented clients, inputs are appended one at a
243 - 'line': meant for line-oriented clients, inputs are appended one at a
238 time to the internal buffer and the whole buffer is compiled.
244 time to the internal buffer and the whole buffer is compiled.
239
245
240 - 'cell': meant for clients that can edit multi-line 'cells' of text at
246 - 'cell': meant for clients that can edit multi-line 'cells' of text at
241 a time. A cell can contain one or more blocks that can be compile in
247 a time. A cell can contain one or more blocks that can be compile in
242 'single' mode by Python. In this mode, each new input new input
248 'single' mode by Python. In this mode, each new input new input
243 completely replaces all prior inputs. Cell mode is thus equivalent
249 completely replaces all prior inputs. Cell mode is thus equivalent
244 to prepending a full reset() to every push() call.
250 to prepending a full reset() to every push() call.
245 """
251 """
246 self._buffer = []
252 self._buffer = []
247 self._compile = codeop.CommandCompiler()
253 self._compile = codeop.CommandCompiler()
248 self.encoding = get_input_encoding()
254 self.encoding = get_input_encoding()
249 self.input_mode = InputSplitter.input_mode if input_mode is None \
255 self.input_mode = InputSplitter.input_mode if input_mode is None \
250 else input_mode
256 else input_mode
251
257
252 def reset(self):
258 def reset(self):
253 """Reset the input buffer and associated state."""
259 """Reset the input buffer and associated state."""
254 self.indent_spaces = 0
260 self.indent_spaces = 0
255 self._buffer[:] = []
261 self._buffer[:] = []
256 self.source = ''
262 self.source = ''
257 self.code = None
263 self.code = None
258 self._is_complete = False
264 self._is_complete = False
259 self._full_dedent = False
265 self._full_dedent = False
260
266
261 def source_reset(self):
267 def source_reset(self):
262 """Return the input source and perform a full reset.
268 """Return the input source and perform a full reset.
263 """
269 """
264 out = self.source
270 out = self.source
265 self.reset()
271 self.reset()
266 return out
272 return out
267
273
268 def push(self, lines):
274 def push(self, lines):
269 """Push one or more lines of input.
275 """Push one or more lines of input.
270
276
271 This stores the given lines and returns a status code indicating
277 This stores the given lines and returns a status code indicating
272 whether the code forms a complete Python block or not.
278 whether the code forms a complete Python block or not.
273
279
274 Any exceptions generated in compilation are swallowed, but if an
280 Any exceptions generated in compilation are swallowed, but if an
275 exception was produced, the method returns True.
281 exception was produced, the method returns True.
276
282
277 Parameters
283 Parameters
278 ----------
284 ----------
279 lines : string
285 lines : string
280 One or more lines of Python input.
286 One or more lines of Python input.
281
287
282 Returns
288 Returns
283 -------
289 -------
284 is_complete : boolean
290 is_complete : boolean
285 True if the current input source (the result of the current input
291 True if the current input source (the result of the current input
286 plus prior inputs) forms a complete Python execution block. Note that
292 plus prior inputs) forms a complete Python execution block. Note that
287 this value is also stored as a private attribute (_is_complete), so it
293 this value is also stored as a private attribute (_is_complete), so it
288 can be queried at any time.
294 can be queried at any time.
289 """
295 """
290 if self.input_mode == 'cell':
296 if self.input_mode == 'cell':
291 self.reset()
297 self.reset()
292
298
293 self._store(lines)
299 self._store(lines)
294 source = self.source
300 source = self.source
295
301
296 # Before calling _compile(), reset the code object to None so that if an
302 # Before calling _compile(), reset the code object to None so that if an
297 # exception is raised in compilation, we don't mislead by having
303 # exception is raised in compilation, we don't mislead by having
298 # inconsistent code/source attributes.
304 # inconsistent code/source attributes.
299 self.code, self._is_complete = None, None
305 self.code, self._is_complete = None, None
300
306
301 # Honor termination lines properly
307 # Honor termination lines properly
302 if source.rstrip().endswith('\\'):
308 if source.rstrip().endswith('\\'):
303 return False
309 return False
304
310
305 self._update_indent(lines)
311 self._update_indent(lines)
306 try:
312 try:
307 self.code = self._compile(source)
313 self.code = self._compile(source)
308 # Invalid syntax can produce any of a number of different errors from
314 # Invalid syntax can produce any of a number of different errors from
309 # inside the compiler, so we have to catch them all. Syntax errors
315 # inside the compiler, so we have to catch them all. Syntax errors
310 # immediately produce a 'ready' block, so the invalid Python can be
316 # immediately produce a 'ready' block, so the invalid Python can be
311 # sent to the kernel for evaluation with possible ipython
317 # sent to the kernel for evaluation with possible ipython
312 # special-syntax conversion.
318 # special-syntax conversion.
313 except (SyntaxError, OverflowError, ValueError, TypeError,
319 except (SyntaxError, OverflowError, ValueError, TypeError,
314 MemoryError):
320 MemoryError):
315 self._is_complete = True
321 self._is_complete = True
316 else:
322 else:
317 # Compilation didn't produce any exceptions (though it may not have
323 # Compilation didn't produce any exceptions (though it may not have
318 # given a complete code object)
324 # given a complete code object)
319 self._is_complete = self.code is not None
325 self._is_complete = self.code is not None
320
326
321 return self._is_complete
327 return self._is_complete
322
328
323 def push_accepts_more(self):
329 def push_accepts_more(self):
324 """Return whether a block of interactive input can accept more input.
330 """Return whether a block of interactive input can accept more input.
325
331
326 This method is meant to be used by line-oriented frontends, who need to
332 This method is meant to be used by line-oriented frontends, who need to
327 guess whether a block is complete or not based solely on prior and
333 guess whether a block is complete or not based solely on prior and
328 current input lines. The InputSplitter considers it has a complete
334 current input lines. The InputSplitter considers it has a complete
329 interactive block and will not accept more input only when either a
335 interactive block and will not accept more input only when either a
330 SyntaxError is raised, or *all* of the following are true:
336 SyntaxError is raised, or *all* of the following are true:
331
337
332 1. The input compiles to a complete statement.
338 1. The input compiles to a complete statement.
333
339
334 2. The indentation level is flush-left (because if we are indented,
340 2. The indentation level is flush-left (because if we are indented,
335 like inside a function definition or for loop, we need to keep
341 like inside a function definition or for loop, we need to keep
336 reading new input).
342 reading new input).
337
343
338 3. There is one extra line consisting only of whitespace.
344 3. There is one extra line consisting only of whitespace.
339
345
340 Because of condition #3, this method should be used only by
346 Because of condition #3, this method should be used only by
341 *line-oriented* frontends, since it means that intermediate blank lines
347 *line-oriented* frontends, since it means that intermediate blank lines
342 are not allowed in function definitions (or any other indented block).
348 are not allowed in function definitions (or any other indented block).
343
349
344 Block-oriented frontends that have a separate keyboard event to
350 Block-oriented frontends that have a separate keyboard event to
345 indicate execution should use the :meth:`split_blocks` method instead.
351 indicate execution should use the :meth:`split_blocks` method instead.
346
352
347 If the current input produces a syntax error, this method immediately
353 If the current input produces a syntax error, this method immediately
348 returns False but does *not* raise the syntax error exception, as
354 returns False but does *not* raise the syntax error exception, as
349 typically clients will want to send invalid syntax to an execution
355 typically clients will want to send invalid syntax to an execution
350 backend which might convert the invalid syntax into valid Python via
356 backend which might convert the invalid syntax into valid Python via
351 one of the dynamic IPython mechanisms.
357 one of the dynamic IPython mechanisms.
352 """
358 """
353
359
354 # With incomplete input, unconditionally accept more
360 # With incomplete input, unconditionally accept more
355 if not self._is_complete:
361 if not self._is_complete:
356 return True
362 return True
357
363
358 # If we already have complete input and we're flush left, the answer
364 # If we already have complete input and we're flush left, the answer
359 # depends. In line mode, if there hasn't been any indentation,
365 # depends. In line mode, if there hasn't been any indentation,
360 # that's it. If we've come back from some indentation, we need
366 # that's it. If we've come back from some indentation, we need
361 # the blank final line to finish.
367 # the blank final line to finish.
362 # In cell mode, we need to check how many blocks the input so far
368 # In cell mode, we need to check how many blocks the input so far
363 # compiles into, because if there's already more than one full
369 # compiles into, because if there's already more than one full
364 # independent block of input, then the client has entered full
370 # independent block of input, then the client has entered full
365 # 'cell' mode and is feeding lines that each is complete. In this
371 # 'cell' mode and is feeding lines that each is complete. In this
366 # case we should then keep accepting. The Qt terminal-like console
372 # case we should then keep accepting. The Qt terminal-like console
367 # does precisely this, to provide the convenience of terminal-like
373 # does precisely this, to provide the convenience of terminal-like
368 # input of single expressions, but allowing the user (with a
374 # input of single expressions, but allowing the user (with a
369 # separate keystroke) to switch to 'cell' mode and type multiple
375 # separate keystroke) to switch to 'cell' mode and type multiple
370 # expressions in one shot.
376 # expressions in one shot.
371 if self.indent_spaces==0:
377 if self.indent_spaces==0:
372 if self.input_mode=='line':
378 if self.input_mode=='line':
373 if not self._full_dedent:
379 if not self._full_dedent:
374 return False
380 return False
375 else:
381 else:
376 try:
382 try:
377 code_ast = ast.parse(u''.join(self._buffer))
383 code_ast = ast.parse(u''.join(self._buffer))
378 except Exception:
384 except Exception:
379 return False
385 return False
380 else:
386 else:
381 if len(code_ast.body) == 1:
387 if len(code_ast.body) == 1:
382 return False
388 return False
383
389
384 # When input is complete, then termination is marked by an extra blank
390 # When input is complete, then termination is marked by an extra blank
385 # line at the end.
391 # line at the end.
386 last_line = self.source.splitlines()[-1]
392 last_line = self.source.splitlines()[-1]
387 return bool(last_line and not last_line.isspace())
393 return bool(last_line and not last_line.isspace())
388
394
389 #------------------------------------------------------------------------
395 #------------------------------------------------------------------------
390 # Private interface
396 # Private interface
391 #------------------------------------------------------------------------
397 #------------------------------------------------------------------------
392
398
393 def _find_indent(self, line):
399 def _find_indent(self, line):
394 """Compute the new indentation level for a single line.
400 """Compute the new indentation level for a single line.
395
401
396 Parameters
402 Parameters
397 ----------
403 ----------
398 line : str
404 line : str
399 A single new line of non-whitespace, non-comment Python input.
405 A single new line of non-whitespace, non-comment Python input.
400
406
401 Returns
407 Returns
402 -------
408 -------
403 indent_spaces : int
409 indent_spaces : int
404 New value for the indent level (it may be equal to self.indent_spaces
410 New value for the indent level (it may be equal to self.indent_spaces
405 if indentation doesn't change.
411 if indentation doesn't change.
406
412
407 full_dedent : boolean
413 full_dedent : boolean
408 Whether the new line causes a full flush-left dedent.
414 Whether the new line causes a full flush-left dedent.
409 """
415 """
410 indent_spaces = self.indent_spaces
416 indent_spaces = self.indent_spaces
411 full_dedent = self._full_dedent
417 full_dedent = self._full_dedent
412
418
413 inisp = num_ini_spaces(line)
419 inisp = num_ini_spaces(line)
414 if inisp < indent_spaces:
420 if inisp < indent_spaces:
415 indent_spaces = inisp
421 indent_spaces = inisp
416 if indent_spaces <= 0:
422 if indent_spaces <= 0:
417 #print 'Full dedent in text',self.source # dbg
423 #print 'Full dedent in text',self.source # dbg
418 full_dedent = True
424 full_dedent = True
419
425
420 if line[-1] == ':':
426 if line[-1] == ':':
421 indent_spaces += 4
427 indent_spaces += 4
422 elif dedent_re.match(line):
428 elif dedent_re.match(line):
423 indent_spaces -= 4
429 indent_spaces -= 4
424 if indent_spaces <= 0:
430 if indent_spaces <= 0:
425 full_dedent = True
431 full_dedent = True
426
432
427 # Safety
433 # Safety
428 if indent_spaces < 0:
434 if indent_spaces < 0:
429 indent_spaces = 0
435 indent_spaces = 0
430 #print 'safety' # dbg
436 #print 'safety' # dbg
431
437
432 return indent_spaces, full_dedent
438 return indent_spaces, full_dedent
433
439
434 def _update_indent(self, lines):
440 def _update_indent(self, lines):
435 for line in remove_comments(lines).splitlines():
441 for line in remove_comments(lines).splitlines():
436 if line and not line.isspace():
442 if line and not line.isspace():
437 self.indent_spaces, self._full_dedent = self._find_indent(line)
443 self.indent_spaces, self._full_dedent = self._find_indent(line)
438
444
439 def _store(self, lines, buffer=None, store='source'):
445 def _store(self, lines, buffer=None, store='source'):
440 """Store one or more lines of input.
446 """Store one or more lines of input.
441
447
442 If input lines are not newline-terminated, a newline is automatically
448 If input lines are not newline-terminated, a newline is automatically
443 appended."""
449 appended."""
444
450
445 if buffer is None:
451 if buffer is None:
446 buffer = self._buffer
452 buffer = self._buffer
447
453
448 if lines.endswith('\n'):
454 if lines.endswith('\n'):
449 buffer.append(lines)
455 buffer.append(lines)
450 else:
456 else:
451 buffer.append(lines+'\n')
457 buffer.append(lines+'\n')
452 setattr(self, store, self._set_source(buffer))
458 setattr(self, store, self._set_source(buffer))
453
459
454 def _set_source(self, buffer):
460 def _set_source(self, buffer):
455 return u''.join(buffer)
461 return u''.join(buffer)
456
462
457
463
458 #-----------------------------------------------------------------------------
464 #-----------------------------------------------------------------------------
459 # Functions and classes for IPython-specific syntactic support
465 # Functions and classes for IPython-specific syntactic support
460 #-----------------------------------------------------------------------------
466 #-----------------------------------------------------------------------------
461
467
462 # RegExp for splitting line contents into pre-char//first word-method//rest.
468 # RegExp for splitting line contents into pre-char//first word-method//rest.
463 # For clarity, each group in on one line.
469 # For clarity, each group in on one line.
464
470
465 line_split = re.compile("""
471 line_split = re.compile("""
466 ^(\s*) # any leading space
472 ^(\s*) # any leading space
467 ([,;/%]|!!?|\?\??) # escape character or characters
473 ([,;/%]|!!?|\?\??) # escape character or characters
468 \s*(%?[\w\.\*]*) # function/method, possibly with leading %
474 \s*(%?[\w\.\*]*) # function/method, possibly with leading %
469 # to correctly treat things like '?%magic'
475 # to correctly treat things like '?%magic'
470 (\s+.*$|$) # rest of line
476 (\s+.*$|$) # rest of line
471 """, re.VERBOSE)
477 """, re.VERBOSE)
472
478
473
479
474 def split_user_input(line):
480 def split_user_input(line):
475 """Split user input into early whitespace, esc-char, function part and rest.
481 """Split user input into early whitespace, esc-char, function part and rest.
476
482
477 This is currently handles lines with '=' in them in a very inconsistent
483 This is currently handles lines with '=' in them in a very inconsistent
478 manner.
484 manner.
479
485
480 Examples
486 Examples
481 ========
487 ========
482 >>> split_user_input('x=1')
488 >>> split_user_input('x=1')
483 ('', '', 'x=1', '')
489 ('', '', 'x=1', '')
484 >>> split_user_input('?')
490 >>> split_user_input('?')
485 ('', '?', '', '')
491 ('', '?', '', '')
486 >>> split_user_input('??')
492 >>> split_user_input('??')
487 ('', '??', '', '')
493 ('', '??', '', '')
488 >>> split_user_input(' ?')
494 >>> split_user_input(' ?')
489 (' ', '?', '', '')
495 (' ', '?', '', '')
490 >>> split_user_input(' ??')
496 >>> split_user_input(' ??')
491 (' ', '??', '', '')
497 (' ', '??', '', '')
492 >>> split_user_input('??x')
498 >>> split_user_input('??x')
493 ('', '??', 'x', '')
499 ('', '??', 'x', '')
494 >>> split_user_input('?x=1')
500 >>> split_user_input('?x=1')
495 ('', '', '?x=1', '')
501 ('', '', '?x=1', '')
496 >>> split_user_input('!ls')
502 >>> split_user_input('!ls')
497 ('', '!', 'ls', '')
503 ('', '!', 'ls', '')
498 >>> split_user_input(' !ls')
504 >>> split_user_input(' !ls')
499 (' ', '!', 'ls', '')
505 (' ', '!', 'ls', '')
500 >>> split_user_input('!!ls')
506 >>> split_user_input('!!ls')
501 ('', '!!', 'ls', '')
507 ('', '!!', 'ls', '')
502 >>> split_user_input(' !!ls')
508 >>> split_user_input(' !!ls')
503 (' ', '!!', 'ls', '')
509 (' ', '!!', 'ls', '')
504 >>> split_user_input(',ls')
510 >>> split_user_input(',ls')
505 ('', ',', 'ls', '')
511 ('', ',', 'ls', '')
506 >>> split_user_input(';ls')
512 >>> split_user_input(';ls')
507 ('', ';', 'ls', '')
513 ('', ';', 'ls', '')
508 >>> split_user_input(' ;ls')
514 >>> split_user_input(' ;ls')
509 (' ', ';', 'ls', '')
515 (' ', ';', 'ls', '')
510 >>> split_user_input('f.g(x)')
516 >>> split_user_input('f.g(x)')
511 ('', '', 'f.g(x)', '')
517 ('', '', 'f.g(x)', '')
512 >>> split_user_input('f.g (x)')
518 >>> split_user_input('f.g (x)')
513 ('', '', 'f.g', '(x)')
519 ('', '', 'f.g', '(x)')
514 >>> split_user_input('?%hist')
520 >>> split_user_input('?%hist')
515 ('', '?', '%hist', '')
521 ('', '?', '%hist', '')
516 >>> split_user_input('?x*')
522 >>> split_user_input('?x*')
517 ('', '?', 'x*', '')
523 ('', '?', 'x*', '')
518 """
524 """
519 match = line_split.match(line)
525 match = line_split.match(line)
520 if match:
526 if match:
521 lspace, esc, fpart, rest = match.groups()
527 lspace, esc, fpart, rest = match.groups()
522 else:
528 else:
523 # print "match failed for line '%s'" % line
529 # print "match failed for line '%s'" % line
524 try:
530 try:
525 fpart, rest = line.split(None, 1)
531 fpart, rest = line.split(None, 1)
526 except ValueError:
532 except ValueError:
527 # print "split failed for line '%s'" % line
533 # print "split failed for line '%s'" % line
528 fpart, rest = line,''
534 fpart, rest = line,''
529 lspace = re.match('^(\s*)(.*)', line).groups()[0]
535 lspace = re.match('^(\s*)(.*)', line).groups()[0]
530 esc = ''
536 esc = ''
531
537
532 # fpart has to be a valid python identifier, so it better be only pure
538 # fpart has to be a valid python identifier, so it better be only pure
533 # ascii, no unicode:
539 # ascii, no unicode:
534 try:
540 try:
535 fpart = fpart.encode('ascii')
541 fpart = fpart.encode('ascii')
536 except UnicodeEncodeError:
542 except UnicodeEncodeError:
537 lspace = unicode(lspace)
543 lspace = unicode(lspace)
538 rest = fpart + u' ' + rest
544 rest = fpart + u' ' + rest
539 fpart = u''
545 fpart = u''
540
546
541 #print 'line:<%s>' % line # dbg
547 #print 'line:<%s>' % line # dbg
542 #print 'esc <%s> fpart <%s> rest <%s>' % (esc,fpart.strip(),rest) # dbg
548 #print 'esc <%s> fpart <%s> rest <%s>' % (esc,fpart.strip(),rest) # dbg
543 return lspace, esc, fpart.strip(), rest.lstrip()
549 return lspace, esc, fpart.strip(), rest.lstrip()
544
550
545
551
546 # The escaped translators ALL receive a line where their own escape has been
552 # The escaped translators ALL receive a line where their own escape has been
547 # stripped. Only '?' is valid at the end of the line, all others can only be
553 # stripped. Only '?' is valid at the end of the line, all others can only be
548 # placed at the start.
554 # placed at the start.
549
555
550 class LineInfo(object):
556 class LineInfo(object):
551 """A single line of input and associated info.
557 """A single line of input and associated info.
552
558
553 This is a utility class that mostly wraps the output of
559 This is a utility class that mostly wraps the output of
554 :func:`split_user_input` into a convenient object to be passed around
560 :func:`split_user_input` into a convenient object to be passed around
555 during input transformations.
561 during input transformations.
556
562
557 Includes the following as properties:
563 Includes the following as properties:
558
564
559 line
565 line
560 The original, raw line
566 The original, raw line
561
567
562 lspace
568 lspace
563 Any early whitespace before actual text starts.
569 Any early whitespace before actual text starts.
564
570
565 esc
571 esc
566 The initial esc character (or characters, for double-char escapes like
572 The initial esc character (or characters, for double-char escapes like
567 '??' or '!!').
573 '??' or '!!').
568
574
569 fpart
575 fpart
570 The 'function part', which is basically the maximal initial sequence
576 The 'function part', which is basically the maximal initial sequence
571 of valid python identifiers and the '.' character. This is what is
577 of valid python identifiers and the '.' character. This is what is
572 checked for alias and magic transformations, used for auto-calling,
578 checked for alias and magic transformations, used for auto-calling,
573 etc.
579 etc.
574
580
575 rest
581 rest
576 Everything else on the line.
582 Everything else on the line.
577 """
583 """
578 def __init__(self, line):
584 def __init__(self, line):
579 self.line = line
585 self.line = line
580 self.lspace, self.esc, self.fpart, self.rest = \
586 self.lspace, self.esc, self.fpart, self.rest = \
581 split_user_input(line)
587 split_user_input(line)
582
588
583 def __str__(self):
589 def __str__(self):
584 return "LineInfo [%s|%s|%s|%s]" % (self.lspace, self.esc,
590 return "LineInfo [%s|%s|%s|%s]" % (self.lspace, self.esc,
585 self.fpart, self.rest)
591 self.fpart, self.rest)
586
592
587
593
588 # Transformations of the special syntaxes that don't rely on an explicit escape
594 # Transformations of the special syntaxes that don't rely on an explicit escape
589 # character but instead on patterns on the input line
595 # character but instead on patterns on the input line
590
596
591 # The core transformations are implemented as standalone functions that can be
597 # The core transformations are implemented as standalone functions that can be
592 # tested and validated in isolation. Each of these uses a regexp, we
598 # tested and validated in isolation. Each of these uses a regexp, we
593 # pre-compile these and keep them close to each function definition for clarity
599 # pre-compile these and keep them close to each function definition for clarity
594
600
595 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
601 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
596 r'\s*=\s*!\s*(?P<cmd>.*)')
602 r'\s*=\s*!\s*(?P<cmd>.*)')
597
603
598 def transform_assign_system(line):
604 def transform_assign_system(line):
599 """Handle the `files = !ls` syntax."""
605 """Handle the `files = !ls` syntax."""
600 m = _assign_system_re.match(line)
606 m = _assign_system_re.match(line)
601 if m is not None:
607 if m is not None:
602 cmd = m.group('cmd')
608 cmd = m.group('cmd')
603 lhs = m.group('lhs')
609 lhs = m.group('lhs')
604 expr = make_quoted_expr(cmd)
610 expr = make_quoted_expr(cmd)
605 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
611 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
606 return new_line
612 return new_line
607 return line
613 return line
608
614
609
615
610 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
616 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
611 r'\s*=\s*%\s*(?P<cmd>.*)')
617 r'\s*=\s*%\s*(?P<cmd>.*)')
612
618
613 def transform_assign_magic(line):
619 def transform_assign_magic(line):
614 """Handle the `a = %who` syntax."""
620 """Handle the `a = %who` syntax."""
615 m = _assign_magic_re.match(line)
621 m = _assign_magic_re.match(line)
616 if m is not None:
622 if m is not None:
617 cmd = m.group('cmd')
623 cmd = m.group('cmd')
618 lhs = m.group('lhs')
624 lhs = m.group('lhs')
619 expr = make_quoted_expr(cmd)
625 expr = make_quoted_expr(cmd)
620 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
626 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
621 return new_line
627 return new_line
622 return line
628 return line
623
629
624
630
625 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
631 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
626
632
627 def transform_classic_prompt(line):
633 def transform_classic_prompt(line):
628 """Handle inputs that start with '>>> ' syntax."""
634 """Handle inputs that start with '>>> ' syntax."""
629
635
630 if not line or line.isspace():
636 if not line or line.isspace():
631 return line
637 return line
632 m = _classic_prompt_re.match(line)
638 m = _classic_prompt_re.match(line)
633 if m:
639 if m:
634 return line[len(m.group(0)):]
640 return line[len(m.group(0)):]
635 else:
641 else:
636 return line
642 return line
637
643
638
644
639 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
645 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
640
646
641 def transform_ipy_prompt(line):
647 def transform_ipy_prompt(line):
642 """Handle inputs that start classic IPython prompt syntax."""
648 """Handle inputs that start classic IPython prompt syntax."""
643
649
644 if not line or line.isspace():
650 if not line or line.isspace():
645 return line
651 return line
646 #print 'LINE: %r' % line # dbg
652 #print 'LINE: %r' % line # dbg
647 m = _ipy_prompt_re.match(line)
653 m = _ipy_prompt_re.match(line)
648 if m:
654 if m:
649 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
655 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
650 return line[len(m.group(0)):]
656 return line[len(m.group(0)):]
651 else:
657 else:
652 return line
658 return line
653
659
654
660
655 class EscapedTransformer(object):
661 class EscapedTransformer(object):
656 """Class to transform lines that are explicitly escaped out."""
662 """Class to transform lines that are explicitly escaped out."""
657
663
658 def __init__(self):
664 def __init__(self):
659 tr = { ESC_SHELL : self._tr_system,
665 tr = { ESC_SHELL : self._tr_system,
660 ESC_SH_CAP : self._tr_system2,
666 ESC_SH_CAP : self._tr_system2,
661 ESC_HELP : self._tr_help,
667 ESC_HELP : self._tr_help,
662 ESC_HELP2 : self._tr_help,
668 ESC_HELP2 : self._tr_help,
663 ESC_MAGIC : self._tr_magic,
669 ESC_MAGIC : self._tr_magic,
664 ESC_QUOTE : self._tr_quote,
670 ESC_QUOTE : self._tr_quote,
665 ESC_QUOTE2 : self._tr_quote2,
671 ESC_QUOTE2 : self._tr_quote2,
666 ESC_PAREN : self._tr_paren }
672 ESC_PAREN : self._tr_paren }
667 self.tr = tr
673 self.tr = tr
668
674
669 # Support for syntax transformations that use explicit escapes typed by the
675 # Support for syntax transformations that use explicit escapes typed by the
670 # user at the beginning of a line
676 # user at the beginning of a line
671 @staticmethod
677 @staticmethod
672 def _tr_system(line_info):
678 def _tr_system(line_info):
673 "Translate lines escaped with: !"
679 "Translate lines escaped with: !"
674 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
680 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
675 return '%sget_ipython().system(%s)' % (line_info.lspace,
681 return '%sget_ipython().system(%s)' % (line_info.lspace,
676 make_quoted_expr(cmd))
682 make_quoted_expr(cmd))
677
683
678 @staticmethod
684 @staticmethod
679 def _tr_system2(line_info):
685 def _tr_system2(line_info):
680 "Translate lines escaped with: !!"
686 "Translate lines escaped with: !!"
681 cmd = line_info.line.lstrip()[2:]
687 cmd = line_info.line.lstrip()[2:]
682 return '%sget_ipython().getoutput(%s)' % (line_info.lspace,
688 return '%sget_ipython().getoutput(%s)' % (line_info.lspace,
683 make_quoted_expr(cmd))
689 make_quoted_expr(cmd))
684
690
685 @staticmethod
691 @staticmethod
686 def _tr_help(line_info):
692 def _tr_help(line_info):
687 "Translate lines escaped with: ?/??"
693 "Translate lines escaped with: ?/??"
688 # A naked help line should just fire the intro help screen
694 # A naked help line should just fire the intro help screen
689 if not line_info.line[1:]:
695 if not line_info.line[1:]:
690 return 'get_ipython().show_usage()'
696 return 'get_ipython().show_usage()'
691
697
692 # There may be one or two '?' at the end, move them to the front so that
698 # There may be one or two '?' at the end, move them to the front so that
693 # the rest of the logic can assume escapes are at the start
699 # the rest of the logic can assume escapes are at the start
694 l_ori = line_info
700 l_ori = line_info
695 line = line_info.line
701 line = line_info.line
696 if line.endswith('?'):
702 if line.endswith('?'):
697 line = line[-1] + line[:-1]
703 line = line[-1] + line[:-1]
698 if line.endswith('?'):
704 if line.endswith('?'):
699 line = line[-1] + line[:-1]
705 line = line[-1] + line[:-1]
700 line_info = LineInfo(line)
706 line_info = LineInfo(line)
701
707
702 # From here on, simply choose which level of detail to get, and
708 # From here on, simply choose which level of detail to get, and
703 # special-case the psearch syntax
709 # special-case the psearch syntax
704 pinfo = 'pinfo' # default
710 pinfo = 'pinfo' # default
705 if '*' in line_info.line:
711 if '*' in line_info.line:
706 pinfo = 'psearch'
712 pinfo = 'psearch'
707 elif line_info.esc == '??':
713 elif line_info.esc == '??':
708 pinfo = 'pinfo2'
714 pinfo = 'pinfo2'
709
715
710 tpl = '%sget_ipython().magic(u"%s %s")'
716 tpl = '%sget_ipython().magic(u"%s %s")'
711 return tpl % (line_info.lspace, pinfo,
717 return tpl % (line_info.lspace, pinfo,
712 ' '.join([line_info.fpart, line_info.rest]).strip())
718 ' '.join([line_info.fpart, line_info.rest]).strip())
713
719
714 @staticmethod
720 @staticmethod
715 def _tr_magic(line_info):
721 def _tr_magic(line_info):
716 "Translate lines escaped with: %"
722 "Translate lines escaped with: %"
717 tpl = '%sget_ipython().magic(%s)'
723 tpl = '%sget_ipython().magic(%s)'
718 cmd = make_quoted_expr(' '.join([line_info.fpart,
724 cmd = make_quoted_expr(' '.join([line_info.fpart,
719 line_info.rest]).strip())
725 line_info.rest]).strip())
720 return tpl % (line_info.lspace, cmd)
726 return tpl % (line_info.lspace, cmd)
721
727
722 @staticmethod
728 @staticmethod
723 def _tr_quote(line_info):
729 def _tr_quote(line_info):
724 "Translate lines escaped with: ,"
730 "Translate lines escaped with: ,"
725 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
731 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
726 '", "'.join(line_info.rest.split()) )
732 '", "'.join(line_info.rest.split()) )
727
733
728 @staticmethod
734 @staticmethod
729 def _tr_quote2(line_info):
735 def _tr_quote2(line_info):
730 "Translate lines escaped with: ;"
736 "Translate lines escaped with: ;"
731 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
737 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
732 line_info.rest)
738 line_info.rest)
733
739
734 @staticmethod
740 @staticmethod
735 def _tr_paren(line_info):
741 def _tr_paren(line_info):
736 "Translate lines escaped with: /"
742 "Translate lines escaped with: /"
737 return '%s%s(%s)' % (line_info.lspace, line_info.fpart,
743 return '%s%s(%s)' % (line_info.lspace, line_info.fpart,
738 ", ".join(line_info.rest.split()))
744 ", ".join(line_info.rest.split()))
739
745
740 def __call__(self, line):
746 def __call__(self, line):
741 """Class to transform lines that are explicitly escaped out.
747 """Class to transform lines that are explicitly escaped out.
742
748
743 This calls the above _tr_* static methods for the actual line
749 This calls the above _tr_* static methods for the actual line
744 translations."""
750 translations."""
745
751
746 # Empty lines just get returned unmodified
752 # Empty lines just get returned unmodified
747 if not line or line.isspace():
753 if not line or line.isspace():
748 return line
754 return line
749
755
750 # Get line endpoints, where the escapes can be
756 # Get line endpoints, where the escapes can be
751 line_info = LineInfo(line)
757 line_info = LineInfo(line)
752
758
753 # If the escape is not at the start, only '?' needs to be special-cased.
759 # If the escape is not at the start, only '?' needs to be special-cased.
754 # All other escapes are only valid at the start
760 # All other escapes are only valid at the start
755 if not line_info.esc in self.tr:
761 if not line_info.esc in self.tr:
756 if line.endswith(ESC_HELP):
762 if line.endswith(ESC_HELP):
757 return self._tr_help(line_info)
763 return self._tr_help(line_info)
758 else:
764 else:
759 # If we don't recognize the escape, don't modify the line
765 # If we don't recognize the escape, don't modify the line
760 return line
766 return line
761
767
762 return self.tr[line_info.esc](line_info)
768 return self.tr[line_info.esc](line_info)
763
769
764
770
765 # A function-looking object to be used by the rest of the code. The purpose of
771 # A function-looking object to be used by the rest of the code. The purpose of
766 # the class in this case is to organize related functionality, more than to
772 # the class in this case is to organize related functionality, more than to
767 # manage state.
773 # manage state.
768 transform_escaped = EscapedTransformer()
774 transform_escaped = EscapedTransformer()
769
775
770
776
771 class IPythonInputSplitter(InputSplitter):
777 class IPythonInputSplitter(InputSplitter):
772 """An input splitter that recognizes all of IPython's special syntax."""
778 """An input splitter that recognizes all of IPython's special syntax."""
773
779
774 # String with raw, untransformed input.
780 # String with raw, untransformed input.
775 source_raw = ''
781 source_raw = ''
776
782
777 # Private attributes
783 # Private attributes
778
784
779 # List with lines of raw input accumulated so far.
785 # List with lines of raw input accumulated so far.
780 _buffer_raw = None
786 _buffer_raw = None
781
787
782 def __init__(self, input_mode=None):
788 def __init__(self, input_mode=None):
783 InputSplitter.__init__(self, input_mode)
789 InputSplitter.__init__(self, input_mode)
784 self._buffer_raw = []
790 self._buffer_raw = []
785
791
786 def reset(self):
792 def reset(self):
787 """Reset the input buffer and associated state."""
793 """Reset the input buffer and associated state."""
788 InputSplitter.reset(self)
794 InputSplitter.reset(self)
789 self._buffer_raw[:] = []
795 self._buffer_raw[:] = []
790 self.source_raw = ''
796 self.source_raw = ''
791
797
792 def source_raw_reset(self):
798 def source_raw_reset(self):
793 """Return input and raw source and perform a full reset.
799 """Return input and raw source and perform a full reset.
794 """
800 """
795 out = self.source
801 out = self.source
796 out_r = self.source_raw
802 out_r = self.source_raw
797 self.reset()
803 self.reset()
798 return out, out_r
804 return out, out_r
799
805
800 def push(self, lines):
806 def push(self, lines):
801 """Push one or more lines of IPython input.
807 """Push one or more lines of IPython input.
802 """
808 """
803 if not lines:
809 if not lines:
804 return super(IPythonInputSplitter, self).push(lines)
810 return super(IPythonInputSplitter, self).push(lines)
805
811
806 # We must ensure all input is pure unicode
812 # We must ensure all input is pure unicode
807 if type(lines)==str:
813 if type(lines)==str:
808 lines = lines.decode(self.encoding)
814 lines = lines.decode(self.encoding)
809
815
810 lines_list = lines.splitlines()
816 lines_list = lines.splitlines()
811
817
812 transforms = [transform_escaped, transform_assign_system,
818 transforms = [transform_escaped, transform_assign_system,
813 transform_assign_magic, transform_ipy_prompt,
819 transform_assign_magic, transform_ipy_prompt,
814 transform_classic_prompt]
820 transform_classic_prompt]
815
821
816 # Transform logic
822 # Transform logic
817 #
823 #
818 # We only apply the line transformers to the input if we have either no
824 # We only apply the line transformers to the input if we have either no
819 # input yet, or complete input, or if the last line of the buffer ends
825 # input yet, or complete input, or if the last line of the buffer ends
820 # with ':' (opening an indented block). This prevents the accidental
826 # with ':' (opening an indented block). This prevents the accidental
821 # transformation of escapes inside multiline expressions like
827 # transformation of escapes inside multiline expressions like
822 # triple-quoted strings or parenthesized expressions.
828 # triple-quoted strings or parenthesized expressions.
823 #
829 #
824 # The last heuristic, while ugly, ensures that the first line of an
830 # The last heuristic, while ugly, ensures that the first line of an
825 # indented block is correctly transformed.
831 # indented block is correctly transformed.
826 #
832 #
827 # FIXME: try to find a cleaner approach for this last bit.
833 # FIXME: try to find a cleaner approach for this last bit.
828
834
829 # If we were in 'block' mode, since we're going to pump the parent
835 # If we were in 'block' mode, since we're going to pump the parent
830 # class by hand line by line, we need to temporarily switch out to
836 # class by hand line by line, we need to temporarily switch out to
831 # 'line' mode, do a single manual reset and then feed the lines one
837 # 'line' mode, do a single manual reset and then feed the lines one
832 # by one. Note that this only matters if the input has more than one
838 # by one. Note that this only matters if the input has more than one
833 # line.
839 # line.
834 changed_input_mode = False
840 changed_input_mode = False
835
841
836 if self.input_mode == 'cell':
842 if self.input_mode == 'cell':
837 self.reset()
843 self.reset()
838 changed_input_mode = True
844 changed_input_mode = True
839 saved_input_mode = 'cell'
845 saved_input_mode = 'cell'
840 self.input_mode = 'line'
846 self.input_mode = 'line'
841
847
842 # Store raw source before applying any transformations to it. Note
848 # Store raw source before applying any transformations to it. Note
843 # that this must be done *after* the reset() call that would otherwise
849 # that this must be done *after* the reset() call that would otherwise
844 # flush the buffer.
850 # flush the buffer.
845 self._store(lines, self._buffer_raw, 'source_raw')
851 self._store(lines, self._buffer_raw, 'source_raw')
846
852
847 try:
853 try:
848 push = super(IPythonInputSplitter, self).push
854 push = super(IPythonInputSplitter, self).push
849 for line in lines_list:
855 for line in lines_list:
850 if self._is_complete or not self._buffer or \
856 if self._is_complete or not self._buffer or \
851 (self._buffer and self._buffer[-1].rstrip().endswith(':')):
857 (self._buffer and self._buffer[-1].rstrip().endswith(':')):
852 for f in transforms:
858 for f in transforms:
853 line = f(line)
859 line = f(line)
854
860
855 out = push(line)
861 out = push(line)
856 finally:
862 finally:
857 if changed_input_mode:
863 if changed_input_mode:
858 self.input_mode = saved_input_mode
864 self.input_mode = saved_input_mode
859 return out
865 return out
General Comments 0
You need to be logged in to leave comments. Login now