##// END OF EJS Templates
Reorder inputsplitter transformers so that prompts pasted into the input are removed before other transforms are applied.
Thomas Kluyver -
Show More
@@ -1,862 +1,862 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 from __future__ import print_function
63 from __future__ import print_function
64
64
65 #-----------------------------------------------------------------------------
65 #-----------------------------------------------------------------------------
66 # Imports
66 # Imports
67 #-----------------------------------------------------------------------------
67 #-----------------------------------------------------------------------------
68 # stdlib
68 # stdlib
69 import ast
69 import ast
70 import codeop
70 import codeop
71 import re
71 import re
72 import sys
72 import sys
73
73
74 # IPython modules
74 # IPython modules
75 from IPython.utils.text import make_quoted_expr
75 from IPython.utils.text import make_quoted_expr
76
76
77 #-----------------------------------------------------------------------------
77 #-----------------------------------------------------------------------------
78 # Globals
78 # Globals
79 #-----------------------------------------------------------------------------
79 #-----------------------------------------------------------------------------
80
80
81 # The escape sequences that define the syntax transformations IPython will
81 # The escape sequences that define the syntax transformations IPython will
82 # apply to user input. These can NOT be just changed here: many regular
82 # apply to user input. These can NOT be just changed here: many regular
83 # expressions and other parts of the code may use their hardcoded values, and
83 # expressions and other parts of the code may use their hardcoded values, and
84 # for all intents and purposes they constitute the 'IPython syntax', so they
84 # for all intents and purposes they constitute the 'IPython syntax', so they
85 # should be considered fixed.
85 # should be considered fixed.
86
86
87 ESC_SHELL = '!' # Send line to underlying system shell
87 ESC_SHELL = '!' # Send line to underlying system shell
88 ESC_SH_CAP = '!!' # Send line to system shell and capture output
88 ESC_SH_CAP = '!!' # Send line to system shell and capture output
89 ESC_HELP = '?' # Find information about object
89 ESC_HELP = '?' # Find information about object
90 ESC_HELP2 = '??' # Find extra-detailed information about object
90 ESC_HELP2 = '??' # Find extra-detailed information about object
91 ESC_MAGIC = '%' # Call magic function
91 ESC_MAGIC = '%' # Call magic function
92 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
92 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
93 ESC_QUOTE2 = ';' # Quote all args as a single string, call
93 ESC_QUOTE2 = ';' # Quote all args as a single string, call
94 ESC_PAREN = '/' # Call first argument with rest of line as arguments
94 ESC_PAREN = '/' # Call first argument with rest of line as arguments
95
95
96 #-----------------------------------------------------------------------------
96 #-----------------------------------------------------------------------------
97 # Utilities
97 # Utilities
98 #-----------------------------------------------------------------------------
98 #-----------------------------------------------------------------------------
99
99
100 # FIXME: These are general-purpose utilities that later can be moved to the
100 # FIXME: These are general-purpose utilities that later can be moved to the
101 # general ward. Kept here for now because we're being very strict about test
101 # general ward. Kept here for now because we're being very strict about test
102 # coverage with this code, and this lets us ensure that we keep 100% coverage
102 # coverage with this code, and this lets us ensure that we keep 100% coverage
103 # while developing.
103 # while developing.
104
104
105 # compiled regexps for autoindent management
105 # compiled regexps for autoindent management
106 dedent_re = re.compile('|'.join([
106 dedent_re = re.compile('|'.join([
107 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
107 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
108 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
108 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
109 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
109 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
110 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
110 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
111 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
111 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
112 ]))
112 ]))
113 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
113 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
114
114
115 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
115 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
116 # before pure comments
116 # before pure comments
117 comment_line_re = re.compile('^\s*\#')
117 comment_line_re = re.compile('^\s*\#')
118
118
119
119
120 def num_ini_spaces(s):
120 def num_ini_spaces(s):
121 """Return the number of initial spaces in a string.
121 """Return the number of initial spaces in a string.
122
122
123 Note that tabs are counted as a single space. For now, we do *not* support
123 Note that tabs are counted as a single space. For now, we do *not* support
124 mixing of tabs and spaces in the user's input.
124 mixing of tabs and spaces in the user's input.
125
125
126 Parameters
126 Parameters
127 ----------
127 ----------
128 s : string
128 s : string
129
129
130 Returns
130 Returns
131 -------
131 -------
132 n : int
132 n : int
133 """
133 """
134
134
135 ini_spaces = ini_spaces_re.match(s)
135 ini_spaces = ini_spaces_re.match(s)
136 if ini_spaces:
136 if ini_spaces:
137 return ini_spaces.end()
137 return ini_spaces.end()
138 else:
138 else:
139 return 0
139 return 0
140
140
141
141
142 def remove_comments(src):
142 def remove_comments(src):
143 """Remove all comments from input source.
143 """Remove all comments from input source.
144
144
145 Note: comments are NOT recognized inside of strings!
145 Note: comments are NOT recognized inside of strings!
146
146
147 Parameters
147 Parameters
148 ----------
148 ----------
149 src : string
149 src : string
150 A single or multiline input string.
150 A single or multiline input string.
151
151
152 Returns
152 Returns
153 -------
153 -------
154 String with all Python comments removed.
154 String with all Python comments removed.
155 """
155 """
156
156
157 return re.sub('#.*', '', src)
157 return re.sub('#.*', '', src)
158
158
159
159
160 def get_input_encoding():
160 def get_input_encoding():
161 """Return the default standard input encoding.
161 """Return the default standard input encoding.
162
162
163 If sys.stdin has no encoding, 'ascii' is returned."""
163 If sys.stdin has no encoding, 'ascii' is returned."""
164 # There are strange environments for which sys.stdin.encoding is None. We
164 # There are strange environments for which sys.stdin.encoding is None. We
165 # ensure that a valid encoding is returned.
165 # ensure that a valid encoding is returned.
166 encoding = getattr(sys.stdin, 'encoding', None)
166 encoding = getattr(sys.stdin, 'encoding', None)
167 if encoding is None:
167 if encoding is None:
168 encoding = 'ascii'
168 encoding = 'ascii'
169 return encoding
169 return encoding
170
170
171 #-----------------------------------------------------------------------------
171 #-----------------------------------------------------------------------------
172 # Classes and functions for normal Python syntax handling
172 # Classes and functions for normal Python syntax handling
173 #-----------------------------------------------------------------------------
173 #-----------------------------------------------------------------------------
174
174
175 class InputSplitter(object):
175 class InputSplitter(object):
176 """An object that can accumulate lines of Python source before execution.
176 """An object that can accumulate lines of Python source before execution.
177
177
178 This object is designed to be fed python source line-by-line, using
178 This object is designed to be fed python source line-by-line, using
179 :meth:`push`. It will return on each push whether the currently pushed
179 :meth:`push`. It will return on each push whether the currently pushed
180 code could be executed already. In addition, it provides a method called
180 code could be executed already. In addition, it provides a method called
181 :meth:`push_accepts_more` that can be used to query whether more input
181 :meth:`push_accepts_more` that can be used to query whether more input
182 can be pushed into a single interactive block.
182 can be pushed into a single interactive block.
183
183
184 This is a simple example of how an interactive terminal-based client can use
184 This is a simple example of how an interactive terminal-based client can use
185 this tool::
185 this tool::
186
186
187 isp = InputSplitter()
187 isp = InputSplitter()
188 while isp.push_accepts_more():
188 while isp.push_accepts_more():
189 indent = ' '*isp.indent_spaces
189 indent = ' '*isp.indent_spaces
190 prompt = '>>> ' + indent
190 prompt = '>>> ' + indent
191 line = indent + raw_input(prompt)
191 line = indent + raw_input(prompt)
192 isp.push(line)
192 isp.push(line)
193 print 'Input source was:\n', isp.source_reset(),
193 print 'Input source was:\n', isp.source_reset(),
194 """
194 """
195 # Number of spaces of indentation computed from input that has been pushed
195 # Number of spaces of indentation computed from input that has been pushed
196 # so far. This is the attributes callers should query to get the current
196 # so far. This is the attributes callers should query to get the current
197 # indentation level, in order to provide auto-indent facilities.
197 # indentation level, in order to provide auto-indent facilities.
198 indent_spaces = 0
198 indent_spaces = 0
199 # String, indicating the default input encoding. It is computed by default
199 # String, indicating the default input encoding. It is computed by default
200 # at initialization time via get_input_encoding(), but it can be reset by a
200 # at initialization time via get_input_encoding(), but it can be reset by a
201 # client with specific knowledge of the encoding.
201 # client with specific knowledge of the encoding.
202 encoding = ''
202 encoding = ''
203 # String where the current full source input is stored, properly encoded.
203 # String where the current full source input is stored, properly encoded.
204 # Reading this attribute is the normal way of querying the currently pushed
204 # Reading this attribute is the normal way of querying the currently pushed
205 # source code, that has been properly encoded.
205 # source code, that has been properly encoded.
206 source = ''
206 source = ''
207 # Code object corresponding to the current source. It is automatically
207 # Code object corresponding to the current source. It is automatically
208 # synced to the source, so it can be queried at any time to obtain the code
208 # synced to the source, so it can be queried at any time to obtain the code
209 # object; it will be None if the source doesn't compile to valid Python.
209 # object; it will be None if the source doesn't compile to valid Python.
210 code = None
210 code = None
211 # Input mode
211 # Input mode
212 input_mode = 'line'
212 input_mode = 'line'
213
213
214 # Private attributes
214 # Private attributes
215
215
216 # List with lines of input accumulated so far
216 # List with lines of input accumulated so far
217 _buffer = None
217 _buffer = None
218 # Command compiler
218 # Command compiler
219 _compile = None
219 _compile = None
220 # Mark when input has changed indentation all the way back to flush-left
220 # Mark when input has changed indentation all the way back to flush-left
221 _full_dedent = False
221 _full_dedent = False
222 # Boolean indicating whether the current block is complete
222 # Boolean indicating whether the current block is complete
223 _is_complete = None
223 _is_complete = None
224
224
225 def __init__(self, input_mode=None):
225 def __init__(self, input_mode=None):
226 """Create a new InputSplitter instance.
226 """Create a new InputSplitter instance.
227
227
228 Parameters
228 Parameters
229 ----------
229 ----------
230 input_mode : str
230 input_mode : str
231
231
232 One of ['line', 'cell']; default is 'line'.
232 One of ['line', 'cell']; default is 'line'.
233
233
234 The input_mode parameter controls how new inputs are used when fed via
234 The input_mode parameter controls how new inputs are used when fed via
235 the :meth:`push` method:
235 the :meth:`push` method:
236
236
237 - 'line': meant for line-oriented clients, inputs are appended one at a
237 - 'line': meant for line-oriented clients, inputs are appended one at a
238 time to the internal buffer and the whole buffer is compiled.
238 time to the internal buffer and the whole buffer is compiled.
239
239
240 - 'cell': meant for clients that can edit multi-line 'cells' of text at
240 - 'cell': meant for clients that can edit multi-line 'cells' of text at
241 a time. A cell can contain one or more blocks that can be compile in
241 a time. A cell can contain one or more blocks that can be compile in
242 'single' mode by Python. In this mode, each new input new input
242 'single' mode by Python. In this mode, each new input new input
243 completely replaces all prior inputs. Cell mode is thus equivalent
243 completely replaces all prior inputs. Cell mode is thus equivalent
244 to prepending a full reset() to every push() call.
244 to prepending a full reset() to every push() call.
245 """
245 """
246 self._buffer = []
246 self._buffer = []
247 self._compile = codeop.CommandCompiler()
247 self._compile = codeop.CommandCompiler()
248 self.encoding = get_input_encoding()
248 self.encoding = get_input_encoding()
249 self.input_mode = InputSplitter.input_mode if input_mode is None \
249 self.input_mode = InputSplitter.input_mode if input_mode is None \
250 else input_mode
250 else input_mode
251
251
252 def reset(self):
252 def reset(self):
253 """Reset the input buffer and associated state."""
253 """Reset the input buffer and associated state."""
254 self.indent_spaces = 0
254 self.indent_spaces = 0
255 self._buffer[:] = []
255 self._buffer[:] = []
256 self.source = ''
256 self.source = ''
257 self.code = None
257 self.code = None
258 self._is_complete = False
258 self._is_complete = False
259 self._full_dedent = False
259 self._full_dedent = False
260
260
261 def source_reset(self):
261 def source_reset(self):
262 """Return the input source and perform a full reset.
262 """Return the input source and perform a full reset.
263 """
263 """
264 out = self.source
264 out = self.source
265 self.reset()
265 self.reset()
266 return out
266 return out
267
267
268 def push(self, lines):
268 def push(self, lines):
269 """Push one or more lines of input.
269 """Push one or more lines of input.
270
270
271 This stores the given lines and returns a status code indicating
271 This stores the given lines and returns a status code indicating
272 whether the code forms a complete Python block or not.
272 whether the code forms a complete Python block or not.
273
273
274 Any exceptions generated in compilation are swallowed, but if an
274 Any exceptions generated in compilation are swallowed, but if an
275 exception was produced, the method returns True.
275 exception was produced, the method returns True.
276
276
277 Parameters
277 Parameters
278 ----------
278 ----------
279 lines : string
279 lines : string
280 One or more lines of Python input.
280 One or more lines of Python input.
281
281
282 Returns
282 Returns
283 -------
283 -------
284 is_complete : boolean
284 is_complete : boolean
285 True if the current input source (the result of the current input
285 True if the current input source (the result of the current input
286 plus prior inputs) forms a complete Python execution block. Note that
286 plus prior inputs) forms a complete Python execution block. Note that
287 this value is also stored as a private attribute (_is_complete), so it
287 this value is also stored as a private attribute (_is_complete), so it
288 can be queried at any time.
288 can be queried at any time.
289 """
289 """
290 if self.input_mode == 'cell':
290 if self.input_mode == 'cell':
291 self.reset()
291 self.reset()
292
292
293 self._store(lines)
293 self._store(lines)
294 source = self.source
294 source = self.source
295
295
296 # Before calling _compile(), reset the code object to None so that if an
296 # Before calling _compile(), reset the code object to None so that if an
297 # exception is raised in compilation, we don't mislead by having
297 # exception is raised in compilation, we don't mislead by having
298 # inconsistent code/source attributes.
298 # inconsistent code/source attributes.
299 self.code, self._is_complete = None, None
299 self.code, self._is_complete = None, None
300
300
301 # Honor termination lines properly
301 # Honor termination lines properly
302 if source.rstrip().endswith('\\'):
302 if source.rstrip().endswith('\\'):
303 return False
303 return False
304
304
305 self._update_indent(lines)
305 self._update_indent(lines)
306 try:
306 try:
307 self.code = self._compile(source, symbol="exec")
307 self.code = self._compile(source, symbol="exec")
308 # Invalid syntax can produce any of a number of different errors from
308 # Invalid syntax can produce any of a number of different errors from
309 # inside the compiler, so we have to catch them all. Syntax errors
309 # inside the compiler, so we have to catch them all. Syntax errors
310 # immediately produce a 'ready' block, so the invalid Python can be
310 # immediately produce a 'ready' block, so the invalid Python can be
311 # sent to the kernel for evaluation with possible ipython
311 # sent to the kernel for evaluation with possible ipython
312 # special-syntax conversion.
312 # special-syntax conversion.
313 except (SyntaxError, OverflowError, ValueError, TypeError,
313 except (SyntaxError, OverflowError, ValueError, TypeError,
314 MemoryError):
314 MemoryError):
315 self._is_complete = True
315 self._is_complete = True
316 else:
316 else:
317 # Compilation didn't produce any exceptions (though it may not have
317 # Compilation didn't produce any exceptions (though it may not have
318 # given a complete code object)
318 # given a complete code object)
319 self._is_complete = self.code is not None
319 self._is_complete = self.code is not None
320
320
321 return self._is_complete
321 return self._is_complete
322
322
323 def push_accepts_more(self):
323 def push_accepts_more(self):
324 """Return whether a block of interactive input can accept more input.
324 """Return whether a block of interactive input can accept more input.
325
325
326 This method is meant to be used by line-oriented frontends, who need to
326 This method is meant to be used by line-oriented frontends, who need to
327 guess whether a block is complete or not based solely on prior and
327 guess whether a block is complete or not based solely on prior and
328 current input lines. The InputSplitter considers it has a complete
328 current input lines. The InputSplitter considers it has a complete
329 interactive block and will not accept more input only when either a
329 interactive block and will not accept more input only when either a
330 SyntaxError is raised, or *all* of the following are true:
330 SyntaxError is raised, or *all* of the following are true:
331
331
332 1. The input compiles to a complete statement.
332 1. The input compiles to a complete statement.
333
333
334 2. The indentation level is flush-left (because if we are indented,
334 2. The indentation level is flush-left (because if we are indented,
335 like inside a function definition or for loop, we need to keep
335 like inside a function definition or for loop, we need to keep
336 reading new input).
336 reading new input).
337
337
338 3. There is one extra line consisting only of whitespace.
338 3. There is one extra line consisting only of whitespace.
339
339
340 Because of condition #3, this method should be used only by
340 Because of condition #3, this method should be used only by
341 *line-oriented* frontends, since it means that intermediate blank lines
341 *line-oriented* frontends, since it means that intermediate blank lines
342 are not allowed in function definitions (or any other indented block).
342 are not allowed in function definitions (or any other indented block).
343
343
344 Block-oriented frontends that have a separate keyboard event to
344 Block-oriented frontends that have a separate keyboard event to
345 indicate execution should use the :meth:`split_blocks` method instead.
345 indicate execution should use the :meth:`split_blocks` method instead.
346
346
347 If the current input produces a syntax error, this method immediately
347 If the current input produces a syntax error, this method immediately
348 returns False but does *not* raise the syntax error exception, as
348 returns False but does *not* raise the syntax error exception, as
349 typically clients will want to send invalid syntax to an execution
349 typically clients will want to send invalid syntax to an execution
350 backend which might convert the invalid syntax into valid Python via
350 backend which might convert the invalid syntax into valid Python via
351 one of the dynamic IPython mechanisms.
351 one of the dynamic IPython mechanisms.
352 """
352 """
353
353
354 # With incomplete input, unconditionally accept more
354 # With incomplete input, unconditionally accept more
355 if not self._is_complete:
355 if not self._is_complete:
356 return True
356 return True
357
357
358 # If we already have complete input and we're flush left, the answer
358 # If we already have complete input and we're flush left, the answer
359 # depends. In line mode, if there hasn't been any indentation,
359 # depends. In line mode, if there hasn't been any indentation,
360 # that's it. If we've come back from some indentation, we need
360 # that's it. If we've come back from some indentation, we need
361 # the blank final line to finish.
361 # the blank final line to finish.
362 # In cell mode, we need to check how many blocks the input so far
362 # In cell mode, we need to check how many blocks the input so far
363 # compiles into, because if there's already more than one full
363 # compiles into, because if there's already more than one full
364 # independent block of input, then the client has entered full
364 # independent block of input, then the client has entered full
365 # 'cell' mode and is feeding lines that each is complete. In this
365 # 'cell' mode and is feeding lines that each is complete. In this
366 # case we should then keep accepting. The Qt terminal-like console
366 # case we should then keep accepting. The Qt terminal-like console
367 # does precisely this, to provide the convenience of terminal-like
367 # does precisely this, to provide the convenience of terminal-like
368 # input of single expressions, but allowing the user (with a
368 # input of single expressions, but allowing the user (with a
369 # separate keystroke) to switch to 'cell' mode and type multiple
369 # separate keystroke) to switch to 'cell' mode and type multiple
370 # expressions in one shot.
370 # expressions in one shot.
371 if self.indent_spaces==0:
371 if self.indent_spaces==0:
372 if self.input_mode=='line':
372 if self.input_mode=='line':
373 if not self._full_dedent:
373 if not self._full_dedent:
374 return False
374 return False
375 else:
375 else:
376 try:
376 try:
377 code_ast = ast.parse(u''.join(self._buffer))
377 code_ast = ast.parse(u''.join(self._buffer))
378 except Exception:
378 except Exception:
379 return False
379 return False
380 else:
380 else:
381 if len(code_ast.body) == 1:
381 if len(code_ast.body) == 1:
382 return False
382 return False
383
383
384 # When input is complete, then termination is marked by an extra blank
384 # When input is complete, then termination is marked by an extra blank
385 # line at the end.
385 # line at the end.
386 last_line = self.source.splitlines()[-1]
386 last_line = self.source.splitlines()[-1]
387 return bool(last_line and not last_line.isspace())
387 return bool(last_line and not last_line.isspace())
388
388
389 #------------------------------------------------------------------------
389 #------------------------------------------------------------------------
390 # Private interface
390 # Private interface
391 #------------------------------------------------------------------------
391 #------------------------------------------------------------------------
392
392
393 def _find_indent(self, line):
393 def _find_indent(self, line):
394 """Compute the new indentation level for a single line.
394 """Compute the new indentation level for a single line.
395
395
396 Parameters
396 Parameters
397 ----------
397 ----------
398 line : str
398 line : str
399 A single new line of non-whitespace, non-comment Python input.
399 A single new line of non-whitespace, non-comment Python input.
400
400
401 Returns
401 Returns
402 -------
402 -------
403 indent_spaces : int
403 indent_spaces : int
404 New value for the indent level (it may be equal to self.indent_spaces
404 New value for the indent level (it may be equal to self.indent_spaces
405 if indentation doesn't change.
405 if indentation doesn't change.
406
406
407 full_dedent : boolean
407 full_dedent : boolean
408 Whether the new line causes a full flush-left dedent.
408 Whether the new line causes a full flush-left dedent.
409 """
409 """
410 indent_spaces = self.indent_spaces
410 indent_spaces = self.indent_spaces
411 full_dedent = self._full_dedent
411 full_dedent = self._full_dedent
412
412
413 inisp = num_ini_spaces(line)
413 inisp = num_ini_spaces(line)
414 if inisp < indent_spaces:
414 if inisp < indent_spaces:
415 indent_spaces = inisp
415 indent_spaces = inisp
416 if indent_spaces <= 0:
416 if indent_spaces <= 0:
417 #print 'Full dedent in text',self.source # dbg
417 #print 'Full dedent in text',self.source # dbg
418 full_dedent = True
418 full_dedent = True
419
419
420 if line[-1] == ':':
420 if line[-1] == ':':
421 indent_spaces += 4
421 indent_spaces += 4
422 elif dedent_re.match(line):
422 elif dedent_re.match(line):
423 indent_spaces -= 4
423 indent_spaces -= 4
424 if indent_spaces <= 0:
424 if indent_spaces <= 0:
425 full_dedent = True
425 full_dedent = True
426
426
427 # Safety
427 # Safety
428 if indent_spaces < 0:
428 if indent_spaces < 0:
429 indent_spaces = 0
429 indent_spaces = 0
430 #print 'safety' # dbg
430 #print 'safety' # dbg
431
431
432 return indent_spaces, full_dedent
432 return indent_spaces, full_dedent
433
433
434 def _update_indent(self, lines):
434 def _update_indent(self, lines):
435 for line in remove_comments(lines).splitlines():
435 for line in remove_comments(lines).splitlines():
436 if line and not line.isspace():
436 if line and not line.isspace():
437 self.indent_spaces, self._full_dedent = self._find_indent(line)
437 self.indent_spaces, self._full_dedent = self._find_indent(line)
438
438
439 def _store(self, lines, buffer=None, store='source'):
439 def _store(self, lines, buffer=None, store='source'):
440 """Store one or more lines of input.
440 """Store one or more lines of input.
441
441
442 If input lines are not newline-terminated, a newline is automatically
442 If input lines are not newline-terminated, a newline is automatically
443 appended."""
443 appended."""
444
444
445 if buffer is None:
445 if buffer is None:
446 buffer = self._buffer
446 buffer = self._buffer
447
447
448 if lines.endswith('\n'):
448 if lines.endswith('\n'):
449 buffer.append(lines)
449 buffer.append(lines)
450 else:
450 else:
451 buffer.append(lines+'\n')
451 buffer.append(lines+'\n')
452 setattr(self, store, self._set_source(buffer))
452 setattr(self, store, self._set_source(buffer))
453
453
454 def _set_source(self, buffer):
454 def _set_source(self, buffer):
455 return u''.join(buffer)
455 return u''.join(buffer)
456
456
457
457
458 #-----------------------------------------------------------------------------
458 #-----------------------------------------------------------------------------
459 # Functions and classes for IPython-specific syntactic support
459 # Functions and classes for IPython-specific syntactic support
460 #-----------------------------------------------------------------------------
460 #-----------------------------------------------------------------------------
461
461
462 # RegExp for splitting line contents into pre-char//first word-method//rest.
462 # RegExp for splitting line contents into pre-char//first word-method//rest.
463 # For clarity, each group in on one line.
463 # For clarity, each group in on one line.
464
464
465 line_split = re.compile("""
465 line_split = re.compile("""
466 ^(\s*) # any leading space
466 ^(\s*) # any leading space
467 ([,;/%]|!!?|\?\??) # escape character or characters
467 ([,;/%]|!!?|\?\??) # escape character or characters
468 \s*(%?[\w\.\*]*) # function/method, possibly with leading %
468 \s*(%?[\w\.\*]*) # function/method, possibly with leading %
469 # to correctly treat things like '?%magic'
469 # to correctly treat things like '?%magic'
470 (\s+.*$|$) # rest of line
470 (\s+.*$|$) # rest of line
471 """, re.VERBOSE)
471 """, re.VERBOSE)
472
472
473
473
474 def split_user_input(line):
474 def split_user_input(line):
475 """Split user input into early whitespace, esc-char, function part and rest.
475 """Split user input into early whitespace, esc-char, function part and rest.
476
476
477 This is currently handles lines with '=' in them in a very inconsistent
477 This is currently handles lines with '=' in them in a very inconsistent
478 manner.
478 manner.
479
479
480 Examples
480 Examples
481 ========
481 ========
482 >>> split_user_input('x=1')
482 >>> split_user_input('x=1')
483 ('', '', 'x=1', '')
483 ('', '', 'x=1', '')
484 >>> split_user_input('?')
484 >>> split_user_input('?')
485 ('', '?', '', '')
485 ('', '?', '', '')
486 >>> split_user_input('??')
486 >>> split_user_input('??')
487 ('', '??', '', '')
487 ('', '??', '', '')
488 >>> split_user_input(' ?')
488 >>> split_user_input(' ?')
489 (' ', '?', '', '')
489 (' ', '?', '', '')
490 >>> split_user_input(' ??')
490 >>> split_user_input(' ??')
491 (' ', '??', '', '')
491 (' ', '??', '', '')
492 >>> split_user_input('??x')
492 >>> split_user_input('??x')
493 ('', '??', 'x', '')
493 ('', '??', 'x', '')
494 >>> split_user_input('?x=1')
494 >>> split_user_input('?x=1')
495 ('', '', '?x=1', '')
495 ('', '', '?x=1', '')
496 >>> split_user_input('!ls')
496 >>> split_user_input('!ls')
497 ('', '!', 'ls', '')
497 ('', '!', 'ls', '')
498 >>> split_user_input(' !ls')
498 >>> split_user_input(' !ls')
499 (' ', '!', 'ls', '')
499 (' ', '!', 'ls', '')
500 >>> split_user_input('!!ls')
500 >>> split_user_input('!!ls')
501 ('', '!!', 'ls', '')
501 ('', '!!', 'ls', '')
502 >>> split_user_input(' !!ls')
502 >>> split_user_input(' !!ls')
503 (' ', '!!', 'ls', '')
503 (' ', '!!', 'ls', '')
504 >>> split_user_input(',ls')
504 >>> split_user_input(',ls')
505 ('', ',', 'ls', '')
505 ('', ',', 'ls', '')
506 >>> split_user_input(';ls')
506 >>> split_user_input(';ls')
507 ('', ';', 'ls', '')
507 ('', ';', 'ls', '')
508 >>> split_user_input(' ;ls')
508 >>> split_user_input(' ;ls')
509 (' ', ';', 'ls', '')
509 (' ', ';', 'ls', '')
510 >>> split_user_input('f.g(x)')
510 >>> split_user_input('f.g(x)')
511 ('', '', 'f.g(x)', '')
511 ('', '', 'f.g(x)', '')
512 >>> split_user_input('f.g (x)')
512 >>> split_user_input('f.g (x)')
513 ('', '', 'f.g', '(x)')
513 ('', '', 'f.g', '(x)')
514 >>> split_user_input('?%hist')
514 >>> split_user_input('?%hist')
515 ('', '?', '%hist', '')
515 ('', '?', '%hist', '')
516 >>> split_user_input('?x*')
516 >>> split_user_input('?x*')
517 ('', '?', 'x*', '')
517 ('', '?', 'x*', '')
518 """
518 """
519 match = line_split.match(line)
519 match = line_split.match(line)
520 if match:
520 if match:
521 lspace, esc, fpart, rest = match.groups()
521 lspace, esc, fpart, rest = match.groups()
522 else:
522 else:
523 # print "match failed for line '%s'" % line
523 # print "match failed for line '%s'" % line
524 try:
524 try:
525 fpart, rest = line.split(None, 1)
525 fpart, rest = line.split(None, 1)
526 except ValueError:
526 except ValueError:
527 # print "split failed for line '%s'" % line
527 # print "split failed for line '%s'" % line
528 fpart, rest = line,''
528 fpart, rest = line,''
529 lspace = re.match('^(\s*)(.*)', line).groups()[0]
529 lspace = re.match('^(\s*)(.*)', line).groups()[0]
530 esc = ''
530 esc = ''
531
531
532 # fpart has to be a valid python identifier, so it better be only pure
532 # fpart has to be a valid python identifier, so it better be only pure
533 # ascii, no unicode:
533 # ascii, no unicode:
534 try:
534 try:
535 fpart = fpart.encode('ascii')
535 fpart = fpart.encode('ascii')
536 except UnicodeEncodeError:
536 except UnicodeEncodeError:
537 lspace = unicode(lspace)
537 lspace = unicode(lspace)
538 rest = fpart + u' ' + rest
538 rest = fpart + u' ' + rest
539 fpart = u''
539 fpart = u''
540
540
541 #print 'line:<%s>' % line # dbg
541 #print 'line:<%s>' % line # dbg
542 #print 'esc <%s> fpart <%s> rest <%s>' % (esc,fpart.strip(),rest) # dbg
542 #print 'esc <%s> fpart <%s> rest <%s>' % (esc,fpart.strip(),rest) # dbg
543 return lspace, esc, fpart.strip(), rest.lstrip()
543 return lspace, esc, fpart.strip(), rest.lstrip()
544
544
545
545
546 # The escaped translators ALL receive a line where their own escape has been
546 # The escaped translators ALL receive a line where their own escape has been
547 # stripped. Only '?' is valid at the end of the line, all others can only be
547 # stripped. Only '?' is valid at the end of the line, all others can only be
548 # placed at the start.
548 # placed at the start.
549
549
550 class LineInfo(object):
550 class LineInfo(object):
551 """A single line of input and associated info.
551 """A single line of input and associated info.
552
552
553 This is a utility class that mostly wraps the output of
553 This is a utility class that mostly wraps the output of
554 :func:`split_user_input` into a convenient object to be passed around
554 :func:`split_user_input` into a convenient object to be passed around
555 during input transformations.
555 during input transformations.
556
556
557 Includes the following as properties:
557 Includes the following as properties:
558
558
559 line
559 line
560 The original, raw line
560 The original, raw line
561
561
562 lspace
562 lspace
563 Any early whitespace before actual text starts.
563 Any early whitespace before actual text starts.
564
564
565 esc
565 esc
566 The initial esc character (or characters, for double-char escapes like
566 The initial esc character (or characters, for double-char escapes like
567 '??' or '!!').
567 '??' or '!!').
568
568
569 fpart
569 fpart
570 The 'function part', which is basically the maximal initial sequence
570 The 'function part', which is basically the maximal initial sequence
571 of valid python identifiers and the '.' character. This is what is
571 of valid python identifiers and the '.' character. This is what is
572 checked for alias and magic transformations, used for auto-calling,
572 checked for alias and magic transformations, used for auto-calling,
573 etc.
573 etc.
574
574
575 rest
575 rest
576 Everything else on the line.
576 Everything else on the line.
577 """
577 """
578 def __init__(self, line):
578 def __init__(self, line):
579 self.line = line
579 self.line = line
580 self.lspace, self.esc, self.fpart, self.rest = \
580 self.lspace, self.esc, self.fpart, self.rest = \
581 split_user_input(line)
581 split_user_input(line)
582
582
583 def __str__(self):
583 def __str__(self):
584 return "LineInfo [%s|%s|%s|%s]" % (self.lspace, self.esc,
584 return "LineInfo [%s|%s|%s|%s]" % (self.lspace, self.esc,
585 self.fpart, self.rest)
585 self.fpart, self.rest)
586
586
587
587
588 # Transformations of the special syntaxes that don't rely on an explicit escape
588 # Transformations of the special syntaxes that don't rely on an explicit escape
589 # character but instead on patterns on the input line
589 # character but instead on patterns on the input line
590
590
591 # The core transformations are implemented as standalone functions that can be
591 # The core transformations are implemented as standalone functions that can be
592 # tested and validated in isolation. Each of these uses a regexp, we
592 # tested and validated in isolation. Each of these uses a regexp, we
593 # pre-compile these and keep them close to each function definition for clarity
593 # pre-compile these and keep them close to each function definition for clarity
594
594
595 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
595 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
596 r'\s*=\s*!\s*(?P<cmd>.*)')
596 r'\s*=\s*!\s*(?P<cmd>.*)')
597
597
598 def transform_assign_system(line):
598 def transform_assign_system(line):
599 """Handle the `files = !ls` syntax."""
599 """Handle the `files = !ls` syntax."""
600 m = _assign_system_re.match(line)
600 m = _assign_system_re.match(line)
601 if m is not None:
601 if m is not None:
602 cmd = m.group('cmd')
602 cmd = m.group('cmd')
603 lhs = m.group('lhs')
603 lhs = m.group('lhs')
604 expr = make_quoted_expr(cmd)
604 expr = make_quoted_expr(cmd)
605 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
605 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
606 return new_line
606 return new_line
607 return line
607 return line
608
608
609
609
610 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
610 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
611 r'\s*=\s*%\s*(?P<cmd>.*)')
611 r'\s*=\s*%\s*(?P<cmd>.*)')
612
612
613 def transform_assign_magic(line):
613 def transform_assign_magic(line):
614 """Handle the `a = %who` syntax."""
614 """Handle the `a = %who` syntax."""
615 m = _assign_magic_re.match(line)
615 m = _assign_magic_re.match(line)
616 if m is not None:
616 if m is not None:
617 cmd = m.group('cmd')
617 cmd = m.group('cmd')
618 lhs = m.group('lhs')
618 lhs = m.group('lhs')
619 expr = make_quoted_expr(cmd)
619 expr = make_quoted_expr(cmd)
620 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
620 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
621 return new_line
621 return new_line
622 return line
622 return line
623
623
624
624
625 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
625 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
626
626
627 def transform_classic_prompt(line):
627 def transform_classic_prompt(line):
628 """Handle inputs that start with '>>> ' syntax."""
628 """Handle inputs that start with '>>> ' syntax."""
629
629
630 if not line or line.isspace():
630 if not line or line.isspace():
631 return line
631 return line
632 m = _classic_prompt_re.match(line)
632 m = _classic_prompt_re.match(line)
633 if m:
633 if m:
634 return line[len(m.group(0)):]
634 return line[len(m.group(0)):]
635 else:
635 else:
636 return line
636 return line
637
637
638
638
639 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
639 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
640
640
641 def transform_ipy_prompt(line):
641 def transform_ipy_prompt(line):
642 """Handle inputs that start classic IPython prompt syntax."""
642 """Handle inputs that start classic IPython prompt syntax."""
643
643
644 if not line or line.isspace():
644 if not line or line.isspace():
645 return line
645 return line
646 #print 'LINE: %r' % line # dbg
646 #print 'LINE: %r' % line # dbg
647 m = _ipy_prompt_re.match(line)
647 m = _ipy_prompt_re.match(line)
648 if m:
648 if m:
649 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
649 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
650 return line[len(m.group(0)):]
650 return line[len(m.group(0)):]
651 else:
651 else:
652 return line
652 return line
653
653
654
654
655 def _make_help_call(target, esc, lspace):
655 def _make_help_call(target, esc, lspace):
656 """Prepares a pinfo(2)/psearch call from a target name and the escape
656 """Prepares a pinfo(2)/psearch call from a target name and the escape
657 (i.e. ? or ??)"""
657 (i.e. ? or ??)"""
658 method = 'pinfo2' if esc == '??' \
658 method = 'pinfo2' if esc == '??' \
659 else 'psearch' if '*' in target \
659 else 'psearch' if '*' in target \
660 else 'pinfo'
660 else 'pinfo'
661
661
662 tpl = '%sget_ipython().magic(u"%s %s")'
662 tpl = '%sget_ipython().magic(u"%s %s")'
663 return tpl % (lspace, method, target)
663 return tpl % (lspace, method, target)
664
664
665 _initial_space_re = re.compile(r'\s*')
665 _initial_space_re = re.compile(r'\s*')
666 _help_end_re = re.compile(r"""(%?
666 _help_end_re = re.compile(r"""(%?
667 [a-zA-Z_*][a-zA-Z0-9_*]* # Variable name
667 [a-zA-Z_*][a-zA-Z0-9_*]* # Variable name
668 (.[a-zA-Z_*][a-zA-Z0-9_*]*)* # .etc.etc
668 (\.[a-zA-Z_*][a-zA-Z0-9_*]*)* # .etc.etc
669 )
669 )
670 (\?\??)$ # ? or ??""",
670 (\?\??)$ # ? or ??""",
671 re.VERBOSE)
671 re.VERBOSE)
672 def transform_help_end(line):
672 def transform_help_end(line):
673 """Translate lines with ?/?? at the end"""
673 """Translate lines with ?/?? at the end"""
674 m = _help_end_re.search(line)
674 m = _help_end_re.search(line)
675 if m is None:
675 if m is None:
676 return line
676 return line
677 target = m.group(1)
677 target = m.group(1)
678 esc = m.group(3)
678 esc = m.group(3)
679 lspace = _initial_space_re.match(line).group(0)
679 lspace = _initial_space_re.match(line).group(0)
680 return _make_help_call(target, esc, lspace)
680 return _make_help_call(target, esc, lspace)
681
681
682
682
683 class EscapedTransformer(object):
683 class EscapedTransformer(object):
684 """Class to transform lines that are explicitly escaped out."""
684 """Class to transform lines that are explicitly escaped out."""
685
685
686 def __init__(self):
686 def __init__(self):
687 tr = { ESC_SHELL : self._tr_system,
687 tr = { ESC_SHELL : self._tr_system,
688 ESC_SH_CAP : self._tr_system2,
688 ESC_SH_CAP : self._tr_system2,
689 ESC_HELP : self._tr_help,
689 ESC_HELP : self._tr_help,
690 ESC_HELP2 : self._tr_help,
690 ESC_HELP2 : self._tr_help,
691 ESC_MAGIC : self._tr_magic,
691 ESC_MAGIC : self._tr_magic,
692 ESC_QUOTE : self._tr_quote,
692 ESC_QUOTE : self._tr_quote,
693 ESC_QUOTE2 : self._tr_quote2,
693 ESC_QUOTE2 : self._tr_quote2,
694 ESC_PAREN : self._tr_paren }
694 ESC_PAREN : self._tr_paren }
695 self.tr = tr
695 self.tr = tr
696
696
697 # Support for syntax transformations that use explicit escapes typed by the
697 # Support for syntax transformations that use explicit escapes typed by the
698 # user at the beginning of a line
698 # user at the beginning of a line
699 @staticmethod
699 @staticmethod
700 def _tr_system(line_info):
700 def _tr_system(line_info):
701 "Translate lines escaped with: !"
701 "Translate lines escaped with: !"
702 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
702 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
703 return '%sget_ipython().system(%s)' % (line_info.lspace,
703 return '%sget_ipython().system(%s)' % (line_info.lspace,
704 make_quoted_expr(cmd))
704 make_quoted_expr(cmd))
705
705
706 @staticmethod
706 @staticmethod
707 def _tr_system2(line_info):
707 def _tr_system2(line_info):
708 "Translate lines escaped with: !!"
708 "Translate lines escaped with: !!"
709 cmd = line_info.line.lstrip()[2:]
709 cmd = line_info.line.lstrip()[2:]
710 return '%sget_ipython().getoutput(%s)' % (line_info.lspace,
710 return '%sget_ipython().getoutput(%s)' % (line_info.lspace,
711 make_quoted_expr(cmd))
711 make_quoted_expr(cmd))
712
712
713 @staticmethod
713 @staticmethod
714 def _tr_help(line_info):
714 def _tr_help(line_info):
715 "Translate lines escaped with: ?/??"
715 "Translate lines escaped with: ?/??"
716 # A naked help line should just fire the intro help screen
716 # A naked help line should just fire the intro help screen
717 if not line_info.line[1:]:
717 if not line_info.line[1:]:
718 return 'get_ipython().show_usage()'
718 return 'get_ipython().show_usage()'
719
719
720 return _make_help_call(line_info.fpart, line_info.esc, line_info.lspace)
720 return _make_help_call(line_info.fpart, line_info.esc, line_info.lspace)
721
721
722 @staticmethod
722 @staticmethod
723 def _tr_magic(line_info):
723 def _tr_magic(line_info):
724 "Translate lines escaped with: %"
724 "Translate lines escaped with: %"
725 tpl = '%sget_ipython().magic(%s)'
725 tpl = '%sget_ipython().magic(%s)'
726 cmd = make_quoted_expr(' '.join([line_info.fpart,
726 cmd = make_quoted_expr(' '.join([line_info.fpart,
727 line_info.rest]).strip())
727 line_info.rest]).strip())
728 return tpl % (line_info.lspace, cmd)
728 return tpl % (line_info.lspace, cmd)
729
729
730 @staticmethod
730 @staticmethod
731 def _tr_quote(line_info):
731 def _tr_quote(line_info):
732 "Translate lines escaped with: ,"
732 "Translate lines escaped with: ,"
733 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
733 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
734 '", "'.join(line_info.rest.split()) )
734 '", "'.join(line_info.rest.split()) )
735
735
736 @staticmethod
736 @staticmethod
737 def _tr_quote2(line_info):
737 def _tr_quote2(line_info):
738 "Translate lines escaped with: ;"
738 "Translate lines escaped with: ;"
739 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
739 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
740 line_info.rest)
740 line_info.rest)
741
741
742 @staticmethod
742 @staticmethod
743 def _tr_paren(line_info):
743 def _tr_paren(line_info):
744 "Translate lines escaped with: /"
744 "Translate lines escaped with: /"
745 return '%s%s(%s)' % (line_info.lspace, line_info.fpart,
745 return '%s%s(%s)' % (line_info.lspace, line_info.fpart,
746 ", ".join(line_info.rest.split()))
746 ", ".join(line_info.rest.split()))
747
747
748 def __call__(self, line):
748 def __call__(self, line):
749 """Class to transform lines that are explicitly escaped out.
749 """Class to transform lines that are explicitly escaped out.
750
750
751 This calls the above _tr_* static methods for the actual line
751 This calls the above _tr_* static methods for the actual line
752 translations."""
752 translations."""
753
753
754 # Empty lines just get returned unmodified
754 # Empty lines just get returned unmodified
755 if not line or line.isspace():
755 if not line or line.isspace():
756 return line
756 return line
757
757
758 # Get line endpoints, where the escapes can be
758 # Get line endpoints, where the escapes can be
759 line_info = LineInfo(line)
759 line_info = LineInfo(line)
760
760
761 if not line_info.esc in self.tr:
761 if not line_info.esc in self.tr:
762 # If we don't recognize the escape, don't modify the line
762 # If we don't recognize the escape, don't modify the line
763 return line
763 return line
764
764
765 return self.tr[line_info.esc](line_info)
765 return self.tr[line_info.esc](line_info)
766
766
767
767
768 # A function-looking object to be used by the rest of the code. The purpose of
768 # A function-looking object to be used by the rest of the code. The purpose of
769 # the class in this case is to organize related functionality, more than to
769 # the class in this case is to organize related functionality, more than to
770 # manage state.
770 # manage state.
771 transform_escaped = EscapedTransformer()
771 transform_escaped = EscapedTransformer()
772
772
773
773
774 class IPythonInputSplitter(InputSplitter):
774 class IPythonInputSplitter(InputSplitter):
775 """An input splitter that recognizes all of IPython's special syntax."""
775 """An input splitter that recognizes all of IPython's special syntax."""
776
776
777 # String with raw, untransformed input.
777 # String with raw, untransformed input.
778 source_raw = ''
778 source_raw = ''
779
779
780 # Private attributes
780 # Private attributes
781
781
782 # List with lines of raw input accumulated so far.
782 # List with lines of raw input accumulated so far.
783 _buffer_raw = None
783 _buffer_raw = None
784
784
785 def __init__(self, input_mode=None):
785 def __init__(self, input_mode=None):
786 InputSplitter.__init__(self, input_mode)
786 InputSplitter.__init__(self, input_mode)
787 self._buffer_raw = []
787 self._buffer_raw = []
788
788
789 def reset(self):
789 def reset(self):
790 """Reset the input buffer and associated state."""
790 """Reset the input buffer and associated state."""
791 InputSplitter.reset(self)
791 InputSplitter.reset(self)
792 self._buffer_raw[:] = []
792 self._buffer_raw[:] = []
793 self.source_raw = ''
793 self.source_raw = ''
794
794
795 def source_raw_reset(self):
795 def source_raw_reset(self):
796 """Return input and raw source and perform a full reset.
796 """Return input and raw source and perform a full reset.
797 """
797 """
798 out = self.source
798 out = self.source
799 out_r = self.source_raw
799 out_r = self.source_raw
800 self.reset()
800 self.reset()
801 return out, out_r
801 return out, out_r
802
802
803 def push(self, lines):
803 def push(self, lines):
804 """Push one or more lines of IPython input.
804 """Push one or more lines of IPython input.
805 """
805 """
806 if not lines:
806 if not lines:
807 return super(IPythonInputSplitter, self).push(lines)
807 return super(IPythonInputSplitter, self).push(lines)
808
808
809 # We must ensure all input is pure unicode
809 # We must ensure all input is pure unicode
810 if type(lines)==str:
810 if type(lines)==str:
811 lines = lines.decode(self.encoding)
811 lines = lines.decode(self.encoding)
812
812
813 lines_list = lines.splitlines()
813 lines_list = lines.splitlines()
814
814
815 transforms = [transform_escaped, transform_help_end,
815 transforms = [transform_ipy_prompt, transform_classic_prompt,
816 transform_assign_system, transform_assign_magic,
816 transform_escaped, transform_help_end,
817 transform_ipy_prompt, transform_classic_prompt]
817 transform_assign_system, transform_assign_magic]
818
818
819 # Transform logic
819 # Transform logic
820 #
820 #
821 # We only apply the line transformers to the input if we have either no
821 # We only apply the line transformers to the input if we have either no
822 # input yet, or complete input, or if the last line of the buffer ends
822 # input yet, or complete input, or if the last line of the buffer ends
823 # with ':' (opening an indented block). This prevents the accidental
823 # with ':' (opening an indented block). This prevents the accidental
824 # transformation of escapes inside multiline expressions like
824 # transformation of escapes inside multiline expressions like
825 # triple-quoted strings or parenthesized expressions.
825 # triple-quoted strings or parenthesized expressions.
826 #
826 #
827 # The last heuristic, while ugly, ensures that the first line of an
827 # The last heuristic, while ugly, ensures that the first line of an
828 # indented block is correctly transformed.
828 # indented block is correctly transformed.
829 #
829 #
830 # FIXME: try to find a cleaner approach for this last bit.
830 # FIXME: try to find a cleaner approach for this last bit.
831
831
832 # If we were in 'block' mode, since we're going to pump the parent
832 # If we were in 'block' mode, since we're going to pump the parent
833 # class by hand line by line, we need to temporarily switch out to
833 # class by hand line by line, we need to temporarily switch out to
834 # 'line' mode, do a single manual reset and then feed the lines one
834 # 'line' mode, do a single manual reset and then feed the lines one
835 # by one. Note that this only matters if the input has more than one
835 # by one. Note that this only matters if the input has more than one
836 # line.
836 # line.
837 changed_input_mode = False
837 changed_input_mode = False
838
838
839 if self.input_mode == 'cell':
839 if self.input_mode == 'cell':
840 self.reset()
840 self.reset()
841 changed_input_mode = True
841 changed_input_mode = True
842 saved_input_mode = 'cell'
842 saved_input_mode = 'cell'
843 self.input_mode = 'line'
843 self.input_mode = 'line'
844
844
845 # Store raw source before applying any transformations to it. Note
845 # Store raw source before applying any transformations to it. Note
846 # that this must be done *after* the reset() call that would otherwise
846 # that this must be done *after* the reset() call that would otherwise
847 # flush the buffer.
847 # flush the buffer.
848 self._store(lines, self._buffer_raw, 'source_raw')
848 self._store(lines, self._buffer_raw, 'source_raw')
849
849
850 try:
850 try:
851 push = super(IPythonInputSplitter, self).push
851 push = super(IPythonInputSplitter, self).push
852 for line in lines_list:
852 for line in lines_list:
853 if self._is_complete or not self._buffer or \
853 if self._is_complete or not self._buffer or \
854 (self._buffer and self._buffer[-1].rstrip().endswith(':')):
854 (self._buffer and self._buffer[-1].rstrip().endswith(':')):
855 for f in transforms:
855 for f in transforms:
856 line = f(line)
856 line = f(line)
857
857
858 out = push(line)
858 out = push(line)
859 finally:
859 finally:
860 if changed_input_mode:
860 if changed_input_mode:
861 self.input_mode = saved_input_mode
861 self.input_mode = saved_input_mode
862 return out
862 return out
@@ -1,664 +1,672 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """Tests for the inputsplitter module.
2 """Tests for the inputsplitter module.
3
3
4 Authors
4 Authors
5 -------
5 -------
6 * Fernando Perez
6 * Fernando Perez
7 * Robert Kern
7 * Robert Kern
8 """
8 """
9 #-----------------------------------------------------------------------------
9 #-----------------------------------------------------------------------------
10 # Copyright (C) 2010 The IPython Development Team
10 # Copyright (C) 2010 The IPython Development Team
11 #
11 #
12 # Distributed under the terms of the BSD License. The full license is in
12 # Distributed under the terms of the BSD License. The full license is in
13 # the file COPYING, distributed as part of this software.
13 # the file COPYING, distributed as part of this software.
14 #-----------------------------------------------------------------------------
14 #-----------------------------------------------------------------------------
15
15
16 #-----------------------------------------------------------------------------
16 #-----------------------------------------------------------------------------
17 # Imports
17 # Imports
18 #-----------------------------------------------------------------------------
18 #-----------------------------------------------------------------------------
19 # stdlib
19 # stdlib
20 import unittest
20 import unittest
21 import sys
21 import sys
22
22
23 # Third party
23 # Third party
24 import nose.tools as nt
24 import nose.tools as nt
25
25
26 # Our own
26 # Our own
27 from IPython.core import inputsplitter as isp
27 from IPython.core import inputsplitter as isp
28
28
29 #-----------------------------------------------------------------------------
29 #-----------------------------------------------------------------------------
30 # Semi-complete examples (also used as tests)
30 # Semi-complete examples (also used as tests)
31 #-----------------------------------------------------------------------------
31 #-----------------------------------------------------------------------------
32
32
33 # Note: at the bottom, there's a slightly more complete version of this that
33 # Note: at the bottom, there's a slightly more complete version of this that
34 # can be useful during development of code here.
34 # can be useful during development of code here.
35
35
36 def mini_interactive_loop(input_func):
36 def mini_interactive_loop(input_func):
37 """Minimal example of the logic of an interactive interpreter loop.
37 """Minimal example of the logic of an interactive interpreter loop.
38
38
39 This serves as an example, and it is used by the test system with a fake
39 This serves as an example, and it is used by the test system with a fake
40 raw_input that simulates interactive input."""
40 raw_input that simulates interactive input."""
41
41
42 from IPython.core.inputsplitter import InputSplitter
42 from IPython.core.inputsplitter import InputSplitter
43
43
44 isp = InputSplitter()
44 isp = InputSplitter()
45 # In practice, this input loop would be wrapped in an outside loop to read
45 # In practice, this input loop would be wrapped in an outside loop to read
46 # input indefinitely, until some exit/quit command was issued. Here we
46 # input indefinitely, until some exit/quit command was issued. Here we
47 # only illustrate the basic inner loop.
47 # only illustrate the basic inner loop.
48 while isp.push_accepts_more():
48 while isp.push_accepts_more():
49 indent = ' '*isp.indent_spaces
49 indent = ' '*isp.indent_spaces
50 prompt = '>>> ' + indent
50 prompt = '>>> ' + indent
51 line = indent + input_func(prompt)
51 line = indent + input_func(prompt)
52 isp.push(line)
52 isp.push(line)
53
53
54 # Here we just return input so we can use it in a test suite, but a real
54 # Here we just return input so we can use it in a test suite, but a real
55 # interpreter would instead send it for execution somewhere.
55 # interpreter would instead send it for execution somewhere.
56 src = isp.source_reset()
56 src = isp.source_reset()
57 #print 'Input source was:\n', src # dbg
57 #print 'Input source was:\n', src # dbg
58 return src
58 return src
59
59
60 #-----------------------------------------------------------------------------
60 #-----------------------------------------------------------------------------
61 # Test utilities, just for local use
61 # Test utilities, just for local use
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63
63
64 def assemble(block):
64 def assemble(block):
65 """Assemble a block into multi-line sub-blocks."""
65 """Assemble a block into multi-line sub-blocks."""
66 return ['\n'.join(sub_block)+'\n' for sub_block in block]
66 return ['\n'.join(sub_block)+'\n' for sub_block in block]
67
67
68
68
69 def pseudo_input(lines):
69 def pseudo_input(lines):
70 """Return a function that acts like raw_input but feeds the input list."""
70 """Return a function that acts like raw_input but feeds the input list."""
71 ilines = iter(lines)
71 ilines = iter(lines)
72 def raw_in(prompt):
72 def raw_in(prompt):
73 try:
73 try:
74 return next(ilines)
74 return next(ilines)
75 except StopIteration:
75 except StopIteration:
76 return ''
76 return ''
77 return raw_in
77 return raw_in
78
78
79 #-----------------------------------------------------------------------------
79 #-----------------------------------------------------------------------------
80 # Tests
80 # Tests
81 #-----------------------------------------------------------------------------
81 #-----------------------------------------------------------------------------
82 def test_spaces():
82 def test_spaces():
83 tests = [('', 0),
83 tests = [('', 0),
84 (' ', 1),
84 (' ', 1),
85 ('\n', 0),
85 ('\n', 0),
86 (' \n', 1),
86 (' \n', 1),
87 ('x', 0),
87 ('x', 0),
88 (' x', 1),
88 (' x', 1),
89 (' x',2),
89 (' x',2),
90 (' x',4),
90 (' x',4),
91 # Note: tabs are counted as a single whitespace!
91 # Note: tabs are counted as a single whitespace!
92 ('\tx', 1),
92 ('\tx', 1),
93 ('\t x', 2),
93 ('\t x', 2),
94 ]
94 ]
95
95
96 for s, nsp in tests:
96 for s, nsp in tests:
97 nt.assert_equal(isp.num_ini_spaces(s), nsp)
97 nt.assert_equal(isp.num_ini_spaces(s), nsp)
98
98
99
99
100 def test_remove_comments():
100 def test_remove_comments():
101 tests = [('text', 'text'),
101 tests = [('text', 'text'),
102 ('text # comment', 'text '),
102 ('text # comment', 'text '),
103 ('text # comment\n', 'text \n'),
103 ('text # comment\n', 'text \n'),
104 ('text # comment \n', 'text \n'),
104 ('text # comment \n', 'text \n'),
105 ('line # c \nline\n','line \nline\n'),
105 ('line # c \nline\n','line \nline\n'),
106 ('line # c \nline#c2 \nline\nline #c\n\n',
106 ('line # c \nline#c2 \nline\nline #c\n\n',
107 'line \nline\nline\nline \n\n'),
107 'line \nline\nline\nline \n\n'),
108 ]
108 ]
109
109
110 for inp, out in tests:
110 for inp, out in tests:
111 nt.assert_equal(isp.remove_comments(inp), out)
111 nt.assert_equal(isp.remove_comments(inp), out)
112
112
113
113
114 def test_get_input_encoding():
114 def test_get_input_encoding():
115 encoding = isp.get_input_encoding()
115 encoding = isp.get_input_encoding()
116 nt.assert_true(isinstance(encoding, basestring))
116 nt.assert_true(isinstance(encoding, basestring))
117 # simple-minded check that at least encoding a simple string works with the
117 # simple-minded check that at least encoding a simple string works with the
118 # encoding we got.
118 # encoding we got.
119 nt.assert_equal('test'.encode(encoding), 'test')
119 nt.assert_equal('test'.encode(encoding), 'test')
120
120
121
121
122 class NoInputEncodingTestCase(unittest.TestCase):
122 class NoInputEncodingTestCase(unittest.TestCase):
123 def setUp(self):
123 def setUp(self):
124 self.old_stdin = sys.stdin
124 self.old_stdin = sys.stdin
125 class X: pass
125 class X: pass
126 fake_stdin = X()
126 fake_stdin = X()
127 sys.stdin = fake_stdin
127 sys.stdin = fake_stdin
128
128
129 def test(self):
129 def test(self):
130 # Verify that if sys.stdin has no 'encoding' attribute we do the right
130 # Verify that if sys.stdin has no 'encoding' attribute we do the right
131 # thing
131 # thing
132 enc = isp.get_input_encoding()
132 enc = isp.get_input_encoding()
133 self.assertEqual(enc, 'ascii')
133 self.assertEqual(enc, 'ascii')
134
134
135 def tearDown(self):
135 def tearDown(self):
136 sys.stdin = self.old_stdin
136 sys.stdin = self.old_stdin
137
137
138
138
139 class InputSplitterTestCase(unittest.TestCase):
139 class InputSplitterTestCase(unittest.TestCase):
140 def setUp(self):
140 def setUp(self):
141 self.isp = isp.InputSplitter()
141 self.isp = isp.InputSplitter()
142
142
143 def test_reset(self):
143 def test_reset(self):
144 isp = self.isp
144 isp = self.isp
145 isp.push('x=1')
145 isp.push('x=1')
146 isp.reset()
146 isp.reset()
147 self.assertEqual(isp._buffer, [])
147 self.assertEqual(isp._buffer, [])
148 self.assertEqual(isp.indent_spaces, 0)
148 self.assertEqual(isp.indent_spaces, 0)
149 self.assertEqual(isp.source, '')
149 self.assertEqual(isp.source, '')
150 self.assertEqual(isp.code, None)
150 self.assertEqual(isp.code, None)
151 self.assertEqual(isp._is_complete, False)
151 self.assertEqual(isp._is_complete, False)
152
152
153 def test_source(self):
153 def test_source(self):
154 self.isp._store('1')
154 self.isp._store('1')
155 self.isp._store('2')
155 self.isp._store('2')
156 self.assertEqual(self.isp.source, '1\n2\n')
156 self.assertEqual(self.isp.source, '1\n2\n')
157 self.assertTrue(len(self.isp._buffer)>0)
157 self.assertTrue(len(self.isp._buffer)>0)
158 self.assertEqual(self.isp.source_reset(), '1\n2\n')
158 self.assertEqual(self.isp.source_reset(), '1\n2\n')
159 self.assertEqual(self.isp._buffer, [])
159 self.assertEqual(self.isp._buffer, [])
160 self.assertEqual(self.isp.source, '')
160 self.assertEqual(self.isp.source, '')
161
161
162 def test_indent(self):
162 def test_indent(self):
163 isp = self.isp # shorthand
163 isp = self.isp # shorthand
164 isp.push('x=1')
164 isp.push('x=1')
165 self.assertEqual(isp.indent_spaces, 0)
165 self.assertEqual(isp.indent_spaces, 0)
166 isp.push('if 1:\n x=1')
166 isp.push('if 1:\n x=1')
167 self.assertEqual(isp.indent_spaces, 4)
167 self.assertEqual(isp.indent_spaces, 4)
168 isp.push('y=2\n')
168 isp.push('y=2\n')
169 self.assertEqual(isp.indent_spaces, 0)
169 self.assertEqual(isp.indent_spaces, 0)
170
170
171 def test_indent2(self):
171 def test_indent2(self):
172 # In cell mode, inputs must be fed in whole blocks, so skip this test
172 # In cell mode, inputs must be fed in whole blocks, so skip this test
173 if self.isp.input_mode == 'cell': return
173 if self.isp.input_mode == 'cell': return
174
174
175 isp = self.isp
175 isp = self.isp
176 isp.push('if 1:')
176 isp.push('if 1:')
177 self.assertEqual(isp.indent_spaces, 4)
177 self.assertEqual(isp.indent_spaces, 4)
178 isp.push(' x=1')
178 isp.push(' x=1')
179 self.assertEqual(isp.indent_spaces, 4)
179 self.assertEqual(isp.indent_spaces, 4)
180 # Blank lines shouldn't change the indent level
180 # Blank lines shouldn't change the indent level
181 isp.push(' '*2)
181 isp.push(' '*2)
182 self.assertEqual(isp.indent_spaces, 4)
182 self.assertEqual(isp.indent_spaces, 4)
183
183
184 def test_indent3(self):
184 def test_indent3(self):
185 # In cell mode, inputs must be fed in whole blocks, so skip this test
185 # In cell mode, inputs must be fed in whole blocks, so skip this test
186 if self.isp.input_mode == 'cell': return
186 if self.isp.input_mode == 'cell': return
187
187
188 isp = self.isp
188 isp = self.isp
189 # When a multiline statement contains parens or multiline strings, we
189 # When a multiline statement contains parens or multiline strings, we
190 # shouldn't get confused.
190 # shouldn't get confused.
191 isp.push("if 1:")
191 isp.push("if 1:")
192 isp.push(" x = (1+\n 2)")
192 isp.push(" x = (1+\n 2)")
193 self.assertEqual(isp.indent_spaces, 4)
193 self.assertEqual(isp.indent_spaces, 4)
194
194
195 def test_dedent_pass(self):
195 def test_dedent_pass(self):
196 isp = self.isp # shorthand
196 isp = self.isp # shorthand
197 # should NOT cause dedent
197 # should NOT cause dedent
198 isp.push('if 1:\n passes = 5')
198 isp.push('if 1:\n passes = 5')
199 self.assertEqual(isp.indent_spaces, 4)
199 self.assertEqual(isp.indent_spaces, 4)
200 isp.push('if 1:\n pass')
200 isp.push('if 1:\n pass')
201 self.assertEqual(isp.indent_spaces, 0)
201 self.assertEqual(isp.indent_spaces, 0)
202 isp.push('if 1:\n pass ')
202 isp.push('if 1:\n pass ')
203 self.assertEqual(isp.indent_spaces, 0)
203 self.assertEqual(isp.indent_spaces, 0)
204
204
205 def test_dedent_raise(self):
205 def test_dedent_raise(self):
206 isp = self.isp # shorthand
206 isp = self.isp # shorthand
207 # should NOT cause dedent
207 # should NOT cause dedent
208 isp.push('if 1:\n raised = 4')
208 isp.push('if 1:\n raised = 4')
209 self.assertEqual(isp.indent_spaces, 4)
209 self.assertEqual(isp.indent_spaces, 4)
210 isp.push('if 1:\n raise TypeError()')
210 isp.push('if 1:\n raise TypeError()')
211 self.assertEqual(isp.indent_spaces, 0)
211 self.assertEqual(isp.indent_spaces, 0)
212 isp.push('if 1:\n raise')
212 isp.push('if 1:\n raise')
213 self.assertEqual(isp.indent_spaces, 0)
213 self.assertEqual(isp.indent_spaces, 0)
214 isp.push('if 1:\n raise ')
214 isp.push('if 1:\n raise ')
215 self.assertEqual(isp.indent_spaces, 0)
215 self.assertEqual(isp.indent_spaces, 0)
216
216
217 def test_dedent_return(self):
217 def test_dedent_return(self):
218 isp = self.isp # shorthand
218 isp = self.isp # shorthand
219 # should NOT cause dedent
219 # should NOT cause dedent
220 isp.push('if 1:\n returning = 4')
220 isp.push('if 1:\n returning = 4')
221 self.assertEqual(isp.indent_spaces, 4)
221 self.assertEqual(isp.indent_spaces, 4)
222 isp.push('if 1:\n return 5 + 493')
222 isp.push('if 1:\n return 5 + 493')
223 self.assertEqual(isp.indent_spaces, 0)
223 self.assertEqual(isp.indent_spaces, 0)
224 isp.push('if 1:\n return')
224 isp.push('if 1:\n return')
225 self.assertEqual(isp.indent_spaces, 0)
225 self.assertEqual(isp.indent_spaces, 0)
226 isp.push('if 1:\n return ')
226 isp.push('if 1:\n return ')
227 self.assertEqual(isp.indent_spaces, 0)
227 self.assertEqual(isp.indent_spaces, 0)
228 isp.push('if 1:\n return(0)')
228 isp.push('if 1:\n return(0)')
229 self.assertEqual(isp.indent_spaces, 0)
229 self.assertEqual(isp.indent_spaces, 0)
230
230
231 def test_push(self):
231 def test_push(self):
232 isp = self.isp
232 isp = self.isp
233 self.assertTrue(isp.push('x=1'))
233 self.assertTrue(isp.push('x=1'))
234
234
235 def test_push2(self):
235 def test_push2(self):
236 isp = self.isp
236 isp = self.isp
237 self.assertFalse(isp.push('if 1:'))
237 self.assertFalse(isp.push('if 1:'))
238 for line in [' x=1', '# a comment', ' y=2']:
238 for line in [' x=1', '# a comment', ' y=2']:
239 self.assertTrue(isp.push(line))
239 self.assertTrue(isp.push(line))
240
240
241 def test_push3(self):
241 def test_push3(self):
242 isp = self.isp
242 isp = self.isp
243 isp.push('if True:')
243 isp.push('if True:')
244 isp.push(' a = 1')
244 isp.push(' a = 1')
245 self.assertFalse(isp.push('b = [1,'))
245 self.assertFalse(isp.push('b = [1,'))
246
246
247 def test_replace_mode(self):
247 def test_replace_mode(self):
248 isp = self.isp
248 isp = self.isp
249 isp.input_mode = 'cell'
249 isp.input_mode = 'cell'
250 isp.push('x=1')
250 isp.push('x=1')
251 self.assertEqual(isp.source, 'x=1\n')
251 self.assertEqual(isp.source, 'x=1\n')
252 isp.push('x=2')
252 isp.push('x=2')
253 self.assertEqual(isp.source, 'x=2\n')
253 self.assertEqual(isp.source, 'x=2\n')
254
254
255 def test_push_accepts_more(self):
255 def test_push_accepts_more(self):
256 isp = self.isp
256 isp = self.isp
257 isp.push('x=1')
257 isp.push('x=1')
258 self.assertFalse(isp.push_accepts_more())
258 self.assertFalse(isp.push_accepts_more())
259
259
260 def test_push_accepts_more2(self):
260 def test_push_accepts_more2(self):
261 # In cell mode, inputs must be fed in whole blocks, so skip this test
261 # In cell mode, inputs must be fed in whole blocks, so skip this test
262 if self.isp.input_mode == 'cell': return
262 if self.isp.input_mode == 'cell': return
263
263
264 isp = self.isp
264 isp = self.isp
265 isp.push('if 1:')
265 isp.push('if 1:')
266 self.assertTrue(isp.push_accepts_more())
266 self.assertTrue(isp.push_accepts_more())
267 isp.push(' x=1')
267 isp.push(' x=1')
268 self.assertTrue(isp.push_accepts_more())
268 self.assertTrue(isp.push_accepts_more())
269 isp.push('')
269 isp.push('')
270 self.assertFalse(isp.push_accepts_more())
270 self.assertFalse(isp.push_accepts_more())
271
271
272 def test_push_accepts_more3(self):
272 def test_push_accepts_more3(self):
273 isp = self.isp
273 isp = self.isp
274 isp.push("x = (2+\n3)")
274 isp.push("x = (2+\n3)")
275 self.assertFalse(isp.push_accepts_more())
275 self.assertFalse(isp.push_accepts_more())
276
276
277 def test_push_accepts_more4(self):
277 def test_push_accepts_more4(self):
278 # In cell mode, inputs must be fed in whole blocks, so skip this test
278 # In cell mode, inputs must be fed in whole blocks, so skip this test
279 if self.isp.input_mode == 'cell': return
279 if self.isp.input_mode == 'cell': return
280
280
281 isp = self.isp
281 isp = self.isp
282 # When a multiline statement contains parens or multiline strings, we
282 # When a multiline statement contains parens or multiline strings, we
283 # shouldn't get confused.
283 # shouldn't get confused.
284 # FIXME: we should be able to better handle de-dents in statements like
284 # FIXME: we should be able to better handle de-dents in statements like
285 # multiline strings and multiline expressions (continued with \ or
285 # multiline strings and multiline expressions (continued with \ or
286 # parens). Right now we aren't handling the indentation tracking quite
286 # parens). Right now we aren't handling the indentation tracking quite
287 # correctly with this, though in practice it may not be too much of a
287 # correctly with this, though in practice it may not be too much of a
288 # problem. We'll need to see.
288 # problem. We'll need to see.
289 isp.push("if 1:")
289 isp.push("if 1:")
290 isp.push(" x = (2+")
290 isp.push(" x = (2+")
291 isp.push(" 3)")
291 isp.push(" 3)")
292 self.assertTrue(isp.push_accepts_more())
292 self.assertTrue(isp.push_accepts_more())
293 isp.push(" y = 3")
293 isp.push(" y = 3")
294 self.assertTrue(isp.push_accepts_more())
294 self.assertTrue(isp.push_accepts_more())
295 isp.push('')
295 isp.push('')
296 self.assertFalse(isp.push_accepts_more())
296 self.assertFalse(isp.push_accepts_more())
297
297
298 def test_push_accepts_more5(self):
298 def test_push_accepts_more5(self):
299 # In cell mode, inputs must be fed in whole blocks, so skip this test
299 # In cell mode, inputs must be fed in whole blocks, so skip this test
300 if self.isp.input_mode == 'cell': return
300 if self.isp.input_mode == 'cell': return
301
301
302 isp = self.isp
302 isp = self.isp
303 isp.push('try:')
303 isp.push('try:')
304 isp.push(' a = 5')
304 isp.push(' a = 5')
305 isp.push('except:')
305 isp.push('except:')
306 isp.push(' raise')
306 isp.push(' raise')
307 self.assertTrue(isp.push_accepts_more())
307 self.assertTrue(isp.push_accepts_more())
308
308
309 def test_continuation(self):
309 def test_continuation(self):
310 isp = self.isp
310 isp = self.isp
311 isp.push("import os, \\")
311 isp.push("import os, \\")
312 self.assertTrue(isp.push_accepts_more())
312 self.assertTrue(isp.push_accepts_more())
313 isp.push("sys")
313 isp.push("sys")
314 self.assertFalse(isp.push_accepts_more())
314 self.assertFalse(isp.push_accepts_more())
315
315
316 def test_syntax_error(self):
316 def test_syntax_error(self):
317 isp = self.isp
317 isp = self.isp
318 # Syntax errors immediately produce a 'ready' block, so the invalid
318 # Syntax errors immediately produce a 'ready' block, so the invalid
319 # Python can be sent to the kernel for evaluation with possible ipython
319 # Python can be sent to the kernel for evaluation with possible ipython
320 # special-syntax conversion.
320 # special-syntax conversion.
321 isp.push('run foo')
321 isp.push('run foo')
322 self.assertFalse(isp.push_accepts_more())
322 self.assertFalse(isp.push_accepts_more())
323
323
324 def test_unicode(self):
324 def test_unicode(self):
325 self.isp.push(u"PΓ©rez")
325 self.isp.push(u"PΓ©rez")
326 self.isp.push(u'\xc3\xa9')
326 self.isp.push(u'\xc3\xa9')
327 self.isp.push(u"u'\xc3\xa9'")
327 self.isp.push(u"u'\xc3\xa9'")
328
328
329 class InteractiveLoopTestCase(unittest.TestCase):
329 class InteractiveLoopTestCase(unittest.TestCase):
330 """Tests for an interactive loop like a python shell.
330 """Tests for an interactive loop like a python shell.
331 """
331 """
332 def check_ns(self, lines, ns):
332 def check_ns(self, lines, ns):
333 """Validate that the given input lines produce the resulting namespace.
333 """Validate that the given input lines produce the resulting namespace.
334
334
335 Note: the input lines are given exactly as they would be typed in an
335 Note: the input lines are given exactly as they would be typed in an
336 auto-indenting environment, as mini_interactive_loop above already does
336 auto-indenting environment, as mini_interactive_loop above already does
337 auto-indenting and prepends spaces to the input.
337 auto-indenting and prepends spaces to the input.
338 """
338 """
339 src = mini_interactive_loop(pseudo_input(lines))
339 src = mini_interactive_loop(pseudo_input(lines))
340 test_ns = {}
340 test_ns = {}
341 exec src in test_ns
341 exec src in test_ns
342 # We can't check that the provided ns is identical to the test_ns,
342 # We can't check that the provided ns is identical to the test_ns,
343 # because Python fills test_ns with extra keys (copyright, etc). But
343 # because Python fills test_ns with extra keys (copyright, etc). But
344 # we can check that the given dict is *contained* in test_ns
344 # we can check that the given dict is *contained* in test_ns
345 for k,v in ns.iteritems():
345 for k,v in ns.iteritems():
346 self.assertEqual(test_ns[k], v)
346 self.assertEqual(test_ns[k], v)
347
347
348 def test_simple(self):
348 def test_simple(self):
349 self.check_ns(['x=1'], dict(x=1))
349 self.check_ns(['x=1'], dict(x=1))
350
350
351 def test_simple2(self):
351 def test_simple2(self):
352 self.check_ns(['if 1:', 'x=2'], dict(x=2))
352 self.check_ns(['if 1:', 'x=2'], dict(x=2))
353
353
354 def test_xy(self):
354 def test_xy(self):
355 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
355 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
356
356
357 def test_abc(self):
357 def test_abc(self):
358 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
358 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
359
359
360 def test_multi(self):
360 def test_multi(self):
361 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
361 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
362
362
363
363
364 def test_LineInfo():
364 def test_LineInfo():
365 """Simple test for LineInfo construction and str()"""
365 """Simple test for LineInfo construction and str()"""
366 linfo = isp.LineInfo(' %cd /home')
366 linfo = isp.LineInfo(' %cd /home')
367 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
367 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
368
368
369
369
370 def test_split_user_input():
370 def test_split_user_input():
371 """Unicode test - split_user_input already has good doctests"""
371 """Unicode test - split_user_input already has good doctests"""
372 line = u"PΓ©rez Fernando"
372 line = u"PΓ©rez Fernando"
373 parts = isp.split_user_input(line)
373 parts = isp.split_user_input(line)
374 parts_expected = (u'', u'', u'', line)
374 parts_expected = (u'', u'', u'', line)
375 nt.assert_equal(parts, parts_expected)
375 nt.assert_equal(parts, parts_expected)
376
376
377
377
378 # Transformer tests
378 # Transformer tests
379 def transform_checker(tests, func):
379 def transform_checker(tests, func):
380 """Utility to loop over test inputs"""
380 """Utility to loop over test inputs"""
381 for inp, tr in tests:
381 for inp, tr in tests:
382 nt.assert_equals(func(inp), tr)
382 nt.assert_equals(func(inp), tr)
383
383
384 # Data for all the syntax tests in the form of lists of pairs of
384 # Data for all the syntax tests in the form of lists of pairs of
385 # raw/transformed input. We store it here as a global dict so that we can use
385 # raw/transformed input. We store it here as a global dict so that we can use
386 # it both within single-function tests and also to validate the behavior of the
386 # it both within single-function tests and also to validate the behavior of the
387 # larger objects
387 # larger objects
388
388
389 syntax = \
389 syntax = \
390 dict(assign_system =
390 dict(assign_system =
391 [('a =! ls', 'a = get_ipython().getoutput(u"ls")'),
391 [('a =! ls', 'a = get_ipython().getoutput(u"ls")'),
392 ('b = !ls', 'b = get_ipython().getoutput(u"ls")'),
392 ('b = !ls', 'b = get_ipython().getoutput(u"ls")'),
393 ('x=1', 'x=1'), # normal input is unmodified
393 ('x=1', 'x=1'), # normal input is unmodified
394 (' ',' '), # blank lines are kept intact
394 (' ',' '), # blank lines are kept intact
395 ],
395 ],
396
396
397 assign_magic =
397 assign_magic =
398 [('a =% who', 'a = get_ipython().magic(u"who")'),
398 [('a =% who', 'a = get_ipython().magic(u"who")'),
399 ('b = %who', 'b = get_ipython().magic(u"who")'),
399 ('b = %who', 'b = get_ipython().magic(u"who")'),
400 ('x=1', 'x=1'), # normal input is unmodified
400 ('x=1', 'x=1'), # normal input is unmodified
401 (' ',' '), # blank lines are kept intact
401 (' ',' '), # blank lines are kept intact
402 ],
402 ],
403
403
404 classic_prompt =
404 classic_prompt =
405 [('>>> x=1', 'x=1'),
405 [('>>> x=1', 'x=1'),
406 ('x=1', 'x=1'), # normal input is unmodified
406 ('x=1', 'x=1'), # normal input is unmodified
407 (' ', ' '), # blank lines are kept intact
407 (' ', ' '), # blank lines are kept intact
408 ('... ', ''), # continuation prompts
408 ('... ', ''), # continuation prompts
409 ],
409 ],
410
410
411 ipy_prompt =
411 ipy_prompt =
412 [('In [1]: x=1', 'x=1'),
412 [('In [1]: x=1', 'x=1'),
413 ('x=1', 'x=1'), # normal input is unmodified
413 ('x=1', 'x=1'), # normal input is unmodified
414 (' ',' '), # blank lines are kept intact
414 (' ',' '), # blank lines are kept intact
415 (' ....: ', ''), # continuation prompts
415 (' ....: ', ''), # continuation prompts
416 ],
416 ],
417
417
418 # Tests for the escape transformer to leave normal code alone
418 # Tests for the escape transformer to leave normal code alone
419 escaped_noesc =
419 escaped_noesc =
420 [ (' ', ' '),
420 [ (' ', ' '),
421 ('x=1', 'x=1'),
421 ('x=1', 'x=1'),
422 ],
422 ],
423
423
424 # System calls
424 # System calls
425 escaped_shell =
425 escaped_shell =
426 [ ('!ls', 'get_ipython().system(u"ls")'),
426 [ ('!ls', 'get_ipython().system(u"ls")'),
427 # Double-escape shell, this means to capture the output of the
427 # Double-escape shell, this means to capture the output of the
428 # subprocess and return it
428 # subprocess and return it
429 ('!!ls', 'get_ipython().getoutput(u"ls")'),
429 ('!!ls', 'get_ipython().getoutput(u"ls")'),
430 ],
430 ],
431
431
432 # Help/object info
432 # Help/object info
433 escaped_help =
433 escaped_help =
434 [ ('?', 'get_ipython().show_usage()'),
434 [ ('?', 'get_ipython().show_usage()'),
435 ('?x1', 'get_ipython().magic(u"pinfo x1")'),
435 ('?x1', 'get_ipython().magic(u"pinfo x1")'),
436 ('??x2', 'get_ipython().magic(u"pinfo2 x2")'),
436 ('??x2', 'get_ipython().magic(u"pinfo2 x2")'),
437 ('?a.*s', 'get_ipython().magic(u"psearch a.*s")'),
437 ('?a.*s', 'get_ipython().magic(u"psearch a.*s")'),
438 ('?%hist', 'get_ipython().magic(u"pinfo %hist")'),
438 ('?%hist', 'get_ipython().magic(u"pinfo %hist")'),
439 ('?abc = qwe', 'get_ipython().magic(u"pinfo abc")'),
439 ('?abc = qwe', 'get_ipython().magic(u"pinfo abc")'),
440 ],
440 ],
441
441
442 end_help =
442 end_help =
443 [ ('x3?', 'get_ipython().magic(u"pinfo x3")'),
443 [ ('x3?', 'get_ipython().magic(u"pinfo x3")'),
444 ('x4??', 'get_ipython().magic(u"pinfo2 x4")'),
444 ('x4??', 'get_ipython().magic(u"pinfo2 x4")'),
445 ('%hist?', 'get_ipython().magic(u"pinfo %hist")'),
445 ('%hist?', 'get_ipython().magic(u"pinfo %hist")'),
446 ('f*?', 'get_ipython().magic(u"psearch f*")'),
446 ('f*?', 'get_ipython().magic(u"psearch f*")'),
447 ('ax.*aspe*?', 'get_ipython().magic(u"psearch ax.*aspe*")'),
447 ('ax.*aspe*?', 'get_ipython().magic(u"psearch ax.*aspe*")'),
448 ('a = abc?', 'get_ipython().magic(u"pinfo abc")'),
448 ('a = abc?', 'get_ipython().magic(u"pinfo abc")'),
449 ('a = abc.qe??', 'get_ipython().magic(u"pinfo2 abc.qe")'),
449 ('a = abc.qe??', 'get_ipython().magic(u"pinfo2 abc.qe")'),
450 ('a = *.items?', 'get_ipython().magic(u"psearch *.items")'),
450 ('a = *.items?', 'get_ipython().magic(u"psearch *.items")'),
451 ],
451 ],
452
452
453 # Explicit magic calls
453 # Explicit magic calls
454 escaped_magic =
454 escaped_magic =
455 [ ('%cd', 'get_ipython().magic(u"cd")'),
455 [ ('%cd', 'get_ipython().magic(u"cd")'),
456 ('%cd /home', 'get_ipython().magic(u"cd /home")'),
456 ('%cd /home', 'get_ipython().magic(u"cd /home")'),
457 (' %magic', ' get_ipython().magic(u"magic")'),
457 (' %magic', ' get_ipython().magic(u"magic")'),
458 ],
458 ],
459
459
460 # Quoting with separate arguments
460 # Quoting with separate arguments
461 escaped_quote =
461 escaped_quote =
462 [ (',f', 'f("")'),
462 [ (',f', 'f("")'),
463 (',f x', 'f("x")'),
463 (',f x', 'f("x")'),
464 (' ,f y', ' f("y")'),
464 (' ,f y', ' f("y")'),
465 (',f a b', 'f("a", "b")'),
465 (',f a b', 'f("a", "b")'),
466 ],
466 ],
467
467
468 # Quoting with single argument
468 # Quoting with single argument
469 escaped_quote2 =
469 escaped_quote2 =
470 [ (';f', 'f("")'),
470 [ (';f', 'f("")'),
471 (';f x', 'f("x")'),
471 (';f x', 'f("x")'),
472 (' ;f y', ' f("y")'),
472 (' ;f y', ' f("y")'),
473 (';f a b', 'f("a b")'),
473 (';f a b', 'f("a b")'),
474 ],
474 ],
475
475
476 # Simply apply parens
476 # Simply apply parens
477 escaped_paren =
477 escaped_paren =
478 [ ('/f', 'f()'),
478 [ ('/f', 'f()'),
479 ('/f x', 'f(x)'),
479 ('/f x', 'f(x)'),
480 (' /f y', ' f(y)'),
480 (' /f y', ' f(y)'),
481 ('/f a b', 'f(a, b)'),
481 ('/f a b', 'f(a, b)'),
482 ],
482 ],
483
483
484 # Check that we transform prompts before other transforms
485 mixed =
486 [ ('In [1]: %lsmagic', 'get_ipython().magic(u"lsmagic")'),
487 ('>>> %lsmagic', 'get_ipython().magic(u"lsmagic")'),
488 ('In [2]: !ls', 'get_ipython().system(u"ls")'),
489 ('In [3]: abs?', 'get_ipython().magic(u"pinfo abs")'),
490 ('In [4]: b = %who', 'b = get_ipython().magic(u"who")'),
491 ],
484 )
492 )
485
493
486 # multiline syntax examples. Each of these should be a list of lists, with
494 # multiline syntax examples. Each of these should be a list of lists, with
487 # each entry itself having pairs of raw/transformed input. The union (with
495 # each entry itself having pairs of raw/transformed input. The union (with
488 # '\n'.join() of the transformed inputs is what the splitter should produce
496 # '\n'.join() of the transformed inputs is what the splitter should produce
489 # when fed the raw lines one at a time via push.
497 # when fed the raw lines one at a time via push.
490 syntax_ml = \
498 syntax_ml = \
491 dict(classic_prompt =
499 dict(classic_prompt =
492 [ [('>>> for i in range(10):','for i in range(10):'),
500 [ [('>>> for i in range(10):','for i in range(10):'),
493 ('... print i',' print i'),
501 ('... print i',' print i'),
494 ('... ', ''),
502 ('... ', ''),
495 ],
503 ],
496 ],
504 ],
497
505
498 ipy_prompt =
506 ipy_prompt =
499 [ [('In [24]: for i in range(10):','for i in range(10):'),
507 [ [('In [24]: for i in range(10):','for i in range(10):'),
500 (' ....: print i',' print i'),
508 (' ....: print i',' print i'),
501 (' ....: ', ''),
509 (' ....: ', ''),
502 ],
510 ],
503 ],
511 ],
504 )
512 )
505
513
506
514
507 def test_assign_system():
515 def test_assign_system():
508 transform_checker(syntax['assign_system'], isp.transform_assign_system)
516 transform_checker(syntax['assign_system'], isp.transform_assign_system)
509
517
510
518
511 def test_assign_magic():
519 def test_assign_magic():
512 transform_checker(syntax['assign_magic'], isp.transform_assign_magic)
520 transform_checker(syntax['assign_magic'], isp.transform_assign_magic)
513
521
514
522
515 def test_classic_prompt():
523 def test_classic_prompt():
516 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
524 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
517 for example in syntax_ml['classic_prompt']:
525 for example in syntax_ml['classic_prompt']:
518 transform_checker(example, isp.transform_classic_prompt)
526 transform_checker(example, isp.transform_classic_prompt)
519
527
520
528
521 def test_ipy_prompt():
529 def test_ipy_prompt():
522 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
530 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
523 for example in syntax_ml['ipy_prompt']:
531 for example in syntax_ml['ipy_prompt']:
524 transform_checker(example, isp.transform_ipy_prompt)
532 transform_checker(example, isp.transform_ipy_prompt)
525
533
526 def test_end_help():
534 def test_end_help():
527 transform_checker(syntax['end_help'], isp.transform_help_end)
535 transform_checker(syntax['end_help'], isp.transform_help_end)
528
536
529 def test_escaped_noesc():
537 def test_escaped_noesc():
530 transform_checker(syntax['escaped_noesc'], isp.transform_escaped)
538 transform_checker(syntax['escaped_noesc'], isp.transform_escaped)
531
539
532
540
533 def test_escaped_shell():
541 def test_escaped_shell():
534 transform_checker(syntax['escaped_shell'], isp.transform_escaped)
542 transform_checker(syntax['escaped_shell'], isp.transform_escaped)
535
543
536
544
537 def test_escaped_help():
545 def test_escaped_help():
538 transform_checker(syntax['escaped_help'], isp.transform_escaped)
546 transform_checker(syntax['escaped_help'], isp.transform_escaped)
539
547
540
548
541 def test_escaped_magic():
549 def test_escaped_magic():
542 transform_checker(syntax['escaped_magic'], isp.transform_escaped)
550 transform_checker(syntax['escaped_magic'], isp.transform_escaped)
543
551
544
552
545 def test_escaped_quote():
553 def test_escaped_quote():
546 transform_checker(syntax['escaped_quote'], isp.transform_escaped)
554 transform_checker(syntax['escaped_quote'], isp.transform_escaped)
547
555
548
556
549 def test_escaped_quote2():
557 def test_escaped_quote2():
550 transform_checker(syntax['escaped_quote2'], isp.transform_escaped)
558 transform_checker(syntax['escaped_quote2'], isp.transform_escaped)
551
559
552
560
553 def test_escaped_paren():
561 def test_escaped_paren():
554 transform_checker(syntax['escaped_paren'], isp.transform_escaped)
562 transform_checker(syntax['escaped_paren'], isp.transform_escaped)
555
563
556
564
557 class IPythonInputTestCase(InputSplitterTestCase):
565 class IPythonInputTestCase(InputSplitterTestCase):
558 """By just creating a new class whose .isp is a different instance, we
566 """By just creating a new class whose .isp is a different instance, we
559 re-run the same test battery on the new input splitter.
567 re-run the same test battery on the new input splitter.
560
568
561 In addition, this runs the tests over the syntax and syntax_ml dicts that
569 In addition, this runs the tests over the syntax and syntax_ml dicts that
562 were tested by individual functions, as part of the OO interface.
570 were tested by individual functions, as part of the OO interface.
563
571
564 It also makes some checks on the raw buffer storage.
572 It also makes some checks on the raw buffer storage.
565 """
573 """
566
574
567 def setUp(self):
575 def setUp(self):
568 self.isp = isp.IPythonInputSplitter(input_mode='line')
576 self.isp = isp.IPythonInputSplitter(input_mode='line')
569
577
570 def test_syntax(self):
578 def test_syntax(self):
571 """Call all single-line syntax tests from the main object"""
579 """Call all single-line syntax tests from the main object"""
572 isp = self.isp
580 isp = self.isp
573 for example in syntax.itervalues():
581 for example in syntax.itervalues():
574 for raw, out_t in example:
582 for raw, out_t in example:
575 if raw.startswith(' '):
583 if raw.startswith(' '):
576 continue
584 continue
577
585
578 isp.push(raw)
586 isp.push(raw)
579 out, out_raw = isp.source_raw_reset()
587 out, out_raw = isp.source_raw_reset()
580 self.assertEqual(out.rstrip(), out_t)
588 self.assertEqual(out.rstrip(), out_t)
581 self.assertEqual(out_raw.rstrip(), raw.rstrip())
589 self.assertEqual(out_raw.rstrip(), raw.rstrip())
582
590
583 def test_syntax_multiline(self):
591 def test_syntax_multiline(self):
584 isp = self.isp
592 isp = self.isp
585 for example in syntax_ml.itervalues():
593 for example in syntax_ml.itervalues():
586 out_t_parts = []
594 out_t_parts = []
587 raw_parts = []
595 raw_parts = []
588 for line_pairs in example:
596 for line_pairs in example:
589 for lraw, out_t_part in line_pairs:
597 for lraw, out_t_part in line_pairs:
590 isp.push(lraw)
598 isp.push(lraw)
591 out_t_parts.append(out_t_part)
599 out_t_parts.append(out_t_part)
592 raw_parts.append(lraw)
600 raw_parts.append(lraw)
593
601
594 out, out_raw = isp.source_raw_reset()
602 out, out_raw = isp.source_raw_reset()
595 out_t = '\n'.join(out_t_parts).rstrip()
603 out_t = '\n'.join(out_t_parts).rstrip()
596 raw = '\n'.join(raw_parts).rstrip()
604 raw = '\n'.join(raw_parts).rstrip()
597 self.assertEqual(out.rstrip(), out_t)
605 self.assertEqual(out.rstrip(), out_t)
598 self.assertEqual(out_raw.rstrip(), raw)
606 self.assertEqual(out_raw.rstrip(), raw)
599
607
600
608
601 class BlockIPythonInputTestCase(IPythonInputTestCase):
609 class BlockIPythonInputTestCase(IPythonInputTestCase):
602
610
603 # Deactivate tests that don't make sense for the block mode
611 # Deactivate tests that don't make sense for the block mode
604 test_push3 = test_split = lambda s: None
612 test_push3 = test_split = lambda s: None
605
613
606 def setUp(self):
614 def setUp(self):
607 self.isp = isp.IPythonInputSplitter(input_mode='cell')
615 self.isp = isp.IPythonInputSplitter(input_mode='cell')
608
616
609 def test_syntax_multiline(self):
617 def test_syntax_multiline(self):
610 isp = self.isp
618 isp = self.isp
611 for example in syntax_ml.itervalues():
619 for example in syntax_ml.itervalues():
612 raw_parts = []
620 raw_parts = []
613 out_t_parts = []
621 out_t_parts = []
614 for line_pairs in example:
622 for line_pairs in example:
615 for raw, out_t_part in line_pairs:
623 for raw, out_t_part in line_pairs:
616 raw_parts.append(raw)
624 raw_parts.append(raw)
617 out_t_parts.append(out_t_part)
625 out_t_parts.append(out_t_part)
618
626
619 raw = '\n'.join(raw_parts)
627 raw = '\n'.join(raw_parts)
620 out_t = '\n'.join(out_t_parts)
628 out_t = '\n'.join(out_t_parts)
621
629
622 isp.push(raw)
630 isp.push(raw)
623 out, out_raw = isp.source_raw_reset()
631 out, out_raw = isp.source_raw_reset()
624 # Match ignoring trailing whitespace
632 # Match ignoring trailing whitespace
625 self.assertEqual(out.rstrip(), out_t.rstrip())
633 self.assertEqual(out.rstrip(), out_t.rstrip())
626 self.assertEqual(out_raw.rstrip(), raw.rstrip())
634 self.assertEqual(out_raw.rstrip(), raw.rstrip())
627
635
628
636
629 #-----------------------------------------------------------------------------
637 #-----------------------------------------------------------------------------
630 # Main - use as a script, mostly for developer experiments
638 # Main - use as a script, mostly for developer experiments
631 #-----------------------------------------------------------------------------
639 #-----------------------------------------------------------------------------
632
640
633 if __name__ == '__main__':
641 if __name__ == '__main__':
634 # A simple demo for interactive experimentation. This code will not get
642 # A simple demo for interactive experimentation. This code will not get
635 # picked up by any test suite.
643 # picked up by any test suite.
636 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
644 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
637
645
638 # configure here the syntax to use, prompt and whether to autoindent
646 # configure here the syntax to use, prompt and whether to autoindent
639 #isp, start_prompt = InputSplitter(), '>>> '
647 #isp, start_prompt = InputSplitter(), '>>> '
640 isp, start_prompt = IPythonInputSplitter(), 'In> '
648 isp, start_prompt = IPythonInputSplitter(), 'In> '
641
649
642 autoindent = True
650 autoindent = True
643 #autoindent = False
651 #autoindent = False
644
652
645 try:
653 try:
646 while True:
654 while True:
647 prompt = start_prompt
655 prompt = start_prompt
648 while isp.push_accepts_more():
656 while isp.push_accepts_more():
649 indent = ' '*isp.indent_spaces
657 indent = ' '*isp.indent_spaces
650 if autoindent:
658 if autoindent:
651 line = indent + raw_input(prompt+indent)
659 line = indent + raw_input(prompt+indent)
652 else:
660 else:
653 line = raw_input(prompt)
661 line = raw_input(prompt)
654 isp.push(line)
662 isp.push(line)
655 prompt = '... '
663 prompt = '... '
656
664
657 # Here we just return input so we can use it in a test suite, but a
665 # Here we just return input so we can use it in a test suite, but a
658 # real interpreter would instead send it for execution somewhere.
666 # real interpreter would instead send it for execution somewhere.
659 #src = isp.source; raise EOFError # dbg
667 #src = isp.source; raise EOFError # dbg
660 src, raw = isp.source_raw_reset()
668 src, raw = isp.source_raw_reset()
661 print 'Input source was:\n', src
669 print 'Input source was:\n', src
662 print 'Raw source was:\n', raw
670 print 'Raw source was:\n', raw
663 except EOFError:
671 except EOFError:
664 print 'Bye'
672 print 'Bye'
General Comments 0
You need to be logged in to leave comments. Login now