##// END OF EJS Templates
Add a few more fixes to cell/line input code, switch approaches....
Fernando Perez -
Show More
@@ -1,993 +1,1004 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 from __future__ import print_function
63 from __future__ import print_function
64
64
65 #-----------------------------------------------------------------------------
65 #-----------------------------------------------------------------------------
66 # Imports
66 # Imports
67 #-----------------------------------------------------------------------------
67 #-----------------------------------------------------------------------------
68 # stdlib
68 # stdlib
69 import ast
69 import ast
70 import codeop
70 import codeop
71 import re
71 import re
72 import sys
72 import sys
73 import tokenize
73 import tokenize
74 from StringIO import StringIO
74 from StringIO import StringIO
75
75
76 # IPython modules
76 # IPython modules
77 from IPython.core.splitinput import split_user_input, LineInfo
77 from IPython.core.splitinput import split_user_input, LineInfo
78 from IPython.utils.py3compat import cast_unicode
78 from IPython.utils.py3compat import cast_unicode
79
79
80 #-----------------------------------------------------------------------------
80 #-----------------------------------------------------------------------------
81 # Globals
81 # Globals
82 #-----------------------------------------------------------------------------
82 #-----------------------------------------------------------------------------
83
83
84 # The escape sequences that define the syntax transformations IPython will
84 # The escape sequences that define the syntax transformations IPython will
85 # apply to user input. These can NOT be just changed here: many regular
85 # apply to user input. These can NOT be just changed here: many regular
86 # expressions and other parts of the code may use their hardcoded values, and
86 # expressions and other parts of the code may use their hardcoded values, and
87 # for all intents and purposes they constitute the 'IPython syntax', so they
87 # for all intents and purposes they constitute the 'IPython syntax', so they
88 # should be considered fixed.
88 # should be considered fixed.
89
89
90 ESC_SHELL = '!' # Send line to underlying system shell
90 ESC_SHELL = '!' # Send line to underlying system shell
91 ESC_SH_CAP = '!!' # Send line to system shell and capture output
91 ESC_SH_CAP = '!!' # Send line to system shell and capture output
92 ESC_HELP = '?' # Find information about object
92 ESC_HELP = '?' # Find information about object
93 ESC_HELP2 = '??' # Find extra-detailed information about object
93 ESC_HELP2 = '??' # Find extra-detailed information about object
94 ESC_MAGIC = '%' # Call magic function
94 ESC_MAGIC = '%' # Call magic function
95 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
95 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
96 ESC_QUOTE2 = ';' # Quote all args as a single string, call
96 ESC_QUOTE2 = ';' # Quote all args as a single string, call
97 ESC_PAREN = '/' # Call first argument with rest of line as arguments
97 ESC_PAREN = '/' # Call first argument with rest of line as arguments
98
98
99 #-----------------------------------------------------------------------------
99 #-----------------------------------------------------------------------------
100 # Utilities
100 # Utilities
101 #-----------------------------------------------------------------------------
101 #-----------------------------------------------------------------------------
102
102
103 # FIXME: These are general-purpose utilities that later can be moved to the
103 # FIXME: These are general-purpose utilities that later can be moved to the
104 # general ward. Kept here for now because we're being very strict about test
104 # general ward. Kept here for now because we're being very strict about test
105 # coverage with this code, and this lets us ensure that we keep 100% coverage
105 # coverage with this code, and this lets us ensure that we keep 100% coverage
106 # while developing.
106 # while developing.
107
107
108 # compiled regexps for autoindent management
108 # compiled regexps for autoindent management
109 dedent_re = re.compile('|'.join([
109 dedent_re = re.compile('|'.join([
110 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
110 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
111 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
111 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
112 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
112 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
113 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
113 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
114 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
114 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
115 ]))
115 ]))
116 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
116 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
117
117
118 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
118 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
119 # before pure comments
119 # before pure comments
120 comment_line_re = re.compile('^\s*\#')
120 comment_line_re = re.compile('^\s*\#')
121
121
122
122
123 def num_ini_spaces(s):
123 def num_ini_spaces(s):
124 """Return the number of initial spaces in a string.
124 """Return the number of initial spaces in a string.
125
125
126 Note that tabs are counted as a single space. For now, we do *not* support
126 Note that tabs are counted as a single space. For now, we do *not* support
127 mixing of tabs and spaces in the user's input.
127 mixing of tabs and spaces in the user's input.
128
128
129 Parameters
129 Parameters
130 ----------
130 ----------
131 s : string
131 s : string
132
132
133 Returns
133 Returns
134 -------
134 -------
135 n : int
135 n : int
136 """
136 """
137
137
138 ini_spaces = ini_spaces_re.match(s)
138 ini_spaces = ini_spaces_re.match(s)
139 if ini_spaces:
139 if ini_spaces:
140 return ini_spaces.end()
140 return ini_spaces.end()
141 else:
141 else:
142 return 0
142 return 0
143
143
144 last_blank_re = re.compile(r'^.*\n\s+$', re.MULTILINE)
145
146 def last_blank(src):
144 def last_blank(src):
147 """Determine if the input source ends in a blank.
145 """Determine if the input source ends in a blank.
148
146
149 A blank is either a newline or a line consisting of whitespace.
147 A blank is either a newline or a line consisting of whitespace.
150
148
151 Parameters
149 Parameters
152 ----------
150 ----------
153 src : string
151 src : string
154 A single or multiline string.
152 A single or multiline string.
155 """
153 """
156 return src == '\n' or bool(last_blank_re.match(src))
154 if not src: return False
155 ll = src.splitlines()[-1]
156 return (ll == '') or ll.isspace()
157
157
158
158
159 last_two_blanks_re = re.compile(r'^\n\s*\n\s*$', re.MULTILINE)
159 last_two_blanks_re = re.compile(r'\n\s*\n\s*$', re.MULTILINE)
160 last_two_blanks_re2 = re.compile(r'^.+\n\s*\n\s+$', re.MULTILINE)
160 last_two_blanks_re2 = re.compile(r'.+\n\s*\n\s+$', re.MULTILINE)
161
161
162 def last_two_blanks(src):
162 def last_two_blanks(src):
163 """Determine if the input source ends in two blanks.
163 """Determine if the input source ends in two blanks.
164
164
165 A blank is either a newline or a line consisting of whitespace.
165 A blank is either a newline or a line consisting of whitespace.
166
166
167 Parameters
167 Parameters
168 ----------
168 ----------
169 src : string
169 src : string
170 A single or multiline string.
170 A single or multiline string.
171 """
171 """
172 return (bool(last_two_blanks_re.match(src)) or
172 if not src: return False
173 bool(last_two_blanks_re2.match(src)) )
173 # The logic here is tricky: I couldn't get a regexp to work and pass all
174 # the tests, so I took a different approach: split the source by lines,
175 # grab the last two and prepend '###\n' as a stand-in for whatever was in
176 # the body before the last two lines. Then, with that structure, it's
177 # possible to analyze with two regexps. Not the most elegant solution, but
178 # it works. If anyone tries to change this logic, make sure to validate
179 # the whole test suite first!
180 new_src = '\n'.join(['###\n'] + src.splitlines()[-2:])
181 return (bool(last_two_blanks_re.match(new_src)) or
182 bool(last_two_blanks_re2.match(new_src)) )
174
183
175
184
176 def remove_comments(src):
185 def remove_comments(src):
177 """Remove all comments from input source.
186 """Remove all comments from input source.
178
187
179 Note: comments are NOT recognized inside of strings!
188 Note: comments are NOT recognized inside of strings!
180
189
181 Parameters
190 Parameters
182 ----------
191 ----------
183 src : string
192 src : string
184 A single or multiline input string.
193 A single or multiline input string.
185
194
186 Returns
195 Returns
187 -------
196 -------
188 String with all Python comments removed.
197 String with all Python comments removed.
189 """
198 """
190
199
191 return re.sub('#.*', '', src)
200 return re.sub('#.*', '', src)
192
201
193 def has_comment(src):
202 def has_comment(src):
194 """Indicate whether an input line has (i.e. ends in, or is) a comment.
203 """Indicate whether an input line has (i.e. ends in, or is) a comment.
195
204
196 This uses tokenize, so it can distinguish comments from # inside strings.
205 This uses tokenize, so it can distinguish comments from # inside strings.
197
206
198 Parameters
207 Parameters
199 ----------
208 ----------
200 src : string
209 src : string
201 A single line input string.
210 A single line input string.
202
211
203 Returns
212 Returns
204 -------
213 -------
205 Boolean: True if source has a comment.
214 Boolean: True if source has a comment.
206 """
215 """
207 readline = StringIO(src).readline
216 readline = StringIO(src).readline
208 toktypes = set()
217 toktypes = set()
209 try:
218 try:
210 for t in tokenize.generate_tokens(readline):
219 for t in tokenize.generate_tokens(readline):
211 toktypes.add(t[0])
220 toktypes.add(t[0])
212 except tokenize.TokenError:
221 except tokenize.TokenError:
213 pass
222 pass
214 return(tokenize.COMMENT in toktypes)
223 return(tokenize.COMMENT in toktypes)
215
224
216
225
217 def get_input_encoding():
226 def get_input_encoding():
218 """Return the default standard input encoding.
227 """Return the default standard input encoding.
219
228
220 If sys.stdin has no encoding, 'ascii' is returned."""
229 If sys.stdin has no encoding, 'ascii' is returned."""
221 # There are strange environments for which sys.stdin.encoding is None. We
230 # There are strange environments for which sys.stdin.encoding is None. We
222 # ensure that a valid encoding is returned.
231 # ensure that a valid encoding is returned.
223 encoding = getattr(sys.stdin, 'encoding', None)
232 encoding = getattr(sys.stdin, 'encoding', None)
224 if encoding is None:
233 if encoding is None:
225 encoding = 'ascii'
234 encoding = 'ascii'
226 return encoding
235 return encoding
227
236
228 #-----------------------------------------------------------------------------
237 #-----------------------------------------------------------------------------
229 # Classes and functions for normal Python syntax handling
238 # Classes and functions for normal Python syntax handling
230 #-----------------------------------------------------------------------------
239 #-----------------------------------------------------------------------------
231
240
232 class InputSplitter(object):
241 class InputSplitter(object):
233 """An object that can accumulate lines of Python source before execution.
242 """An object that can accumulate lines of Python source before execution.
234
243
235 This object is designed to be fed python source line-by-line, using
244 This object is designed to be fed python source line-by-line, using
236 :meth:`push`. It will return on each push whether the currently pushed
245 :meth:`push`. It will return on each push whether the currently pushed
237 code could be executed already. In addition, it provides a method called
246 code could be executed already. In addition, it provides a method called
238 :meth:`push_accepts_more` that can be used to query whether more input
247 :meth:`push_accepts_more` that can be used to query whether more input
239 can be pushed into a single interactive block.
248 can be pushed into a single interactive block.
240
249
241 This is a simple example of how an interactive terminal-based client can use
250 This is a simple example of how an interactive terminal-based client can use
242 this tool::
251 this tool::
243
252
244 isp = InputSplitter()
253 isp = InputSplitter()
245 while isp.push_accepts_more():
254 while isp.push_accepts_more():
246 indent = ' '*isp.indent_spaces
255 indent = ' '*isp.indent_spaces
247 prompt = '>>> ' + indent
256 prompt = '>>> ' + indent
248 line = indent + raw_input(prompt)
257 line = indent + raw_input(prompt)
249 isp.push(line)
258 isp.push(line)
250 print 'Input source was:\n', isp.source_reset(),
259 print 'Input source was:\n', isp.source_reset(),
251 """
260 """
252 # Number of spaces of indentation computed from input that has been pushed
261 # Number of spaces of indentation computed from input that has been pushed
253 # so far. This is the attributes callers should query to get the current
262 # so far. This is the attributes callers should query to get the current
254 # indentation level, in order to provide auto-indent facilities.
263 # indentation level, in order to provide auto-indent facilities.
255 indent_spaces = 0
264 indent_spaces = 0
256 # String, indicating the default input encoding. It is computed by default
265 # String, indicating the default input encoding. It is computed by default
257 # at initialization time via get_input_encoding(), but it can be reset by a
266 # at initialization time via get_input_encoding(), but it can be reset by a
258 # client with specific knowledge of the encoding.
267 # client with specific knowledge of the encoding.
259 encoding = ''
268 encoding = ''
260 # String where the current full source input is stored, properly encoded.
269 # String where the current full source input is stored, properly encoded.
261 # Reading this attribute is the normal way of querying the currently pushed
270 # Reading this attribute is the normal way of querying the currently pushed
262 # source code, that has been properly encoded.
271 # source code, that has been properly encoded.
263 source = ''
272 source = ''
264 # Code object corresponding to the current source. It is automatically
273 # Code object corresponding to the current source. It is automatically
265 # synced to the source, so it can be queried at any time to obtain the code
274 # synced to the source, so it can be queried at any time to obtain the code
266 # object; it will be None if the source doesn't compile to valid Python.
275 # object; it will be None if the source doesn't compile to valid Python.
267 code = None
276 code = None
268 # Input mode
277 # Input mode
269 input_mode = 'line'
278 input_mode = 'line'
270
279
271 # Private attributes
280 # Private attributes
272
281
273 # List with lines of input accumulated so far
282 # List with lines of input accumulated so far
274 _buffer = None
283 _buffer = None
275 # Command compiler
284 # Command compiler
276 _compile = None
285 _compile = None
277 # Mark when input has changed indentation all the way back to flush-left
286 # Mark when input has changed indentation all the way back to flush-left
278 _full_dedent = False
287 _full_dedent = False
279 # Boolean indicating whether the current block is complete
288 # Boolean indicating whether the current block is complete
280 _is_complete = None
289 _is_complete = None
281
290
282 def __init__(self, input_mode=None):
291 def __init__(self, input_mode=None):
283 """Create a new InputSplitter instance.
292 """Create a new InputSplitter instance.
284
293
285 Parameters
294 Parameters
286 ----------
295 ----------
287 input_mode : str
296 input_mode : str
288
297
289 One of ['line', 'cell']; default is 'line'.
298 One of ['line', 'cell']; default is 'line'.
290
299
291 The input_mode parameter controls how new inputs are used when fed via
300 The input_mode parameter controls how new inputs are used when fed via
292 the :meth:`push` method:
301 the :meth:`push` method:
293
302
294 - 'line': meant for line-oriented clients, inputs are appended one at a
303 - 'line': meant for line-oriented clients, inputs are appended one at a
295 time to the internal buffer and the whole buffer is compiled.
304 time to the internal buffer and the whole buffer is compiled.
296
305
297 - 'cell': meant for clients that can edit multi-line 'cells' of text at
306 - 'cell': meant for clients that can edit multi-line 'cells' of text at
298 a time. A cell can contain one or more blocks that can be compile in
307 a time. A cell can contain one or more blocks that can be compile in
299 'single' mode by Python. In this mode, each new input new input
308 'single' mode by Python. In this mode, each new input new input
300 completely replaces all prior inputs. Cell mode is thus equivalent
309 completely replaces all prior inputs. Cell mode is thus equivalent
301 to prepending a full reset() to every push() call.
310 to prepending a full reset() to every push() call.
302 """
311 """
303 self._buffer = []
312 self._buffer = []
304 self._compile = codeop.CommandCompiler()
313 self._compile = codeop.CommandCompiler()
305 self.encoding = get_input_encoding()
314 self.encoding = get_input_encoding()
306 self.input_mode = InputSplitter.input_mode if input_mode is None \
315 self.input_mode = InputSplitter.input_mode if input_mode is None \
307 else input_mode
316 else input_mode
308
317
309 def reset(self):
318 def reset(self):
310 """Reset the input buffer and associated state."""
319 """Reset the input buffer and associated state."""
311 self.indent_spaces = 0
320 self.indent_spaces = 0
312 self._buffer[:] = []
321 self._buffer[:] = []
313 self.source = ''
322 self.source = ''
314 self.code = None
323 self.code = None
315 self._is_complete = False
324 self._is_complete = False
316 self._full_dedent = False
325 self._full_dedent = False
317
326
318 def source_reset(self):
327 def source_reset(self):
319 """Return the input source and perform a full reset.
328 """Return the input source and perform a full reset.
320 """
329 """
321 out = self.source
330 out = self.source
322 self.reset()
331 self.reset()
323 return out
332 return out
324
333
325 def push(self, lines):
334 def push(self, lines):
326 """Push one or more lines of input.
335 """Push one or more lines of input.
327
336
328 This stores the given lines and returns a status code indicating
337 This stores the given lines and returns a status code indicating
329 whether the code forms a complete Python block or not.
338 whether the code forms a complete Python block or not.
330
339
331 Any exceptions generated in compilation are swallowed, but if an
340 Any exceptions generated in compilation are swallowed, but if an
332 exception was produced, the method returns True.
341 exception was produced, the method returns True.
333
342
334 Parameters
343 Parameters
335 ----------
344 ----------
336 lines : string
345 lines : string
337 One or more lines of Python input.
346 One or more lines of Python input.
338
347
339 Returns
348 Returns
340 -------
349 -------
341 is_complete : boolean
350 is_complete : boolean
342 True if the current input source (the result of the current input
351 True if the current input source (the result of the current input
343 plus prior inputs) forms a complete Python execution block. Note that
352 plus prior inputs) forms a complete Python execution block. Note that
344 this value is also stored as a private attribute (_is_complete), so it
353 this value is also stored as a private attribute (_is_complete), so it
345 can be queried at any time.
354 can be queried at any time.
346 """
355 """
347 if self.input_mode == 'cell':
356 if self.input_mode == 'cell':
348 self.reset()
357 self.reset()
349
358
350 self._store(lines)
359 self._store(lines)
351 source = self.source
360 source = self.source
352
361
353 # Before calling _compile(), reset the code object to None so that if an
362 # Before calling _compile(), reset the code object to None so that if an
354 # exception is raised in compilation, we don't mislead by having
363 # exception is raised in compilation, we don't mislead by having
355 # inconsistent code/source attributes.
364 # inconsistent code/source attributes.
356 self.code, self._is_complete = None, None
365 self.code, self._is_complete = None, None
357
366
358 # Honor termination lines properly
367 # Honor termination lines properly
359 if source.rstrip().endswith('\\'):
368 if source.rstrip().endswith('\\'):
360 return False
369 return False
361
370
362 self._update_indent(lines)
371 self._update_indent(lines)
363 try:
372 try:
364 self.code = self._compile(source, symbol="exec")
373 self.code = self._compile(source, symbol="exec")
365 # Invalid syntax can produce any of a number of different errors from
374 # Invalid syntax can produce any of a number of different errors from
366 # inside the compiler, so we have to catch them all. Syntax errors
375 # inside the compiler, so we have to catch them all. Syntax errors
367 # immediately produce a 'ready' block, so the invalid Python can be
376 # immediately produce a 'ready' block, so the invalid Python can be
368 # sent to the kernel for evaluation with possible ipython
377 # sent to the kernel for evaluation with possible ipython
369 # special-syntax conversion.
378 # special-syntax conversion.
370 except (SyntaxError, OverflowError, ValueError, TypeError,
379 except (SyntaxError, OverflowError, ValueError, TypeError,
371 MemoryError):
380 MemoryError):
372 self._is_complete = True
381 self._is_complete = True
373 else:
382 else:
374 # Compilation didn't produce any exceptions (though it may not have
383 # Compilation didn't produce any exceptions (though it may not have
375 # given a complete code object)
384 # given a complete code object)
376 self._is_complete = self.code is not None
385 self._is_complete = self.code is not None
377
386
378 return self._is_complete
387 return self._is_complete
379
388
380 def push_accepts_more(self):
389 def push_accepts_more(self):
381 """Return whether a block of interactive input can accept more input.
390 """Return whether a block of interactive input can accept more input.
382
391
383 This method is meant to be used by line-oriented frontends, who need to
392 This method is meant to be used by line-oriented frontends, who need to
384 guess whether a block is complete or not based solely on prior and
393 guess whether a block is complete or not based solely on prior and
385 current input lines. The InputSplitter considers it has a complete
394 current input lines. The InputSplitter considers it has a complete
386 interactive block and will not accept more input only when either a
395 interactive block and will not accept more input only when either a
387 SyntaxError is raised, or *all* of the following are true:
396 SyntaxError is raised, or *all* of the following are true:
388
397
389 1. The input compiles to a complete statement.
398 1. The input compiles to a complete statement.
390
399
391 2. The indentation level is flush-left (because if we are indented,
400 2. The indentation level is flush-left (because if we are indented,
392 like inside a function definition or for loop, we need to keep
401 like inside a function definition or for loop, we need to keep
393 reading new input).
402 reading new input).
394
403
395 3. There is one extra line consisting only of whitespace.
404 3. There is one extra line consisting only of whitespace.
396
405
397 Because of condition #3, this method should be used only by
406 Because of condition #3, this method should be used only by
398 *line-oriented* frontends, since it means that intermediate blank lines
407 *line-oriented* frontends, since it means that intermediate blank lines
399 are not allowed in function definitions (or any other indented block).
408 are not allowed in function definitions (or any other indented block).
400
409
401 If the current input produces a syntax error, this method immediately
410 If the current input produces a syntax error, this method immediately
402 returns False but does *not* raise the syntax error exception, as
411 returns False but does *not* raise the syntax error exception, as
403 typically clients will want to send invalid syntax to an execution
412 typically clients will want to send invalid syntax to an execution
404 backend which might convert the invalid syntax into valid Python via
413 backend which might convert the invalid syntax into valid Python via
405 one of the dynamic IPython mechanisms.
414 one of the dynamic IPython mechanisms.
406 """
415 """
407
416
408 # With incomplete input, unconditionally accept more
417 # With incomplete input, unconditionally accept more
409 if not self._is_complete:
418 if not self._is_complete:
410 return True
419 return True
411
420
412 # If we already have complete input and we're flush left, the answer
421 # If we already have complete input and we're flush left, the answer
413 # depends. In line mode, if there hasn't been any indentation,
422 # depends. In line mode, if there hasn't been any indentation,
414 # that's it. If we've come back from some indentation, we need
423 # that's it. If we've come back from some indentation, we need
415 # the blank final line to finish.
424 # the blank final line to finish.
416 # In cell mode, we need to check how many blocks the input so far
425 # In cell mode, we need to check how many blocks the input so far
417 # compiles into, because if there's already more than one full
426 # compiles into, because if there's already more than one full
418 # independent block of input, then the client has entered full
427 # independent block of input, then the client has entered full
419 # 'cell' mode and is feeding lines that each is complete. In this
428 # 'cell' mode and is feeding lines that each is complete. In this
420 # case we should then keep accepting. The Qt terminal-like console
429 # case we should then keep accepting. The Qt terminal-like console
421 # does precisely this, to provide the convenience of terminal-like
430 # does precisely this, to provide the convenience of terminal-like
422 # input of single expressions, but allowing the user (with a
431 # input of single expressions, but allowing the user (with a
423 # separate keystroke) to switch to 'cell' mode and type multiple
432 # separate keystroke) to switch to 'cell' mode and type multiple
424 # expressions in one shot.
433 # expressions in one shot.
425 if self.indent_spaces==0:
434 if self.indent_spaces==0:
426 if self.input_mode=='line':
435 if self.input_mode=='line':
427 if not self._full_dedent:
436 if not self._full_dedent:
428 return False
437 return False
429 else:
438 else:
430 try:
439 try:
431 code_ast = ast.parse(u''.join(self._buffer))
440 code_ast = ast.parse(u''.join(self._buffer))
432 except Exception:
441 except Exception:
433 return False
442 return False
434 else:
443 else:
435 if len(code_ast.body) == 1:
444 if len(code_ast.body) == 1:
436 return False
445 return False
437
446
438 # When input is complete, then termination is marked by an extra blank
447 # When input is complete, then termination is marked by an extra blank
439 # line at the end.
448 # line at the end.
440 last_line = self.source.splitlines()[-1]
449 last_line = self.source.splitlines()[-1]
441 return bool(last_line and not last_line.isspace())
450 return bool(last_line and not last_line.isspace())
442
451
443 #------------------------------------------------------------------------
452 #------------------------------------------------------------------------
444 # Private interface
453 # Private interface
445 #------------------------------------------------------------------------
454 #------------------------------------------------------------------------
446
455
447 def _find_indent(self, line):
456 def _find_indent(self, line):
448 """Compute the new indentation level for a single line.
457 """Compute the new indentation level for a single line.
449
458
450 Parameters
459 Parameters
451 ----------
460 ----------
452 line : str
461 line : str
453 A single new line of non-whitespace, non-comment Python input.
462 A single new line of non-whitespace, non-comment Python input.
454
463
455 Returns
464 Returns
456 -------
465 -------
457 indent_spaces : int
466 indent_spaces : int
458 New value for the indent level (it may be equal to self.indent_spaces
467 New value for the indent level (it may be equal to self.indent_spaces
459 if indentation doesn't change.
468 if indentation doesn't change.
460
469
461 full_dedent : boolean
470 full_dedent : boolean
462 Whether the new line causes a full flush-left dedent.
471 Whether the new line causes a full flush-left dedent.
463 """
472 """
464 indent_spaces = self.indent_spaces
473 indent_spaces = self.indent_spaces
465 full_dedent = self._full_dedent
474 full_dedent = self._full_dedent
466
475
467 inisp = num_ini_spaces(line)
476 inisp = num_ini_spaces(line)
468 if inisp < indent_spaces:
477 if inisp < indent_spaces:
469 indent_spaces = inisp
478 indent_spaces = inisp
470 if indent_spaces <= 0:
479 if indent_spaces <= 0:
471 #print 'Full dedent in text',self.source # dbg
480 #print 'Full dedent in text',self.source # dbg
472 full_dedent = True
481 full_dedent = True
473
482
474 if line.rstrip()[-1] == ':':
483 if line.rstrip()[-1] == ':':
475 indent_spaces += 4
484 indent_spaces += 4
476 elif dedent_re.match(line):
485 elif dedent_re.match(line):
477 indent_spaces -= 4
486 indent_spaces -= 4
478 if indent_spaces <= 0:
487 if indent_spaces <= 0:
479 full_dedent = True
488 full_dedent = True
480
489
481 # Safety
490 # Safety
482 if indent_spaces < 0:
491 if indent_spaces < 0:
483 indent_spaces = 0
492 indent_spaces = 0
484 #print 'safety' # dbg
493 #print 'safety' # dbg
485
494
486 return indent_spaces, full_dedent
495 return indent_spaces, full_dedent
487
496
488 def _update_indent(self, lines):
497 def _update_indent(self, lines):
489 for line in remove_comments(lines).splitlines():
498 for line in remove_comments(lines).splitlines():
490 if line and not line.isspace():
499 if line and not line.isspace():
491 self.indent_spaces, self._full_dedent = self._find_indent(line)
500 self.indent_spaces, self._full_dedent = self._find_indent(line)
492
501
493 def _store(self, lines, buffer=None, store='source'):
502 def _store(self, lines, buffer=None, store='source'):
494 """Store one or more lines of input.
503 """Store one or more lines of input.
495
504
496 If input lines are not newline-terminated, a newline is automatically
505 If input lines are not newline-terminated, a newline is automatically
497 appended."""
506 appended."""
498
507
499 if buffer is None:
508 if buffer is None:
500 buffer = self._buffer
509 buffer = self._buffer
501
510
502 if lines.endswith('\n'):
511 if lines.endswith('\n'):
503 buffer.append(lines)
512 buffer.append(lines)
504 else:
513 else:
505 buffer.append(lines+'\n')
514 buffer.append(lines+'\n')
506 setattr(self, store, self._set_source(buffer))
515 setattr(self, store, self._set_source(buffer))
507
516
508 def _set_source(self, buffer):
517 def _set_source(self, buffer):
509 return u''.join(buffer)
518 return u''.join(buffer)
510
519
511
520
512 #-----------------------------------------------------------------------------
521 #-----------------------------------------------------------------------------
513 # Functions and classes for IPython-specific syntactic support
522 # Functions and classes for IPython-specific syntactic support
514 #-----------------------------------------------------------------------------
523 #-----------------------------------------------------------------------------
515
524
516 # The escaped translators ALL receive a line where their own escape has been
525 # The escaped translators ALL receive a line where their own escape has been
517 # stripped. Only '?' is valid at the end of the line, all others can only be
526 # stripped. Only '?' is valid at the end of the line, all others can only be
518 # placed at the start.
527 # placed at the start.
519
528
520 # Transformations of the special syntaxes that don't rely on an explicit escape
529 # Transformations of the special syntaxes that don't rely on an explicit escape
521 # character but instead on patterns on the input line
530 # character but instead on patterns on the input line
522
531
523 # The core transformations are implemented as standalone functions that can be
532 # The core transformations are implemented as standalone functions that can be
524 # tested and validated in isolation. Each of these uses a regexp, we
533 # tested and validated in isolation. Each of these uses a regexp, we
525 # pre-compile these and keep them close to each function definition for clarity
534 # pre-compile these and keep them close to each function definition for clarity
526
535
527 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
536 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
528 r'\s*=\s*!\s*(?P<cmd>.*)')
537 r'\s*=\s*!\s*(?P<cmd>.*)')
529
538
530 def transform_assign_system(line):
539 def transform_assign_system(line):
531 """Handle the `files = !ls` syntax."""
540 """Handle the `files = !ls` syntax."""
532 m = _assign_system_re.match(line)
541 m = _assign_system_re.match(line)
533 if m is not None:
542 if m is not None:
534 cmd = m.group('cmd')
543 cmd = m.group('cmd')
535 lhs = m.group('lhs')
544 lhs = m.group('lhs')
536 new_line = '%s = get_ipython().getoutput(%r)' % (lhs, cmd)
545 new_line = '%s = get_ipython().getoutput(%r)' % (lhs, cmd)
537 return new_line
546 return new_line
538 return line
547 return line
539
548
540
549
541 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
550 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
542 r'\s*=\s*%\s*(?P<cmd>.*)')
551 r'\s*=\s*%\s*(?P<cmd>.*)')
543
552
544 def transform_assign_magic(line):
553 def transform_assign_magic(line):
545 """Handle the `a = %who` syntax."""
554 """Handle the `a = %who` syntax."""
546 m = _assign_magic_re.match(line)
555 m = _assign_magic_re.match(line)
547 if m is not None:
556 if m is not None:
548 cmd = m.group('cmd')
557 cmd = m.group('cmd')
549 lhs = m.group('lhs')
558 lhs = m.group('lhs')
550 new_line = '%s = get_ipython().magic(%r)' % (lhs, cmd)
559 new_line = '%s = get_ipython().magic(%r)' % (lhs, cmd)
551 return new_line
560 return new_line
552 return line
561 return line
553
562
554
563
555 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
564 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
556
565
557 def transform_classic_prompt(line):
566 def transform_classic_prompt(line):
558 """Handle inputs that start with '>>> ' syntax."""
567 """Handle inputs that start with '>>> ' syntax."""
559
568
560 if not line or line.isspace():
569 if not line or line.isspace():
561 return line
570 return line
562 m = _classic_prompt_re.match(line)
571 m = _classic_prompt_re.match(line)
563 if m:
572 if m:
564 return line[len(m.group(0)):]
573 return line[len(m.group(0)):]
565 else:
574 else:
566 return line
575 return line
567
576
568
577
569 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
578 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
570
579
571 def transform_ipy_prompt(line):
580 def transform_ipy_prompt(line):
572 """Handle inputs that start classic IPython prompt syntax."""
581 """Handle inputs that start classic IPython prompt syntax."""
573
582
574 if not line or line.isspace():
583 if not line or line.isspace():
575 return line
584 return line
576 #print 'LINE: %r' % line # dbg
585 #print 'LINE: %r' % line # dbg
577 m = _ipy_prompt_re.match(line)
586 m = _ipy_prompt_re.match(line)
578 if m:
587 if m:
579 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
588 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
580 return line[len(m.group(0)):]
589 return line[len(m.group(0)):]
581 else:
590 else:
582 return line
591 return line
583
592
584
593
585 def _make_help_call(target, esc, lspace, next_input=None):
594 def _make_help_call(target, esc, lspace, next_input=None):
586 """Prepares a pinfo(2)/psearch call from a target name and the escape
595 """Prepares a pinfo(2)/psearch call from a target name and the escape
587 (i.e. ? or ??)"""
596 (i.e. ? or ??)"""
588 method = 'pinfo2' if esc == '??' \
597 method = 'pinfo2' if esc == '??' \
589 else 'psearch' if '*' in target \
598 else 'psearch' if '*' in target \
590 else 'pinfo'
599 else 'pinfo'
591 arg = " ".join([method, target])
600 arg = " ".join([method, target])
592
601
593 if next_input is None:
602 if next_input is None:
594 return '%sget_ipython().magic(%r)' % (lspace, arg)
603 return '%sget_ipython().magic(%r)' % (lspace, arg)
595 else:
604 else:
596 return '%sget_ipython().set_next_input(%r);get_ipython().magic(%r)' % \
605 return '%sget_ipython().set_next_input(%r);get_ipython().magic(%r)' % \
597 (lspace, next_input, arg)
606 (lspace, next_input, arg)
598
607
599
608
600 _initial_space_re = re.compile(r'\s*')
609 _initial_space_re = re.compile(r'\s*')
601
610
602 _help_end_re = re.compile(r"""(%?
611 _help_end_re = re.compile(r"""(%?
603 [a-zA-Z_*][\w*]* # Variable name
612 [a-zA-Z_*][\w*]* # Variable name
604 (\.[a-zA-Z_*][\w*]*)* # .etc.etc
613 (\.[a-zA-Z_*][\w*]*)* # .etc.etc
605 )
614 )
606 (\?\??)$ # ? or ??""",
615 (\?\??)$ # ? or ??""",
607 re.VERBOSE)
616 re.VERBOSE)
608
617
609
618
610 def transform_help_end(line):
619 def transform_help_end(line):
611 """Translate lines with ?/?? at the end"""
620 """Translate lines with ?/?? at the end"""
612 m = _help_end_re.search(line)
621 m = _help_end_re.search(line)
613 if m is None or has_comment(line):
622 if m is None or has_comment(line):
614 return line
623 return line
615 target = m.group(1)
624 target = m.group(1)
616 esc = m.group(3)
625 esc = m.group(3)
617 lspace = _initial_space_re.match(line).group(0)
626 lspace = _initial_space_re.match(line).group(0)
618
627
619 # If we're mid-command, put it back on the next prompt for the user.
628 # If we're mid-command, put it back on the next prompt for the user.
620 next_input = line.rstrip('?') if line.strip() != m.group(0) else None
629 next_input = line.rstrip('?') if line.strip() != m.group(0) else None
621
630
622 return _make_help_call(target, esc, lspace, next_input)
631 return _make_help_call(target, esc, lspace, next_input)
623
632
624
633
625 class EscapedTransformer(object):
634 class EscapedTransformer(object):
626 """Class to transform lines that are explicitly escaped out."""
635 """Class to transform lines that are explicitly escaped out."""
627
636
628 def __init__(self):
637 def __init__(self):
629 tr = { ESC_SHELL : self._tr_system,
638 tr = { ESC_SHELL : self._tr_system,
630 ESC_SH_CAP : self._tr_system2,
639 ESC_SH_CAP : self._tr_system2,
631 ESC_HELP : self._tr_help,
640 ESC_HELP : self._tr_help,
632 ESC_HELP2 : self._tr_help,
641 ESC_HELP2 : self._tr_help,
633 ESC_MAGIC : self._tr_magic,
642 ESC_MAGIC : self._tr_magic,
634 ESC_QUOTE : self._tr_quote,
643 ESC_QUOTE : self._tr_quote,
635 ESC_QUOTE2 : self._tr_quote2,
644 ESC_QUOTE2 : self._tr_quote2,
636 ESC_PAREN : self._tr_paren }
645 ESC_PAREN : self._tr_paren }
637 self.tr = tr
646 self.tr = tr
638
647
639 # Support for syntax transformations that use explicit escapes typed by the
648 # Support for syntax transformations that use explicit escapes typed by the
640 # user at the beginning of a line
649 # user at the beginning of a line
641 @staticmethod
650 @staticmethod
642 def _tr_system(line_info):
651 def _tr_system(line_info):
643 "Translate lines escaped with: !"
652 "Translate lines escaped with: !"
644 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
653 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
645 return '%sget_ipython().system(%r)' % (line_info.pre, cmd)
654 return '%sget_ipython().system(%r)' % (line_info.pre, cmd)
646
655
647 @staticmethod
656 @staticmethod
648 def _tr_system2(line_info):
657 def _tr_system2(line_info):
649 "Translate lines escaped with: !!"
658 "Translate lines escaped with: !!"
650 cmd = line_info.line.lstrip()[2:]
659 cmd = line_info.line.lstrip()[2:]
651 return '%sget_ipython().getoutput(%r)' % (line_info.pre, cmd)
660 return '%sget_ipython().getoutput(%r)' % (line_info.pre, cmd)
652
661
653 @staticmethod
662 @staticmethod
654 def _tr_help(line_info):
663 def _tr_help(line_info):
655 "Translate lines escaped with: ?/??"
664 "Translate lines escaped with: ?/??"
656 # A naked help line should just fire the intro help screen
665 # A naked help line should just fire the intro help screen
657 if not line_info.line[1:]:
666 if not line_info.line[1:]:
658 return 'get_ipython().show_usage()'
667 return 'get_ipython().show_usage()'
659
668
660 return _make_help_call(line_info.ifun, line_info.esc, line_info.pre)
669 return _make_help_call(line_info.ifun, line_info.esc, line_info.pre)
661
670
662 @staticmethod
671 @staticmethod
663 def _tr_magic(line_info):
672 def _tr_magic(line_info):
664 "Translate lines escaped with: %"
673 "Translate lines escaped with: %"
665 tpl = '%sget_ipython().magic(%r)'
674 tpl = '%sget_ipython().magic(%r)'
666 cmd = ' '.join([line_info.ifun, line_info.the_rest]).strip()
675 cmd = ' '.join([line_info.ifun, line_info.the_rest]).strip()
667 return tpl % (line_info.pre, cmd)
676 return tpl % (line_info.pre, cmd)
668
677
669 @staticmethod
678 @staticmethod
670 def _tr_quote(line_info):
679 def _tr_quote(line_info):
671 "Translate lines escaped with: ,"
680 "Translate lines escaped with: ,"
672 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
681 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
673 '", "'.join(line_info.the_rest.split()) )
682 '", "'.join(line_info.the_rest.split()) )
674
683
675 @staticmethod
684 @staticmethod
676 def _tr_quote2(line_info):
685 def _tr_quote2(line_info):
677 "Translate lines escaped with: ;"
686 "Translate lines escaped with: ;"
678 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
687 return '%s%s("%s")' % (line_info.pre, line_info.ifun,
679 line_info.the_rest)
688 line_info.the_rest)
680
689
681 @staticmethod
690 @staticmethod
682 def _tr_paren(line_info):
691 def _tr_paren(line_info):
683 "Translate lines escaped with: /"
692 "Translate lines escaped with: /"
684 return '%s%s(%s)' % (line_info.pre, line_info.ifun,
693 return '%s%s(%s)' % (line_info.pre, line_info.ifun,
685 ", ".join(line_info.the_rest.split()))
694 ", ".join(line_info.the_rest.split()))
686
695
687 def __call__(self, line):
696 def __call__(self, line):
688 """Class to transform lines that are explicitly escaped out.
697 """Class to transform lines that are explicitly escaped out.
689
698
690 This calls the above _tr_* static methods for the actual line
699 This calls the above _tr_* static methods for the actual line
691 translations."""
700 translations."""
692
701
693 # Empty lines just get returned unmodified
702 # Empty lines just get returned unmodified
694 if not line or line.isspace():
703 if not line or line.isspace():
695 return line
704 return line
696
705
697 # Get line endpoints, where the escapes can be
706 # Get line endpoints, where the escapes can be
698 line_info = LineInfo(line)
707 line_info = LineInfo(line)
699
708
700 if not line_info.esc in self.tr:
709 if not line_info.esc in self.tr:
701 # If we don't recognize the escape, don't modify the line
710 # If we don't recognize the escape, don't modify the line
702 return line
711 return line
703
712
704 return self.tr[line_info.esc](line_info)
713 return self.tr[line_info.esc](line_info)
705
714
706
715
707 # A function-looking object to be used by the rest of the code. The purpose of
716 # A function-looking object to be used by the rest of the code. The purpose of
708 # the class in this case is to organize related functionality, more than to
717 # the class in this case is to organize related functionality, more than to
709 # manage state.
718 # manage state.
710 transform_escaped = EscapedTransformer()
719 transform_escaped = EscapedTransformer()
711
720
712
721
713 class IPythonInputSplitter(InputSplitter):
722 class IPythonInputSplitter(InputSplitter):
714 """An input splitter that recognizes all of IPython's special syntax."""
723 """An input splitter that recognizes all of IPython's special syntax."""
715
724
716 # String with raw, untransformed input.
725 # String with raw, untransformed input.
717 source_raw = ''
726 source_raw = ''
718
727
719 cell_magic_parts = []
728 cell_magic_parts = []
720
729
721 cell_magic_mode = False
730 cell_magic_mode = False
722
731
723 # Private attributes
732 # Private attributes
724
733
725 # List with lines of raw input accumulated so far.
734 # List with lines of raw input accumulated so far.
726 _buffer_raw = None
735 _buffer_raw = None
727
736
728 def __init__(self, input_mode=None):
737 def __init__(self, input_mode=None):
729 super(IPythonInputSplitter, self).__init__(input_mode)
738 super(IPythonInputSplitter, self).__init__(input_mode)
730 self._buffer_raw = []
739 self._buffer_raw = []
731 self._validate = True
740 self._validate = True
732
741
733 def reset(self):
742 def reset(self):
734 """Reset the input buffer and associated state."""
743 """Reset the input buffer and associated state."""
735 super(IPythonInputSplitter, self).reset()
744 super(IPythonInputSplitter, self).reset()
736 self._buffer_raw[:] = []
745 self._buffer_raw[:] = []
737 self.source_raw = ''
746 self.source_raw = ''
738 self.cell_magic_parts = []
747 self.cell_magic_parts = []
739 self.cell_magic_mode = False
748 self.cell_magic_mode = False
740
749
741 def source_raw_reset(self):
750 def source_raw_reset(self):
742 """Return input and raw source and perform a full reset.
751 """Return input and raw source and perform a full reset.
743 """
752 """
744 out = self.source
753 out = self.source
745 out_r = self.source_raw
754 out_r = self.source_raw
746 self.reset()
755 self.reset()
747 return out, out_r
756 return out, out_r
748
757
749 def push_accepts_more(self):
758 def push_accepts_more(self):
750 if self.cell_magic_mode:
759 if self.cell_magic_mode:
751 return not self._is_complete
760 return not self._is_complete
752 else:
761 else:
753 return super(IPythonInputSplitter, self).push_accepts_more()
762 return super(IPythonInputSplitter, self).push_accepts_more()
754
763
755 def _push_line_mode(self, lines):
764 def _push_line_mode(self, lines):
756 """Push in line mode.
765 """Push in line mode.
757
766
758 This means that we only get individual 'lines' with each call, though
767 This means that we only get individual 'lines' with each call, though
759 in practice each input may be multiline. But this is in contrast to
768 in practice each input may be multiline. But this is in contrast to
760 cell mode, which feeds the entirety of the cell from the start with
769 cell mode, which feeds the entirety of the cell from the start with
761 each call.
770 each call.
762 """
771 """
763 # cell magic support
772 # cell magic support
764 #print('#'*10)
773 #print('#'*10)
765 #print(lines+'\n---') # dbg
774 #print(lines+'\n---') # dbg
766 #print (repr(lines)+'\n+++')
775 #print (repr(lines)+'\n+++')
767 #print('raw', self._buffer_raw, 'validate', self.cell_magic_mode)
776 #print('raw', self._buffer_raw, 'validate', self.cell_magic_mode)
768 # Only trigger this block if we're at a 'fresh' pumping start.
777 # Only trigger this block if we're at a 'fresh' pumping start.
769 if lines.startswith('%%') and (not self.cell_magic_mode) and \
778 if lines.startswith('%%') and (not self.cell_magic_mode) and \
770 not self._buffer_raw:
779 not self._buffer_raw:
771 # Cell magics bypass all further transformations
780 # Cell magics bypass all further transformations
772 self.cell_magic_mode = True
781 self.cell_magic_mode = True
773 first, _, body = lines.partition('\n')
782 first, _, body = lines.partition('\n')
774 magic_name, _, line = first.partition(' ')
783 magic_name, _, line = first.partition(' ')
775 magic_name = magic_name.lstrip(ESC_MAGIC)
784 magic_name = magic_name.lstrip(ESC_MAGIC)
776 # We store the body of the cell and create a call to a method that
785 # We store the body of the cell and create a call to a method that
777 # will use this stored value. This is ugly, but it's a first cut to
786 # will use this stored value. This is ugly, but it's a first cut to
778 # get it all working, as right now changing the return API of our
787 # get it all working, as right now changing the return API of our
779 # methods would require major refactoring.
788 # methods would require major refactoring.
780 self.cell_magic_parts = [body]
789 self.cell_magic_parts = [body]
781 tpl = 'get_ipython()._cell_magic(%r, %r)'
790 tpl = 'get_ipython()._cell_magic(%r, %r)'
782 tlines = tpl % (magic_name, line)
791 tlines = tpl % (magic_name, line)
783 self._store(tlines)
792 self._store(tlines)
784 self._store(lines, self._buffer_raw, 'source_raw')
793 self._store(lines, self._buffer_raw, 'source_raw')
785 self._is_complete = False
794 self._is_complete = False
786 return False
795 return False
787
796
788 if self.cell_magic_mode:
797 if self.cell_magic_mode:
798 #print('c2 lines', repr(lines)) # dbg
789 # Find out if the last stored block has a whitespace line as its
799 # Find out if the last stored block has a whitespace line as its
790 # last line and also this line is whitespace, case in which we're
800 # last line and also this line is whitespace, case in which we're
791 # done (two contiguous blank lines signal termination). Note that
801 # done (two contiguous blank lines signal termination). Note that
792 # the storage logic *enforces* that every stored block is
802 # the storage logic *enforces* that every stored block is
793 # newline-terminated, so we grab everything but the last character
803 # newline-terminated, so we grab everything but the last character
794 # so we can have the body of the block alone.
804 # so we can have the body of the block alone.
795 last_block = self.cell_magic_parts[-1]
805 last_block = self.cell_magic_parts[-1]
796 self._is_complete = last_blank(last_block) and lines.isspace()
806 self._is_complete = last_blank(last_block) and lines.isspace()
797 # Only store the raw input. For lines beyond the first one, we
807 # Only store the raw input. For lines beyond the first one, we
798 # only store them for history purposes, and for execution we want
808 # only store them for history purposes, and for execution we want
799 # the caller to only receive the _cell_magic() call.
809 # the caller to only receive the _cell_magic() call.
800 self._store(lines, self._buffer_raw, 'source_raw')
810 self._store(lines, self._buffer_raw, 'source_raw')
801 self.cell_magic_parts.append(lines)
811 self.cell_magic_parts.append(lines)
802 return self._is_complete
812 return self._is_complete
803
813
804 lines_list = lines.splitlines()
814 lines_list = lines.splitlines()
805
815
806 transforms = [transform_ipy_prompt, transform_classic_prompt,
816 transforms = [transform_ipy_prompt, transform_classic_prompt,
807 transform_help_end, transform_escaped,
817 transform_help_end, transform_escaped,
808 transform_assign_system, transform_assign_magic]
818 transform_assign_system, transform_assign_magic]
809
819
810 # Transform logic
820 # Transform logic
811 #
821 #
812 # We only apply the line transformers to the input if we have either no
822 # We only apply the line transformers to the input if we have either no
813 # input yet, or complete input, or if the last line of the buffer ends
823 # input yet, or complete input, or if the last line of the buffer ends
814 # with ':' (opening an indented block). This prevents the accidental
824 # with ':' (opening an indented block). This prevents the accidental
815 # transformation of escapes inside multiline expressions like
825 # transformation of escapes inside multiline expressions like
816 # triple-quoted strings or parenthesized expressions.
826 # triple-quoted strings or parenthesized expressions.
817 #
827 #
818 # The last heuristic, while ugly, ensures that the first line of an
828 # The last heuristic, while ugly, ensures that the first line of an
819 # indented block is correctly transformed.
829 # indented block is correctly transformed.
820 #
830 #
821 # FIXME: try to find a cleaner approach for this last bit.
831 # FIXME: try to find a cleaner approach for this last bit.
822
832
823 # Store raw source before applying any transformations to it. Note
833 # Store raw source before applying any transformations to it. Note
824 # that this must be done *after* the reset() call that would otherwise
834 # that this must be done *after* the reset() call that would otherwise
825 # flush the buffer.
835 # flush the buffer.
826 self._store(lines, self._buffer_raw, 'source_raw')
836 self._store(lines, self._buffer_raw, 'source_raw')
827
837
828 push = super(IPythonInputSplitter, self).push
838 push = super(IPythonInputSplitter, self).push
829 buf = self._buffer
839 buf = self._buffer
830 for line in lines_list:
840 for line in lines_list:
831 if self._is_complete or not buf or \
841 if self._is_complete or not buf or \
832 (buf and buf[-1].rstrip().endswith((':', ','))):
842 (buf and buf[-1].rstrip().endswith((':', ','))):
833 for f in transforms:
843 for f in transforms:
834 line = f(line)
844 line = f(line)
835
845
836 out = push(line)
846 out = push(line)
837 return out
847 return out
838
848
839
849
840 def _push_cell_mode(self, lines):
850 def _push_cell_mode(self, lines):
841 """Push in cell mode.
851 """Push in cell mode.
842
852
843 This means that we get the entire cell with each call. Between resets,
853 This means that we get the entire cell with each call. Between resets,
844 the calls simply add more text to the input."""
854 the calls simply add more text to the input."""
845
855 print('lines', repr(lines)) # dbg
846 if lines.startswith('%%'):
856 if lines.startswith('%%'):
847 # Cell magics bypass all further transformations
857 # Cell magics bypass all further transformations
848 self.cell_magic_mode = True
858 self.cell_magic_mode = True
849 first, _, body = lines.partition('\n')
859 first, _, body = lines.partition('\n')
850 magic_name, _, line = first.partition(' ')
860 magic_name, _, line = first.partition(' ')
851 magic_name = magic_name.lstrip(ESC_MAGIC)
861 magic_name = magic_name.lstrip(ESC_MAGIC)
852 # We store the body of the cell and create a call to a method that
862 # We store the body of the cell and create a call to a method that
853 # will use this stored value. This is ugly, but it's a first cut to
863 # will use this stored value. This is ugly, but it's a first cut to
854 # get it all working, as right now changing the return API of our
864 # get it all working, as right now changing the return API of our
855 # methods would require major refactoring.
865 # methods would require major refactoring.
856 self.cell_magic_parts = [body]
866 self.cell_magic_parts = [body]
857 tpl = 'get_ipython()._cell_magic(%r, %r)'
867 tpl = 'get_ipython()._cell_magic(%r, %r)'
858 tlines = tpl % (magic_name, line)
868 tlines = tpl % (magic_name, line)
859 self._store(tlines)
869 self._store(tlines)
860 self._store(lines, self._buffer_raw, 'source_raw')
870 self._store(lines, self._buffer_raw, 'source_raw')
861 self._is_complete = last_two_blanks(lines)
871 self._is_complete = last_two_blanks(lines)
872 print('IC', self._is_complete) # dbg
862 return self._is_complete
873 return self._is_complete
863
874
864 lines_list = lines.splitlines()
875 lines_list = lines.splitlines()
865
876
866 transforms = [transform_ipy_prompt, transform_classic_prompt,
877 transforms = [transform_ipy_prompt, transform_classic_prompt,
867 transform_help_end, transform_escaped,
878 transform_help_end, transform_escaped,
868 transform_assign_system, transform_assign_magic]
879 transform_assign_system, transform_assign_magic]
869
880
870 # Transform logic
881 # Transform logic
871 #
882 #
872 # We only apply the line transformers to the input if we have either no
883 # We only apply the line transformers to the input if we have either no
873 # input yet, or complete input, or if the last line of the buffer ends
884 # input yet, or complete input, or if the last line of the buffer ends
874 # with ':' (opening an indented block). This prevents the accidental
885 # with ':' (opening an indented block). This prevents the accidental
875 # transformation of escapes inside multiline expressions like
886 # transformation of escapes inside multiline expressions like
876 # triple-quoted strings or parenthesized expressions.
887 # triple-quoted strings or parenthesized expressions.
877 #
888 #
878 # The last heuristic, while ugly, ensures that the first line of an
889 # The last heuristic, while ugly, ensures that the first line of an
879 # indented block is correctly transformed.
890 # indented block is correctly transformed.
880 #
891 #
881 # FIXME: try to find a cleaner approach for this last bit.
892 # FIXME: try to find a cleaner approach for this last bit.
882
893
883 # In cell mode, since we're going to pump the parent class by hand line
894 # In cell mode, since we're going to pump the parent class by hand line
884 # by line, we need to temporarily switch out to 'line' mode, do a
895 # by line, we need to temporarily switch out to 'line' mode, do a
885 # single manual reset and then feed the lines one by one. Note that
896 # single manual reset and then feed the lines one by one. Note that
886 # this only matters if the input has more than one line.
897 # this only matters if the input has more than one line.
887 self.reset()
898 self.reset()
888 self.input_mode = 'line'
899 self.input_mode = 'line'
889
900
890 # Store raw source before applying any transformations to it. Note
901 # Store raw source before applying any transformations to it. Note
891 # that this must be done *after* the reset() call that would otherwise
902 # that this must be done *after* the reset() call that would otherwise
892 # flush the buffer.
903 # flush the buffer.
893 self._store(lines, self._buffer_raw, 'source_raw')
904 self._store(lines, self._buffer_raw, 'source_raw')
894
905
895 try:
906 try:
896 push = super(IPythonInputSplitter, self).push
907 push = super(IPythonInputSplitter, self).push
897 buf = self._buffer
908 buf = self._buffer
898 for line in lines_list:
909 for line in lines_list:
899 if self._is_complete or not buf or \
910 if self._is_complete or not buf or \
900 (buf and buf[-1].rstrip().endswith((':', ','))):
911 (buf and buf[-1].rstrip().endswith((':', ','))):
901 for f in transforms:
912 for f in transforms:
902 line = f(line)
913 line = f(line)
903
914
904 out = push(line)
915 out = push(line)
905 finally:
916 finally:
906 self.input_mode = 'cell'
917 self.input_mode = 'cell'
907 return out
918 return out
908
919
909 def push(self, lines):
920 def push(self, lines):
910 """Push one or more lines of IPython input.
921 """Push one or more lines of IPython input.
911
922
912 This stores the given lines and returns a status code indicating
923 This stores the given lines and returns a status code indicating
913 whether the code forms a complete Python block or not, after processing
924 whether the code forms a complete Python block or not, after processing
914 all input lines for special IPython syntax.
925 all input lines for special IPython syntax.
915
926
916 Any exceptions generated in compilation are swallowed, but if an
927 Any exceptions generated in compilation are swallowed, but if an
917 exception was produced, the method returns True.
928 exception was produced, the method returns True.
918
929
919 Parameters
930 Parameters
920 ----------
931 ----------
921 lines : string
932 lines : string
922 One or more lines of Python input.
933 One or more lines of Python input.
923
934
924 Returns
935 Returns
925 -------
936 -------
926 is_complete : boolean
937 is_complete : boolean
927 True if the current input source (the result of the current input
938 True if the current input source (the result of the current input
928 plus prior inputs) forms a complete Python execution block. Note that
939 plus prior inputs) forms a complete Python execution block. Note that
929 this value is also stored as a private attribute (_is_complete), so it
940 this value is also stored as a private attribute (_is_complete), so it
930 can be queried at any time.
941 can be queried at any time.
931 """
942 """
932 if not lines:
943 if not lines:
933 return super(IPythonInputSplitter, self).push(lines)
944 return super(IPythonInputSplitter, self).push(lines)
934
945
935 # We must ensure all input is pure unicode
946 # We must ensure all input is pure unicode
936 lines = cast_unicode(lines, self.encoding)
947 lines = cast_unicode(lines, self.encoding)
937
948
938 if self.input_mode == 'line':
949 if self.input_mode == 'line':
939 return self._push_line_mode(lines)
950 return self._push_line_mode(lines)
940 else:
951 else:
941 return self._push_cell_mode(lines)
952 return self._push_cell_mode(lines)
942
953
943 lines_list = lines.splitlines()
954 lines_list = lines.splitlines()
944
955
945 transforms = [transform_ipy_prompt, transform_classic_prompt,
956 transforms = [transform_ipy_prompt, transform_classic_prompt,
946 transform_help_end, transform_escaped,
957 transform_help_end, transform_escaped,
947 transform_assign_system, transform_assign_magic]
958 transform_assign_system, transform_assign_magic]
948
959
949 # Transform logic
960 # Transform logic
950 #
961 #
951 # We only apply the line transformers to the input if we have either no
962 # We only apply the line transformers to the input if we have either no
952 # input yet, or complete input, or if the last line of the buffer ends
963 # input yet, or complete input, or if the last line of the buffer ends
953 # with ':' (opening an indented block). This prevents the accidental
964 # with ':' (opening an indented block). This prevents the accidental
954 # transformation of escapes inside multiline expressions like
965 # transformation of escapes inside multiline expressions like
955 # triple-quoted strings or parenthesized expressions.
966 # triple-quoted strings or parenthesized expressions.
956 #
967 #
957 # The last heuristic, while ugly, ensures that the first line of an
968 # The last heuristic, while ugly, ensures that the first line of an
958 # indented block is correctly transformed.
969 # indented block is correctly transformed.
959 #
970 #
960 # FIXME: try to find a cleaner approach for this last bit.
971 # FIXME: try to find a cleaner approach for this last bit.
961
972
962 # If we were in 'block' mode, since we're going to pump the parent
973 # If we were in 'block' mode, since we're going to pump the parent
963 # class by hand line by line, we need to temporarily switch out to
974 # class by hand line by line, we need to temporarily switch out to
964 # 'line' mode, do a single manual reset and then feed the lines one
975 # 'line' mode, do a single manual reset and then feed the lines one
965 # by one. Note that this only matters if the input has more than one
976 # by one. Note that this only matters if the input has more than one
966 # line.
977 # line.
967 changed_input_mode = False
978 changed_input_mode = False
968
979
969 if self.input_mode == 'cell':
980 if self.input_mode == 'cell':
970 self.reset()
981 self.reset()
971 changed_input_mode = True
982 changed_input_mode = True
972 saved_input_mode = 'cell'
983 saved_input_mode = 'cell'
973 self.input_mode = 'line'
984 self.input_mode = 'line'
974
985
975 # Store raw source before applying any transformations to it. Note
986 # Store raw source before applying any transformations to it. Note
976 # that this must be done *after* the reset() call that would otherwise
987 # that this must be done *after* the reset() call that would otherwise
977 # flush the buffer.
988 # flush the buffer.
978 self._store(lines, self._buffer_raw, 'source_raw')
989 self._store(lines, self._buffer_raw, 'source_raw')
979
990
980 try:
991 try:
981 push = super(IPythonInputSplitter, self).push
992 push = super(IPythonInputSplitter, self).push
982 buf = self._buffer
993 buf = self._buffer
983 for line in lines_list:
994 for line in lines_list:
984 if self._is_complete or not buf or \
995 if self._is_complete or not buf or \
985 (buf and buf[-1].rstrip().endswith((':', ','))):
996 (buf and buf[-1].rstrip().endswith((':', ','))):
986 for f in transforms:
997 for f in transforms:
987 line = f(line)
998 line = f(line)
988
999
989 out = push(line)
1000 out = push(line)
990 finally:
1001 finally:
991 if changed_input_mode:
1002 if changed_input_mode:
992 self.input_mode = saved_input_mode
1003 self.input_mode = saved_input_mode
993 return out
1004 return out
@@ -1,789 +1,794 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """Tests for the inputsplitter module.
2 """Tests for the inputsplitter module.
3
3
4 Authors
4 Authors
5 -------
5 -------
6 * Fernando Perez
6 * Fernando Perez
7 * Robert Kern
7 * Robert Kern
8 """
8 """
9 #-----------------------------------------------------------------------------
9 #-----------------------------------------------------------------------------
10 # Copyright (C) 2010-2011 The IPython Development Team
10 # Copyright (C) 2010-2011 The IPython Development Team
11 #
11 #
12 # Distributed under the terms of the BSD License. The full license is in
12 # Distributed under the terms of the BSD License. The full license is in
13 # the file COPYING, distributed as part of this software.
13 # the file COPYING, distributed as part of this software.
14 #-----------------------------------------------------------------------------
14 #-----------------------------------------------------------------------------
15
15
16 #-----------------------------------------------------------------------------
16 #-----------------------------------------------------------------------------
17 # Imports
17 # Imports
18 #-----------------------------------------------------------------------------
18 #-----------------------------------------------------------------------------
19 # stdlib
19 # stdlib
20 import unittest
20 import unittest
21 import sys
21 import sys
22
22
23 # Third party
23 # Third party
24 import nose.tools as nt
24 import nose.tools as nt
25
25
26 # Our own
26 # Our own
27 from IPython.core import inputsplitter as isp
27 from IPython.core import inputsplitter as isp
28 from IPython.testing import tools as tt
28 from IPython.testing import tools as tt
29 from IPython.utils import py3compat
29 from IPython.utils import py3compat
30
30
31 #-----------------------------------------------------------------------------
31 #-----------------------------------------------------------------------------
32 # Semi-complete examples (also used as tests)
32 # Semi-complete examples (also used as tests)
33 #-----------------------------------------------------------------------------
33 #-----------------------------------------------------------------------------
34
34
35 # Note: at the bottom, there's a slightly more complete version of this that
35 # Note: at the bottom, there's a slightly more complete version of this that
36 # can be useful during development of code here.
36 # can be useful during development of code here.
37
37
38 def mini_interactive_loop(input_func):
38 def mini_interactive_loop(input_func):
39 """Minimal example of the logic of an interactive interpreter loop.
39 """Minimal example of the logic of an interactive interpreter loop.
40
40
41 This serves as an example, and it is used by the test system with a fake
41 This serves as an example, and it is used by the test system with a fake
42 raw_input that simulates interactive input."""
42 raw_input that simulates interactive input."""
43
43
44 from IPython.core.inputsplitter import InputSplitter
44 from IPython.core.inputsplitter import InputSplitter
45
45
46 isp = InputSplitter()
46 isp = InputSplitter()
47 # In practice, this input loop would be wrapped in an outside loop to read
47 # In practice, this input loop would be wrapped in an outside loop to read
48 # input indefinitely, until some exit/quit command was issued. Here we
48 # input indefinitely, until some exit/quit command was issued. Here we
49 # only illustrate the basic inner loop.
49 # only illustrate the basic inner loop.
50 while isp.push_accepts_more():
50 while isp.push_accepts_more():
51 indent = ' '*isp.indent_spaces
51 indent = ' '*isp.indent_spaces
52 prompt = '>>> ' + indent
52 prompt = '>>> ' + indent
53 line = indent + input_func(prompt)
53 line = indent + input_func(prompt)
54 isp.push(line)
54 isp.push(line)
55
55
56 # Here we just return input so we can use it in a test suite, but a real
56 # Here we just return input so we can use it in a test suite, but a real
57 # interpreter would instead send it for execution somewhere.
57 # interpreter would instead send it for execution somewhere.
58 src = isp.source_reset()
58 src = isp.source_reset()
59 #print 'Input source was:\n', src # dbg
59 #print 'Input source was:\n', src # dbg
60 return src
60 return src
61
61
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 # Test utilities, just for local use
63 # Test utilities, just for local use
64 #-----------------------------------------------------------------------------
64 #-----------------------------------------------------------------------------
65
65
66 def assemble(block):
66 def assemble(block):
67 """Assemble a block into multi-line sub-blocks."""
67 """Assemble a block into multi-line sub-blocks."""
68 return ['\n'.join(sub_block)+'\n' for sub_block in block]
68 return ['\n'.join(sub_block)+'\n' for sub_block in block]
69
69
70
70
71 def pseudo_input(lines):
71 def pseudo_input(lines):
72 """Return a function that acts like raw_input but feeds the input list."""
72 """Return a function that acts like raw_input but feeds the input list."""
73 ilines = iter(lines)
73 ilines = iter(lines)
74 def raw_in(prompt):
74 def raw_in(prompt):
75 try:
75 try:
76 return next(ilines)
76 return next(ilines)
77 except StopIteration:
77 except StopIteration:
78 return ''
78 return ''
79 return raw_in
79 return raw_in
80
80
81 #-----------------------------------------------------------------------------
81 #-----------------------------------------------------------------------------
82 # Tests
82 # Tests
83 #-----------------------------------------------------------------------------
83 #-----------------------------------------------------------------------------
84 def test_spaces():
84 def test_spaces():
85 tests = [('', 0),
85 tests = [('', 0),
86 (' ', 1),
86 (' ', 1),
87 ('\n', 0),
87 ('\n', 0),
88 (' \n', 1),
88 (' \n', 1),
89 ('x', 0),
89 ('x', 0),
90 (' x', 1),
90 (' x', 1),
91 (' x',2),
91 (' x',2),
92 (' x',4),
92 (' x',4),
93 # Note: tabs are counted as a single whitespace!
93 # Note: tabs are counted as a single whitespace!
94 ('\tx', 1),
94 ('\tx', 1),
95 ('\t x', 2),
95 ('\t x', 2),
96 ]
96 ]
97 tt.check_pairs(isp.num_ini_spaces, tests)
97 tt.check_pairs(isp.num_ini_spaces, tests)
98
98
99
99
100 def test_remove_comments():
100 def test_remove_comments():
101 tests = [('text', 'text'),
101 tests = [('text', 'text'),
102 ('text # comment', 'text '),
102 ('text # comment', 'text '),
103 ('text # comment\n', 'text \n'),
103 ('text # comment\n', 'text \n'),
104 ('text # comment \n', 'text \n'),
104 ('text # comment \n', 'text \n'),
105 ('line # c \nline\n','line \nline\n'),
105 ('line # c \nline\n','line \nline\n'),
106 ('line # c \nline#c2 \nline\nline #c\n\n',
106 ('line # c \nline#c2 \nline\nline #c\n\n',
107 'line \nline\nline\nline \n\n'),
107 'line \nline\nline\nline \n\n'),
108 ]
108 ]
109 tt.check_pairs(isp.remove_comments, tests)
109 tt.check_pairs(isp.remove_comments, tests)
110
110
111 def test_has_comment():
111 def test_has_comment():
112 tests = [('text', False),
112 tests = [('text', False),
113 ('text #comment', True),
113 ('text #comment', True),
114 ('text #comment\n', True),
114 ('text #comment\n', True),
115 ('#comment', True),
115 ('#comment', True),
116 ('#comment\n', True),
116 ('#comment\n', True),
117 ('a = "#string"', False),
117 ('a = "#string"', False),
118 ('a = "#string" # comment', True),
118 ('a = "#string" # comment', True),
119 ('a #comment not "string"', True),
119 ('a #comment not "string"', True),
120 ]
120 ]
121 tt.check_pairs(isp.has_comment, tests)
121 tt.check_pairs(isp.has_comment, tests)
122
122
123
123
124 def test_get_input_encoding():
124 def test_get_input_encoding():
125 encoding = isp.get_input_encoding()
125 encoding = isp.get_input_encoding()
126 nt.assert_true(isinstance(encoding, basestring))
126 nt.assert_true(isinstance(encoding, basestring))
127 # simple-minded check that at least encoding a simple string works with the
127 # simple-minded check that at least encoding a simple string works with the
128 # encoding we got.
128 # encoding we got.
129 nt.assert_equal(u'test'.encode(encoding), b'test')
129 nt.assert_equal(u'test'.encode(encoding), b'test')
130
130
131
131
132 class NoInputEncodingTestCase(unittest.TestCase):
132 class NoInputEncodingTestCase(unittest.TestCase):
133 def setUp(self):
133 def setUp(self):
134 self.old_stdin = sys.stdin
134 self.old_stdin = sys.stdin
135 class X: pass
135 class X: pass
136 fake_stdin = X()
136 fake_stdin = X()
137 sys.stdin = fake_stdin
137 sys.stdin = fake_stdin
138
138
139 def test(self):
139 def test(self):
140 # Verify that if sys.stdin has no 'encoding' attribute we do the right
140 # Verify that if sys.stdin has no 'encoding' attribute we do the right
141 # thing
141 # thing
142 enc = isp.get_input_encoding()
142 enc = isp.get_input_encoding()
143 self.assertEqual(enc, 'ascii')
143 self.assertEqual(enc, 'ascii')
144
144
145 def tearDown(self):
145 def tearDown(self):
146 sys.stdin = self.old_stdin
146 sys.stdin = self.old_stdin
147
147
148
148
149 class InputSplitterTestCase(unittest.TestCase):
149 class InputSplitterTestCase(unittest.TestCase):
150 def setUp(self):
150 def setUp(self):
151 self.isp = isp.InputSplitter()
151 self.isp = isp.InputSplitter()
152
152
153 def test_reset(self):
153 def test_reset(self):
154 isp = self.isp
154 isp = self.isp
155 isp.push('x=1')
155 isp.push('x=1')
156 isp.reset()
156 isp.reset()
157 self.assertEqual(isp._buffer, [])
157 self.assertEqual(isp._buffer, [])
158 self.assertEqual(isp.indent_spaces, 0)
158 self.assertEqual(isp.indent_spaces, 0)
159 self.assertEqual(isp.source, '')
159 self.assertEqual(isp.source, '')
160 self.assertEqual(isp.code, None)
160 self.assertEqual(isp.code, None)
161 self.assertEqual(isp._is_complete, False)
161 self.assertEqual(isp._is_complete, False)
162
162
163 def test_source(self):
163 def test_source(self):
164 self.isp._store('1')
164 self.isp._store('1')
165 self.isp._store('2')
165 self.isp._store('2')
166 self.assertEqual(self.isp.source, '1\n2\n')
166 self.assertEqual(self.isp.source, '1\n2\n')
167 self.assertTrue(len(self.isp._buffer)>0)
167 self.assertTrue(len(self.isp._buffer)>0)
168 self.assertEqual(self.isp.source_reset(), '1\n2\n')
168 self.assertEqual(self.isp.source_reset(), '1\n2\n')
169 self.assertEqual(self.isp._buffer, [])
169 self.assertEqual(self.isp._buffer, [])
170 self.assertEqual(self.isp.source, '')
170 self.assertEqual(self.isp.source, '')
171
171
172 def test_indent(self):
172 def test_indent(self):
173 isp = self.isp # shorthand
173 isp = self.isp # shorthand
174 isp.push('x=1')
174 isp.push('x=1')
175 self.assertEqual(isp.indent_spaces, 0)
175 self.assertEqual(isp.indent_spaces, 0)
176 isp.push('if 1:\n x=1')
176 isp.push('if 1:\n x=1')
177 self.assertEqual(isp.indent_spaces, 4)
177 self.assertEqual(isp.indent_spaces, 4)
178 isp.push('y=2\n')
178 isp.push('y=2\n')
179 self.assertEqual(isp.indent_spaces, 0)
179 self.assertEqual(isp.indent_spaces, 0)
180
180
181 def test_indent2(self):
181 def test_indent2(self):
182 # In cell mode, inputs must be fed in whole blocks, so skip this test
182 # In cell mode, inputs must be fed in whole blocks, so skip this test
183 if self.isp.input_mode == 'cell': return
183 if self.isp.input_mode == 'cell': return
184
184
185 isp = self.isp
185 isp = self.isp
186 isp.push('if 1:')
186 isp.push('if 1:')
187 self.assertEqual(isp.indent_spaces, 4)
187 self.assertEqual(isp.indent_spaces, 4)
188 isp.push(' x=1')
188 isp.push(' x=1')
189 self.assertEqual(isp.indent_spaces, 4)
189 self.assertEqual(isp.indent_spaces, 4)
190 # Blank lines shouldn't change the indent level
190 # Blank lines shouldn't change the indent level
191 isp.push(' '*2)
191 isp.push(' '*2)
192 self.assertEqual(isp.indent_spaces, 4)
192 self.assertEqual(isp.indent_spaces, 4)
193
193
194 def test_indent3(self):
194 def test_indent3(self):
195 # In cell mode, inputs must be fed in whole blocks, so skip this test
195 # In cell mode, inputs must be fed in whole blocks, so skip this test
196 if self.isp.input_mode == 'cell': return
196 if self.isp.input_mode == 'cell': return
197
197
198 isp = self.isp
198 isp = self.isp
199 # When a multiline statement contains parens or multiline strings, we
199 # When a multiline statement contains parens or multiline strings, we
200 # shouldn't get confused.
200 # shouldn't get confused.
201 isp.push("if 1:")
201 isp.push("if 1:")
202 isp.push(" x = (1+\n 2)")
202 isp.push(" x = (1+\n 2)")
203 self.assertEqual(isp.indent_spaces, 4)
203 self.assertEqual(isp.indent_spaces, 4)
204
204
205 def test_indent4(self):
205 def test_indent4(self):
206 # In cell mode, inputs must be fed in whole blocks, so skip this test
206 # In cell mode, inputs must be fed in whole blocks, so skip this test
207 if self.isp.input_mode == 'cell': return
207 if self.isp.input_mode == 'cell': return
208
208
209 isp = self.isp
209 isp = self.isp
210 # whitespace after ':' should not screw up indent level
210 # whitespace after ':' should not screw up indent level
211 isp.push('if 1: \n x=1')
211 isp.push('if 1: \n x=1')
212 self.assertEqual(isp.indent_spaces, 4)
212 self.assertEqual(isp.indent_spaces, 4)
213 isp.push('y=2\n')
213 isp.push('y=2\n')
214 self.assertEqual(isp.indent_spaces, 0)
214 self.assertEqual(isp.indent_spaces, 0)
215 isp.push('if 1:\t\n x=1')
215 isp.push('if 1:\t\n x=1')
216 self.assertEqual(isp.indent_spaces, 4)
216 self.assertEqual(isp.indent_spaces, 4)
217 isp.push('y=2\n')
217 isp.push('y=2\n')
218 self.assertEqual(isp.indent_spaces, 0)
218 self.assertEqual(isp.indent_spaces, 0)
219
219
220 def test_dedent_pass(self):
220 def test_dedent_pass(self):
221 isp = self.isp # shorthand
221 isp = self.isp # shorthand
222 # should NOT cause dedent
222 # should NOT cause dedent
223 isp.push('if 1:\n passes = 5')
223 isp.push('if 1:\n passes = 5')
224 self.assertEqual(isp.indent_spaces, 4)
224 self.assertEqual(isp.indent_spaces, 4)
225 isp.push('if 1:\n pass')
225 isp.push('if 1:\n pass')
226 self.assertEqual(isp.indent_spaces, 0)
226 self.assertEqual(isp.indent_spaces, 0)
227 isp.push('if 1:\n pass ')
227 isp.push('if 1:\n pass ')
228 self.assertEqual(isp.indent_spaces, 0)
228 self.assertEqual(isp.indent_spaces, 0)
229
229
230 def test_dedent_raise(self):
230 def test_dedent_raise(self):
231 isp = self.isp # shorthand
231 isp = self.isp # shorthand
232 # should NOT cause dedent
232 # should NOT cause dedent
233 isp.push('if 1:\n raised = 4')
233 isp.push('if 1:\n raised = 4')
234 self.assertEqual(isp.indent_spaces, 4)
234 self.assertEqual(isp.indent_spaces, 4)
235 isp.push('if 1:\n raise TypeError()')
235 isp.push('if 1:\n raise TypeError()')
236 self.assertEqual(isp.indent_spaces, 0)
236 self.assertEqual(isp.indent_spaces, 0)
237 isp.push('if 1:\n raise')
237 isp.push('if 1:\n raise')
238 self.assertEqual(isp.indent_spaces, 0)
238 self.assertEqual(isp.indent_spaces, 0)
239 isp.push('if 1:\n raise ')
239 isp.push('if 1:\n raise ')
240 self.assertEqual(isp.indent_spaces, 0)
240 self.assertEqual(isp.indent_spaces, 0)
241
241
242 def test_dedent_return(self):
242 def test_dedent_return(self):
243 isp = self.isp # shorthand
243 isp = self.isp # shorthand
244 # should NOT cause dedent
244 # should NOT cause dedent
245 isp.push('if 1:\n returning = 4')
245 isp.push('if 1:\n returning = 4')
246 self.assertEqual(isp.indent_spaces, 4)
246 self.assertEqual(isp.indent_spaces, 4)
247 isp.push('if 1:\n return 5 + 493')
247 isp.push('if 1:\n return 5 + 493')
248 self.assertEqual(isp.indent_spaces, 0)
248 self.assertEqual(isp.indent_spaces, 0)
249 isp.push('if 1:\n return')
249 isp.push('if 1:\n return')
250 self.assertEqual(isp.indent_spaces, 0)
250 self.assertEqual(isp.indent_spaces, 0)
251 isp.push('if 1:\n return ')
251 isp.push('if 1:\n return ')
252 self.assertEqual(isp.indent_spaces, 0)
252 self.assertEqual(isp.indent_spaces, 0)
253 isp.push('if 1:\n return(0)')
253 isp.push('if 1:\n return(0)')
254 self.assertEqual(isp.indent_spaces, 0)
254 self.assertEqual(isp.indent_spaces, 0)
255
255
256 def test_push(self):
256 def test_push(self):
257 isp = self.isp
257 isp = self.isp
258 self.assertTrue(isp.push('x=1'))
258 self.assertTrue(isp.push('x=1'))
259
259
260 def test_push2(self):
260 def test_push2(self):
261 isp = self.isp
261 isp = self.isp
262 self.assertFalse(isp.push('if 1:'))
262 self.assertFalse(isp.push('if 1:'))
263 for line in [' x=1', '# a comment', ' y=2']:
263 for line in [' x=1', '# a comment', ' y=2']:
264 self.assertTrue(isp.push(line))
264 self.assertTrue(isp.push(line))
265
265
266 def test_push3(self):
266 def test_push3(self):
267 isp = self.isp
267 isp = self.isp
268 isp.push('if True:')
268 isp.push('if True:')
269 isp.push(' a = 1')
269 isp.push(' a = 1')
270 self.assertFalse(isp.push('b = [1,'))
270 self.assertFalse(isp.push('b = [1,'))
271
271
272 def test_replace_mode(self):
272 def test_replace_mode(self):
273 isp = self.isp
273 isp = self.isp
274 isp.input_mode = 'cell'
274 isp.input_mode = 'cell'
275 isp.push('x=1')
275 isp.push('x=1')
276 self.assertEqual(isp.source, 'x=1\n')
276 self.assertEqual(isp.source, 'x=1\n')
277 isp.push('x=2')
277 isp.push('x=2')
278 self.assertEqual(isp.source, 'x=2\n')
278 self.assertEqual(isp.source, 'x=2\n')
279
279
280 def test_push_accepts_more(self):
280 def test_push_accepts_more(self):
281 isp = self.isp
281 isp = self.isp
282 isp.push('x=1')
282 isp.push('x=1')
283 self.assertFalse(isp.push_accepts_more())
283 self.assertFalse(isp.push_accepts_more())
284
284
285 def test_push_accepts_more2(self):
285 def test_push_accepts_more2(self):
286 # In cell mode, inputs must be fed in whole blocks, so skip this test
286 # In cell mode, inputs must be fed in whole blocks, so skip this test
287 if self.isp.input_mode == 'cell': return
287 if self.isp.input_mode == 'cell': return
288
288
289 isp = self.isp
289 isp = self.isp
290 isp.push('if 1:')
290 isp.push('if 1:')
291 self.assertTrue(isp.push_accepts_more())
291 self.assertTrue(isp.push_accepts_more())
292 isp.push(' x=1')
292 isp.push(' x=1')
293 self.assertTrue(isp.push_accepts_more())
293 self.assertTrue(isp.push_accepts_more())
294 isp.push('')
294 isp.push('')
295 self.assertFalse(isp.push_accepts_more())
295 self.assertFalse(isp.push_accepts_more())
296
296
297 def test_push_accepts_more3(self):
297 def test_push_accepts_more3(self):
298 isp = self.isp
298 isp = self.isp
299 isp.push("x = (2+\n3)")
299 isp.push("x = (2+\n3)")
300 self.assertFalse(isp.push_accepts_more())
300 self.assertFalse(isp.push_accepts_more())
301
301
302 def test_push_accepts_more4(self):
302 def test_push_accepts_more4(self):
303 # In cell mode, inputs must be fed in whole blocks, so skip this test
303 # In cell mode, inputs must be fed in whole blocks, so skip this test
304 if self.isp.input_mode == 'cell': return
304 if self.isp.input_mode == 'cell': return
305
305
306 isp = self.isp
306 isp = self.isp
307 # When a multiline statement contains parens or multiline strings, we
307 # When a multiline statement contains parens or multiline strings, we
308 # shouldn't get confused.
308 # shouldn't get confused.
309 # FIXME: we should be able to better handle de-dents in statements like
309 # FIXME: we should be able to better handle de-dents in statements like
310 # multiline strings and multiline expressions (continued with \ or
310 # multiline strings and multiline expressions (continued with \ or
311 # parens). Right now we aren't handling the indentation tracking quite
311 # parens). Right now we aren't handling the indentation tracking quite
312 # correctly with this, though in practice it may not be too much of a
312 # correctly with this, though in practice it may not be too much of a
313 # problem. We'll need to see.
313 # problem. We'll need to see.
314 isp.push("if 1:")
314 isp.push("if 1:")
315 isp.push(" x = (2+")
315 isp.push(" x = (2+")
316 isp.push(" 3)")
316 isp.push(" 3)")
317 self.assertTrue(isp.push_accepts_more())
317 self.assertTrue(isp.push_accepts_more())
318 isp.push(" y = 3")
318 isp.push(" y = 3")
319 self.assertTrue(isp.push_accepts_more())
319 self.assertTrue(isp.push_accepts_more())
320 isp.push('')
320 isp.push('')
321 self.assertFalse(isp.push_accepts_more())
321 self.assertFalse(isp.push_accepts_more())
322
322
323 def test_push_accepts_more5(self):
323 def test_push_accepts_more5(self):
324 # In cell mode, inputs must be fed in whole blocks, so skip this test
324 # In cell mode, inputs must be fed in whole blocks, so skip this test
325 if self.isp.input_mode == 'cell': return
325 if self.isp.input_mode == 'cell': return
326
326
327 isp = self.isp
327 isp = self.isp
328 isp.push('try:')
328 isp.push('try:')
329 isp.push(' a = 5')
329 isp.push(' a = 5')
330 isp.push('except:')
330 isp.push('except:')
331 isp.push(' raise')
331 isp.push(' raise')
332 self.assertTrue(isp.push_accepts_more())
332 self.assertTrue(isp.push_accepts_more())
333
333
334 def test_continuation(self):
334 def test_continuation(self):
335 isp = self.isp
335 isp = self.isp
336 isp.push("import os, \\")
336 isp.push("import os, \\")
337 self.assertTrue(isp.push_accepts_more())
337 self.assertTrue(isp.push_accepts_more())
338 isp.push("sys")
338 isp.push("sys")
339 self.assertFalse(isp.push_accepts_more())
339 self.assertFalse(isp.push_accepts_more())
340
340
341 def test_syntax_error(self):
341 def test_syntax_error(self):
342 isp = self.isp
342 isp = self.isp
343 # Syntax errors immediately produce a 'ready' block, so the invalid
343 # Syntax errors immediately produce a 'ready' block, so the invalid
344 # Python can be sent to the kernel for evaluation with possible ipython
344 # Python can be sent to the kernel for evaluation with possible ipython
345 # special-syntax conversion.
345 # special-syntax conversion.
346 isp.push('run foo')
346 isp.push('run foo')
347 self.assertFalse(isp.push_accepts_more())
347 self.assertFalse(isp.push_accepts_more())
348
348
349 def test_unicode(self):
349 def test_unicode(self):
350 self.isp.push(u"PΓ©rez")
350 self.isp.push(u"PΓ©rez")
351 self.isp.push(u'\xc3\xa9')
351 self.isp.push(u'\xc3\xa9')
352 self.isp.push(u"u'\xc3\xa9'")
352 self.isp.push(u"u'\xc3\xa9'")
353
353
354 class InteractiveLoopTestCase(unittest.TestCase):
354 class InteractiveLoopTestCase(unittest.TestCase):
355 """Tests for an interactive loop like a python shell.
355 """Tests for an interactive loop like a python shell.
356 """
356 """
357 def check_ns(self, lines, ns):
357 def check_ns(self, lines, ns):
358 """Validate that the given input lines produce the resulting namespace.
358 """Validate that the given input lines produce the resulting namespace.
359
359
360 Note: the input lines are given exactly as they would be typed in an
360 Note: the input lines are given exactly as they would be typed in an
361 auto-indenting environment, as mini_interactive_loop above already does
361 auto-indenting environment, as mini_interactive_loop above already does
362 auto-indenting and prepends spaces to the input.
362 auto-indenting and prepends spaces to the input.
363 """
363 """
364 src = mini_interactive_loop(pseudo_input(lines))
364 src = mini_interactive_loop(pseudo_input(lines))
365 test_ns = {}
365 test_ns = {}
366 exec src in test_ns
366 exec src in test_ns
367 # We can't check that the provided ns is identical to the test_ns,
367 # We can't check that the provided ns is identical to the test_ns,
368 # because Python fills test_ns with extra keys (copyright, etc). But
368 # because Python fills test_ns with extra keys (copyright, etc). But
369 # we can check that the given dict is *contained* in test_ns
369 # we can check that the given dict is *contained* in test_ns
370 for k,v in ns.iteritems():
370 for k,v in ns.iteritems():
371 self.assertEqual(test_ns[k], v)
371 self.assertEqual(test_ns[k], v)
372
372
373 def test_simple(self):
373 def test_simple(self):
374 self.check_ns(['x=1'], dict(x=1))
374 self.check_ns(['x=1'], dict(x=1))
375
375
376 def test_simple2(self):
376 def test_simple2(self):
377 self.check_ns(['if 1:', 'x=2'], dict(x=2))
377 self.check_ns(['if 1:', 'x=2'], dict(x=2))
378
378
379 def test_xy(self):
379 def test_xy(self):
380 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
380 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
381
381
382 def test_abc(self):
382 def test_abc(self):
383 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
383 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
384
384
385 def test_multi(self):
385 def test_multi(self):
386 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
386 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
387
387
388
388
389 def test_LineInfo():
389 def test_LineInfo():
390 """Simple test for LineInfo construction and str()"""
390 """Simple test for LineInfo construction and str()"""
391 linfo = isp.LineInfo(' %cd /home')
391 linfo = isp.LineInfo(' %cd /home')
392 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
392 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
393
393
394 # Transformer tests
394 # Transformer tests
395 def transform_checker(tests, func):
395 def transform_checker(tests, func):
396 """Utility to loop over test inputs"""
396 """Utility to loop over test inputs"""
397 for inp, tr in tests:
397 for inp, tr in tests:
398 nt.assert_equals(func(inp), tr)
398 nt.assert_equals(func(inp), tr)
399
399
400 # Data for all the syntax tests in the form of lists of pairs of
400 # Data for all the syntax tests in the form of lists of pairs of
401 # raw/transformed input. We store it here as a global dict so that we can use
401 # raw/transformed input. We store it here as a global dict so that we can use
402 # it both within single-function tests and also to validate the behavior of the
402 # it both within single-function tests and also to validate the behavior of the
403 # larger objects
403 # larger objects
404
404
405 syntax = \
405 syntax = \
406 dict(assign_system =
406 dict(assign_system =
407 [(i,py3compat.u_format(o)) for i,o in \
407 [(i,py3compat.u_format(o)) for i,o in \
408 [(u'a =! ls', "a = get_ipython().getoutput({u}'ls')"),
408 [(u'a =! ls', "a = get_ipython().getoutput({u}'ls')"),
409 (u'b = !ls', "b = get_ipython().getoutput({u}'ls')"),
409 (u'b = !ls', "b = get_ipython().getoutput({u}'ls')"),
410 ('x=1', 'x=1'), # normal input is unmodified
410 ('x=1', 'x=1'), # normal input is unmodified
411 (' ',' '), # blank lines are kept intact
411 (' ',' '), # blank lines are kept intact
412 ]],
412 ]],
413
413
414 assign_magic =
414 assign_magic =
415 [(i,py3compat.u_format(o)) for i,o in \
415 [(i,py3compat.u_format(o)) for i,o in \
416 [(u'a =% who', "a = get_ipython().magic({u}'who')"),
416 [(u'a =% who', "a = get_ipython().magic({u}'who')"),
417 (u'b = %who', "b = get_ipython().magic({u}'who')"),
417 (u'b = %who', "b = get_ipython().magic({u}'who')"),
418 ('x=1', 'x=1'), # normal input is unmodified
418 ('x=1', 'x=1'), # normal input is unmodified
419 (' ',' '), # blank lines are kept intact
419 (' ',' '), # blank lines are kept intact
420 ]],
420 ]],
421
421
422 classic_prompt =
422 classic_prompt =
423 [('>>> x=1', 'x=1'),
423 [('>>> x=1', 'x=1'),
424 ('x=1', 'x=1'), # normal input is unmodified
424 ('x=1', 'x=1'), # normal input is unmodified
425 (' ', ' '), # blank lines are kept intact
425 (' ', ' '), # blank lines are kept intact
426 ('... ', ''), # continuation prompts
426 ('... ', ''), # continuation prompts
427 ],
427 ],
428
428
429 ipy_prompt =
429 ipy_prompt =
430 [('In [1]: x=1', 'x=1'),
430 [('In [1]: x=1', 'x=1'),
431 ('x=1', 'x=1'), # normal input is unmodified
431 ('x=1', 'x=1'), # normal input is unmodified
432 (' ',' '), # blank lines are kept intact
432 (' ',' '), # blank lines are kept intact
433 (' ....: ', ''), # continuation prompts
433 (' ....: ', ''), # continuation prompts
434 ],
434 ],
435
435
436 # Tests for the escape transformer to leave normal code alone
436 # Tests for the escape transformer to leave normal code alone
437 escaped_noesc =
437 escaped_noesc =
438 [ (' ', ' '),
438 [ (' ', ' '),
439 ('x=1', 'x=1'),
439 ('x=1', 'x=1'),
440 ],
440 ],
441
441
442 # System calls
442 # System calls
443 escaped_shell =
443 escaped_shell =
444 [(i,py3compat.u_format(o)) for i,o in \
444 [(i,py3compat.u_format(o)) for i,o in \
445 [ (u'!ls', "get_ipython().system({u}'ls')"),
445 [ (u'!ls', "get_ipython().system({u}'ls')"),
446 # Double-escape shell, this means to capture the output of the
446 # Double-escape shell, this means to capture the output of the
447 # subprocess and return it
447 # subprocess and return it
448 (u'!!ls', "get_ipython().getoutput({u}'ls')"),
448 (u'!!ls', "get_ipython().getoutput({u}'ls')"),
449 ]],
449 ]],
450
450
451 # Help/object info
451 # Help/object info
452 escaped_help =
452 escaped_help =
453 [(i,py3compat.u_format(o)) for i,o in \
453 [(i,py3compat.u_format(o)) for i,o in \
454 [ (u'?', 'get_ipython().show_usage()'),
454 [ (u'?', 'get_ipython().show_usage()'),
455 (u'?x1', "get_ipython().magic({u}'pinfo x1')"),
455 (u'?x1', "get_ipython().magic({u}'pinfo x1')"),
456 (u'??x2', "get_ipython().magic({u}'pinfo2 x2')"),
456 (u'??x2', "get_ipython().magic({u}'pinfo2 x2')"),
457 (u'?a.*s', "get_ipython().magic({u}'psearch a.*s')"),
457 (u'?a.*s', "get_ipython().magic({u}'psearch a.*s')"),
458 (u'?%hist', "get_ipython().magic({u}'pinfo %hist')"),
458 (u'?%hist', "get_ipython().magic({u}'pinfo %hist')"),
459 (u'?abc = qwe', "get_ipython().magic({u}'pinfo abc')"),
459 (u'?abc = qwe', "get_ipython().magic({u}'pinfo abc')"),
460 ]],
460 ]],
461
461
462 end_help =
462 end_help =
463 [(i,py3compat.u_format(o)) for i,o in \
463 [(i,py3compat.u_format(o)) for i,o in \
464 [ (u'x3?', "get_ipython().magic({u}'pinfo x3')"),
464 [ (u'x3?', "get_ipython().magic({u}'pinfo x3')"),
465 (u'x4??', "get_ipython().magic({u}'pinfo2 x4')"),
465 (u'x4??', "get_ipython().magic({u}'pinfo2 x4')"),
466 (u'%hist?', "get_ipython().magic({u}'pinfo %hist')"),
466 (u'%hist?', "get_ipython().magic({u}'pinfo %hist')"),
467 (u'f*?', "get_ipython().magic({u}'psearch f*')"),
467 (u'f*?', "get_ipython().magic({u}'psearch f*')"),
468 (u'ax.*aspe*?', "get_ipython().magic({u}'psearch ax.*aspe*')"),
468 (u'ax.*aspe*?', "get_ipython().magic({u}'psearch ax.*aspe*')"),
469 (u'a = abc?', "get_ipython().set_next_input({u}'a = abc');"
469 (u'a = abc?', "get_ipython().set_next_input({u}'a = abc');"
470 "get_ipython().magic({u}'pinfo abc')"),
470 "get_ipython().magic({u}'pinfo abc')"),
471 (u'a = abc.qe??', "get_ipython().set_next_input({u}'a = abc.qe');"
471 (u'a = abc.qe??', "get_ipython().set_next_input({u}'a = abc.qe');"
472 "get_ipython().magic({u}'pinfo2 abc.qe')"),
472 "get_ipython().magic({u}'pinfo2 abc.qe')"),
473 (u'a = *.items?', "get_ipython().set_next_input({u}'a = *.items');"
473 (u'a = *.items?', "get_ipython().set_next_input({u}'a = *.items');"
474 "get_ipython().magic({u}'psearch *.items')"),
474 "get_ipython().magic({u}'psearch *.items')"),
475 (u'plot(a?', "get_ipython().set_next_input({u}'plot(a');"
475 (u'plot(a?', "get_ipython().set_next_input({u}'plot(a');"
476 "get_ipython().magic({u}'pinfo a')"),
476 "get_ipython().magic({u}'pinfo a')"),
477 (u'a*2 #comment?', 'a*2 #comment?'),
477 (u'a*2 #comment?', 'a*2 #comment?'),
478 ]],
478 ]],
479
479
480 # Explicit magic calls
480 # Explicit magic calls
481 escaped_magic =
481 escaped_magic =
482 [(i,py3compat.u_format(o)) for i,o in \
482 [(i,py3compat.u_format(o)) for i,o in \
483 [ (u'%cd', "get_ipython().magic({u}'cd')"),
483 [ (u'%cd', "get_ipython().magic({u}'cd')"),
484 (u'%cd /home', "get_ipython().magic({u}'cd /home')"),
484 (u'%cd /home', "get_ipython().magic({u}'cd /home')"),
485 # Backslashes need to be escaped.
485 # Backslashes need to be escaped.
486 (u'%cd C:\\User', "get_ipython().magic({u}'cd C:\\\\User')"),
486 (u'%cd C:\\User', "get_ipython().magic({u}'cd C:\\\\User')"),
487 (u' %magic', " get_ipython().magic({u}'magic')"),
487 (u' %magic', " get_ipython().magic({u}'magic')"),
488 ]],
488 ]],
489
489
490 # Quoting with separate arguments
490 # Quoting with separate arguments
491 escaped_quote =
491 escaped_quote =
492 [ (',f', 'f("")'),
492 [ (',f', 'f("")'),
493 (',f x', 'f("x")'),
493 (',f x', 'f("x")'),
494 (' ,f y', ' f("y")'),
494 (' ,f y', ' f("y")'),
495 (',f a b', 'f("a", "b")'),
495 (',f a b', 'f("a", "b")'),
496 ],
496 ],
497
497
498 # Quoting with single argument
498 # Quoting with single argument
499 escaped_quote2 =
499 escaped_quote2 =
500 [ (';f', 'f("")'),
500 [ (';f', 'f("")'),
501 (';f x', 'f("x")'),
501 (';f x', 'f("x")'),
502 (' ;f y', ' f("y")'),
502 (' ;f y', ' f("y")'),
503 (';f a b', 'f("a b")'),
503 (';f a b', 'f("a b")'),
504 ],
504 ],
505
505
506 # Simply apply parens
506 # Simply apply parens
507 escaped_paren =
507 escaped_paren =
508 [ ('/f', 'f()'),
508 [ ('/f', 'f()'),
509 ('/f x', 'f(x)'),
509 ('/f x', 'f(x)'),
510 (' /f y', ' f(y)'),
510 (' /f y', ' f(y)'),
511 ('/f a b', 'f(a, b)'),
511 ('/f a b', 'f(a, b)'),
512 ],
512 ],
513
513
514 # Check that we transform prompts before other transforms
514 # Check that we transform prompts before other transforms
515 mixed =
515 mixed =
516 [(i,py3compat.u_format(o)) for i,o in \
516 [(i,py3compat.u_format(o)) for i,o in \
517 [ (u'In [1]: %lsmagic', "get_ipython().magic({u}'lsmagic')"),
517 [ (u'In [1]: %lsmagic', "get_ipython().magic({u}'lsmagic')"),
518 (u'>>> %lsmagic', "get_ipython().magic({u}'lsmagic')"),
518 (u'>>> %lsmagic', "get_ipython().magic({u}'lsmagic')"),
519 (u'In [2]: !ls', "get_ipython().system({u}'ls')"),
519 (u'In [2]: !ls', "get_ipython().system({u}'ls')"),
520 (u'In [3]: abs?', "get_ipython().magic({u}'pinfo abs')"),
520 (u'In [3]: abs?', "get_ipython().magic({u}'pinfo abs')"),
521 (u'In [4]: b = %who', "b = get_ipython().magic({u}'who')"),
521 (u'In [4]: b = %who', "b = get_ipython().magic({u}'who')"),
522 ]],
522 ]],
523 )
523 )
524
524
525 # multiline syntax examples. Each of these should be a list of lists, with
525 # multiline syntax examples. Each of these should be a list of lists, with
526 # each entry itself having pairs of raw/transformed input. The union (with
526 # each entry itself having pairs of raw/transformed input. The union (with
527 # '\n'.join() of the transformed inputs is what the splitter should produce
527 # '\n'.join() of the transformed inputs is what the splitter should produce
528 # when fed the raw lines one at a time via push.
528 # when fed the raw lines one at a time via push.
529 syntax_ml = \
529 syntax_ml = \
530 dict(classic_prompt =
530 dict(classic_prompt =
531 [ [('>>> for i in range(10):','for i in range(10):'),
531 [ [('>>> for i in range(10):','for i in range(10):'),
532 ('... print i',' print i'),
532 ('... print i',' print i'),
533 ('... ', ''),
533 ('... ', ''),
534 ],
534 ],
535 ],
535 ],
536
536
537 ipy_prompt =
537 ipy_prompt =
538 [ [('In [24]: for i in range(10):','for i in range(10):'),
538 [ [('In [24]: for i in range(10):','for i in range(10):'),
539 (' ....: print i',' print i'),
539 (' ....: print i',' print i'),
540 (' ....: ', ''),
540 (' ....: ', ''),
541 ],
541 ],
542 ],
542 ],
543
543
544 multiline_datastructure =
544 multiline_datastructure =
545 [ [('>>> a = [1,','a = [1,'),
545 [ [('>>> a = [1,','a = [1,'),
546 ('... 2]','2]'),
546 ('... 2]','2]'),
547 ],
547 ],
548 ],
548 ],
549 )
549 )
550
550
551
551
552 def test_assign_system():
552 def test_assign_system():
553 tt.check_pairs(isp.transform_assign_system, syntax['assign_system'])
553 tt.check_pairs(isp.transform_assign_system, syntax['assign_system'])
554
554
555
555
556 def test_assign_magic():
556 def test_assign_magic():
557 tt.check_pairs(isp.transform_assign_magic, syntax['assign_magic'])
557 tt.check_pairs(isp.transform_assign_magic, syntax['assign_magic'])
558
558
559
559
560 def test_classic_prompt():
560 def test_classic_prompt():
561 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
561 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
562 for example in syntax_ml['classic_prompt']:
562 for example in syntax_ml['classic_prompt']:
563 transform_checker(example, isp.transform_classic_prompt)
563 transform_checker(example, isp.transform_classic_prompt)
564
564
565
565
566 def test_ipy_prompt():
566 def test_ipy_prompt():
567 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
567 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
568 for example in syntax_ml['ipy_prompt']:
568 for example in syntax_ml['ipy_prompt']:
569 transform_checker(example, isp.transform_ipy_prompt)
569 transform_checker(example, isp.transform_ipy_prompt)
570
570
571 def test_end_help():
571 def test_end_help():
572 tt.check_pairs(isp.transform_help_end, syntax['end_help'])
572 tt.check_pairs(isp.transform_help_end, syntax['end_help'])
573
573
574 def test_escaped_noesc():
574 def test_escaped_noesc():
575 tt.check_pairs(isp.transform_escaped, syntax['escaped_noesc'])
575 tt.check_pairs(isp.transform_escaped, syntax['escaped_noesc'])
576
576
577
577
578 def test_escaped_shell():
578 def test_escaped_shell():
579 tt.check_pairs(isp.transform_escaped, syntax['escaped_shell'])
579 tt.check_pairs(isp.transform_escaped, syntax['escaped_shell'])
580
580
581
581
582 def test_escaped_help():
582 def test_escaped_help():
583 tt.check_pairs(isp.transform_escaped, syntax['escaped_help'])
583 tt.check_pairs(isp.transform_escaped, syntax['escaped_help'])
584
584
585
585
586 def test_escaped_magic():
586 def test_escaped_magic():
587 tt.check_pairs(isp.transform_escaped, syntax['escaped_magic'])
587 tt.check_pairs(isp.transform_escaped, syntax['escaped_magic'])
588
588
589
589
590 def test_escaped_quote():
590 def test_escaped_quote():
591 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote'])
591 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote'])
592
592
593
593
594 def test_escaped_quote2():
594 def test_escaped_quote2():
595 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote2'])
595 tt.check_pairs(isp.transform_escaped, syntax['escaped_quote2'])
596
596
597
597
598 def test_escaped_paren():
598 def test_escaped_paren():
599 tt.check_pairs(isp.transform_escaped, syntax['escaped_paren'])
599 tt.check_pairs(isp.transform_escaped, syntax['escaped_paren'])
600
600
601
601
602 def test_last_blank():
602 def test_last_blank():
603 nt.assert_false(isp.last_blank(''))
603 nt.assert_false(isp.last_blank(''))
604 nt.assert_false(isp.last_blank('abc'))
604 nt.assert_false(isp.last_blank('abc'))
605 nt.assert_false(isp.last_blank('abc\n'))
605 nt.assert_false(isp.last_blank('abc\n'))
606 nt.assert_false(isp.last_blank('abc\na'))
606 nt.assert_false(isp.last_blank('abc\na'))
607
607
608 nt.assert_true(isp.last_blank('\n'))
608 nt.assert_true(isp.last_blank('\n'))
609 nt.assert_true(isp.last_blank('\n '))
609 nt.assert_true(isp.last_blank('\n '))
610 nt.assert_true(isp.last_blank('abc\n '))
610 nt.assert_true(isp.last_blank('abc\n '))
611 nt.assert_true(isp.last_blank('abc\n\n'))
611 nt.assert_true(isp.last_blank('abc\n\n'))
612 nt.assert_true(isp.last_blank('abc\nd\n\n'))
613 nt.assert_true(isp.last_blank('abc\nd\ne\n\n'))
614 nt.assert_true(isp.last_blank('abc \n \n \n\n'))
612
615
613
616
614 def test_last_two_blanks():
617 def test_last_two_blanks():
615 nt.assert_false(isp.last_two_blanks(''))
618 nt.assert_false(isp.last_two_blanks(''))
616 nt.assert_false(isp.last_two_blanks('abc'))
619 nt.assert_false(isp.last_two_blanks('abc'))
617 nt.assert_false(isp.last_two_blanks('abc\n'))
620 nt.assert_false(isp.last_two_blanks('abc\n'))
618 nt.assert_false(isp.last_two_blanks('abc\n\na'))
621 nt.assert_false(isp.last_two_blanks('abc\n\na'))
619 nt.assert_false(isp.last_two_blanks('abc\n \n'))
622 nt.assert_false(isp.last_two_blanks('abc\n \n'))
620 nt.assert_false(isp.last_two_blanks('abc\n\n'))
623 nt.assert_false(isp.last_two_blanks('abc\n\n'))
621
624
622 nt.assert_true(isp.last_two_blanks('\n\n'))
625 nt.assert_true(isp.last_two_blanks('\n\n'))
623 nt.assert_true(isp.last_two_blanks('\n\n '))
626 nt.assert_true(isp.last_two_blanks('\n\n '))
624 nt.assert_true(isp.last_two_blanks('\n \n'))
627 nt.assert_true(isp.last_two_blanks('\n \n'))
625 nt.assert_true(isp.last_two_blanks('abc\n\n '))
628 nt.assert_true(isp.last_two_blanks('abc\n\n '))
626 nt.assert_true(isp.last_two_blanks('abc\n\n\n'))
629 nt.assert_true(isp.last_two_blanks('abc\n\n\n'))
627 nt.assert_true(isp.last_two_blanks('abc\n\n \n'))
630 nt.assert_true(isp.last_two_blanks('abc\n\n \n'))
628 nt.assert_true(isp.last_two_blanks('abc\n\n \n '))
631 nt.assert_true(isp.last_two_blanks('abc\n\n \n '))
629 nt.assert_true(isp.last_two_blanks('abc\n\n \n \n'))
632 nt.assert_true(isp.last_two_blanks('abc\n\n \n \n'))
633 nt.assert_true(isp.last_two_blanks('abc\nd\n\n\n'))
634 nt.assert_true(isp.last_two_blanks('abc\nd\ns\nds\n\n\n'))
630
635
631
636
632 def test_cell_magics_line_mode():
637 def test_cell_magics_line_mode():
633
638
634 cell = """\
639 cell = """\
635 %%cellm line
640 %%cellm line
636 body
641 body
637 """
642 """
638 sp = isp.IPythonInputSplitter(input_mode='line')
643 sp = isp.IPythonInputSplitter(input_mode='line')
639 sp.push(cell)
644 sp.push(cell)
640 nt.assert_equal(sp.cell_magic_parts, ['body\n'])
645 nt.assert_equal(sp.cell_magic_parts, ['body\n'])
641 out = sp.source
646 out = sp.source
642 ref = u"get_ipython()._cell_magic(u'cellm', u'line')\n"
647 ref = u"get_ipython()._cell_magic(u'cellm', u'line')\n"
643 nt.assert_equal(out, ref)
648 nt.assert_equal(out, ref)
644
649
645 sp.reset()
650 sp.reset()
646
651
647 sp.push('%%cellm line2\n')
652 sp.push('%%cellm line2\n')
648 nt.assert_true(sp.push_accepts_more()) #1
653 nt.assert_true(sp.push_accepts_more()) #1
649 sp.push('\n')
654 sp.push('\n')
650 nt.assert_true(sp.push_accepts_more()) #2
655 nt.assert_true(sp.push_accepts_more()) #2
651 sp.push('\n')
656 sp.push('\n')
652 nt.assert_false(sp.push_accepts_more()) #3
657 nt.assert_false(sp.push_accepts_more()) #3
653
658
654
659
655 def test_cell_magics_cell_mode():
660 def test_cell_magics_cell_mode():
656
661
657 cell = """\
662 cell = """\
658 %%cellm line
663 %%cellm line
659 body
664 body
660 """
665 """
661 sp = isp.IPythonInputSplitter(input_mode='cell')
666 sp = isp.IPythonInputSplitter(input_mode='cell')
662 sp.push(cell)
667 sp.push(cell)
663 nt.assert_equal(sp.cell_magic_parts, ['body\n'])
668 nt.assert_equal(sp.cell_magic_parts, ['body\n'])
664 out = sp.source
669 out = sp.source
665 ref = u"get_ipython()._cell_magic(u'cellm', u'line')\n"
670 ref = u"get_ipython()._cell_magic(u'cellm', u'line')\n"
666 nt.assert_equal(out, ref)
671 nt.assert_equal(out, ref)
667
672
668 sp.reset()
673 sp.reset()
669
674
670 src = '%%cellm line2\n'
675 src = '%%cellm line2\n'
671 sp.push(src)
676 sp.push(src)
672 nt.assert_true(sp.push_accepts_more()) #1
677 nt.assert_true(sp.push_accepts_more()) #1
673 src += '\n'
678 src += '\n'
674 sp.push(src)
679 sp.push(src)
675 nt.assert_true(sp.push_accepts_more()) #2
680 nt.assert_true(sp.push_accepts_more()) #2
676 src += '\n'
681 src += '\n'
677 sp.push(src)
682 sp.push(src)
678 nt.assert_false(sp.push_accepts_more()) #3
683 nt.assert_false(sp.push_accepts_more()) #3
679
684
680
685
681 class IPythonInputTestCase(InputSplitterTestCase):
686 class IPythonInputTestCase(InputSplitterTestCase):
682 """By just creating a new class whose .isp is a different instance, we
687 """By just creating a new class whose .isp is a different instance, we
683 re-run the same test battery on the new input splitter.
688 re-run the same test battery on the new input splitter.
684
689
685 In addition, this runs the tests over the syntax and syntax_ml dicts that
690 In addition, this runs the tests over the syntax and syntax_ml dicts that
686 were tested by individual functions, as part of the OO interface.
691 were tested by individual functions, as part of the OO interface.
687
692
688 It also makes some checks on the raw buffer storage.
693 It also makes some checks on the raw buffer storage.
689 """
694 """
690
695
691 def setUp(self):
696 def setUp(self):
692 self.isp = isp.IPythonInputSplitter(input_mode='line')
697 self.isp = isp.IPythonInputSplitter(input_mode='line')
693
698
694 def test_syntax(self):
699 def test_syntax(self):
695 """Call all single-line syntax tests from the main object"""
700 """Call all single-line syntax tests from the main object"""
696 isp = self.isp
701 isp = self.isp
697 for example in syntax.itervalues():
702 for example in syntax.itervalues():
698 for raw, out_t in example:
703 for raw, out_t in example:
699 if raw.startswith(' '):
704 if raw.startswith(' '):
700 continue
705 continue
701
706
702 isp.push(raw)
707 isp.push(raw)
703 out, out_raw = isp.source_raw_reset()
708 out, out_raw = isp.source_raw_reset()
704 self.assertEqual(out.rstrip(), out_t,
709 self.assertEqual(out.rstrip(), out_t,
705 tt.pair_fail_msg.format("inputsplitter",raw, out_t, out))
710 tt.pair_fail_msg.format("inputsplitter",raw, out_t, out))
706 self.assertEqual(out_raw.rstrip(), raw.rstrip())
711 self.assertEqual(out_raw.rstrip(), raw.rstrip())
707
712
708 def test_syntax_multiline(self):
713 def test_syntax_multiline(self):
709 isp = self.isp
714 isp = self.isp
710 for example in syntax_ml.itervalues():
715 for example in syntax_ml.itervalues():
711 out_t_parts = []
716 out_t_parts = []
712 raw_parts = []
717 raw_parts = []
713 for line_pairs in example:
718 for line_pairs in example:
714 for lraw, out_t_part in line_pairs:
719 for lraw, out_t_part in line_pairs:
715 isp.push(lraw)
720 isp.push(lraw)
716 out_t_parts.append(out_t_part)
721 out_t_parts.append(out_t_part)
717 raw_parts.append(lraw)
722 raw_parts.append(lraw)
718
723
719 out, out_raw = isp.source_raw_reset()
724 out, out_raw = isp.source_raw_reset()
720 out_t = '\n'.join(out_t_parts).rstrip()
725 out_t = '\n'.join(out_t_parts).rstrip()
721 raw = '\n'.join(raw_parts).rstrip()
726 raw = '\n'.join(raw_parts).rstrip()
722 self.assertEqual(out.rstrip(), out_t)
727 self.assertEqual(out.rstrip(), out_t)
723 self.assertEqual(out_raw.rstrip(), raw)
728 self.assertEqual(out_raw.rstrip(), raw)
724
729
725
730
726 class BlockIPythonInputTestCase(IPythonInputTestCase):
731 class BlockIPythonInputTestCase(IPythonInputTestCase):
727
732
728 # Deactivate tests that don't make sense for the block mode
733 # Deactivate tests that don't make sense for the block mode
729 test_push3 = test_split = lambda s: None
734 test_push3 = test_split = lambda s: None
730
735
731 def setUp(self):
736 def setUp(self):
732 self.isp = isp.IPythonInputSplitter(input_mode='cell')
737 self.isp = isp.IPythonInputSplitter(input_mode='cell')
733
738
734 def test_syntax_multiline(self):
739 def test_syntax_multiline(self):
735 isp = self.isp
740 isp = self.isp
736 for example in syntax_ml.itervalues():
741 for example in syntax_ml.itervalues():
737 raw_parts = []
742 raw_parts = []
738 out_t_parts = []
743 out_t_parts = []
739 for line_pairs in example:
744 for line_pairs in example:
740 for raw, out_t_part in line_pairs:
745 for raw, out_t_part in line_pairs:
741 raw_parts.append(raw)
746 raw_parts.append(raw)
742 out_t_parts.append(out_t_part)
747 out_t_parts.append(out_t_part)
743
748
744 raw = '\n'.join(raw_parts)
749 raw = '\n'.join(raw_parts)
745 out_t = '\n'.join(out_t_parts)
750 out_t = '\n'.join(out_t_parts)
746
751
747 isp.push(raw)
752 isp.push(raw)
748 out, out_raw = isp.source_raw_reset()
753 out, out_raw = isp.source_raw_reset()
749 # Match ignoring trailing whitespace
754 # Match ignoring trailing whitespace
750 self.assertEqual(out.rstrip(), out_t.rstrip())
755 self.assertEqual(out.rstrip(), out_t.rstrip())
751 self.assertEqual(out_raw.rstrip(), raw.rstrip())
756 self.assertEqual(out_raw.rstrip(), raw.rstrip())
752
757
753
758
754 #-----------------------------------------------------------------------------
759 #-----------------------------------------------------------------------------
755 # Main - use as a script, mostly for developer experiments
760 # Main - use as a script, mostly for developer experiments
756 #-----------------------------------------------------------------------------
761 #-----------------------------------------------------------------------------
757
762
758 if __name__ == '__main__':
763 if __name__ == '__main__':
759 # A simple demo for interactive experimentation. This code will not get
764 # A simple demo for interactive experimentation. This code will not get
760 # picked up by any test suite.
765 # picked up by any test suite.
761 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
766 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
762
767
763 # configure here the syntax to use, prompt and whether to autoindent
768 # configure here the syntax to use, prompt and whether to autoindent
764 #isp, start_prompt = InputSplitter(), '>>> '
769 #isp, start_prompt = InputSplitter(), '>>> '
765 isp, start_prompt = IPythonInputSplitter(), 'In> '
770 isp, start_prompt = IPythonInputSplitter(), 'In> '
766
771
767 autoindent = True
772 autoindent = True
768 #autoindent = False
773 #autoindent = False
769
774
770 try:
775 try:
771 while True:
776 while True:
772 prompt = start_prompt
777 prompt = start_prompt
773 while isp.push_accepts_more():
778 while isp.push_accepts_more():
774 indent = ' '*isp.indent_spaces
779 indent = ' '*isp.indent_spaces
775 if autoindent:
780 if autoindent:
776 line = indent + raw_input(prompt+indent)
781 line = indent + raw_input(prompt+indent)
777 else:
782 else:
778 line = raw_input(prompt)
783 line = raw_input(prompt)
779 isp.push(line)
784 isp.push(line)
780 prompt = '... '
785 prompt = '... '
781
786
782 # Here we just return input so we can use it in a test suite, but a
787 # Here we just return input so we can use it in a test suite, but a
783 # real interpreter would instead send it for execution somewhere.
788 # real interpreter would instead send it for execution somewhere.
784 #src = isp.source; raise EOFError # dbg
789 #src = isp.source; raise EOFError # dbg
785 src, raw = isp.source_raw_reset()
790 src, raw = isp.source_raw_reset()
786 print 'Input source was:\n', src
791 print 'Input source was:\n', src
787 print 'Raw source was:\n', raw
792 print 'Raw source was:\n', raw
788 except EOFError:
793 except EOFError:
789 print 'Bye'
794 print 'Bye'
General Comments 0
You need to be logged in to leave comments. Login now