##// END OF EJS Templates
Allow getting help mid-way through a command (e.g. a = abc.def?).
Thomas Kluyver -
Show More
@@ -1,865 +1,868 b''
1 """Analysis of text input into executable blocks.
1 """Analysis of text input into executable blocks.
2
2
3 The main class in this module, :class:`InputSplitter`, is designed to break
3 The main class in this module, :class:`InputSplitter`, is designed to break
4 input from either interactive, line-by-line environments or block-based ones,
4 input from either interactive, line-by-line environments or block-based ones,
5 into standalone blocks that can be executed by Python as 'single' statements
5 into standalone blocks that can be executed by Python as 'single' statements
6 (thus triggering sys.displayhook).
6 (thus triggering sys.displayhook).
7
7
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
8 A companion, :class:`IPythonInputSplitter`, provides the same functionality but
9 with full support for the extended IPython syntax (magics, system calls, etc).
9 with full support for the extended IPython syntax (magics, system calls, etc).
10
10
11 For more details, see the class docstring below.
11 For more details, see the class docstring below.
12
12
13 Syntax Transformations
13 Syntax Transformations
14 ----------------------
14 ----------------------
15
15
16 One of the main jobs of the code in this file is to apply all syntax
16 One of the main jobs of the code in this file is to apply all syntax
17 transformations that make up 'the IPython language', i.e. magics, shell
17 transformations that make up 'the IPython language', i.e. magics, shell
18 escapes, etc. All transformations should be implemented as *fully stateless*
18 escapes, etc. All transformations should be implemented as *fully stateless*
19 entities, that simply take one line as their input and return a line.
19 entities, that simply take one line as their input and return a line.
20 Internally for implementation purposes they may be a normal function or a
20 Internally for implementation purposes they may be a normal function or a
21 callable object, but the only input they receive will be a single line and they
21 callable object, but the only input they receive will be a single line and they
22 should only return a line, without holding any data-dependent state between
22 should only return a line, without holding any data-dependent state between
23 calls.
23 calls.
24
24
25 As an example, the EscapedTransformer is a class so we can more clearly group
25 As an example, the EscapedTransformer is a class so we can more clearly group
26 together the functionality of dispatching to individual functions based on the
26 together the functionality of dispatching to individual functions based on the
27 starting escape character, but the only method for public use is its call
27 starting escape character, but the only method for public use is its call
28 method.
28 method.
29
29
30
30
31 ToDo
31 ToDo
32 ----
32 ----
33
33
34 - Should we make push() actually raise an exception once push_accepts_more()
34 - Should we make push() actually raise an exception once push_accepts_more()
35 returns False?
35 returns False?
36
36
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
37 - Naming cleanups. The tr_* names aren't the most elegant, though now they are
38 at least just attributes of a class so not really very exposed.
38 at least just attributes of a class so not really very exposed.
39
39
40 - Think about the best way to support dynamic things: automagic, autocall,
40 - Think about the best way to support dynamic things: automagic, autocall,
41 macros, etc.
41 macros, etc.
42
42
43 - Think of a better heuristic for the application of the transforms in
43 - Think of a better heuristic for the application of the transforms in
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
44 IPythonInputSplitter.push() than looking at the buffer ending in ':'. Idea:
45 track indentation change events (indent, dedent, nothing) and apply them only
45 track indentation change events (indent, dedent, nothing) and apply them only
46 if the indentation went up, but not otherwise.
46 if the indentation went up, but not otherwise.
47
47
48 - Think of the cleanest way for supporting user-specified transformations (the
48 - Think of the cleanest way for supporting user-specified transformations (the
49 user prefilters we had before).
49 user prefilters we had before).
50
50
51 Authors
51 Authors
52 -------
52 -------
53
53
54 * Fernando Perez
54 * Fernando Perez
55 * Brian Granger
55 * Brian Granger
56 """
56 """
57 #-----------------------------------------------------------------------------
57 #-----------------------------------------------------------------------------
58 # Copyright (C) 2010 The IPython Development Team
58 # Copyright (C) 2010 The IPython Development Team
59 #
59 #
60 # Distributed under the terms of the BSD License. The full license is in
60 # Distributed under the terms of the BSD License. The full license is in
61 # the file COPYING, distributed as part of this software.
61 # the file COPYING, distributed as part of this software.
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63 from __future__ import print_function
63 from __future__ import print_function
64
64
65 #-----------------------------------------------------------------------------
65 #-----------------------------------------------------------------------------
66 # Imports
66 # Imports
67 #-----------------------------------------------------------------------------
67 #-----------------------------------------------------------------------------
68 # stdlib
68 # stdlib
69 import ast
69 import ast
70 import codeop
70 import codeop
71 import re
71 import re
72 import sys
72 import sys
73
73
74 # IPython modules
74 # IPython modules
75 from IPython.utils.text import make_quoted_expr
75 from IPython.utils.text import make_quoted_expr
76
76
77 #-----------------------------------------------------------------------------
77 #-----------------------------------------------------------------------------
78 # Globals
78 # Globals
79 #-----------------------------------------------------------------------------
79 #-----------------------------------------------------------------------------
80
80
81 # The escape sequences that define the syntax transformations IPython will
81 # The escape sequences that define the syntax transformations IPython will
82 # apply to user input. These can NOT be just changed here: many regular
82 # apply to user input. These can NOT be just changed here: many regular
83 # expressions and other parts of the code may use their hardcoded values, and
83 # expressions and other parts of the code may use their hardcoded values, and
84 # for all intents and purposes they constitute the 'IPython syntax', so they
84 # for all intents and purposes they constitute the 'IPython syntax', so they
85 # should be considered fixed.
85 # should be considered fixed.
86
86
87 ESC_SHELL = '!' # Send line to underlying system shell
87 ESC_SHELL = '!' # Send line to underlying system shell
88 ESC_SH_CAP = '!!' # Send line to system shell and capture output
88 ESC_SH_CAP = '!!' # Send line to system shell and capture output
89 ESC_HELP = '?' # Find information about object
89 ESC_HELP = '?' # Find information about object
90 ESC_HELP2 = '??' # Find extra-detailed information about object
90 ESC_HELP2 = '??' # Find extra-detailed information about object
91 ESC_MAGIC = '%' # Call magic function
91 ESC_MAGIC = '%' # Call magic function
92 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
92 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
93 ESC_QUOTE2 = ';' # Quote all args as a single string, call
93 ESC_QUOTE2 = ';' # Quote all args as a single string, call
94 ESC_PAREN = '/' # Call first argument with rest of line as arguments
94 ESC_PAREN = '/' # Call first argument with rest of line as arguments
95
95
96 #-----------------------------------------------------------------------------
96 #-----------------------------------------------------------------------------
97 # Utilities
97 # Utilities
98 #-----------------------------------------------------------------------------
98 #-----------------------------------------------------------------------------
99
99
100 # FIXME: These are general-purpose utilities that later can be moved to the
100 # FIXME: These are general-purpose utilities that later can be moved to the
101 # general ward. Kept here for now because we're being very strict about test
101 # general ward. Kept here for now because we're being very strict about test
102 # coverage with this code, and this lets us ensure that we keep 100% coverage
102 # coverage with this code, and this lets us ensure that we keep 100% coverage
103 # while developing.
103 # while developing.
104
104
105 # compiled regexps for autoindent management
105 # compiled regexps for autoindent management
106 dedent_re = re.compile('|'.join([
106 dedent_re = re.compile('|'.join([
107 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
107 r'^\s+raise(\s.*)?$', # raise statement (+ space + other stuff, maybe)
108 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
108 r'^\s+raise\([^\)]*\).*$', # wacky raise with immediate open paren
109 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
109 r'^\s+return(\s.*)?$', # normal return (+ space + other stuff, maybe)
110 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
110 r'^\s+return\([^\)]*\).*$', # wacky return with immediate open paren
111 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
111 r'^\s+pass\s*$' # pass (optionally followed by trailing spaces)
112 ]))
112 ]))
113 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
113 ini_spaces_re = re.compile(r'^([ \t\r\f\v]+)')
114
114
115 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
115 # regexp to match pure comment lines so we don't accidentally insert 'if 1:'
116 # before pure comments
116 # before pure comments
117 comment_line_re = re.compile('^\s*\#')
117 comment_line_re = re.compile('^\s*\#')
118
118
119
119
120 def num_ini_spaces(s):
120 def num_ini_spaces(s):
121 """Return the number of initial spaces in a string.
121 """Return the number of initial spaces in a string.
122
122
123 Note that tabs are counted as a single space. For now, we do *not* support
123 Note that tabs are counted as a single space. For now, we do *not* support
124 mixing of tabs and spaces in the user's input.
124 mixing of tabs and spaces in the user's input.
125
125
126 Parameters
126 Parameters
127 ----------
127 ----------
128 s : string
128 s : string
129
129
130 Returns
130 Returns
131 -------
131 -------
132 n : int
132 n : int
133 """
133 """
134
134
135 ini_spaces = ini_spaces_re.match(s)
135 ini_spaces = ini_spaces_re.match(s)
136 if ini_spaces:
136 if ini_spaces:
137 return ini_spaces.end()
137 return ini_spaces.end()
138 else:
138 else:
139 return 0
139 return 0
140
140
141
141
142 def remove_comments(src):
142 def remove_comments(src):
143 """Remove all comments from input source.
143 """Remove all comments from input source.
144
144
145 Note: comments are NOT recognized inside of strings!
145 Note: comments are NOT recognized inside of strings!
146
146
147 Parameters
147 Parameters
148 ----------
148 ----------
149 src : string
149 src : string
150 A single or multiline input string.
150 A single or multiline input string.
151
151
152 Returns
152 Returns
153 -------
153 -------
154 String with all Python comments removed.
154 String with all Python comments removed.
155 """
155 """
156
156
157 return re.sub('#.*', '', src)
157 return re.sub('#.*', '', src)
158
158
159
159
160 def get_input_encoding():
160 def get_input_encoding():
161 """Return the default standard input encoding.
161 """Return the default standard input encoding.
162
162
163 If sys.stdin has no encoding, 'ascii' is returned."""
163 If sys.stdin has no encoding, 'ascii' is returned."""
164 # There are strange environments for which sys.stdin.encoding is None. We
164 # There are strange environments for which sys.stdin.encoding is None. We
165 # ensure that a valid encoding is returned.
165 # ensure that a valid encoding is returned.
166 encoding = getattr(sys.stdin, 'encoding', None)
166 encoding = getattr(sys.stdin, 'encoding', None)
167 if encoding is None:
167 if encoding is None:
168 encoding = 'ascii'
168 encoding = 'ascii'
169 return encoding
169 return encoding
170
170
171 #-----------------------------------------------------------------------------
171 #-----------------------------------------------------------------------------
172 # Classes and functions for normal Python syntax handling
172 # Classes and functions for normal Python syntax handling
173 #-----------------------------------------------------------------------------
173 #-----------------------------------------------------------------------------
174
174
175 class InputSplitter(object):
175 class InputSplitter(object):
176 """An object that can split Python source input in executable blocks.
176 """An object that can split Python source input in executable blocks.
177
177
178 This object is designed to be used in one of two basic modes:
178 This object is designed to be used in one of two basic modes:
179
179
180 1. By feeding it python source line-by-line, using :meth:`push`. In this
180 1. By feeding it python source line-by-line, using :meth:`push`. In this
181 mode, it will return on each push whether the currently pushed code
181 mode, it will return on each push whether the currently pushed code
182 could be executed already. In addition, it provides a method called
182 could be executed already. In addition, it provides a method called
183 :meth:`push_accepts_more` that can be used to query whether more input
183 :meth:`push_accepts_more` that can be used to query whether more input
184 can be pushed into a single interactive block.
184 can be pushed into a single interactive block.
185
185
186 2. By calling :meth:`split_blocks` with a single, multiline Python string,
186 2. By calling :meth:`split_blocks` with a single, multiline Python string,
187 that is then split into blocks each of which can be executed
187 that is then split into blocks each of which can be executed
188 interactively as a single statement.
188 interactively as a single statement.
189
189
190 This is a simple example of how an interactive terminal-based client can use
190 This is a simple example of how an interactive terminal-based client can use
191 this tool::
191 this tool::
192
192
193 isp = InputSplitter()
193 isp = InputSplitter()
194 while isp.push_accepts_more():
194 while isp.push_accepts_more():
195 indent = ' '*isp.indent_spaces
195 indent = ' '*isp.indent_spaces
196 prompt = '>>> ' + indent
196 prompt = '>>> ' + indent
197 line = indent + raw_input(prompt)
197 line = indent + raw_input(prompt)
198 isp.push(line)
198 isp.push(line)
199 print 'Input source was:\n', isp.source_reset(),
199 print 'Input source was:\n', isp.source_reset(),
200 """
200 """
201 # Number of spaces of indentation computed from input that has been pushed
201 # Number of spaces of indentation computed from input that has been pushed
202 # so far. This is the attributes callers should query to get the current
202 # so far. This is the attributes callers should query to get the current
203 # indentation level, in order to provide auto-indent facilities.
203 # indentation level, in order to provide auto-indent facilities.
204 indent_spaces = 0
204 indent_spaces = 0
205 # String, indicating the default input encoding. It is computed by default
205 # String, indicating the default input encoding. It is computed by default
206 # at initialization time via get_input_encoding(), but it can be reset by a
206 # at initialization time via get_input_encoding(), but it can be reset by a
207 # client with specific knowledge of the encoding.
207 # client with specific knowledge of the encoding.
208 encoding = ''
208 encoding = ''
209 # String where the current full source input is stored, properly encoded.
209 # String where the current full source input is stored, properly encoded.
210 # Reading this attribute is the normal way of querying the currently pushed
210 # Reading this attribute is the normal way of querying the currently pushed
211 # source code, that has been properly encoded.
211 # source code, that has been properly encoded.
212 source = ''
212 source = ''
213 # Code object corresponding to the current source. It is automatically
213 # Code object corresponding to the current source. It is automatically
214 # synced to the source, so it can be queried at any time to obtain the code
214 # synced to the source, so it can be queried at any time to obtain the code
215 # object; it will be None if the source doesn't compile to valid Python.
215 # object; it will be None if the source doesn't compile to valid Python.
216 code = None
216 code = None
217 # Input mode
217 # Input mode
218 input_mode = 'line'
218 input_mode = 'line'
219
219
220 # Private attributes
220 # Private attributes
221
221
222 # List with lines of input accumulated so far
222 # List with lines of input accumulated so far
223 _buffer = None
223 _buffer = None
224 # Command compiler
224 # Command compiler
225 _compile = None
225 _compile = None
226 # Mark when input has changed indentation all the way back to flush-left
226 # Mark when input has changed indentation all the way back to flush-left
227 _full_dedent = False
227 _full_dedent = False
228 # Boolean indicating whether the current block is complete
228 # Boolean indicating whether the current block is complete
229 _is_complete = None
229 _is_complete = None
230
230
231 def __init__(self, input_mode=None):
231 def __init__(self, input_mode=None):
232 """Create a new InputSplitter instance.
232 """Create a new InputSplitter instance.
233
233
234 Parameters
234 Parameters
235 ----------
235 ----------
236 input_mode : str
236 input_mode : str
237
237
238 One of ['line', 'cell']; default is 'line'.
238 One of ['line', 'cell']; default is 'line'.
239
239
240 The input_mode parameter controls how new inputs are used when fed via
240 The input_mode parameter controls how new inputs are used when fed via
241 the :meth:`push` method:
241 the :meth:`push` method:
242
242
243 - 'line': meant for line-oriented clients, inputs are appended one at a
243 - 'line': meant for line-oriented clients, inputs are appended one at a
244 time to the internal buffer and the whole buffer is compiled.
244 time to the internal buffer and the whole buffer is compiled.
245
245
246 - 'cell': meant for clients that can edit multi-line 'cells' of text at
246 - 'cell': meant for clients that can edit multi-line 'cells' of text at
247 a time. A cell can contain one or more blocks that can be compile in
247 a time. A cell can contain one or more blocks that can be compile in
248 'single' mode by Python. In this mode, each new input new input
248 'single' mode by Python. In this mode, each new input new input
249 completely replaces all prior inputs. Cell mode is thus equivalent
249 completely replaces all prior inputs. Cell mode is thus equivalent
250 to prepending a full reset() to every push() call.
250 to prepending a full reset() to every push() call.
251 """
251 """
252 self._buffer = []
252 self._buffer = []
253 self._compile = codeop.CommandCompiler()
253 self._compile = codeop.CommandCompiler()
254 self.encoding = get_input_encoding()
254 self.encoding = get_input_encoding()
255 self.input_mode = InputSplitter.input_mode if input_mode is None \
255 self.input_mode = InputSplitter.input_mode if input_mode is None \
256 else input_mode
256 else input_mode
257
257
258 def reset(self):
258 def reset(self):
259 """Reset the input buffer and associated state."""
259 """Reset the input buffer and associated state."""
260 self.indent_spaces = 0
260 self.indent_spaces = 0
261 self._buffer[:] = []
261 self._buffer[:] = []
262 self.source = ''
262 self.source = ''
263 self.code = None
263 self.code = None
264 self._is_complete = False
264 self._is_complete = False
265 self._full_dedent = False
265 self._full_dedent = False
266
266
267 def source_reset(self):
267 def source_reset(self):
268 """Return the input source and perform a full reset.
268 """Return the input source and perform a full reset.
269 """
269 """
270 out = self.source
270 out = self.source
271 self.reset()
271 self.reset()
272 return out
272 return out
273
273
274 def push(self, lines):
274 def push(self, lines):
275 """Push one or more lines of input.
275 """Push one or more lines of input.
276
276
277 This stores the given lines and returns a status code indicating
277 This stores the given lines and returns a status code indicating
278 whether the code forms a complete Python block or not.
278 whether the code forms a complete Python block or not.
279
279
280 Any exceptions generated in compilation are swallowed, but if an
280 Any exceptions generated in compilation are swallowed, but if an
281 exception was produced, the method returns True.
281 exception was produced, the method returns True.
282
282
283 Parameters
283 Parameters
284 ----------
284 ----------
285 lines : string
285 lines : string
286 One or more lines of Python input.
286 One or more lines of Python input.
287
287
288 Returns
288 Returns
289 -------
289 -------
290 is_complete : boolean
290 is_complete : boolean
291 True if the current input source (the result of the current input
291 True if the current input source (the result of the current input
292 plus prior inputs) forms a complete Python execution block. Note that
292 plus prior inputs) forms a complete Python execution block. Note that
293 this value is also stored as a private attribute (_is_complete), so it
293 this value is also stored as a private attribute (_is_complete), so it
294 can be queried at any time.
294 can be queried at any time.
295 """
295 """
296 if self.input_mode == 'cell':
296 if self.input_mode == 'cell':
297 self.reset()
297 self.reset()
298
298
299 self._store(lines)
299 self._store(lines)
300 source = self.source
300 source = self.source
301
301
302 # Before calling _compile(), reset the code object to None so that if an
302 # Before calling _compile(), reset the code object to None so that if an
303 # exception is raised in compilation, we don't mislead by having
303 # exception is raised in compilation, we don't mislead by having
304 # inconsistent code/source attributes.
304 # inconsistent code/source attributes.
305 self.code, self._is_complete = None, None
305 self.code, self._is_complete = None, None
306
306
307 # Honor termination lines properly
307 # Honor termination lines properly
308 if source.rstrip().endswith('\\'):
308 if source.rstrip().endswith('\\'):
309 return False
309 return False
310
310
311 self._update_indent(lines)
311 self._update_indent(lines)
312 try:
312 try:
313 self.code = self._compile(source, symbol="exec")
313 self.code = self._compile(source, symbol="exec")
314 # Invalid syntax can produce any of a number of different errors from
314 # Invalid syntax can produce any of a number of different errors from
315 # inside the compiler, so we have to catch them all. Syntax errors
315 # inside the compiler, so we have to catch them all. Syntax errors
316 # immediately produce a 'ready' block, so the invalid Python can be
316 # immediately produce a 'ready' block, so the invalid Python can be
317 # sent to the kernel for evaluation with possible ipython
317 # sent to the kernel for evaluation with possible ipython
318 # special-syntax conversion.
318 # special-syntax conversion.
319 except (SyntaxError, OverflowError, ValueError, TypeError,
319 except (SyntaxError, OverflowError, ValueError, TypeError,
320 MemoryError):
320 MemoryError):
321 self._is_complete = True
321 self._is_complete = True
322 else:
322 else:
323 # Compilation didn't produce any exceptions (though it may not have
323 # Compilation didn't produce any exceptions (though it may not have
324 # given a complete code object)
324 # given a complete code object)
325 self._is_complete = self.code is not None
325 self._is_complete = self.code is not None
326
326
327 return self._is_complete
327 return self._is_complete
328
328
329 def push_accepts_more(self):
329 def push_accepts_more(self):
330 """Return whether a block of interactive input can accept more input.
330 """Return whether a block of interactive input can accept more input.
331
331
332 This method is meant to be used by line-oriented frontends, who need to
332 This method is meant to be used by line-oriented frontends, who need to
333 guess whether a block is complete or not based solely on prior and
333 guess whether a block is complete or not based solely on prior and
334 current input lines. The InputSplitter considers it has a complete
334 current input lines. The InputSplitter considers it has a complete
335 interactive block and will not accept more input only when either a
335 interactive block and will not accept more input only when either a
336 SyntaxError is raised, or *all* of the following are true:
336 SyntaxError is raised, or *all* of the following are true:
337
337
338 1. The input compiles to a complete statement.
338 1. The input compiles to a complete statement.
339
339
340 2. The indentation level is flush-left (because if we are indented,
340 2. The indentation level is flush-left (because if we are indented,
341 like inside a function definition or for loop, we need to keep
341 like inside a function definition or for loop, we need to keep
342 reading new input).
342 reading new input).
343
343
344 3. There is one extra line consisting only of whitespace.
344 3. There is one extra line consisting only of whitespace.
345
345
346 Because of condition #3, this method should be used only by
346 Because of condition #3, this method should be used only by
347 *line-oriented* frontends, since it means that intermediate blank lines
347 *line-oriented* frontends, since it means that intermediate blank lines
348 are not allowed in function definitions (or any other indented block).
348 are not allowed in function definitions (or any other indented block).
349
349
350 Block-oriented frontends that have a separate keyboard event to
350 Block-oriented frontends that have a separate keyboard event to
351 indicate execution should use the :meth:`split_blocks` method instead.
351 indicate execution should use the :meth:`split_blocks` method instead.
352
352
353 If the current input produces a syntax error, this method immediately
353 If the current input produces a syntax error, this method immediately
354 returns False but does *not* raise the syntax error exception, as
354 returns False but does *not* raise the syntax error exception, as
355 typically clients will want to send invalid syntax to an execution
355 typically clients will want to send invalid syntax to an execution
356 backend which might convert the invalid syntax into valid Python via
356 backend which might convert the invalid syntax into valid Python via
357 one of the dynamic IPython mechanisms.
357 one of the dynamic IPython mechanisms.
358 """
358 """
359
359
360 # With incomplete input, unconditionally accept more
360 # With incomplete input, unconditionally accept more
361 if not self._is_complete:
361 if not self._is_complete:
362 return True
362 return True
363
363
364 # If we already have complete input and we're flush left, the answer
364 # If we already have complete input and we're flush left, the answer
365 # depends. In line mode, if there hasn't been any indentation,
365 # depends. In line mode, if there hasn't been any indentation,
366 # that's it. If we've come back from some indentation, we need
366 # that's it. If we've come back from some indentation, we need
367 # the blank final line to finish.
367 # the blank final line to finish.
368 # In cell mode, we need to check how many blocks the input so far
368 # In cell mode, we need to check how many blocks the input so far
369 # compiles into, because if there's already more than one full
369 # compiles into, because if there's already more than one full
370 # independent block of input, then the client has entered full
370 # independent block of input, then the client has entered full
371 # 'cell' mode and is feeding lines that each is complete. In this
371 # 'cell' mode and is feeding lines that each is complete. In this
372 # case we should then keep accepting. The Qt terminal-like console
372 # case we should then keep accepting. The Qt terminal-like console
373 # does precisely this, to provide the convenience of terminal-like
373 # does precisely this, to provide the convenience of terminal-like
374 # input of single expressions, but allowing the user (with a
374 # input of single expressions, but allowing the user (with a
375 # separate keystroke) to switch to 'cell' mode and type multiple
375 # separate keystroke) to switch to 'cell' mode and type multiple
376 # expressions in one shot.
376 # expressions in one shot.
377 if self.indent_spaces==0:
377 if self.indent_spaces==0:
378 if self.input_mode=='line':
378 if self.input_mode=='line':
379 if not self._full_dedent:
379 if not self._full_dedent:
380 return False
380 return False
381 else:
381 else:
382 try:
382 try:
383 code_ast = ast.parse(u''.join(self._buffer))
383 code_ast = ast.parse(u''.join(self._buffer))
384 except Exception:
384 except Exception:
385 return False
385 return False
386 else:
386 else:
387 if len(code_ast.body) == 1:
387 if len(code_ast.body) == 1:
388 return False
388 return False
389
389
390 # When input is complete, then termination is marked by an extra blank
390 # When input is complete, then termination is marked by an extra blank
391 # line at the end.
391 # line at the end.
392 last_line = self.source.splitlines()[-1]
392 last_line = self.source.splitlines()[-1]
393 return bool(last_line and not last_line.isspace())
393 return bool(last_line and not last_line.isspace())
394
394
395 #------------------------------------------------------------------------
395 #------------------------------------------------------------------------
396 # Private interface
396 # Private interface
397 #------------------------------------------------------------------------
397 #------------------------------------------------------------------------
398
398
399 def _find_indent(self, line):
399 def _find_indent(self, line):
400 """Compute the new indentation level for a single line.
400 """Compute the new indentation level for a single line.
401
401
402 Parameters
402 Parameters
403 ----------
403 ----------
404 line : str
404 line : str
405 A single new line of non-whitespace, non-comment Python input.
405 A single new line of non-whitespace, non-comment Python input.
406
406
407 Returns
407 Returns
408 -------
408 -------
409 indent_spaces : int
409 indent_spaces : int
410 New value for the indent level (it may be equal to self.indent_spaces
410 New value for the indent level (it may be equal to self.indent_spaces
411 if indentation doesn't change.
411 if indentation doesn't change.
412
412
413 full_dedent : boolean
413 full_dedent : boolean
414 Whether the new line causes a full flush-left dedent.
414 Whether the new line causes a full flush-left dedent.
415 """
415 """
416 indent_spaces = self.indent_spaces
416 indent_spaces = self.indent_spaces
417 full_dedent = self._full_dedent
417 full_dedent = self._full_dedent
418
418
419 inisp = num_ini_spaces(line)
419 inisp = num_ini_spaces(line)
420 if inisp < indent_spaces:
420 if inisp < indent_spaces:
421 indent_spaces = inisp
421 indent_spaces = inisp
422 if indent_spaces <= 0:
422 if indent_spaces <= 0:
423 #print 'Full dedent in text',self.source # dbg
423 #print 'Full dedent in text',self.source # dbg
424 full_dedent = True
424 full_dedent = True
425
425
426 if line[-1] == ':':
426 if line[-1] == ':':
427 indent_spaces += 4
427 indent_spaces += 4
428 elif dedent_re.match(line):
428 elif dedent_re.match(line):
429 indent_spaces -= 4
429 indent_spaces -= 4
430 if indent_spaces <= 0:
430 if indent_spaces <= 0:
431 full_dedent = True
431 full_dedent = True
432
432
433 # Safety
433 # Safety
434 if indent_spaces < 0:
434 if indent_spaces < 0:
435 indent_spaces = 0
435 indent_spaces = 0
436 #print 'safety' # dbg
436 #print 'safety' # dbg
437
437
438 return indent_spaces, full_dedent
438 return indent_spaces, full_dedent
439
439
440 def _update_indent(self, lines):
440 def _update_indent(self, lines):
441 for line in remove_comments(lines).splitlines():
441 for line in remove_comments(lines).splitlines():
442 if line and not line.isspace():
442 if line and not line.isspace():
443 self.indent_spaces, self._full_dedent = self._find_indent(line)
443 self.indent_spaces, self._full_dedent = self._find_indent(line)
444
444
445 def _store(self, lines, buffer=None, store='source'):
445 def _store(self, lines, buffer=None, store='source'):
446 """Store one or more lines of input.
446 """Store one or more lines of input.
447
447
448 If input lines are not newline-terminated, a newline is automatically
448 If input lines are not newline-terminated, a newline is automatically
449 appended."""
449 appended."""
450
450
451 if buffer is None:
451 if buffer is None:
452 buffer = self._buffer
452 buffer = self._buffer
453
453
454 if lines.endswith('\n'):
454 if lines.endswith('\n'):
455 buffer.append(lines)
455 buffer.append(lines)
456 else:
456 else:
457 buffer.append(lines+'\n')
457 buffer.append(lines+'\n')
458 setattr(self, store, self._set_source(buffer))
458 setattr(self, store, self._set_source(buffer))
459
459
460 def _set_source(self, buffer):
460 def _set_source(self, buffer):
461 return u''.join(buffer)
461 return u''.join(buffer)
462
462
463
463
464 #-----------------------------------------------------------------------------
464 #-----------------------------------------------------------------------------
465 # Functions and classes for IPython-specific syntactic support
465 # Functions and classes for IPython-specific syntactic support
466 #-----------------------------------------------------------------------------
466 #-----------------------------------------------------------------------------
467
467
468 # RegExp for splitting line contents into pre-char//first word-method//rest.
468 # RegExp for splitting line contents into pre-char//first word-method//rest.
469 # For clarity, each group in on one line.
469 # For clarity, each group in on one line.
470
470
471 line_split = re.compile("""
471 line_split = re.compile("""
472 ^(\s*) # any leading space
472 ^(\s*) # any leading space
473 ([,;/%]|!!?|\?\??) # escape character or characters
473 ([,;/%]|!!?|\?\??) # escape character or characters
474 \s*(%?[\w\.\*]*) # function/method, possibly with leading %
474 \s*(%?[\w\.\*]*) # function/method, possibly with leading %
475 # to correctly treat things like '?%magic'
475 # to correctly treat things like '?%magic'
476 (\s+.*$|$) # rest of line
476 (\s+.*$|$) # rest of line
477 """, re.VERBOSE)
477 """, re.VERBOSE)
478
478
479
479
480 def split_user_input(line):
480 def split_user_input(line):
481 """Split user input into early whitespace, esc-char, function part and rest.
481 """Split user input into early whitespace, esc-char, function part and rest.
482
482
483 This is currently handles lines with '=' in them in a very inconsistent
483 This is currently handles lines with '=' in them in a very inconsistent
484 manner.
484 manner.
485
485
486 Examples
486 Examples
487 ========
487 ========
488 >>> split_user_input('x=1')
488 >>> split_user_input('x=1')
489 ('', '', 'x=1', '')
489 ('', '', 'x=1', '')
490 >>> split_user_input('?')
490 >>> split_user_input('?')
491 ('', '?', '', '')
491 ('', '?', '', '')
492 >>> split_user_input('??')
492 >>> split_user_input('??')
493 ('', '??', '', '')
493 ('', '??', '', '')
494 >>> split_user_input(' ?')
494 >>> split_user_input(' ?')
495 (' ', '?', '', '')
495 (' ', '?', '', '')
496 >>> split_user_input(' ??')
496 >>> split_user_input(' ??')
497 (' ', '??', '', '')
497 (' ', '??', '', '')
498 >>> split_user_input('??x')
498 >>> split_user_input('??x')
499 ('', '??', 'x', '')
499 ('', '??', 'x', '')
500 >>> split_user_input('?x=1')
500 >>> split_user_input('?x=1')
501 ('', '', '?x=1', '')
501 ('', '', '?x=1', '')
502 >>> split_user_input('!ls')
502 >>> split_user_input('!ls')
503 ('', '!', 'ls', '')
503 ('', '!', 'ls', '')
504 >>> split_user_input(' !ls')
504 >>> split_user_input(' !ls')
505 (' ', '!', 'ls', '')
505 (' ', '!', 'ls', '')
506 >>> split_user_input('!!ls')
506 >>> split_user_input('!!ls')
507 ('', '!!', 'ls', '')
507 ('', '!!', 'ls', '')
508 >>> split_user_input(' !!ls')
508 >>> split_user_input(' !!ls')
509 (' ', '!!', 'ls', '')
509 (' ', '!!', 'ls', '')
510 >>> split_user_input(',ls')
510 >>> split_user_input(',ls')
511 ('', ',', 'ls', '')
511 ('', ',', 'ls', '')
512 >>> split_user_input(';ls')
512 >>> split_user_input(';ls')
513 ('', ';', 'ls', '')
513 ('', ';', 'ls', '')
514 >>> split_user_input(' ;ls')
514 >>> split_user_input(' ;ls')
515 (' ', ';', 'ls', '')
515 (' ', ';', 'ls', '')
516 >>> split_user_input('f.g(x)')
516 >>> split_user_input('f.g(x)')
517 ('', '', 'f.g(x)', '')
517 ('', '', 'f.g(x)', '')
518 >>> split_user_input('f.g (x)')
518 >>> split_user_input('f.g (x)')
519 ('', '', 'f.g', '(x)')
519 ('', '', 'f.g', '(x)')
520 >>> split_user_input('?%hist')
520 >>> split_user_input('?%hist')
521 ('', '?', '%hist', '')
521 ('', '?', '%hist', '')
522 >>> split_user_input('?x*')
522 >>> split_user_input('?x*')
523 ('', '?', 'x*', '')
523 ('', '?', 'x*', '')
524 """
524 """
525 match = line_split.match(line)
525 match = line_split.match(line)
526 if match:
526 if match:
527 lspace, esc, fpart, rest = match.groups()
527 lspace, esc, fpart, rest = match.groups()
528 else:
528 else:
529 # print "match failed for line '%s'" % line
529 # print "match failed for line '%s'" % line
530 try:
530 try:
531 fpart, rest = line.split(None, 1)
531 fpart, rest = line.split(None, 1)
532 except ValueError:
532 except ValueError:
533 # print "split failed for line '%s'" % line
533 # print "split failed for line '%s'" % line
534 fpart, rest = line,''
534 fpart, rest = line,''
535 lspace = re.match('^(\s*)(.*)', line).groups()[0]
535 lspace = re.match('^(\s*)(.*)', line).groups()[0]
536 esc = ''
536 esc = ''
537
537
538 # fpart has to be a valid python identifier, so it better be only pure
538 # fpart has to be a valid python identifier, so it better be only pure
539 # ascii, no unicode:
539 # ascii, no unicode:
540 try:
540 try:
541 fpart = fpart.encode('ascii')
541 fpart = fpart.encode('ascii')
542 except UnicodeEncodeError:
542 except UnicodeEncodeError:
543 lspace = unicode(lspace)
543 lspace = unicode(lspace)
544 rest = fpart + u' ' + rest
544 rest = fpart + u' ' + rest
545 fpart = u''
545 fpart = u''
546
546
547 #print 'line:<%s>' % line # dbg
547 #print 'line:<%s>' % line # dbg
548 #print 'esc <%s> fpart <%s> rest <%s>' % (esc,fpart.strip(),rest) # dbg
548 #print 'esc <%s> fpart <%s> rest <%s>' % (esc,fpart.strip(),rest) # dbg
549 return lspace, esc, fpart.strip(), rest.lstrip()
549 return lspace, esc, fpart.strip(), rest.lstrip()
550
550
551
551
552 # The escaped translators ALL receive a line where their own escape has been
552 # The escaped translators ALL receive a line where their own escape has been
553 # stripped. Only '?' is valid at the end of the line, all others can only be
553 # stripped. Only '?' is valid at the end of the line, all others can only be
554 # placed at the start.
554 # placed at the start.
555
555
556 class LineInfo(object):
556 class LineInfo(object):
557 """A single line of input and associated info.
557 """A single line of input and associated info.
558
558
559 This is a utility class that mostly wraps the output of
559 This is a utility class that mostly wraps the output of
560 :func:`split_user_input` into a convenient object to be passed around
560 :func:`split_user_input` into a convenient object to be passed around
561 during input transformations.
561 during input transformations.
562
562
563 Includes the following as properties:
563 Includes the following as properties:
564
564
565 line
565 line
566 The original, raw line
566 The original, raw line
567
567
568 lspace
568 lspace
569 Any early whitespace before actual text starts.
569 Any early whitespace before actual text starts.
570
570
571 esc
571 esc
572 The initial esc character (or characters, for double-char escapes like
572 The initial esc character (or characters, for double-char escapes like
573 '??' or '!!').
573 '??' or '!!').
574
574
575 fpart
575 fpart
576 The 'function part', which is basically the maximal initial sequence
576 The 'function part', which is basically the maximal initial sequence
577 of valid python identifiers and the '.' character. This is what is
577 of valid python identifiers and the '.' character. This is what is
578 checked for alias and magic transformations, used for auto-calling,
578 checked for alias and magic transformations, used for auto-calling,
579 etc.
579 etc.
580
580
581 rest
581 rest
582 Everything else on the line.
582 Everything else on the line.
583 """
583 """
584 def __init__(self, line):
584 def __init__(self, line):
585 self.line = line
585 self.line = line
586 self.lspace, self.esc, self.fpart, self.rest = \
586 self.lspace, self.esc, self.fpart, self.rest = \
587 split_user_input(line)
587 split_user_input(line)
588
588
589 def __str__(self):
589 def __str__(self):
590 return "LineInfo [%s|%s|%s|%s]" % (self.lspace, self.esc,
590 return "LineInfo [%s|%s|%s|%s]" % (self.lspace, self.esc,
591 self.fpart, self.rest)
591 self.fpart, self.rest)
592
592
593
593
594 # Transformations of the special syntaxes that don't rely on an explicit escape
594 # Transformations of the special syntaxes that don't rely on an explicit escape
595 # character but instead on patterns on the input line
595 # character but instead on patterns on the input line
596
596
597 # The core transformations are implemented as standalone functions that can be
597 # The core transformations are implemented as standalone functions that can be
598 # tested and validated in isolation. Each of these uses a regexp, we
598 # tested and validated in isolation. Each of these uses a regexp, we
599 # pre-compile these and keep them close to each function definition for clarity
599 # pre-compile these and keep them close to each function definition for clarity
600
600
601 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
601 _assign_system_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
602 r'\s*=\s*!\s*(?P<cmd>.*)')
602 r'\s*=\s*!\s*(?P<cmd>.*)')
603
603
604 def transform_assign_system(line):
604 def transform_assign_system(line):
605 """Handle the `files = !ls` syntax."""
605 """Handle the `files = !ls` syntax."""
606 m = _assign_system_re.match(line)
606 m = _assign_system_re.match(line)
607 if m is not None:
607 if m is not None:
608 cmd = m.group('cmd')
608 cmd = m.group('cmd')
609 lhs = m.group('lhs')
609 lhs = m.group('lhs')
610 expr = make_quoted_expr(cmd)
610 expr = make_quoted_expr(cmd)
611 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
611 new_line = '%s = get_ipython().getoutput(%s)' % (lhs, expr)
612 return new_line
612 return new_line
613 return line
613 return line
614
614
615
615
616 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
616 _assign_magic_re = re.compile(r'(?P<lhs>(\s*)([\w\.]+)((\s*,\s*[\w\.]+)*))'
617 r'\s*=\s*%\s*(?P<cmd>.*)')
617 r'\s*=\s*%\s*(?P<cmd>.*)')
618
618
619 def transform_assign_magic(line):
619 def transform_assign_magic(line):
620 """Handle the `a = %who` syntax."""
620 """Handle the `a = %who` syntax."""
621 m = _assign_magic_re.match(line)
621 m = _assign_magic_re.match(line)
622 if m is not None:
622 if m is not None:
623 cmd = m.group('cmd')
623 cmd = m.group('cmd')
624 lhs = m.group('lhs')
624 lhs = m.group('lhs')
625 expr = make_quoted_expr(cmd)
625 expr = make_quoted_expr(cmd)
626 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
626 new_line = '%s = get_ipython().magic(%s)' % (lhs, expr)
627 return new_line
627 return new_line
628 return line
628 return line
629
629
630
630
631 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
631 _classic_prompt_re = re.compile(r'^([ \t]*>>> |^[ \t]*\.\.\. )')
632
632
633 def transform_classic_prompt(line):
633 def transform_classic_prompt(line):
634 """Handle inputs that start with '>>> ' syntax."""
634 """Handle inputs that start with '>>> ' syntax."""
635
635
636 if not line or line.isspace():
636 if not line or line.isspace():
637 return line
637 return line
638 m = _classic_prompt_re.match(line)
638 m = _classic_prompt_re.match(line)
639 if m:
639 if m:
640 return line[len(m.group(0)):]
640 return line[len(m.group(0)):]
641 else:
641 else:
642 return line
642 return line
643
643
644
644
645 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
645 _ipy_prompt_re = re.compile(r'^([ \t]*In \[\d+\]: |^[ \t]*\ \ \ \.\.\.+: )')
646
646
647 def transform_ipy_prompt(line):
647 def transform_ipy_prompt(line):
648 """Handle inputs that start classic IPython prompt syntax."""
648 """Handle inputs that start classic IPython prompt syntax."""
649
649
650 if not line or line.isspace():
650 if not line or line.isspace():
651 return line
651 return line
652 #print 'LINE: %r' % line # dbg
652 #print 'LINE: %r' % line # dbg
653 m = _ipy_prompt_re.match(line)
653 m = _ipy_prompt_re.match(line)
654 if m:
654 if m:
655 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
655 #print 'MATCH! %r -> %r' % (line, line[len(m.group(0)):]) # dbg
656 return line[len(m.group(0)):]
656 return line[len(m.group(0)):]
657 else:
657 else:
658 return line
658 return line
659
659
660
660
661 def _make_help_call(target, esc, lspace):
662 """Prepares a pinfo(2)/psearch call from a target name and the escape
663 (i.e. ? or ??)"""
664 method = 'pinfo2' if esc == '??' \
665 else 'psearch' if '*' in target \
666 else 'pinfo'
667
668 tpl = '%sget_ipython().magic(u"%s %s")'
669 return tpl % (lspace, method, target)
670
671 _initial_space_re = re.compile(r'\s*')
672 _help_end_re = re.compile(r"""(%?
673 [a-zA-Z_*][a-zA-Z0-9_*]* # Variable name
674 (.[a-zA-Z_*][a-zA-Z0-9_*]*)* # .etc.etc
675 )
676 (\?\??)$ # ? or ??""",
677 re.VERBOSE)
678 def transform_help_end(line):
679 """Translate lines with ?/?? at the end"""
680 m = _help_end_re.search(line)
681 if m is None:
682 return line
683 target = m.group(1)
684 esc = m.group(3)
685 lspace = _initial_space_re.match(line).group(0)
686 return _make_help_call(target, esc, lspace)
687
688
661 class EscapedTransformer(object):
689 class EscapedTransformer(object):
662 """Class to transform lines that are explicitly escaped out."""
690 """Class to transform lines that are explicitly escaped out."""
663
691
664 def __init__(self):
692 def __init__(self):
665 tr = { ESC_SHELL : self._tr_system,
693 tr = { ESC_SHELL : self._tr_system,
666 ESC_SH_CAP : self._tr_system2,
694 ESC_SH_CAP : self._tr_system2,
667 ESC_HELP : self._tr_help,
695 ESC_HELP : self._tr_help,
668 ESC_HELP2 : self._tr_help,
696 ESC_HELP2 : self._tr_help,
669 ESC_MAGIC : self._tr_magic,
697 ESC_MAGIC : self._tr_magic,
670 ESC_QUOTE : self._tr_quote,
698 ESC_QUOTE : self._tr_quote,
671 ESC_QUOTE2 : self._tr_quote2,
699 ESC_QUOTE2 : self._tr_quote2,
672 ESC_PAREN : self._tr_paren }
700 ESC_PAREN : self._tr_paren }
673 self.tr = tr
701 self.tr = tr
674
702
675 # Support for syntax transformations that use explicit escapes typed by the
703 # Support for syntax transformations that use explicit escapes typed by the
676 # user at the beginning of a line
704 # user at the beginning of a line
677 @staticmethod
705 @staticmethod
678 def _tr_system(line_info):
706 def _tr_system(line_info):
679 "Translate lines escaped with: !"
707 "Translate lines escaped with: !"
680 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
708 cmd = line_info.line.lstrip().lstrip(ESC_SHELL)
681 return '%sget_ipython().system(%s)' % (line_info.lspace,
709 return '%sget_ipython().system(%s)' % (line_info.lspace,
682 make_quoted_expr(cmd))
710 make_quoted_expr(cmd))
683
711
684 @staticmethod
712 @staticmethod
685 def _tr_system2(line_info):
713 def _tr_system2(line_info):
686 "Translate lines escaped with: !!"
714 "Translate lines escaped with: !!"
687 cmd = line_info.line.lstrip()[2:]
715 cmd = line_info.line.lstrip()[2:]
688 return '%sget_ipython().getoutput(%s)' % (line_info.lspace,
716 return '%sget_ipython().getoutput(%s)' % (line_info.lspace,
689 make_quoted_expr(cmd))
717 make_quoted_expr(cmd))
690
718
691 @staticmethod
719 @staticmethod
692 def _tr_help(line_info):
720 def _tr_help(line_info):
693 "Translate lines escaped with: ?/??"
721 "Translate lines escaped with: ?/??"
694 # A naked help line should just fire the intro help screen
722 # A naked help line should just fire the intro help screen
695 if not line_info.line[1:]:
723 if not line_info.line[1:]:
696 return 'get_ipython().show_usage()'
724 return 'get_ipython().show_usage()'
697
725
698 # There may be one or two '?' at the end, move them to the front so that
726 return _make_help_call(line_info.fpart, line_info.esc, line_info.lspace)
699 # the rest of the logic can assume escapes are at the start
700 l_ori = line_info
701 line = line_info.line
702 if line.endswith('?'):
703 line = line[-1] + line[:-1]
704 if line.endswith('?'):
705 line = line[-1] + line[:-1]
706 line_info = LineInfo(line)
707
708 # From here on, simply choose which level of detail to get, and
709 # special-case the psearch syntax
710 pinfo = 'pinfo' # default
711 if '*' in line_info.line:
712 pinfo = 'psearch'
713 elif line_info.esc == '??':
714 pinfo = 'pinfo2'
715
716 tpl = '%sget_ipython().magic(u"%s %s")'
717 return tpl % (line_info.lspace, pinfo,
718 ' '.join([line_info.fpart, line_info.rest]).strip())
719
727
720 @staticmethod
728 @staticmethod
721 def _tr_magic(line_info):
729 def _tr_magic(line_info):
722 "Translate lines escaped with: %"
730 "Translate lines escaped with: %"
723 tpl = '%sget_ipython().magic(%s)'
731 tpl = '%sget_ipython().magic(%s)'
724 cmd = make_quoted_expr(' '.join([line_info.fpart,
732 cmd = make_quoted_expr(' '.join([line_info.fpart,
725 line_info.rest]).strip())
733 line_info.rest]).strip())
726 return tpl % (line_info.lspace, cmd)
734 return tpl % (line_info.lspace, cmd)
727
735
728 @staticmethod
736 @staticmethod
729 def _tr_quote(line_info):
737 def _tr_quote(line_info):
730 "Translate lines escaped with: ,"
738 "Translate lines escaped with: ,"
731 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
739 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
732 '", "'.join(line_info.rest.split()) )
740 '", "'.join(line_info.rest.split()) )
733
741
734 @staticmethod
742 @staticmethod
735 def _tr_quote2(line_info):
743 def _tr_quote2(line_info):
736 "Translate lines escaped with: ;"
744 "Translate lines escaped with: ;"
737 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
745 return '%s%s("%s")' % (line_info.lspace, line_info.fpart,
738 line_info.rest)
746 line_info.rest)
739
747
740 @staticmethod
748 @staticmethod
741 def _tr_paren(line_info):
749 def _tr_paren(line_info):
742 "Translate lines escaped with: /"
750 "Translate lines escaped with: /"
743 return '%s%s(%s)' % (line_info.lspace, line_info.fpart,
751 return '%s%s(%s)' % (line_info.lspace, line_info.fpart,
744 ", ".join(line_info.rest.split()))
752 ", ".join(line_info.rest.split()))
745
753
746 def __call__(self, line):
754 def __call__(self, line):
747 """Class to transform lines that are explicitly escaped out.
755 """Class to transform lines that are explicitly escaped out.
748
756
749 This calls the above _tr_* static methods for the actual line
757 This calls the above _tr_* static methods for the actual line
750 translations."""
758 translations."""
751
759
752 # Empty lines just get returned unmodified
760 # Empty lines just get returned unmodified
753 if not line or line.isspace():
761 if not line or line.isspace():
754 return line
762 return line
755
763
756 # Get line endpoints, where the escapes can be
764 # Get line endpoints, where the escapes can be
757 line_info = LineInfo(line)
765 line_info = LineInfo(line)
758
766
759 # If the escape is not at the start, only '?' needs to be special-cased.
760 # All other escapes are only valid at the start
761 if not line_info.esc in self.tr:
767 if not line_info.esc in self.tr:
762 if line.endswith(ESC_HELP):
763 return self._tr_help(line_info)
764 else:
765 # If we don't recognize the escape, don't modify the line
768 # If we don't recognize the escape, don't modify the line
766 return line
769 return line
767
770
768 return self.tr[line_info.esc](line_info)
771 return self.tr[line_info.esc](line_info)
769
772
770
773
771 # A function-looking object to be used by the rest of the code. The purpose of
774 # A function-looking object to be used by the rest of the code. The purpose of
772 # the class in this case is to organize related functionality, more than to
775 # the class in this case is to organize related functionality, more than to
773 # manage state.
776 # manage state.
774 transform_escaped = EscapedTransformer()
777 transform_escaped = EscapedTransformer()
775
778
776
779
777 class IPythonInputSplitter(InputSplitter):
780 class IPythonInputSplitter(InputSplitter):
778 """An input splitter that recognizes all of IPython's special syntax."""
781 """An input splitter that recognizes all of IPython's special syntax."""
779
782
780 # String with raw, untransformed input.
783 # String with raw, untransformed input.
781 source_raw = ''
784 source_raw = ''
782
785
783 # Private attributes
786 # Private attributes
784
787
785 # List with lines of raw input accumulated so far.
788 # List with lines of raw input accumulated so far.
786 _buffer_raw = None
789 _buffer_raw = None
787
790
788 def __init__(self, input_mode=None):
791 def __init__(self, input_mode=None):
789 InputSplitter.__init__(self, input_mode)
792 InputSplitter.__init__(self, input_mode)
790 self._buffer_raw = []
793 self._buffer_raw = []
791
794
792 def reset(self):
795 def reset(self):
793 """Reset the input buffer and associated state."""
796 """Reset the input buffer and associated state."""
794 InputSplitter.reset(self)
797 InputSplitter.reset(self)
795 self._buffer_raw[:] = []
798 self._buffer_raw[:] = []
796 self.source_raw = ''
799 self.source_raw = ''
797
800
798 def source_raw_reset(self):
801 def source_raw_reset(self):
799 """Return input and raw source and perform a full reset.
802 """Return input and raw source and perform a full reset.
800 """
803 """
801 out = self.source
804 out = self.source
802 out_r = self.source_raw
805 out_r = self.source_raw
803 self.reset()
806 self.reset()
804 return out, out_r
807 return out, out_r
805
808
806 def push(self, lines):
809 def push(self, lines):
807 """Push one or more lines of IPython input.
810 """Push one or more lines of IPython input.
808 """
811 """
809 if not lines:
812 if not lines:
810 return super(IPythonInputSplitter, self).push(lines)
813 return super(IPythonInputSplitter, self).push(lines)
811
814
812 # We must ensure all input is pure unicode
815 # We must ensure all input is pure unicode
813 if type(lines)==str:
816 if type(lines)==str:
814 lines = lines.decode(self.encoding)
817 lines = lines.decode(self.encoding)
815
818
816 lines_list = lines.splitlines()
819 lines_list = lines.splitlines()
817
820
818 transforms = [transform_escaped, transform_assign_system,
821 transforms = [transform_escaped, transform_help_end,
819 transform_assign_magic, transform_ipy_prompt,
822 transform_assign_system, transform_assign_magic,
820 transform_classic_prompt]
823 transform_ipy_prompt, transform_classic_prompt]
821
824
822 # Transform logic
825 # Transform logic
823 #
826 #
824 # We only apply the line transformers to the input if we have either no
827 # We only apply the line transformers to the input if we have either no
825 # input yet, or complete input, or if the last line of the buffer ends
828 # input yet, or complete input, or if the last line of the buffer ends
826 # with ':' (opening an indented block). This prevents the accidental
829 # with ':' (opening an indented block). This prevents the accidental
827 # transformation of escapes inside multiline expressions like
830 # transformation of escapes inside multiline expressions like
828 # triple-quoted strings or parenthesized expressions.
831 # triple-quoted strings or parenthesized expressions.
829 #
832 #
830 # The last heuristic, while ugly, ensures that the first line of an
833 # The last heuristic, while ugly, ensures that the first line of an
831 # indented block is correctly transformed.
834 # indented block is correctly transformed.
832 #
835 #
833 # FIXME: try to find a cleaner approach for this last bit.
836 # FIXME: try to find a cleaner approach for this last bit.
834
837
835 # If we were in 'block' mode, since we're going to pump the parent
838 # If we were in 'block' mode, since we're going to pump the parent
836 # class by hand line by line, we need to temporarily switch out to
839 # class by hand line by line, we need to temporarily switch out to
837 # 'line' mode, do a single manual reset and then feed the lines one
840 # 'line' mode, do a single manual reset and then feed the lines one
838 # by one. Note that this only matters if the input has more than one
841 # by one. Note that this only matters if the input has more than one
839 # line.
842 # line.
840 changed_input_mode = False
843 changed_input_mode = False
841
844
842 if self.input_mode == 'cell':
845 if self.input_mode == 'cell':
843 self.reset()
846 self.reset()
844 changed_input_mode = True
847 changed_input_mode = True
845 saved_input_mode = 'cell'
848 saved_input_mode = 'cell'
846 self.input_mode = 'line'
849 self.input_mode = 'line'
847
850
848 # Store raw source before applying any transformations to it. Note
851 # Store raw source before applying any transformations to it. Note
849 # that this must be done *after* the reset() call that would otherwise
852 # that this must be done *after* the reset() call that would otherwise
850 # flush the buffer.
853 # flush the buffer.
851 self._store(lines, self._buffer_raw, 'source_raw')
854 self._store(lines, self._buffer_raw, 'source_raw')
852
855
853 try:
856 try:
854 push = super(IPythonInputSplitter, self).push
857 push = super(IPythonInputSplitter, self).push
855 for line in lines_list:
858 for line in lines_list:
856 if self._is_complete or not self._buffer or \
859 if self._is_complete or not self._buffer or \
857 (self._buffer and self._buffer[-1].rstrip().endswith(':')):
860 (self._buffer and self._buffer[-1].rstrip().endswith(':')):
858 for f in transforms:
861 for f in transforms:
859 line = f(line)
862 line = f(line)
860
863
861 out = push(line)
864 out = push(line)
862 finally:
865 finally:
863 if changed_input_mode:
866 if changed_input_mode:
864 self.input_mode = saved_input_mode
867 self.input_mode = saved_input_mode
865 return out
868 return out
@@ -1,653 +1,664 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """Tests for the inputsplitter module.
2 """Tests for the inputsplitter module.
3
3
4 Authors
4 Authors
5 -------
5 -------
6 * Fernando Perez
6 * Fernando Perez
7 * Robert Kern
7 * Robert Kern
8 """
8 """
9 #-----------------------------------------------------------------------------
9 #-----------------------------------------------------------------------------
10 # Copyright (C) 2010 The IPython Development Team
10 # Copyright (C) 2010 The IPython Development Team
11 #
11 #
12 # Distributed under the terms of the BSD License. The full license is in
12 # Distributed under the terms of the BSD License. The full license is in
13 # the file COPYING, distributed as part of this software.
13 # the file COPYING, distributed as part of this software.
14 #-----------------------------------------------------------------------------
14 #-----------------------------------------------------------------------------
15
15
16 #-----------------------------------------------------------------------------
16 #-----------------------------------------------------------------------------
17 # Imports
17 # Imports
18 #-----------------------------------------------------------------------------
18 #-----------------------------------------------------------------------------
19 # stdlib
19 # stdlib
20 import unittest
20 import unittest
21 import sys
21 import sys
22
22
23 # Third party
23 # Third party
24 import nose.tools as nt
24 import nose.tools as nt
25
25
26 # Our own
26 # Our own
27 from IPython.core import inputsplitter as isp
27 from IPython.core import inputsplitter as isp
28
28
29 #-----------------------------------------------------------------------------
29 #-----------------------------------------------------------------------------
30 # Semi-complete examples (also used as tests)
30 # Semi-complete examples (also used as tests)
31 #-----------------------------------------------------------------------------
31 #-----------------------------------------------------------------------------
32
32
33 # Note: at the bottom, there's a slightly more complete version of this that
33 # Note: at the bottom, there's a slightly more complete version of this that
34 # can be useful during development of code here.
34 # can be useful during development of code here.
35
35
36 def mini_interactive_loop(input_func):
36 def mini_interactive_loop(input_func):
37 """Minimal example of the logic of an interactive interpreter loop.
37 """Minimal example of the logic of an interactive interpreter loop.
38
38
39 This serves as an example, and it is used by the test system with a fake
39 This serves as an example, and it is used by the test system with a fake
40 raw_input that simulates interactive input."""
40 raw_input that simulates interactive input."""
41
41
42 from IPython.core.inputsplitter import InputSplitter
42 from IPython.core.inputsplitter import InputSplitter
43
43
44 isp = InputSplitter()
44 isp = InputSplitter()
45 # In practice, this input loop would be wrapped in an outside loop to read
45 # In practice, this input loop would be wrapped in an outside loop to read
46 # input indefinitely, until some exit/quit command was issued. Here we
46 # input indefinitely, until some exit/quit command was issued. Here we
47 # only illustrate the basic inner loop.
47 # only illustrate the basic inner loop.
48 while isp.push_accepts_more():
48 while isp.push_accepts_more():
49 indent = ' '*isp.indent_spaces
49 indent = ' '*isp.indent_spaces
50 prompt = '>>> ' + indent
50 prompt = '>>> ' + indent
51 line = indent + input_func(prompt)
51 line = indent + input_func(prompt)
52 isp.push(line)
52 isp.push(line)
53
53
54 # Here we just return input so we can use it in a test suite, but a real
54 # Here we just return input so we can use it in a test suite, but a real
55 # interpreter would instead send it for execution somewhere.
55 # interpreter would instead send it for execution somewhere.
56 src = isp.source_reset()
56 src = isp.source_reset()
57 #print 'Input source was:\n', src # dbg
57 #print 'Input source was:\n', src # dbg
58 return src
58 return src
59
59
60 #-----------------------------------------------------------------------------
60 #-----------------------------------------------------------------------------
61 # Test utilities, just for local use
61 # Test utilities, just for local use
62 #-----------------------------------------------------------------------------
62 #-----------------------------------------------------------------------------
63
63
64 def assemble(block):
64 def assemble(block):
65 """Assemble a block into multi-line sub-blocks."""
65 """Assemble a block into multi-line sub-blocks."""
66 return ['\n'.join(sub_block)+'\n' for sub_block in block]
66 return ['\n'.join(sub_block)+'\n' for sub_block in block]
67
67
68
68
69 def pseudo_input(lines):
69 def pseudo_input(lines):
70 """Return a function that acts like raw_input but feeds the input list."""
70 """Return a function that acts like raw_input but feeds the input list."""
71 ilines = iter(lines)
71 ilines = iter(lines)
72 def raw_in(prompt):
72 def raw_in(prompt):
73 try:
73 try:
74 return next(ilines)
74 return next(ilines)
75 except StopIteration:
75 except StopIteration:
76 return ''
76 return ''
77 return raw_in
77 return raw_in
78
78
79 #-----------------------------------------------------------------------------
79 #-----------------------------------------------------------------------------
80 # Tests
80 # Tests
81 #-----------------------------------------------------------------------------
81 #-----------------------------------------------------------------------------
82 def test_spaces():
82 def test_spaces():
83 tests = [('', 0),
83 tests = [('', 0),
84 (' ', 1),
84 (' ', 1),
85 ('\n', 0),
85 ('\n', 0),
86 (' \n', 1),
86 (' \n', 1),
87 ('x', 0),
87 ('x', 0),
88 (' x', 1),
88 (' x', 1),
89 (' x',2),
89 (' x',2),
90 (' x',4),
90 (' x',4),
91 # Note: tabs are counted as a single whitespace!
91 # Note: tabs are counted as a single whitespace!
92 ('\tx', 1),
92 ('\tx', 1),
93 ('\t x', 2),
93 ('\t x', 2),
94 ]
94 ]
95
95
96 for s, nsp in tests:
96 for s, nsp in tests:
97 nt.assert_equal(isp.num_ini_spaces(s), nsp)
97 nt.assert_equal(isp.num_ini_spaces(s), nsp)
98
98
99
99
100 def test_remove_comments():
100 def test_remove_comments():
101 tests = [('text', 'text'),
101 tests = [('text', 'text'),
102 ('text # comment', 'text '),
102 ('text # comment', 'text '),
103 ('text # comment\n', 'text \n'),
103 ('text # comment\n', 'text \n'),
104 ('text # comment \n', 'text \n'),
104 ('text # comment \n', 'text \n'),
105 ('line # c \nline\n','line \nline\n'),
105 ('line # c \nline\n','line \nline\n'),
106 ('line # c \nline#c2 \nline\nline #c\n\n',
106 ('line # c \nline#c2 \nline\nline #c\n\n',
107 'line \nline\nline\nline \n\n'),
107 'line \nline\nline\nline \n\n'),
108 ]
108 ]
109
109
110 for inp, out in tests:
110 for inp, out in tests:
111 nt.assert_equal(isp.remove_comments(inp), out)
111 nt.assert_equal(isp.remove_comments(inp), out)
112
112
113
113
114 def test_get_input_encoding():
114 def test_get_input_encoding():
115 encoding = isp.get_input_encoding()
115 encoding = isp.get_input_encoding()
116 nt.assert_true(isinstance(encoding, basestring))
116 nt.assert_true(isinstance(encoding, basestring))
117 # simple-minded check that at least encoding a simple string works with the
117 # simple-minded check that at least encoding a simple string works with the
118 # encoding we got.
118 # encoding we got.
119 nt.assert_equal('test'.encode(encoding), 'test')
119 nt.assert_equal('test'.encode(encoding), 'test')
120
120
121
121
122 class NoInputEncodingTestCase(unittest.TestCase):
122 class NoInputEncodingTestCase(unittest.TestCase):
123 def setUp(self):
123 def setUp(self):
124 self.old_stdin = sys.stdin
124 self.old_stdin = sys.stdin
125 class X: pass
125 class X: pass
126 fake_stdin = X()
126 fake_stdin = X()
127 sys.stdin = fake_stdin
127 sys.stdin = fake_stdin
128
128
129 def test(self):
129 def test(self):
130 # Verify that if sys.stdin has no 'encoding' attribute we do the right
130 # Verify that if sys.stdin has no 'encoding' attribute we do the right
131 # thing
131 # thing
132 enc = isp.get_input_encoding()
132 enc = isp.get_input_encoding()
133 self.assertEqual(enc, 'ascii')
133 self.assertEqual(enc, 'ascii')
134
134
135 def tearDown(self):
135 def tearDown(self):
136 sys.stdin = self.old_stdin
136 sys.stdin = self.old_stdin
137
137
138
138
139 class InputSplitterTestCase(unittest.TestCase):
139 class InputSplitterTestCase(unittest.TestCase):
140 def setUp(self):
140 def setUp(self):
141 self.isp = isp.InputSplitter()
141 self.isp = isp.InputSplitter()
142
142
143 def test_reset(self):
143 def test_reset(self):
144 isp = self.isp
144 isp = self.isp
145 isp.push('x=1')
145 isp.push('x=1')
146 isp.reset()
146 isp.reset()
147 self.assertEqual(isp._buffer, [])
147 self.assertEqual(isp._buffer, [])
148 self.assertEqual(isp.indent_spaces, 0)
148 self.assertEqual(isp.indent_spaces, 0)
149 self.assertEqual(isp.source, '')
149 self.assertEqual(isp.source, '')
150 self.assertEqual(isp.code, None)
150 self.assertEqual(isp.code, None)
151 self.assertEqual(isp._is_complete, False)
151 self.assertEqual(isp._is_complete, False)
152
152
153 def test_source(self):
153 def test_source(self):
154 self.isp._store('1')
154 self.isp._store('1')
155 self.isp._store('2')
155 self.isp._store('2')
156 self.assertEqual(self.isp.source, '1\n2\n')
156 self.assertEqual(self.isp.source, '1\n2\n')
157 self.assertTrue(len(self.isp._buffer)>0)
157 self.assertTrue(len(self.isp._buffer)>0)
158 self.assertEqual(self.isp.source_reset(), '1\n2\n')
158 self.assertEqual(self.isp.source_reset(), '1\n2\n')
159 self.assertEqual(self.isp._buffer, [])
159 self.assertEqual(self.isp._buffer, [])
160 self.assertEqual(self.isp.source, '')
160 self.assertEqual(self.isp.source, '')
161
161
162 def test_indent(self):
162 def test_indent(self):
163 isp = self.isp # shorthand
163 isp = self.isp # shorthand
164 isp.push('x=1')
164 isp.push('x=1')
165 self.assertEqual(isp.indent_spaces, 0)
165 self.assertEqual(isp.indent_spaces, 0)
166 isp.push('if 1:\n x=1')
166 isp.push('if 1:\n x=1')
167 self.assertEqual(isp.indent_spaces, 4)
167 self.assertEqual(isp.indent_spaces, 4)
168 isp.push('y=2\n')
168 isp.push('y=2\n')
169 self.assertEqual(isp.indent_spaces, 0)
169 self.assertEqual(isp.indent_spaces, 0)
170
170
171 def test_indent2(self):
171 def test_indent2(self):
172 # In cell mode, inputs must be fed in whole blocks, so skip this test
172 # In cell mode, inputs must be fed in whole blocks, so skip this test
173 if self.isp.input_mode == 'cell': return
173 if self.isp.input_mode == 'cell': return
174
174
175 isp = self.isp
175 isp = self.isp
176 isp.push('if 1:')
176 isp.push('if 1:')
177 self.assertEqual(isp.indent_spaces, 4)
177 self.assertEqual(isp.indent_spaces, 4)
178 isp.push(' x=1')
178 isp.push(' x=1')
179 self.assertEqual(isp.indent_spaces, 4)
179 self.assertEqual(isp.indent_spaces, 4)
180 # Blank lines shouldn't change the indent level
180 # Blank lines shouldn't change the indent level
181 isp.push(' '*2)
181 isp.push(' '*2)
182 self.assertEqual(isp.indent_spaces, 4)
182 self.assertEqual(isp.indent_spaces, 4)
183
183
184 def test_indent3(self):
184 def test_indent3(self):
185 # In cell mode, inputs must be fed in whole blocks, so skip this test
185 # In cell mode, inputs must be fed in whole blocks, so skip this test
186 if self.isp.input_mode == 'cell': return
186 if self.isp.input_mode == 'cell': return
187
187
188 isp = self.isp
188 isp = self.isp
189 # When a multiline statement contains parens or multiline strings, we
189 # When a multiline statement contains parens or multiline strings, we
190 # shouldn't get confused.
190 # shouldn't get confused.
191 isp.push("if 1:")
191 isp.push("if 1:")
192 isp.push(" x = (1+\n 2)")
192 isp.push(" x = (1+\n 2)")
193 self.assertEqual(isp.indent_spaces, 4)
193 self.assertEqual(isp.indent_spaces, 4)
194
194
195 def test_dedent_pass(self):
195 def test_dedent_pass(self):
196 isp = self.isp # shorthand
196 isp = self.isp # shorthand
197 # should NOT cause dedent
197 # should NOT cause dedent
198 isp.push('if 1:\n passes = 5')
198 isp.push('if 1:\n passes = 5')
199 self.assertEqual(isp.indent_spaces, 4)
199 self.assertEqual(isp.indent_spaces, 4)
200 isp.push('if 1:\n pass')
200 isp.push('if 1:\n pass')
201 self.assertEqual(isp.indent_spaces, 0)
201 self.assertEqual(isp.indent_spaces, 0)
202 isp.push('if 1:\n pass ')
202 isp.push('if 1:\n pass ')
203 self.assertEqual(isp.indent_spaces, 0)
203 self.assertEqual(isp.indent_spaces, 0)
204
204
205 def test_dedent_raise(self):
205 def test_dedent_raise(self):
206 isp = self.isp # shorthand
206 isp = self.isp # shorthand
207 # should NOT cause dedent
207 # should NOT cause dedent
208 isp.push('if 1:\n raised = 4')
208 isp.push('if 1:\n raised = 4')
209 self.assertEqual(isp.indent_spaces, 4)
209 self.assertEqual(isp.indent_spaces, 4)
210 isp.push('if 1:\n raise TypeError()')
210 isp.push('if 1:\n raise TypeError()')
211 self.assertEqual(isp.indent_spaces, 0)
211 self.assertEqual(isp.indent_spaces, 0)
212 isp.push('if 1:\n raise')
212 isp.push('if 1:\n raise')
213 self.assertEqual(isp.indent_spaces, 0)
213 self.assertEqual(isp.indent_spaces, 0)
214 isp.push('if 1:\n raise ')
214 isp.push('if 1:\n raise ')
215 self.assertEqual(isp.indent_spaces, 0)
215 self.assertEqual(isp.indent_spaces, 0)
216
216
217 def test_dedent_return(self):
217 def test_dedent_return(self):
218 isp = self.isp # shorthand
218 isp = self.isp # shorthand
219 # should NOT cause dedent
219 # should NOT cause dedent
220 isp.push('if 1:\n returning = 4')
220 isp.push('if 1:\n returning = 4')
221 self.assertEqual(isp.indent_spaces, 4)
221 self.assertEqual(isp.indent_spaces, 4)
222 isp.push('if 1:\n return 5 + 493')
222 isp.push('if 1:\n return 5 + 493')
223 self.assertEqual(isp.indent_spaces, 0)
223 self.assertEqual(isp.indent_spaces, 0)
224 isp.push('if 1:\n return')
224 isp.push('if 1:\n return')
225 self.assertEqual(isp.indent_spaces, 0)
225 self.assertEqual(isp.indent_spaces, 0)
226 isp.push('if 1:\n return ')
226 isp.push('if 1:\n return ')
227 self.assertEqual(isp.indent_spaces, 0)
227 self.assertEqual(isp.indent_spaces, 0)
228 isp.push('if 1:\n return(0)')
228 isp.push('if 1:\n return(0)')
229 self.assertEqual(isp.indent_spaces, 0)
229 self.assertEqual(isp.indent_spaces, 0)
230
230
231 def test_push(self):
231 def test_push(self):
232 isp = self.isp
232 isp = self.isp
233 self.assertTrue(isp.push('x=1'))
233 self.assertTrue(isp.push('x=1'))
234
234
235 def test_push2(self):
235 def test_push2(self):
236 isp = self.isp
236 isp = self.isp
237 self.assertFalse(isp.push('if 1:'))
237 self.assertFalse(isp.push('if 1:'))
238 for line in [' x=1', '# a comment', ' y=2']:
238 for line in [' x=1', '# a comment', ' y=2']:
239 self.assertTrue(isp.push(line))
239 self.assertTrue(isp.push(line))
240
240
241 def test_push3(self):
241 def test_push3(self):
242 isp = self.isp
242 isp = self.isp
243 isp.push('if True:')
243 isp.push('if True:')
244 isp.push(' a = 1')
244 isp.push(' a = 1')
245 self.assertFalse(isp.push('b = [1,'))
245 self.assertFalse(isp.push('b = [1,'))
246
246
247 def test_replace_mode(self):
247 def test_replace_mode(self):
248 isp = self.isp
248 isp = self.isp
249 isp.input_mode = 'cell'
249 isp.input_mode = 'cell'
250 isp.push('x=1')
250 isp.push('x=1')
251 self.assertEqual(isp.source, 'x=1\n')
251 self.assertEqual(isp.source, 'x=1\n')
252 isp.push('x=2')
252 isp.push('x=2')
253 self.assertEqual(isp.source, 'x=2\n')
253 self.assertEqual(isp.source, 'x=2\n')
254
254
255 def test_push_accepts_more(self):
255 def test_push_accepts_more(self):
256 isp = self.isp
256 isp = self.isp
257 isp.push('x=1')
257 isp.push('x=1')
258 self.assertFalse(isp.push_accepts_more())
258 self.assertFalse(isp.push_accepts_more())
259
259
260 def test_push_accepts_more2(self):
260 def test_push_accepts_more2(self):
261 # In cell mode, inputs must be fed in whole blocks, so skip this test
261 # In cell mode, inputs must be fed in whole blocks, so skip this test
262 if self.isp.input_mode == 'cell': return
262 if self.isp.input_mode == 'cell': return
263
263
264 isp = self.isp
264 isp = self.isp
265 isp.push('if 1:')
265 isp.push('if 1:')
266 self.assertTrue(isp.push_accepts_more())
266 self.assertTrue(isp.push_accepts_more())
267 isp.push(' x=1')
267 isp.push(' x=1')
268 self.assertTrue(isp.push_accepts_more())
268 self.assertTrue(isp.push_accepts_more())
269 isp.push('')
269 isp.push('')
270 self.assertFalse(isp.push_accepts_more())
270 self.assertFalse(isp.push_accepts_more())
271
271
272 def test_push_accepts_more3(self):
272 def test_push_accepts_more3(self):
273 isp = self.isp
273 isp = self.isp
274 isp.push("x = (2+\n3)")
274 isp.push("x = (2+\n3)")
275 self.assertFalse(isp.push_accepts_more())
275 self.assertFalse(isp.push_accepts_more())
276
276
277 def test_push_accepts_more4(self):
277 def test_push_accepts_more4(self):
278 # In cell mode, inputs must be fed in whole blocks, so skip this test
278 # In cell mode, inputs must be fed in whole blocks, so skip this test
279 if self.isp.input_mode == 'cell': return
279 if self.isp.input_mode == 'cell': return
280
280
281 isp = self.isp
281 isp = self.isp
282 # When a multiline statement contains parens or multiline strings, we
282 # When a multiline statement contains parens or multiline strings, we
283 # shouldn't get confused.
283 # shouldn't get confused.
284 # FIXME: we should be able to better handle de-dents in statements like
284 # FIXME: we should be able to better handle de-dents in statements like
285 # multiline strings and multiline expressions (continued with \ or
285 # multiline strings and multiline expressions (continued with \ or
286 # parens). Right now we aren't handling the indentation tracking quite
286 # parens). Right now we aren't handling the indentation tracking quite
287 # correctly with this, though in practice it may not be too much of a
287 # correctly with this, though in practice it may not be too much of a
288 # problem. We'll need to see.
288 # problem. We'll need to see.
289 isp.push("if 1:")
289 isp.push("if 1:")
290 isp.push(" x = (2+")
290 isp.push(" x = (2+")
291 isp.push(" 3)")
291 isp.push(" 3)")
292 self.assertTrue(isp.push_accepts_more())
292 self.assertTrue(isp.push_accepts_more())
293 isp.push(" y = 3")
293 isp.push(" y = 3")
294 self.assertTrue(isp.push_accepts_more())
294 self.assertTrue(isp.push_accepts_more())
295 isp.push('')
295 isp.push('')
296 self.assertFalse(isp.push_accepts_more())
296 self.assertFalse(isp.push_accepts_more())
297
297
298 def test_push_accepts_more5(self):
298 def test_push_accepts_more5(self):
299 # In cell mode, inputs must be fed in whole blocks, so skip this test
299 # In cell mode, inputs must be fed in whole blocks, so skip this test
300 if self.isp.input_mode == 'cell': return
300 if self.isp.input_mode == 'cell': return
301
301
302 isp = self.isp
302 isp = self.isp
303 isp.push('try:')
303 isp.push('try:')
304 isp.push(' a = 5')
304 isp.push(' a = 5')
305 isp.push('except:')
305 isp.push('except:')
306 isp.push(' raise')
306 isp.push(' raise')
307 self.assertTrue(isp.push_accepts_more())
307 self.assertTrue(isp.push_accepts_more())
308
308
309 def test_continuation(self):
309 def test_continuation(self):
310 isp = self.isp
310 isp = self.isp
311 isp.push("import os, \\")
311 isp.push("import os, \\")
312 self.assertTrue(isp.push_accepts_more())
312 self.assertTrue(isp.push_accepts_more())
313 isp.push("sys")
313 isp.push("sys")
314 self.assertFalse(isp.push_accepts_more())
314 self.assertFalse(isp.push_accepts_more())
315
315
316 def test_syntax_error(self):
316 def test_syntax_error(self):
317 isp = self.isp
317 isp = self.isp
318 # Syntax errors immediately produce a 'ready' block, so the invalid
318 # Syntax errors immediately produce a 'ready' block, so the invalid
319 # Python can be sent to the kernel for evaluation with possible ipython
319 # Python can be sent to the kernel for evaluation with possible ipython
320 # special-syntax conversion.
320 # special-syntax conversion.
321 isp.push('run foo')
321 isp.push('run foo')
322 self.assertFalse(isp.push_accepts_more())
322 self.assertFalse(isp.push_accepts_more())
323
323
324 def test_unicode(self):
324 def test_unicode(self):
325 self.isp.push(u"PΓ©rez")
325 self.isp.push(u"PΓ©rez")
326 self.isp.push(u'\xc3\xa9')
326 self.isp.push(u'\xc3\xa9')
327 self.isp.push(u"u'\xc3\xa9'")
327 self.isp.push(u"u'\xc3\xa9'")
328
328
329 class InteractiveLoopTestCase(unittest.TestCase):
329 class InteractiveLoopTestCase(unittest.TestCase):
330 """Tests for an interactive loop like a python shell.
330 """Tests for an interactive loop like a python shell.
331 """
331 """
332 def check_ns(self, lines, ns):
332 def check_ns(self, lines, ns):
333 """Validate that the given input lines produce the resulting namespace.
333 """Validate that the given input lines produce the resulting namespace.
334
334
335 Note: the input lines are given exactly as they would be typed in an
335 Note: the input lines are given exactly as they would be typed in an
336 auto-indenting environment, as mini_interactive_loop above already does
336 auto-indenting environment, as mini_interactive_loop above already does
337 auto-indenting and prepends spaces to the input.
337 auto-indenting and prepends spaces to the input.
338 """
338 """
339 src = mini_interactive_loop(pseudo_input(lines))
339 src = mini_interactive_loop(pseudo_input(lines))
340 test_ns = {}
340 test_ns = {}
341 exec src in test_ns
341 exec src in test_ns
342 # We can't check that the provided ns is identical to the test_ns,
342 # We can't check that the provided ns is identical to the test_ns,
343 # because Python fills test_ns with extra keys (copyright, etc). But
343 # because Python fills test_ns with extra keys (copyright, etc). But
344 # we can check that the given dict is *contained* in test_ns
344 # we can check that the given dict is *contained* in test_ns
345 for k,v in ns.iteritems():
345 for k,v in ns.iteritems():
346 self.assertEqual(test_ns[k], v)
346 self.assertEqual(test_ns[k], v)
347
347
348 def test_simple(self):
348 def test_simple(self):
349 self.check_ns(['x=1'], dict(x=1))
349 self.check_ns(['x=1'], dict(x=1))
350
350
351 def test_simple2(self):
351 def test_simple2(self):
352 self.check_ns(['if 1:', 'x=2'], dict(x=2))
352 self.check_ns(['if 1:', 'x=2'], dict(x=2))
353
353
354 def test_xy(self):
354 def test_xy(self):
355 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
355 self.check_ns(['x=1; y=2'], dict(x=1, y=2))
356
356
357 def test_abc(self):
357 def test_abc(self):
358 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
358 self.check_ns(['if 1:','a=1','b=2','c=3'], dict(a=1, b=2, c=3))
359
359
360 def test_multi(self):
360 def test_multi(self):
361 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
361 self.check_ns(['x =(1+','1+','2)'], dict(x=4))
362
362
363
363
364 def test_LineInfo():
364 def test_LineInfo():
365 """Simple test for LineInfo construction and str()"""
365 """Simple test for LineInfo construction and str()"""
366 linfo = isp.LineInfo(' %cd /home')
366 linfo = isp.LineInfo(' %cd /home')
367 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
367 nt.assert_equals(str(linfo), 'LineInfo [ |%|cd|/home]')
368
368
369
369
370 def test_split_user_input():
370 def test_split_user_input():
371 """Unicode test - split_user_input already has good doctests"""
371 """Unicode test - split_user_input already has good doctests"""
372 line = u"PΓ©rez Fernando"
372 line = u"PΓ©rez Fernando"
373 parts = isp.split_user_input(line)
373 parts = isp.split_user_input(line)
374 parts_expected = (u'', u'', u'', line)
374 parts_expected = (u'', u'', u'', line)
375 nt.assert_equal(parts, parts_expected)
375 nt.assert_equal(parts, parts_expected)
376
376
377
377
378 # Transformer tests
378 # Transformer tests
379 def transform_checker(tests, func):
379 def transform_checker(tests, func):
380 """Utility to loop over test inputs"""
380 """Utility to loop over test inputs"""
381 for inp, tr in tests:
381 for inp, tr in tests:
382 nt.assert_equals(func(inp), tr)
382 nt.assert_equals(func(inp), tr)
383
383
384 # Data for all the syntax tests in the form of lists of pairs of
384 # Data for all the syntax tests in the form of lists of pairs of
385 # raw/transformed input. We store it here as a global dict so that we can use
385 # raw/transformed input. We store it here as a global dict so that we can use
386 # it both within single-function tests and also to validate the behavior of the
386 # it both within single-function tests and also to validate the behavior of the
387 # larger objects
387 # larger objects
388
388
389 syntax = \
389 syntax = \
390 dict(assign_system =
390 dict(assign_system =
391 [('a =! ls', 'a = get_ipython().getoutput(u"ls")'),
391 [('a =! ls', 'a = get_ipython().getoutput(u"ls")'),
392 ('b = !ls', 'b = get_ipython().getoutput(u"ls")'),
392 ('b = !ls', 'b = get_ipython().getoutput(u"ls")'),
393 ('x=1', 'x=1'), # normal input is unmodified
393 ('x=1', 'x=1'), # normal input is unmodified
394 (' ',' '), # blank lines are kept intact
394 (' ',' '), # blank lines are kept intact
395 ],
395 ],
396
396
397 assign_magic =
397 assign_magic =
398 [('a =% who', 'a = get_ipython().magic(u"who")'),
398 [('a =% who', 'a = get_ipython().magic(u"who")'),
399 ('b = %who', 'b = get_ipython().magic(u"who")'),
399 ('b = %who', 'b = get_ipython().magic(u"who")'),
400 ('x=1', 'x=1'), # normal input is unmodified
400 ('x=1', 'x=1'), # normal input is unmodified
401 (' ',' '), # blank lines are kept intact
401 (' ',' '), # blank lines are kept intact
402 ],
402 ],
403
403
404 classic_prompt =
404 classic_prompt =
405 [('>>> x=1', 'x=1'),
405 [('>>> x=1', 'x=1'),
406 ('x=1', 'x=1'), # normal input is unmodified
406 ('x=1', 'x=1'), # normal input is unmodified
407 (' ', ' '), # blank lines are kept intact
407 (' ', ' '), # blank lines are kept intact
408 ('... ', ''), # continuation prompts
408 ('... ', ''), # continuation prompts
409 ],
409 ],
410
410
411 ipy_prompt =
411 ipy_prompt =
412 [('In [1]: x=1', 'x=1'),
412 [('In [1]: x=1', 'x=1'),
413 ('x=1', 'x=1'), # normal input is unmodified
413 ('x=1', 'x=1'), # normal input is unmodified
414 (' ',' '), # blank lines are kept intact
414 (' ',' '), # blank lines are kept intact
415 (' ....: ', ''), # continuation prompts
415 (' ....: ', ''), # continuation prompts
416 ],
416 ],
417
417
418 # Tests for the escape transformer to leave normal code alone
418 # Tests for the escape transformer to leave normal code alone
419 escaped_noesc =
419 escaped_noesc =
420 [ (' ', ' '),
420 [ (' ', ' '),
421 ('x=1', 'x=1'),
421 ('x=1', 'x=1'),
422 ],
422 ],
423
423
424 # System calls
424 # System calls
425 escaped_shell =
425 escaped_shell =
426 [ ('!ls', 'get_ipython().system(u"ls")'),
426 [ ('!ls', 'get_ipython().system(u"ls")'),
427 # Double-escape shell, this means to capture the output of the
427 # Double-escape shell, this means to capture the output of the
428 # subprocess and return it
428 # subprocess and return it
429 ('!!ls', 'get_ipython().getoutput(u"ls")'),
429 ('!!ls', 'get_ipython().getoutput(u"ls")'),
430 ],
430 ],
431
431
432 # Help/object info
432 # Help/object info
433 escaped_help =
433 escaped_help =
434 [ ('?', 'get_ipython().show_usage()'),
434 [ ('?', 'get_ipython().show_usage()'),
435 ('?x1', 'get_ipython().magic(u"pinfo x1")'),
435 ('?x1', 'get_ipython().magic(u"pinfo x1")'),
436 ('??x2', 'get_ipython().magic(u"pinfo2 x2")'),
436 ('??x2', 'get_ipython().magic(u"pinfo2 x2")'),
437 ('x3?', 'get_ipython().magic(u"pinfo x3")'),
437 ('?a.*s', 'get_ipython().magic(u"psearch a.*s")'),
438 ('?%hist', 'get_ipython().magic(u"pinfo %hist")'),
439 ('?abc = qwe', 'get_ipython().magic(u"pinfo abc")'),
440 ],
441
442 end_help =
443 [ ('x3?', 'get_ipython().magic(u"pinfo x3")'),
438 ('x4??', 'get_ipython().magic(u"pinfo2 x4")'),
444 ('x4??', 'get_ipython().magic(u"pinfo2 x4")'),
439 ('%hist?', 'get_ipython().magic(u"pinfo %hist")'),
445 ('%hist?', 'get_ipython().magic(u"pinfo %hist")'),
440 ('f*?', 'get_ipython().magic(u"psearch f*")'),
446 ('f*?', 'get_ipython().magic(u"psearch f*")'),
441 ('ax.*aspe*?', 'get_ipython().magic(u"psearch ax.*aspe*")'),
447 ('ax.*aspe*?', 'get_ipython().magic(u"psearch ax.*aspe*")'),
448 ('a = abc?', 'get_ipython().magic(u"pinfo abc")'),
449 ('a = abc.qe??', 'get_ipython().magic(u"pinfo2 abc.qe")'),
450 ('a = *.items?', 'get_ipython().magic(u"psearch *.items")'),
442 ],
451 ],
443
452
444 # Explicit magic calls
453 # Explicit magic calls
445 escaped_magic =
454 escaped_magic =
446 [ ('%cd', 'get_ipython().magic(u"cd")'),
455 [ ('%cd', 'get_ipython().magic(u"cd")'),
447 ('%cd /home', 'get_ipython().magic(u"cd /home")'),
456 ('%cd /home', 'get_ipython().magic(u"cd /home")'),
448 (' %magic', ' get_ipython().magic(u"magic")'),
457 (' %magic', ' get_ipython().magic(u"magic")'),
449 ],
458 ],
450
459
451 # Quoting with separate arguments
460 # Quoting with separate arguments
452 escaped_quote =
461 escaped_quote =
453 [ (',f', 'f("")'),
462 [ (',f', 'f("")'),
454 (',f x', 'f("x")'),
463 (',f x', 'f("x")'),
455 (' ,f y', ' f("y")'),
464 (' ,f y', ' f("y")'),
456 (',f a b', 'f("a", "b")'),
465 (',f a b', 'f("a", "b")'),
457 ],
466 ],
458
467
459 # Quoting with single argument
468 # Quoting with single argument
460 escaped_quote2 =
469 escaped_quote2 =
461 [ (';f', 'f("")'),
470 [ (';f', 'f("")'),
462 (';f x', 'f("x")'),
471 (';f x', 'f("x")'),
463 (' ;f y', ' f("y")'),
472 (' ;f y', ' f("y")'),
464 (';f a b', 'f("a b")'),
473 (';f a b', 'f("a b")'),
465 ],
474 ],
466
475
467 # Simply apply parens
476 # Simply apply parens
468 escaped_paren =
477 escaped_paren =
469 [ ('/f', 'f()'),
478 [ ('/f', 'f()'),
470 ('/f x', 'f(x)'),
479 ('/f x', 'f(x)'),
471 (' /f y', ' f(y)'),
480 (' /f y', ' f(y)'),
472 ('/f a b', 'f(a, b)'),
481 ('/f a b', 'f(a, b)'),
473 ],
482 ],
474
483
475 )
484 )
476
485
477 # multiline syntax examples. Each of these should be a list of lists, with
486 # multiline syntax examples. Each of these should be a list of lists, with
478 # each entry itself having pairs of raw/transformed input. The union (with
487 # each entry itself having pairs of raw/transformed input. The union (with
479 # '\n'.join() of the transformed inputs is what the splitter should produce
488 # '\n'.join() of the transformed inputs is what the splitter should produce
480 # when fed the raw lines one at a time via push.
489 # when fed the raw lines one at a time via push.
481 syntax_ml = \
490 syntax_ml = \
482 dict(classic_prompt =
491 dict(classic_prompt =
483 [ [('>>> for i in range(10):','for i in range(10):'),
492 [ [('>>> for i in range(10):','for i in range(10):'),
484 ('... print i',' print i'),
493 ('... print i',' print i'),
485 ('... ', ''),
494 ('... ', ''),
486 ],
495 ],
487 ],
496 ],
488
497
489 ipy_prompt =
498 ipy_prompt =
490 [ [('In [24]: for i in range(10):','for i in range(10):'),
499 [ [('In [24]: for i in range(10):','for i in range(10):'),
491 (' ....: print i',' print i'),
500 (' ....: print i',' print i'),
492 (' ....: ', ''),
501 (' ....: ', ''),
493 ],
502 ],
494 ],
503 ],
495 )
504 )
496
505
497
506
498 def test_assign_system():
507 def test_assign_system():
499 transform_checker(syntax['assign_system'], isp.transform_assign_system)
508 transform_checker(syntax['assign_system'], isp.transform_assign_system)
500
509
501
510
502 def test_assign_magic():
511 def test_assign_magic():
503 transform_checker(syntax['assign_magic'], isp.transform_assign_magic)
512 transform_checker(syntax['assign_magic'], isp.transform_assign_magic)
504
513
505
514
506 def test_classic_prompt():
515 def test_classic_prompt():
507 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
516 transform_checker(syntax['classic_prompt'], isp.transform_classic_prompt)
508 for example in syntax_ml['classic_prompt']:
517 for example in syntax_ml['classic_prompt']:
509 transform_checker(example, isp.transform_classic_prompt)
518 transform_checker(example, isp.transform_classic_prompt)
510
519
511
520
512 def test_ipy_prompt():
521 def test_ipy_prompt():
513 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
522 transform_checker(syntax['ipy_prompt'], isp.transform_ipy_prompt)
514 for example in syntax_ml['ipy_prompt']:
523 for example in syntax_ml['ipy_prompt']:
515 transform_checker(example, isp.transform_ipy_prompt)
524 transform_checker(example, isp.transform_ipy_prompt)
516
525
526 def test_end_help():
527 transform_checker(syntax['end_help'], isp.transform_help_end)
517
528
518 def test_escaped_noesc():
529 def test_escaped_noesc():
519 transform_checker(syntax['escaped_noesc'], isp.transform_escaped)
530 transform_checker(syntax['escaped_noesc'], isp.transform_escaped)
520
531
521
532
522 def test_escaped_shell():
533 def test_escaped_shell():
523 transform_checker(syntax['escaped_shell'], isp.transform_escaped)
534 transform_checker(syntax['escaped_shell'], isp.transform_escaped)
524
535
525
536
526 def test_escaped_help():
537 def test_escaped_help():
527 transform_checker(syntax['escaped_help'], isp.transform_escaped)
538 transform_checker(syntax['escaped_help'], isp.transform_escaped)
528
539
529
540
530 def test_escaped_magic():
541 def test_escaped_magic():
531 transform_checker(syntax['escaped_magic'], isp.transform_escaped)
542 transform_checker(syntax['escaped_magic'], isp.transform_escaped)
532
543
533
544
534 def test_escaped_quote():
545 def test_escaped_quote():
535 transform_checker(syntax['escaped_quote'], isp.transform_escaped)
546 transform_checker(syntax['escaped_quote'], isp.transform_escaped)
536
547
537
548
538 def test_escaped_quote2():
549 def test_escaped_quote2():
539 transform_checker(syntax['escaped_quote2'], isp.transform_escaped)
550 transform_checker(syntax['escaped_quote2'], isp.transform_escaped)
540
551
541
552
542 def test_escaped_paren():
553 def test_escaped_paren():
543 transform_checker(syntax['escaped_paren'], isp.transform_escaped)
554 transform_checker(syntax['escaped_paren'], isp.transform_escaped)
544
555
545
556
546 class IPythonInputTestCase(InputSplitterTestCase):
557 class IPythonInputTestCase(InputSplitterTestCase):
547 """By just creating a new class whose .isp is a different instance, we
558 """By just creating a new class whose .isp is a different instance, we
548 re-run the same test battery on the new input splitter.
559 re-run the same test battery on the new input splitter.
549
560
550 In addition, this runs the tests over the syntax and syntax_ml dicts that
561 In addition, this runs the tests over the syntax and syntax_ml dicts that
551 were tested by individual functions, as part of the OO interface.
562 were tested by individual functions, as part of the OO interface.
552
563
553 It also makes some checks on the raw buffer storage.
564 It also makes some checks on the raw buffer storage.
554 """
565 """
555
566
556 def setUp(self):
567 def setUp(self):
557 self.isp = isp.IPythonInputSplitter(input_mode='line')
568 self.isp = isp.IPythonInputSplitter(input_mode='line')
558
569
559 def test_syntax(self):
570 def test_syntax(self):
560 """Call all single-line syntax tests from the main object"""
571 """Call all single-line syntax tests from the main object"""
561 isp = self.isp
572 isp = self.isp
562 for example in syntax.itervalues():
573 for example in syntax.itervalues():
563 for raw, out_t in example:
574 for raw, out_t in example:
564 if raw.startswith(' '):
575 if raw.startswith(' '):
565 continue
576 continue
566
577
567 isp.push(raw)
578 isp.push(raw)
568 out, out_raw = isp.source_raw_reset()
579 out, out_raw = isp.source_raw_reset()
569 self.assertEqual(out.rstrip(), out_t)
580 self.assertEqual(out.rstrip(), out_t)
570 self.assertEqual(out_raw.rstrip(), raw.rstrip())
581 self.assertEqual(out_raw.rstrip(), raw.rstrip())
571
582
572 def test_syntax_multiline(self):
583 def test_syntax_multiline(self):
573 isp = self.isp
584 isp = self.isp
574 for example in syntax_ml.itervalues():
585 for example in syntax_ml.itervalues():
575 out_t_parts = []
586 out_t_parts = []
576 raw_parts = []
587 raw_parts = []
577 for line_pairs in example:
588 for line_pairs in example:
578 for lraw, out_t_part in line_pairs:
589 for lraw, out_t_part in line_pairs:
579 isp.push(lraw)
590 isp.push(lraw)
580 out_t_parts.append(out_t_part)
591 out_t_parts.append(out_t_part)
581 raw_parts.append(lraw)
592 raw_parts.append(lraw)
582
593
583 out, out_raw = isp.source_raw_reset()
594 out, out_raw = isp.source_raw_reset()
584 out_t = '\n'.join(out_t_parts).rstrip()
595 out_t = '\n'.join(out_t_parts).rstrip()
585 raw = '\n'.join(raw_parts).rstrip()
596 raw = '\n'.join(raw_parts).rstrip()
586 self.assertEqual(out.rstrip(), out_t)
597 self.assertEqual(out.rstrip(), out_t)
587 self.assertEqual(out_raw.rstrip(), raw)
598 self.assertEqual(out_raw.rstrip(), raw)
588
599
589
600
590 class BlockIPythonInputTestCase(IPythonInputTestCase):
601 class BlockIPythonInputTestCase(IPythonInputTestCase):
591
602
592 # Deactivate tests that don't make sense for the block mode
603 # Deactivate tests that don't make sense for the block mode
593 test_push3 = test_split = lambda s: None
604 test_push3 = test_split = lambda s: None
594
605
595 def setUp(self):
606 def setUp(self):
596 self.isp = isp.IPythonInputSplitter(input_mode='cell')
607 self.isp = isp.IPythonInputSplitter(input_mode='cell')
597
608
598 def test_syntax_multiline(self):
609 def test_syntax_multiline(self):
599 isp = self.isp
610 isp = self.isp
600 for example in syntax_ml.itervalues():
611 for example in syntax_ml.itervalues():
601 raw_parts = []
612 raw_parts = []
602 out_t_parts = []
613 out_t_parts = []
603 for line_pairs in example:
614 for line_pairs in example:
604 for raw, out_t_part in line_pairs:
615 for raw, out_t_part in line_pairs:
605 raw_parts.append(raw)
616 raw_parts.append(raw)
606 out_t_parts.append(out_t_part)
617 out_t_parts.append(out_t_part)
607
618
608 raw = '\n'.join(raw_parts)
619 raw = '\n'.join(raw_parts)
609 out_t = '\n'.join(out_t_parts)
620 out_t = '\n'.join(out_t_parts)
610
621
611 isp.push(raw)
622 isp.push(raw)
612 out, out_raw = isp.source_raw_reset()
623 out, out_raw = isp.source_raw_reset()
613 # Match ignoring trailing whitespace
624 # Match ignoring trailing whitespace
614 self.assertEqual(out.rstrip(), out_t.rstrip())
625 self.assertEqual(out.rstrip(), out_t.rstrip())
615 self.assertEqual(out_raw.rstrip(), raw.rstrip())
626 self.assertEqual(out_raw.rstrip(), raw.rstrip())
616
627
617
628
618 #-----------------------------------------------------------------------------
629 #-----------------------------------------------------------------------------
619 # Main - use as a script, mostly for developer experiments
630 # Main - use as a script, mostly for developer experiments
620 #-----------------------------------------------------------------------------
631 #-----------------------------------------------------------------------------
621
632
622 if __name__ == '__main__':
633 if __name__ == '__main__':
623 # A simple demo for interactive experimentation. This code will not get
634 # A simple demo for interactive experimentation. This code will not get
624 # picked up by any test suite.
635 # picked up by any test suite.
625 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
636 from IPython.core.inputsplitter import InputSplitter, IPythonInputSplitter
626
637
627 # configure here the syntax to use, prompt and whether to autoindent
638 # configure here the syntax to use, prompt and whether to autoindent
628 #isp, start_prompt = InputSplitter(), '>>> '
639 #isp, start_prompt = InputSplitter(), '>>> '
629 isp, start_prompt = IPythonInputSplitter(), 'In> '
640 isp, start_prompt = IPythonInputSplitter(), 'In> '
630
641
631 autoindent = True
642 autoindent = True
632 #autoindent = False
643 #autoindent = False
633
644
634 try:
645 try:
635 while True:
646 while True:
636 prompt = start_prompt
647 prompt = start_prompt
637 while isp.push_accepts_more():
648 while isp.push_accepts_more():
638 indent = ' '*isp.indent_spaces
649 indent = ' '*isp.indent_spaces
639 if autoindent:
650 if autoindent:
640 line = indent + raw_input(prompt+indent)
651 line = indent + raw_input(prompt+indent)
641 else:
652 else:
642 line = raw_input(prompt)
653 line = raw_input(prompt)
643 isp.push(line)
654 isp.push(line)
644 prompt = '... '
655 prompt = '... '
645
656
646 # Here we just return input so we can use it in a test suite, but a
657 # Here we just return input so we can use it in a test suite, but a
647 # real interpreter would instead send it for execution somewhere.
658 # real interpreter would instead send it for execution somewhere.
648 #src = isp.source; raise EOFError # dbg
659 #src = isp.source; raise EOFError # dbg
649 src, raw = isp.source_raw_reset()
660 src, raw = isp.source_raw_reset()
650 print 'Input source was:\n', src
661 print 'Input source was:\n', src
651 print 'Raw source was:\n', raw
662 print 'Raw source was:\n', raw
652 except EOFError:
663 except EOFError:
653 print 'Bye'
664 print 'Bye'
General Comments 0
You need to be logged in to leave comments. Login now