##// END OF EJS Templates
Merge pull request #10389 from minrk/look_in_string...
Matthias Bussonnier -
r23483:cbb02c5a merge
parent child Browse files
Show More
@@ -1,140 +1,139 b''
1
1
2 ===========================
2 ===========================
3 Custom input transformation
3 Custom input transformation
4 ===========================
4 ===========================
5
5
6 IPython extends Python syntax to allow things like magic commands, and help with
6 IPython extends Python syntax to allow things like magic commands, and help with
7 the ``?`` syntax. There are several ways to customise how the user's input is
7 the ``?`` syntax. There are several ways to customise how the user's input is
8 processed into Python code to be executed.
8 processed into Python code to be executed.
9
9
10 These hooks are mainly for other projects using IPython as the core of their
10 These hooks are mainly for other projects using IPython as the core of their
11 interactive interface. Using them carelessly can easily break IPython!
11 interactive interface. Using them carelessly can easily break IPython!
12
12
13 String based transformations
13 String based transformations
14 ============================
14 ============================
15
15
16 .. currentmodule:: IPython.core.inputtransforms
16 .. currentmodule:: IPython.core.inputtransforms
17
17
18 When the user enters a line of code, it is first processed as a string. By the
18 When the user enters a line of code, it is first processed as a string. By the
19 end of this stage, it must be valid Python syntax.
19 end of this stage, it must be valid Python syntax.
20
20
21 These transformers all subclass :class:`IPython.core.inputtransformer.InputTransformer`,
21 These transformers all subclass :class:`IPython.core.inputtransformer.InputTransformer`,
22 and are used by :class:`IPython.core.inputsplitter.IPythonInputSplitter`.
22 and are used by :class:`IPython.core.inputsplitter.IPythonInputSplitter`.
23
23
24 These transformers act in three groups, stored separately as lists of instances
24 These transformers act in three groups, stored separately as lists of instances
25 in attributes of :class:`~IPython.core.inputsplitter.IPythonInputSplitter`:
25 in attributes of :class:`~IPython.core.inputsplitter.IPythonInputSplitter`:
26
26
27 * ``physical_line_transforms`` act on the lines as the user enters them. For
27 * ``physical_line_transforms`` act on the lines as the user enters them. For
28 example, these strip Python prompts from examples pasted in.
28 example, these strip Python prompts from examples pasted in.
29 * ``logical_line_transforms`` act on lines as connected by explicit line
29 * ``logical_line_transforms`` act on lines as connected by explicit line
30 continuations, i.e. ``\`` at the end of physical lines. They are skipped
30 continuations, i.e. ``\`` at the end of physical lines. They are skipped
31 inside multiline Python statements. This is the point where IPython recognises
31 inside multiline Python statements. This is the point where IPython recognises
32 ``%magic`` commands, for instance.
32 ``%magic`` commands, for instance.
33 * ``python_line_transforms`` act on blocks containing complete Python statements.
33 * ``python_line_transforms`` act on blocks containing complete Python statements.
34 Multi-line strings, lists and function calls are reassembled before being
34 Multi-line strings, lists and function calls are reassembled before being
35 passed to these, but note that function and class *definitions* are still a
35 passed to these, but note that function and class *definitions* are still a
36 series of separate statements. IPython does not use any of these by default.
36 series of separate statements. IPython does not use any of these by default.
37
37
38 An InteractiveShell instance actually has two
38 An InteractiveShell instance actually has two
39 :class:`~IPython.core.inputsplitter.IPythonInputSplitter` instances, as the
39 :class:`~IPython.core.inputsplitter.IPythonInputSplitter` instances, as the
40 attributes :attr:`~IPython.core.interactiveshell.InteractiveShell.input_splitter`,
40 attributes :attr:`~IPython.core.interactiveshell.InteractiveShell.input_splitter`,
41 to tell when a block of input is complete, and
41 to tell when a block of input is complete, and
42 :attr:`~IPython.core.interactiveshell.InteractiveShell.input_transformer_manager`,
42 :attr:`~IPython.core.interactiveshell.InteractiveShell.input_transformer_manager`,
43 to transform complete cells. If you add a transformer, you should make sure that
43 to transform complete cells. If you add a transformer, you should make sure that
44 it gets added to both, e.g.::
44 it gets added to both, e.g.::
45
45
46 ip.input_splitter.logical_line_transforms.append(my_transformer())
46 ip.input_splitter.logical_line_transforms.append(my_transformer())
47 ip.input_transformer_manager.logical_line_transforms.append(my_transformer())
47 ip.input_transformer_manager.logical_line_transforms.append(my_transformer())
48
48
49 These transformers may raise :exc:`SyntaxError` if the input code is invalid, but
49 These transformers may raise :exc:`SyntaxError` if the input code is invalid, but
50 in most cases it is clearer to pass unrecognised code through unmodified and let
50 in most cases it is clearer to pass unrecognised code through unmodified and let
51 Python's own parser decide whether it is valid.
51 Python's own parser decide whether it is valid.
52
52
53 .. versionchanged:: 2.0
53 .. versionchanged:: 2.0
54
54
55 Added the option to raise :exc:`SyntaxError`.
55 Added the option to raise :exc:`SyntaxError`.
56
56
57 Stateless transformations
57 Stateless transformations
58 -------------------------
58 -------------------------
59
59
60 The simplest kind of transformations work one line at a time. Write a function
60 The simplest kind of transformations work one line at a time. Write a function
61 which takes a line and returns a line, and decorate it with
61 which takes a line and returns a line, and decorate it with
62 :meth:`StatelessInputTransformer.wrap`::
62 :meth:`StatelessInputTransformer.wrap`::
63
63
64 @StatelessInputTransformer.wrap
64 @StatelessInputTransformer.wrap
65 def my_special_commands(line):
65 def my_special_commands(line):
66 if line.startswith("Β¬"):
66 if line.startswith("Β¬"):
67 return "specialcommand(" + repr(line) + ")"
67 return "specialcommand(" + repr(line) + ")"
68 return line
68 return line
69
69
70 The decorator returns a factory function which will produce instances of
70 The decorator returns a factory function which will produce instances of
71 :class:`~IPython.core.inputtransformer.StatelessInputTransformer` using your
71 :class:`~IPython.core.inputtransformer.StatelessInputTransformer` using your
72 function.
72 function.
73
73
74 Coroutine transformers
74 Coroutine transformers
75 ----------------------
75 ----------------------
76
76
77 More advanced transformers can be written as coroutines. The coroutine will be
77 More advanced transformers can be written as coroutines. The coroutine will be
78 sent each line in turn, followed by ``None`` to reset it. It can yield lines, or
78 sent each line in turn, followed by ``None`` to reset it. It can yield lines, or
79 ``None`` if it is accumulating text to yield at a later point. When reset, it
79 ``None`` if it is accumulating text to yield at a later point. When reset, it
80 should give up any code it has accumulated.
80 should give up any code it has accumulated.
81
81
82 This code in IPython strips a constant amount of leading indentation from each
82 This code in IPython strips a constant amount of leading indentation from each
83 line in a cell::
83 line in a cell::
84
84
85 @CoroutineInputTransformer.wrap
85 @CoroutineInputTransformer.wrap
86 def leading_indent():
86 def leading_indent():
87 """Remove leading indentation.
87 """Remove leading indentation.
88
88
89 If the first line starts with a spaces or tabs, the same whitespace will be
89 If the first line starts with a spaces or tabs, the same whitespace will be
90 removed from each following line until it is reset.
90 removed from each following line until it is reset.
91 """
91 """
92 space_re = re.compile(r'^[ \t]+')
92 space_re = re.compile(r'^[ \t]+')
93 line = ''
93 line = ''
94 while True:
94 while True:
95 line = (yield line)
95 line = (yield line)
96
96
97 if line is None:
97 if line is None:
98 continue
98 continue
99
99
100 m = space_re.match(line)
100 m = space_re.match(line)
101 if m:
101 if m:
102 space = m.group(0)
102 space = m.group(0)
103 while line is not None:
103 while line is not None:
104 if line.startswith(space):
104 if line.startswith(space):
105 line = line[len(space):]
105 line = line[len(space):]
106 line = (yield line)
106 line = (yield line)
107 else:
107 else:
108 # No leading spaces - wait for reset
108 # No leading spaces - wait for reset
109 while line is not None:
109 while line is not None:
110 line = (yield line)
110 line = (yield line)
111
111
112 leading_indent.look_in_string = True
113
112
114 Token-based transformers
113 Token-based transformers
115 ------------------------
114 ------------------------
116
115
117 There is an experimental framework that takes care of tokenizing and
116 There is an experimental framework that takes care of tokenizing and
118 untokenizing lines of code. Define a function that accepts a list of tokens, and
117 untokenizing lines of code. Define a function that accepts a list of tokens, and
119 returns an iterable of output tokens, and decorate it with
118 returns an iterable of output tokens, and decorate it with
120 :meth:`TokenInputTransformer.wrap`. These should only be used in
119 :meth:`TokenInputTransformer.wrap`. These should only be used in
121 ``python_line_transforms``.
120 ``python_line_transforms``.
122
121
123 AST transformations
122 AST transformations
124 ===================
123 ===================
125
124
126 After the code has been parsed as Python syntax, you can use Python's powerful
125 After the code has been parsed as Python syntax, you can use Python's powerful
127 *Abstract Syntax Tree* tools to modify it. Subclass :class:`ast.NodeTransformer`,
126 *Abstract Syntax Tree* tools to modify it. Subclass :class:`ast.NodeTransformer`,
128 and add an instance to ``shell.ast_transformers``.
127 and add an instance to ``shell.ast_transformers``.
129
128
130 This example wraps integer literals in an ``Integer`` class, which is useful for
129 This example wraps integer literals in an ``Integer`` class, which is useful for
131 mathematical frameworks that want to handle e.g. ``1/3`` as a precise fraction::
130 mathematical frameworks that want to handle e.g. ``1/3`` as a precise fraction::
132
131
133
132
134 class IntegerWrapper(ast.NodeTransformer):
133 class IntegerWrapper(ast.NodeTransformer):
135 """Wraps all integers in a call to Integer()"""
134 """Wraps all integers in a call to Integer()"""
136 def visit_Num(self, node):
135 def visit_Num(self, node):
137 if isinstance(node.n, int):
136 if isinstance(node.n, int):
138 return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()),
137 return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()),
139 args=[node], keywords=[])
138 args=[node], keywords=[])
140 return node
139 return node
General Comments 0
You need to be logged in to leave comments. Login now