Show More
@@ -1,140 +1,139 b'' | |||||
1 |
|
1 | |||
2 | =========================== |
|
2 | =========================== | |
3 | Custom input transformation |
|
3 | Custom input transformation | |
4 | =========================== |
|
4 | =========================== | |
5 |
|
5 | |||
6 | IPython extends Python syntax to allow things like magic commands, and help with |
|
6 | IPython extends Python syntax to allow things like magic commands, and help with | |
7 | the ``?`` syntax. There are several ways to customise how the user's input is |
|
7 | the ``?`` syntax. There are several ways to customise how the user's input is | |
8 | processed into Python code to be executed. |
|
8 | processed into Python code to be executed. | |
9 |
|
9 | |||
10 | These hooks are mainly for other projects using IPython as the core of their |
|
10 | These hooks are mainly for other projects using IPython as the core of their | |
11 | interactive interface. Using them carelessly can easily break IPython! |
|
11 | interactive interface. Using them carelessly can easily break IPython! | |
12 |
|
12 | |||
13 | String based transformations |
|
13 | String based transformations | |
14 | ============================ |
|
14 | ============================ | |
15 |
|
15 | |||
16 | .. currentmodule:: IPython.core.inputtransforms |
|
16 | .. currentmodule:: IPython.core.inputtransforms | |
17 |
|
17 | |||
18 | When the user enters a line of code, it is first processed as a string. By the |
|
18 | When the user enters a line of code, it is first processed as a string. By the | |
19 | end of this stage, it must be valid Python syntax. |
|
19 | end of this stage, it must be valid Python syntax. | |
20 |
|
20 | |||
21 | These transformers all subclass :class:`IPython.core.inputtransformer.InputTransformer`, |
|
21 | These transformers all subclass :class:`IPython.core.inputtransformer.InputTransformer`, | |
22 | and are used by :class:`IPython.core.inputsplitter.IPythonInputSplitter`. |
|
22 | and are used by :class:`IPython.core.inputsplitter.IPythonInputSplitter`. | |
23 |
|
23 | |||
24 | These transformers act in three groups, stored separately as lists of instances |
|
24 | These transformers act in three groups, stored separately as lists of instances | |
25 | in attributes of :class:`~IPython.core.inputsplitter.IPythonInputSplitter`: |
|
25 | in attributes of :class:`~IPython.core.inputsplitter.IPythonInputSplitter`: | |
26 |
|
26 | |||
27 | * ``physical_line_transforms`` act on the lines as the user enters them. For |
|
27 | * ``physical_line_transforms`` act on the lines as the user enters them. For | |
28 | example, these strip Python prompts from examples pasted in. |
|
28 | example, these strip Python prompts from examples pasted in. | |
29 | * ``logical_line_transforms`` act on lines as connected by explicit line |
|
29 | * ``logical_line_transforms`` act on lines as connected by explicit line | |
30 | continuations, i.e. ``\`` at the end of physical lines. They are skipped |
|
30 | continuations, i.e. ``\`` at the end of physical lines. They are skipped | |
31 | inside multiline Python statements. This is the point where IPython recognises |
|
31 | inside multiline Python statements. This is the point where IPython recognises | |
32 | ``%magic`` commands, for instance. |
|
32 | ``%magic`` commands, for instance. | |
33 | * ``python_line_transforms`` act on blocks containing complete Python statements. |
|
33 | * ``python_line_transforms`` act on blocks containing complete Python statements. | |
34 | Multi-line strings, lists and function calls are reassembled before being |
|
34 | Multi-line strings, lists and function calls are reassembled before being | |
35 | passed to these, but note that function and class *definitions* are still a |
|
35 | passed to these, but note that function and class *definitions* are still a | |
36 | series of separate statements. IPython does not use any of these by default. |
|
36 | series of separate statements. IPython does not use any of these by default. | |
37 |
|
37 | |||
38 | An InteractiveShell instance actually has two |
|
38 | An InteractiveShell instance actually has two | |
39 | :class:`~IPython.core.inputsplitter.IPythonInputSplitter` instances, as the |
|
39 | :class:`~IPython.core.inputsplitter.IPythonInputSplitter` instances, as the | |
40 | attributes :attr:`~IPython.core.interactiveshell.InteractiveShell.input_splitter`, |
|
40 | attributes :attr:`~IPython.core.interactiveshell.InteractiveShell.input_splitter`, | |
41 | to tell when a block of input is complete, and |
|
41 | to tell when a block of input is complete, and | |
42 | :attr:`~IPython.core.interactiveshell.InteractiveShell.input_transformer_manager`, |
|
42 | :attr:`~IPython.core.interactiveshell.InteractiveShell.input_transformer_manager`, | |
43 | to transform complete cells. If you add a transformer, you should make sure that |
|
43 | to transform complete cells. If you add a transformer, you should make sure that | |
44 | it gets added to both, e.g.:: |
|
44 | it gets added to both, e.g.:: | |
45 |
|
45 | |||
46 | ip.input_splitter.logical_line_transforms.append(my_transformer()) |
|
46 | ip.input_splitter.logical_line_transforms.append(my_transformer()) | |
47 | ip.input_transformer_manager.logical_line_transforms.append(my_transformer()) |
|
47 | ip.input_transformer_manager.logical_line_transforms.append(my_transformer()) | |
48 |
|
48 | |||
49 | These transformers may raise :exc:`SyntaxError` if the input code is invalid, but |
|
49 | These transformers may raise :exc:`SyntaxError` if the input code is invalid, but | |
50 | in most cases it is clearer to pass unrecognised code through unmodified and let |
|
50 | in most cases it is clearer to pass unrecognised code through unmodified and let | |
51 | Python's own parser decide whether it is valid. |
|
51 | Python's own parser decide whether it is valid. | |
52 |
|
52 | |||
53 | .. versionchanged:: 2.0 |
|
53 | .. versionchanged:: 2.0 | |
54 |
|
54 | |||
55 | Added the option to raise :exc:`SyntaxError`. |
|
55 | Added the option to raise :exc:`SyntaxError`. | |
56 |
|
56 | |||
57 | Stateless transformations |
|
57 | Stateless transformations | |
58 | ------------------------- |
|
58 | ------------------------- | |
59 |
|
59 | |||
60 | The simplest kind of transformations work one line at a time. Write a function |
|
60 | The simplest kind of transformations work one line at a time. Write a function | |
61 | which takes a line and returns a line, and decorate it with |
|
61 | which takes a line and returns a line, and decorate it with | |
62 | :meth:`StatelessInputTransformer.wrap`:: |
|
62 | :meth:`StatelessInputTransformer.wrap`:: | |
63 |
|
63 | |||
64 | @StatelessInputTransformer.wrap |
|
64 | @StatelessInputTransformer.wrap | |
65 | def my_special_commands(line): |
|
65 | def my_special_commands(line): | |
66 | if line.startswith("Β¬"): |
|
66 | if line.startswith("Β¬"): | |
67 | return "specialcommand(" + repr(line) + ")" |
|
67 | return "specialcommand(" + repr(line) + ")" | |
68 | return line |
|
68 | return line | |
69 |
|
69 | |||
70 | The decorator returns a factory function which will produce instances of |
|
70 | The decorator returns a factory function which will produce instances of | |
71 | :class:`~IPython.core.inputtransformer.StatelessInputTransformer` using your |
|
71 | :class:`~IPython.core.inputtransformer.StatelessInputTransformer` using your | |
72 | function. |
|
72 | function. | |
73 |
|
73 | |||
74 | Coroutine transformers |
|
74 | Coroutine transformers | |
75 | ---------------------- |
|
75 | ---------------------- | |
76 |
|
76 | |||
77 | More advanced transformers can be written as coroutines. The coroutine will be |
|
77 | More advanced transformers can be written as coroutines. The coroutine will be | |
78 | sent each line in turn, followed by ``None`` to reset it. It can yield lines, or |
|
78 | sent each line in turn, followed by ``None`` to reset it. It can yield lines, or | |
79 | ``None`` if it is accumulating text to yield at a later point. When reset, it |
|
79 | ``None`` if it is accumulating text to yield at a later point. When reset, it | |
80 | should give up any code it has accumulated. |
|
80 | should give up any code it has accumulated. | |
81 |
|
81 | |||
82 | This code in IPython strips a constant amount of leading indentation from each |
|
82 | This code in IPython strips a constant amount of leading indentation from each | |
83 | line in a cell:: |
|
83 | line in a cell:: | |
84 |
|
84 | |||
85 | @CoroutineInputTransformer.wrap |
|
85 | @CoroutineInputTransformer.wrap | |
86 | def leading_indent(): |
|
86 | def leading_indent(): | |
87 | """Remove leading indentation. |
|
87 | """Remove leading indentation. | |
88 |
|
88 | |||
89 | If the first line starts with a spaces or tabs, the same whitespace will be |
|
89 | If the first line starts with a spaces or tabs, the same whitespace will be | |
90 | removed from each following line until it is reset. |
|
90 | removed from each following line until it is reset. | |
91 | """ |
|
91 | """ | |
92 | space_re = re.compile(r'^[ \t]+') |
|
92 | space_re = re.compile(r'^[ \t]+') | |
93 | line = '' |
|
93 | line = '' | |
94 | while True: |
|
94 | while True: | |
95 | line = (yield line) |
|
95 | line = (yield line) | |
96 |
|
96 | |||
97 | if line is None: |
|
97 | if line is None: | |
98 | continue |
|
98 | continue | |
99 |
|
99 | |||
100 | m = space_re.match(line) |
|
100 | m = space_re.match(line) | |
101 | if m: |
|
101 | if m: | |
102 | space = m.group(0) |
|
102 | space = m.group(0) | |
103 | while line is not None: |
|
103 | while line is not None: | |
104 | if line.startswith(space): |
|
104 | if line.startswith(space): | |
105 | line = line[len(space):] |
|
105 | line = line[len(space):] | |
106 | line = (yield line) |
|
106 | line = (yield line) | |
107 | else: |
|
107 | else: | |
108 | # No leading spaces - wait for reset |
|
108 | # No leading spaces - wait for reset | |
109 | while line is not None: |
|
109 | while line is not None: | |
110 | line = (yield line) |
|
110 | line = (yield line) | |
111 |
|
111 | |||
112 | leading_indent.look_in_string = True |
|
|||
113 |
|
112 | |||
114 | Token-based transformers |
|
113 | Token-based transformers | |
115 | ------------------------ |
|
114 | ------------------------ | |
116 |
|
115 | |||
117 | There is an experimental framework that takes care of tokenizing and |
|
116 | There is an experimental framework that takes care of tokenizing and | |
118 | untokenizing lines of code. Define a function that accepts a list of tokens, and |
|
117 | untokenizing lines of code. Define a function that accepts a list of tokens, and | |
119 | returns an iterable of output tokens, and decorate it with |
|
118 | returns an iterable of output tokens, and decorate it with | |
120 | :meth:`TokenInputTransformer.wrap`. These should only be used in |
|
119 | :meth:`TokenInputTransformer.wrap`. These should only be used in | |
121 | ``python_line_transforms``. |
|
120 | ``python_line_transforms``. | |
122 |
|
121 | |||
123 | AST transformations |
|
122 | AST transformations | |
124 | =================== |
|
123 | =================== | |
125 |
|
124 | |||
126 | After the code has been parsed as Python syntax, you can use Python's powerful |
|
125 | After the code has been parsed as Python syntax, you can use Python's powerful | |
127 | *Abstract Syntax Tree* tools to modify it. Subclass :class:`ast.NodeTransformer`, |
|
126 | *Abstract Syntax Tree* tools to modify it. Subclass :class:`ast.NodeTransformer`, | |
128 | and add an instance to ``shell.ast_transformers``. |
|
127 | and add an instance to ``shell.ast_transformers``. | |
129 |
|
128 | |||
130 | This example wraps integer literals in an ``Integer`` class, which is useful for |
|
129 | This example wraps integer literals in an ``Integer`` class, which is useful for | |
131 | mathematical frameworks that want to handle e.g. ``1/3`` as a precise fraction:: |
|
130 | mathematical frameworks that want to handle e.g. ``1/3`` as a precise fraction:: | |
132 |
|
131 | |||
133 |
|
132 | |||
134 | class IntegerWrapper(ast.NodeTransformer): |
|
133 | class IntegerWrapper(ast.NodeTransformer): | |
135 | """Wraps all integers in a call to Integer()""" |
|
134 | """Wraps all integers in a call to Integer()""" | |
136 | def visit_Num(self, node): |
|
135 | def visit_Num(self, node): | |
137 | if isinstance(node.n, int): |
|
136 | if isinstance(node.n, int): | |
138 | return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()), |
|
137 | return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()), | |
139 | args=[node], keywords=[]) |
|
138 | args=[node], keywords=[]) | |
140 | return node |
|
139 | return node |
General Comments 0
You need to be logged in to leave comments.
Login now