##// END OF EJS Templates
Improve input transformation docs
Thomas Kluyver -
Show More
@@ -1,103 +1,121 b''
1 1
2 2 ===========================
3 3 Custom input transformation
4 4 ===========================
5 5
6 6 IPython extends Python syntax to allow things like magic commands, and help with
7 7 the ``?`` syntax. There are several ways to customise how the user's input is
8 8 processed into Python code to be executed.
9 9
10 10 These hooks are mainly for other projects using IPython as the core of their
11 11 interactive interface. Using them carelessly can easily break IPython!
12 12
13 13 String based transformations
14 14 ============================
15 15
16 .. currentmodule:: IPython.core.inputtransforms
17
16 18 When the user enters a line of code, it is first processed as a string. By the
17 19 end of this stage, it must be valid Python syntax.
18 20
19 21 These transformers all subclass :class:`IPython.core.inputtransformer.InputTransformer`,
20 and are used by :class:`IPython.core.inputsplitter.IPythonInputSplitter`,
21 the ``transform`` attribute of which is a list of instances.
22
23 By default, these transformers are skipped when :class:`~IPython.core.inputsplitter.IPythonInputSplitter`
24 detects that the line starts inside a multi-line string. Some transformers, such
25 as those that remove the prompt markers from pasted examples, need to look
26 inside multiline strings as well - these set the attribute ``look_in_string`` to
27 ``True``.
22 and are used by :class:`IPython.core.inputsplitter.IPythonInputSplitter`.
23
24 These transformers act in three groups, stored separately as lists of instances
25 in attributes of :class:`~IPython.core.inputsplitter.IPythonInputSplitter`:
26
27 * ``physical_line_transforms`` act on the lines as the user enters them. For
28 example, these strip Python prompts from examples pasted in.
29 * ``logical_line_transforms`` act on lines as connected by explicit line
30 continuations, i.e. ``\`` at the end of physical lines. They are skipped
31 inside multiline Python statements. This is the point where IPython recognises
32 ``%magic`` commands, for instance.
33 * ``python_line_transforms`` act on blocks containing complete Python statements.
34 Multi-line strings, lists and function calls are reassembled before being
35 passed to these, but note that function and class *definitions* are still a
36 series of separate statements. IPython does not use any of these by default.
28 37
29 38 Stateless transformations
30 39 -------------------------
31 40
32 41 The simplest kind of transformations work one line at a time. Write a function
33 42 which takes a line and returns a line, and decorate it with
34 :meth:`~IPython.core.inputtransformer.StatelessInputTransformer.wrap`::
43 :meth:`StatelessInputTransformer.wrap`::
35 44
36 45 @StatelessInputTransformer.wrap
37 46 def my_special_commands(line):
38 47 if line.startswith("Β¬"):
39 48 return "specialcommand(" + repr(line) + ")"
40 49 return line
41 50
42 51 The decorator returns a factory function which will produce instances of
43 52 :class:`~IPython.core.inputtransformer.StatelessInputTransformer` using your
44 53 function.
45 54
46 55 Coroutine transformers
47 56 ----------------------
48 57
49 58 More advanced transformers can be written as coroutines. The coroutine will be
50 59 sent each line in turn, followed by ``None`` to reset it. It can yield lines, or
51 60 ``None`` if it is accumulating text to yield at a later point. When reset, it
52 61 should give up any code it has accumulated.
53 62
54 63 This code in IPython strips a constant amount of leading indentation from each
55 64 line in a cell::
56 65
57 66 @CoroutineInputTransformer.wrap
58 67 def leading_indent():
59 68 """Remove leading indentation.
60 69
61 70 If the first line starts with a spaces or tabs, the same whitespace will be
62 71 removed from each following line until it is reset.
63 72 """
64 73 space_re = re.compile(r'^[ \t]+')
65 74 line = ''
66 75 while True:
67 76 line = (yield line)
68 77
69 78 if line is None:
70 79 continue
71 80
72 81 m = space_re.match(line)
73 82 if m:
74 83 space = m.group(0)
75 84 while line is not None:
76 85 if line.startswith(space):
77 86 line = line[len(space):]
78 87 line = (yield line)
79 88 else:
80 89 # No leading spaces - wait for reset
81 90 while line is not None:
82 91 line = (yield line)
83 92
84 93 leading_indent.look_in_string = True
85 94
95 Token-based transformers
96 ------------------------
97
98 There is an experimental framework that takes care of tokenizing and
99 untokenizing lines of code. Define a function that accepts a list of tokens, and
100 returns an iterable of output tokens, and decorate it with
101 :meth:`TokenInputTransformer.wrap`. These should only be used in
102 ``python_line_transforms``.
103
86 104 AST transformations
87 105 ===================
88 106
89 107 After the code has been parsed as Python syntax, you can use Python's powerful
90 108 *Abstract Syntax Tree* tools to modify it. Subclass :class:`ast.NodeTransformer`,
91 109 and add an instance to ``shell.ast_transformers``.
92 110
93 111 This example wraps integer literals in an ``Integer`` class, which is useful for
94 112 mathematical frameworks that want to handle e.g. ``1/3`` as a precise fraction::
95 113
96 114
97 115 class IntegerWrapper(ast.NodeTransformer):
98 116 """Wraps all integers in a call to Integer()"""
99 117 def visit_Num(self, node):
100 118 if isinstance(node.n, int):
101 119 return ast.Call(func=ast.Name(id='Integer', ctx=ast.Load()),
102 120 args=[node], keywords=[])
103 121 return node
General Comments 0
You need to be logged in to leave comments. Login now