##// END OF EJS Templates
darker
richard -
Show More
@@ -1,1257 +1,1260 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 """
2 """
3 Sphinx directive to support embedded IPython code.
3 Sphinx directive to support embedded IPython code.
4
4
5 IPython provides an extension for `Sphinx <http://www.sphinx-doc.org/>`_ to
5 IPython provides an extension for `Sphinx <http://www.sphinx-doc.org/>`_ to
6 highlight and run code.
6 highlight and run code.
7
7
8 This directive allows pasting of entire interactive IPython sessions, prompts
8 This directive allows pasting of entire interactive IPython sessions, prompts
9 and all, and their code will actually get re-executed at doc build time, with
9 and all, and their code will actually get re-executed at doc build time, with
10 all prompts renumbered sequentially. It also allows you to input code as a pure
10 all prompts renumbered sequentially. It also allows you to input code as a pure
11 python input by giving the argument python to the directive. The output looks
11 python input by giving the argument python to the directive. The output looks
12 like an interactive ipython section.
12 like an interactive ipython section.
13
13
14 Here is an example of how the IPython directive can
14 Here is an example of how the IPython directive can
15 **run** python code, at build time.
15 **run** python code, at build time.
16
16
17 .. ipython::
17 .. ipython::
18
18
19 In [1]: 1+1
19 In [1]: 1+1
20
20
21 In [1]: import datetime
21 In [1]: import datetime
22 ...: datetime.datetime.now()
22 ...: datetime.datetime.now()
23
23
24 It supports IPython construct that plain
24 It supports IPython construct that plain
25 Python does not understand (like magics):
25 Python does not understand (like magics):
26
26
27 .. ipython::
27 .. ipython::
28
28
29 In [0]: import time
29 In [0]: import time
30
30
31 In [0]: %timeit time.sleep(0.05)
31 In [0]: %timeit time.sleep(0.05)
32
32
33 This will also support top-level async when using IPython 7.0+
33 This will also support top-level async when using IPython 7.0+
34
34
35 .. ipython::
35 .. ipython::
36
36
37 In [2]: import asyncio
37 In [2]: import asyncio
38 ...: print('before')
38 ...: print('before')
39 ...: await asyncio.sleep(1)
39 ...: await asyncio.sleep(1)
40 ...: print('after')
40 ...: print('after')
41
41
42
42
43 The namespace will persist across multiple code chucks, Let's define a variable:
43 The namespace will persist across multiple code chucks, Let's define a variable:
44
44
45 .. ipython::
45 .. ipython::
46
46
47 In [0]: who = "World"
47 In [0]: who = "World"
48
48
49 And now say hello:
49 And now say hello:
50
50
51 .. ipython::
51 .. ipython::
52
52
53 In [0]: print('Hello,', who)
53 In [0]: print('Hello,', who)
54
54
55 If the current section raises an exception, you can add the ``:okexcept:`` flag
55 If the current section raises an exception, you can add the ``:okexcept:`` flag
56 to the current block, otherwise the build will fail.
56 to the current block, otherwise the build will fail.
57
57
58 .. ipython::
58 .. ipython::
59 :okexcept:
59 :okexcept:
60
60
61 In [1]: 1/0
61 In [1]: 1/0
62
62
63 IPython Sphinx directive module
63 IPython Sphinx directive module
64 ===============================
64 ===============================
65
65
66 To enable this directive, simply list it in your Sphinx ``conf.py`` file
66 To enable this directive, simply list it in your Sphinx ``conf.py`` file
67 (making sure the directory where you placed it is visible to sphinx, as is
67 (making sure the directory where you placed it is visible to sphinx, as is
68 needed for all Sphinx directives). For example, to enable syntax highlighting
68 needed for all Sphinx directives). For example, to enable syntax highlighting
69 and the IPython directive::
69 and the IPython directive::
70
70
71 extensions = ['IPython.sphinxext.ipython_console_highlighting',
71 extensions = ['IPython.sphinxext.ipython_console_highlighting',
72 'IPython.sphinxext.ipython_directive']
72 'IPython.sphinxext.ipython_directive']
73
73
74 The IPython directive outputs code-blocks with the language 'ipython'. So
74 The IPython directive outputs code-blocks with the language 'ipython'. So
75 if you do not have the syntax highlighting extension enabled as well, then
75 if you do not have the syntax highlighting extension enabled as well, then
76 all rendered code-blocks will be uncolored. By default this directive assumes
76 all rendered code-blocks will be uncolored. By default this directive assumes
77 that your prompts are unchanged IPython ones, but this can be customized.
77 that your prompts are unchanged IPython ones, but this can be customized.
78 The configurable options that can be placed in conf.py are:
78 The configurable options that can be placed in conf.py are:
79
79
80 ipython_savefig_dir:
80 ipython_savefig_dir:
81 The directory in which to save the figures. This is relative to the
81 The directory in which to save the figures. This is relative to the
82 Sphinx source directory. The default is `html_static_path`.
82 Sphinx source directory. The default is `html_static_path`.
83 ipython_rgxin:
83 ipython_rgxin:
84 The compiled regular expression to denote the start of IPython input
84 The compiled regular expression to denote the start of IPython input
85 lines. The default is ``re.compile('In \\[(\\d+)\\]:\\s?(.*)\\s*')``. You
85 lines. The default is ``re.compile('In \\[(\\d+)\\]:\\s?(.*)\\s*')``. You
86 shouldn't need to change this.
86 shouldn't need to change this.
87 ipython_warning_is_error: [default to True]
87 ipython_warning_is_error: [default to True]
88 Fail the build if something unexpected happen, for example if a block raise
88 Fail the build if something unexpected happen, for example if a block raise
89 an exception but does not have the `:okexcept:` flag. The exact behavior of
89 an exception but does not have the `:okexcept:` flag. The exact behavior of
90 what is considered strict, may change between the sphinx directive version.
90 what is considered strict, may change between the sphinx directive version.
91 ipython_rgxout:
91 ipython_rgxout:
92 The compiled regular expression to denote the start of IPython output
92 The compiled regular expression to denote the start of IPython output
93 lines. The default is ``re.compile('Out\\[(\\d+)\\]:\\s?(.*)\\s*')``. You
93 lines. The default is ``re.compile('Out\\[(\\d+)\\]:\\s?(.*)\\s*')``. You
94 shouldn't need to change this.
94 shouldn't need to change this.
95 ipython_promptin:
95 ipython_promptin:
96 The string to represent the IPython input prompt in the generated ReST.
96 The string to represent the IPython input prompt in the generated ReST.
97 The default is ``'In [%d]:'``. This expects that the line numbers are used
97 The default is ``'In [%d]:'``. This expects that the line numbers are used
98 in the prompt.
98 in the prompt.
99 ipython_promptout:
99 ipython_promptout:
100 The string to represent the IPython prompt in the generated ReST. The
100 The string to represent the IPython prompt in the generated ReST. The
101 default is ``'Out [%d]:'``. This expects that the line numbers are used
101 default is ``'Out [%d]:'``. This expects that the line numbers are used
102 in the prompt.
102 in the prompt.
103 ipython_mplbackend:
103 ipython_mplbackend:
104 The string which specifies if the embedded Sphinx shell should import
104 The string which specifies if the embedded Sphinx shell should import
105 Matplotlib and set the backend. The value specifies a backend that is
105 Matplotlib and set the backend. The value specifies a backend that is
106 passed to `matplotlib.use()` before any lines in `ipython_execlines` are
106 passed to `matplotlib.use()` before any lines in `ipython_execlines` are
107 executed. If not specified in conf.py, then the default value of 'agg' is
107 executed. If not specified in conf.py, then the default value of 'agg' is
108 used. To use the IPython directive without matplotlib as a dependency, set
108 used. To use the IPython directive without matplotlib as a dependency, set
109 the value to `None`. It may end up that matplotlib is still imported
109 the value to `None`. It may end up that matplotlib is still imported
110 if the user specifies so in `ipython_execlines` or makes use of the
110 if the user specifies so in `ipython_execlines` or makes use of the
111 @savefig pseudo decorator.
111 @savefig pseudo decorator.
112 ipython_execlines:
112 ipython_execlines:
113 A list of strings to be exec'd in the embedded Sphinx shell. Typical
113 A list of strings to be exec'd in the embedded Sphinx shell. Typical
114 usage is to make certain packages always available. Set this to an empty
114 usage is to make certain packages always available. Set this to an empty
115 list if you wish to have no imports always available. If specified in
115 list if you wish to have no imports always available. If specified in
116 ``conf.py`` as `None`, then it has the effect of making no imports available.
116 ``conf.py`` as `None`, then it has the effect of making no imports available.
117 If omitted from conf.py altogether, then the default value of
117 If omitted from conf.py altogether, then the default value of
118 ['import numpy as np', 'import matplotlib.pyplot as plt'] is used.
118 ['import numpy as np', 'import matplotlib.pyplot as plt'] is used.
119 ipython_holdcount
119 ipython_holdcount
120 When the @suppress pseudo-decorator is used, the execution count can be
120 When the @suppress pseudo-decorator is used, the execution count can be
121 incremented or not. The default behavior is to hold the execution count,
121 incremented or not. The default behavior is to hold the execution count,
122 corresponding to a value of `True`. Set this to `False` to increment
122 corresponding to a value of `True`. Set this to `False` to increment
123 the execution count after each suppressed command.
123 the execution count after each suppressed command.
124
124
125 As an example, to use the IPython directive when `matplotlib` is not available,
125 As an example, to use the IPython directive when `matplotlib` is not available,
126 one sets the backend to `None`::
126 one sets the backend to `None`::
127
127
128 ipython_mplbackend = None
128 ipython_mplbackend = None
129
129
130 An example usage of the directive is:
130 An example usage of the directive is:
131
131
132 .. code-block:: rst
132 .. code-block:: rst
133
133
134 .. ipython::
134 .. ipython::
135
135
136 In [1]: x = 1
136 In [1]: x = 1
137
137
138 In [2]: y = x**2
138 In [2]: y = x**2
139
139
140 In [3]: print(y)
140 In [3]: print(y)
141
141
142 See http://matplotlib.org/sampledoc/ipython_directive.html for additional
142 See http://matplotlib.org/sampledoc/ipython_directive.html for additional
143 documentation.
143 documentation.
144
144
145 Pseudo-Decorators
145 Pseudo-Decorators
146 =================
146 =================
147
147
148 Note: Only one decorator is supported per input. If more than one decorator
148 Note: Only one decorator is supported per input. If more than one decorator
149 is specified, then only the last one is used.
149 is specified, then only the last one is used.
150
150
151 In addition to the Pseudo-Decorators/options described at the above link,
151 In addition to the Pseudo-Decorators/options described at the above link,
152 several enhancements have been made. The directive will emit a message to the
152 several enhancements have been made. The directive will emit a message to the
153 console at build-time if code-execution resulted in an exception or warning.
153 console at build-time if code-execution resulted in an exception or warning.
154 You can suppress these on a per-block basis by specifying the :okexcept:
154 You can suppress these on a per-block basis by specifying the :okexcept:
155 or :okwarning: options:
155 or :okwarning: options:
156
156
157 .. code-block:: rst
157 .. code-block:: rst
158
158
159 .. ipython::
159 .. ipython::
160 :okexcept:
160 :okexcept:
161 :okwarning:
161 :okwarning:
162
162
163 In [1]: 1/0
163 In [1]: 1/0
164 In [2]: # raise warning.
164 In [2]: # raise warning.
165
165
166 To Do
166 To Do
167 =====
167 =====
168
168
169 - Turn the ad-hoc test() function into a real test suite.
169 - Turn the ad-hoc test() function into a real test suite.
170 - Break up ipython-specific functionality from matplotlib stuff into better
170 - Break up ipython-specific functionality from matplotlib stuff into better
171 separated code.
171 separated code.
172
172
173 """
173 """
174
174
175 # Authors
175 # Authors
176 # =======
176 # =======
177 #
177 #
178 # - John D Hunter: original author.
178 # - John D Hunter: original author.
179 # - Fernando Perez: refactoring, documentation, cleanups, port to 0.11.
179 # - Fernando Perez: refactoring, documentation, cleanups, port to 0.11.
180 # - VΓ‘clavΕ milauer <eudoxos-AT-arcig.cz>: Prompt generalizations.
180 # - VΓ‘clavΕ milauer <eudoxos-AT-arcig.cz>: Prompt generalizations.
181 # - Skipper Seabold, refactoring, cleanups, pure python addition
181 # - Skipper Seabold, refactoring, cleanups, pure python addition
182
182
183 #-----------------------------------------------------------------------------
183 #-----------------------------------------------------------------------------
184 # Imports
184 # Imports
185 #-----------------------------------------------------------------------------
185 #-----------------------------------------------------------------------------
186
186
187 # Stdlib
187 # Stdlib
188 import atexit
188 import atexit
189 import errno
189 import errno
190 import os
190 import os
191 import pathlib
191 import pathlib
192 import re
192 import re
193 import sys
193 import sys
194 import tempfile
194 import tempfile
195 import ast
195 import ast
196 import warnings
196 import warnings
197 import shutil
197 import shutil
198 from io import StringIO
198 from io import StringIO
199
199
200 # Third-party
200 # Third-party
201 from docutils.parsers.rst import directives
201 from docutils.parsers.rst import directives
202 from docutils.parsers.rst import Directive
202 from docutils.parsers.rst import Directive
203 from sphinx.util import logging
203 from sphinx.util import logging
204
204
205 # Our own
205 # Our own
206 from traitlets.config import Config
206 from traitlets.config import Config
207 from IPython import InteractiveShell
207 from IPython import InteractiveShell
208 from IPython.core.profiledir import ProfileDir
208 from IPython.core.profiledir import ProfileDir
209
209
210 use_matplotlib = False
210 use_matplotlib = False
211 try:
211 try:
212 import matplotlib
212 import matplotlib
213 use_matplotlib = True
213 use_matplotlib = True
214 except Exception:
214 except Exception:
215 pass
215 pass
216
216
217 #-----------------------------------------------------------------------------
217 #-----------------------------------------------------------------------------
218 # Globals
218 # Globals
219 #-----------------------------------------------------------------------------
219 #-----------------------------------------------------------------------------
220 # for tokenizing blocks
220 # for tokenizing blocks
221 COMMENT, INPUT, OUTPUT = range(3)
221 COMMENT, INPUT, OUTPUT = range(3)
222
222
223 #-----------------------------------------------------------------------------
223 #-----------------------------------------------------------------------------
224 # Functions and class declarations
224 # Functions and class declarations
225 #-----------------------------------------------------------------------------
225 #-----------------------------------------------------------------------------
226
226
227 def block_parser(part, rgxin, rgxout, fmtin, fmtout):
227 def block_parser(part, rgxin, rgxout, fmtin, fmtout):
228 """
228 """
229 part is a string of ipython text, comprised of at most one
229 part is a string of ipython text, comprised of at most one
230 input, one output, comments, and blank lines. The block parser
230 input, one output, comments, and blank lines. The block parser
231 parses the text into a list of::
231 parses the text into a list of::
232
232
233 blocks = [ (TOKEN0, data0), (TOKEN1, data1), ...]
233 blocks = [ (TOKEN0, data0), (TOKEN1, data1), ...]
234
234
235 where TOKEN is one of [COMMENT | INPUT | OUTPUT ] and
235 where TOKEN is one of [COMMENT | INPUT | OUTPUT ] and
236 data is, depending on the type of token::
236 data is, depending on the type of token::
237
237
238 COMMENT : the comment string
238 COMMENT : the comment string
239
239
240 INPUT: the (DECORATOR, INPUT_LINE, REST) where
240 INPUT: the (DECORATOR, INPUT_LINE, REST) where
241 DECORATOR: the input decorator (or None)
241 DECORATOR: the input decorator (or None)
242 INPUT_LINE: the input as string (possibly multi-line)
242 INPUT_LINE: the input as string (possibly multi-line)
243 REST : any stdout generated by the input line (not OUTPUT)
243 REST : any stdout generated by the input line (not OUTPUT)
244
244
245 OUTPUT: the output string, possibly multi-line
245 OUTPUT: the output string, possibly multi-line
246
246
247 """
247 """
248 block = []
248 block = []
249 lines = part.split('\n')
249 lines = part.split('\n')
250 N = len(lines)
250 N = len(lines)
251 i = 0
251 i = 0
252 decorator = None
252 decorator = None
253 while 1:
253 while 1:
254
254
255 if i==N:
255 if i==N:
256 # nothing left to parse -- the last line
256 # nothing left to parse -- the last line
257 break
257 break
258
258
259 line = lines[i]
259 line = lines[i]
260 i += 1
260 i += 1
261 line_stripped = line.strip()
261 line_stripped = line.strip()
262 if line_stripped.startswith('#'):
262 if line_stripped.startswith('#'):
263 block.append((COMMENT, line))
263 block.append((COMMENT, line))
264 continue
264 continue
265
265
266 if line_stripped.startswith('@'):
266 if line_stripped.startswith('@'):
267 # Here is where we assume there is, at most, one decorator.
267 # Here is where we assume there is, at most, one decorator.
268 # Might need to rethink this.
268 # Might need to rethink this.
269 decorator = line_stripped
269 decorator = line_stripped
270 continue
270 continue
271
271
272 # does this look like an input line?
272 # does this look like an input line?
273 matchin = rgxin.match(line)
273 matchin = rgxin.match(line)
274 if matchin:
274 if matchin:
275 lineno, inputline = int(matchin.group(1)), matchin.group(2)
275 lineno, inputline = int(matchin.group(1)), matchin.group(2)
276
276
277 # the ....: continuation string
277 # the ....: continuation string
278 continuation = ' %s:'%''.join(['.']*(len(str(lineno))+2))
278 continuation = ' %s:'%''.join(['.']*(len(str(lineno))+2))
279 Nc = len(continuation)
279 Nc = len(continuation)
280 # input lines can continue on for more than one line, if
280 # input lines can continue on for more than one line, if
281 # we have a '\' line continuation char or a function call
281 # we have a '\' line continuation char or a function call
282 # echo line 'print'. The input line can only be
282 # echo line 'print'. The input line can only be
283 # terminated by the end of the block or an output line, so
283 # terminated by the end of the block or an output line, so
284 # we parse out the rest of the input line if it is
284 # we parse out the rest of the input line if it is
285 # multiline as well as any echo text
285 # multiline as well as any echo text
286
286
287 rest = []
287 rest = []
288 while i<N:
288 while i<N:
289
289
290 # look ahead; if the next line is blank, or a comment, or
290 # look ahead; if the next line is blank, or a comment, or
291 # an output line, we're done
291 # an output line, we're done
292
292
293 nextline = lines[i]
293 nextline = lines[i]
294 matchout = rgxout.match(nextline)
294 matchout = rgxout.match(nextline)
295 #print "nextline=%s, continuation=%s, starts=%s"%(nextline, continuation, nextline.startswith(continuation))
295 #print "nextline=%s, continuation=%s, starts=%s"%(nextline, continuation, nextline.startswith(continuation))
296 if matchout or nextline.startswith('#'):
296 if matchout or nextline.startswith('#'):
297 break
297 break
298 elif nextline.startswith(continuation):
298 elif nextline.startswith(continuation):
299 # The default ipython_rgx* treat the space following the colon as optional.
299 # The default ipython_rgx* treat the space following the colon as optional.
300 # However, If the space is there we must consume it or code
300 # However, If the space is there we must consume it or code
301 # employing the cython_magic extension will fail to execute.
301 # employing the cython_magic extension will fail to execute.
302 #
302 #
303 # This works with the default ipython_rgx* patterns,
303 # This works with the default ipython_rgx* patterns,
304 # If you modify them, YMMV.
304 # If you modify them, YMMV.
305 nextline = nextline[Nc:]
305 nextline = nextline[Nc:]
306 if nextline and nextline[0] == ' ':
306 if nextline and nextline[0] == ' ':
307 nextline = nextline[1:]
307 nextline = nextline[1:]
308
308
309 inputline += '\n' + nextline
309 inputline += '\n' + nextline
310 else:
310 else:
311 rest.append(nextline)
311 rest.append(nextline)
312 i+= 1
312 i+= 1
313
313
314 block.append((INPUT, (decorator, inputline, '\n'.join(rest))))
314 block.append((INPUT, (decorator, inputline, '\n'.join(rest))))
315 continue
315 continue
316
316
317 # if it looks like an output line grab all the text to the end
317 # if it looks like an output line grab all the text to the end
318 # of the block
318 # of the block
319 matchout = rgxout.match(line)
319 matchout = rgxout.match(line)
320 if matchout:
320 if matchout:
321 lineno, output = int(matchout.group(1)), matchout.group(2)
321 lineno, output = int(matchout.group(1)), matchout.group(2)
322 if i<N-1:
322 if i<N-1:
323 output = '\n'.join([output] + lines[i:])
323 output = '\n'.join([output] + lines[i:])
324
324
325 block.append((OUTPUT, output))
325 block.append((OUTPUT, output))
326 break
326 break
327
327
328 return block
328 return block
329
329
330
330
331 class EmbeddedSphinxShell(object):
331 class EmbeddedSphinxShell(object):
332 """An embedded IPython instance to run inside Sphinx"""
332 """An embedded IPython instance to run inside Sphinx"""
333
333
334 def __init__(self, exec_lines=None):
334 def __init__(self, exec_lines=None):
335
335
336 self.cout = StringIO()
336 self.cout = StringIO()
337
337
338 if exec_lines is None:
338 if exec_lines is None:
339 exec_lines = []
339 exec_lines = []
340
340
341 # Create config object for IPython
341 # Create config object for IPython
342 config = Config()
342 config = Config()
343 config.HistoryManager.hist_file = ':memory:'
343 config.HistoryManager.hist_file = ':memory:'
344 config.InteractiveShell.autocall = False
344 config.InteractiveShell.autocall = False
345 config.InteractiveShell.autoindent = False
345 config.InteractiveShell.autoindent = False
346 config.InteractiveShell.colors = 'NoColor'
346 config.InteractiveShell.colors = 'NoColor'
347
347
348 # create a profile so instance history isn't saved
348 # create a profile so instance history isn't saved
349 tmp_profile_dir = tempfile.mkdtemp(prefix='profile_')
349 tmp_profile_dir = tempfile.mkdtemp(prefix='profile_')
350 profname = 'auto_profile_sphinx_build'
350 profname = 'auto_profile_sphinx_build'
351 pdir = os.path.join(tmp_profile_dir,profname)
351 pdir = os.path.join(tmp_profile_dir,profname)
352 profile = ProfileDir.create_profile_dir(pdir)
352 profile = ProfileDir.create_profile_dir(pdir)
353
353
354 # Create and initialize global ipython, but don't start its mainloop.
354 # Create and initialize global ipython, but don't start its mainloop.
355 # This will persist across different EmbeddedSphinxShell instances.
355 # This will persist across different EmbeddedSphinxShell instances.
356 IP = InteractiveShell.instance(config=config, profile_dir=profile)
356 IP = InteractiveShell.instance(config=config, profile_dir=profile)
357 atexit.register(self.cleanup)
357 atexit.register(self.cleanup)
358
358
359 # Store a few parts of IPython we'll need.
359 # Store a few parts of IPython we'll need.
360 self.IP = IP
360 self.IP = IP
361 self.user_ns = self.IP.user_ns
361 self.user_ns = self.IP.user_ns
362 self.user_global_ns = self.IP.user_global_ns
362 self.user_global_ns = self.IP.user_global_ns
363
363
364 self.input = ''
364 self.input = ''
365 self.output = ''
365 self.output = ''
366 self.tmp_profile_dir = tmp_profile_dir
366 self.tmp_profile_dir = tmp_profile_dir
367
367
368 self.is_verbatim = False
368 self.is_verbatim = False
369 self.is_doctest = False
369 self.is_doctest = False
370 self.is_suppress = False
370 self.is_suppress = False
371
371
372 # Optionally, provide more detailed information to shell.
372 # Optionally, provide more detailed information to shell.
373 # this is assigned by the SetUp method of IPythonDirective
373 # this is assigned by the SetUp method of IPythonDirective
374 # to point at itself.
374 # to point at itself.
375 #
375 #
376 # So, you can access handy things at self.directive.state
376 # So, you can access handy things at self.directive.state
377 self.directive = None
377 self.directive = None
378
378
379 # on the first call to the savefig decorator, we'll import
379 # on the first call to the savefig decorator, we'll import
380 # pyplot as plt so we can make a call to the plt.gcf().savefig
380 # pyplot as plt so we can make a call to the plt.gcf().savefig
381 self._pyplot_imported = False
381 self._pyplot_imported = False
382
382
383 # Prepopulate the namespace.
383 # Prepopulate the namespace.
384 for line in exec_lines:
384 for line in exec_lines:
385 self.process_input_line(line, store_history=False)
385 self.process_input_line(line, store_history=False)
386
386
387 def cleanup(self):
387 def cleanup(self):
388 shutil.rmtree(self.tmp_profile_dir, ignore_errors=True)
388 shutil.rmtree(self.tmp_profile_dir, ignore_errors=True)
389
389
390 def clear_cout(self):
390 def clear_cout(self):
391 self.cout.seek(0)
391 self.cout.seek(0)
392 self.cout.truncate(0)
392 self.cout.truncate(0)
393
393
394 def process_input_line(self, line, store_history):
394 def process_input_line(self, line, store_history):
395 return self.process_input_lines([line], store_history=store_history)
395 return self.process_input_lines([line], store_history=store_history)
396
396
397 def process_input_lines(self, lines, store_history=True):
397 def process_input_lines(self, lines, store_history=True):
398 """process the input, capturing stdout"""
398 """process the input, capturing stdout"""
399 stdout = sys.stdout
399 stdout = sys.stdout
400 source_raw = '\n'.join(lines)
400 source_raw = '\n'.join(lines)
401 try:
401 try:
402 sys.stdout = self.cout
402 sys.stdout = self.cout
403 self.IP.run_cell(source_raw, store_history=store_history)
403 self.IP.run_cell(source_raw, store_history=store_history)
404 finally:
404 finally:
405 sys.stdout = stdout
405 sys.stdout = stdout
406
406
407 def process_image(self, decorator):
407 def process_image(self, decorator):
408 """
408 """
409 # build out an image directive like
409 # build out an image directive like
410 # .. image:: somefile.png
410 # .. image:: somefile.png
411 # :width 4in
411 # :width 4in
412 #
412 #
413 # from an input like
413 # from an input like
414 # savefig somefile.png width=4in
414 # savefig somefile.png width=4in
415 """
415 """
416 savefig_dir = self.savefig_dir
416 savefig_dir = self.savefig_dir
417 source_dir = self.source_dir
417 source_dir = self.source_dir
418 saveargs = decorator.split(' ')
418 saveargs = decorator.split(' ')
419 filename = saveargs[1]
419 filename = saveargs[1]
420 # insert relative path to image file in source
420 # insert relative path to image file in source
421 # as absolute path for Sphinx
421 # as absolute path for Sphinx
422 # sphinx expects a posix path, even on Windows
422 # sphinx expects a posix path, even on Windows
423 path = pathlib.Path(savefig_dir, filename)
423 path = pathlib.Path(savefig_dir, filename)
424 outfile = '/' + path.relative_to(source_dir).as_posix()
424 outfile = '/' + path.relative_to(source_dir).as_posix()
425
425
426 imagerows = ['.. image:: %s' % outfile]
426 imagerows = ['.. image:: %s' % outfile]
427
427
428 for kwarg in saveargs[2:]:
428 for kwarg in saveargs[2:]:
429 arg, val = kwarg.split('=')
429 arg, val = kwarg.split('=')
430 arg = arg.strip()
430 arg = arg.strip()
431 val = val.strip()
431 val = val.strip()
432 imagerows.append(' :%s: %s'%(arg, val))
432 imagerows.append(' :%s: %s'%(arg, val))
433
433
434 image_file = os.path.basename(outfile) # only return file name
434 image_file = os.path.basename(outfile) # only return file name
435 image_directive = '\n'.join(imagerows)
435 image_directive = '\n'.join(imagerows)
436 return image_file, image_directive
436 return image_file, image_directive
437
437
438 # Callbacks for each type of token
438 # Callbacks for each type of token
439 def process_input(self, data, input_prompt, lineno):
439 def process_input(self, data, input_prompt, lineno):
440 """
440 """
441 Process data block for INPUT token.
441 Process data block for INPUT token.
442
442
443 """
443 """
444 decorator, input, rest = data
444 decorator, input, rest = data
445 image_file = None
445 image_file = None
446 image_directive = None
446 image_directive = None
447
447
448 is_verbatim = decorator=='@verbatim' or self.is_verbatim
448 is_verbatim = decorator=='@verbatim' or self.is_verbatim
449 is_doctest = (decorator is not None and \
449 is_doctest = (decorator is not None and \
450 decorator.startswith('@doctest')) or self.is_doctest
450 decorator.startswith('@doctest')) or self.is_doctest
451 is_suppress = decorator=='@suppress' or self.is_suppress
451 is_suppress = decorator=='@suppress' or self.is_suppress
452 is_okexcept = decorator=='@okexcept' or self.is_okexcept
452 is_okexcept = decorator=='@okexcept' or self.is_okexcept
453 is_okwarning = decorator=='@okwarning' or self.is_okwarning
453 is_okwarning = decorator=='@okwarning' or self.is_okwarning
454 is_savefig = decorator is not None and \
454 is_savefig = decorator is not None and \
455 decorator.startswith('@savefig')
455 decorator.startswith('@savefig')
456
456
457 input_lines = input.split('\n')
457 input_lines = input.split('\n')
458 if len(input_lines) > 1:
458 if len(input_lines) > 1:
459 if input_lines[-1] != "":
459 if input_lines[-1] != "":
460 input_lines.append('') # make sure there's a blank line
460 input_lines.append('') # make sure there's a blank line
461 # so splitter buffer gets reset
461 # so splitter buffer gets reset
462
462
463 continuation = ' %s:'%''.join(['.']*(len(str(lineno))+2))
463 continuation = ' %s:'%''.join(['.']*(len(str(lineno))+2))
464
464
465 if is_savefig:
465 if is_savefig:
466 image_file, image_directive = self.process_image(decorator)
466 image_file, image_directive = self.process_image(decorator)
467
467
468 ret = []
468 ret = []
469 is_semicolon = False
469 is_semicolon = False
470
470
471 # Hold the execution count, if requested to do so.
471 # Hold the execution count, if requested to do so.
472 if is_suppress and self.hold_count:
472 if is_suppress and self.hold_count:
473 store_history = False
473 store_history = False
474 else:
474 else:
475 store_history = True
475 store_history = True
476
476
477 # Note: catch_warnings is not thread safe
477 # Note: catch_warnings is not thread safe
478 with warnings.catch_warnings(record=True) as ws:
478 with warnings.catch_warnings(record=True) as ws:
479 if input_lines[0].endswith(';'):
479 if input_lines[0].endswith(';'):
480 is_semicolon = True
480 is_semicolon = True
481 #for i, line in enumerate(input_lines):
481 #for i, line in enumerate(input_lines):
482
482
483 # process the first input line
483 # process the first input line
484 if is_verbatim:
484 if is_verbatim:
485 self.process_input_lines([''])
485 self.process_input_lines([''])
486 self.IP.execution_count += 1 # increment it anyway
486 self.IP.execution_count += 1 # increment it anyway
487 else:
487 else:
488 # only submit the line in non-verbatim mode
488 # only submit the line in non-verbatim mode
489 self.process_input_lines(input_lines, store_history=store_history)
489 self.process_input_lines(input_lines, store_history=store_history)
490
490
491 if not is_suppress:
491 if not is_suppress:
492 for i, line in enumerate(input_lines):
492 for i, line in enumerate(input_lines):
493 if i == 0:
493 if i == 0:
494 formatted_line = '%s %s'%(input_prompt, line)
494 formatted_line = '%s %s'%(input_prompt, line)
495 else:
495 else:
496 formatted_line = '%s %s'%(continuation, line)
496 formatted_line = '%s %s'%(continuation, line)
497 ret.append(formatted_line)
497 ret.append(formatted_line)
498
498
499 if not is_suppress and len(rest.strip()) and is_verbatim:
499 if not is_suppress and len(rest.strip()) and is_verbatim:
500 # The "rest" is the standard output of the input. This needs to be
500 # The "rest" is the standard output of the input. This needs to be
501 # added when in verbatim mode. If there is no "rest", then we don't
501 # added when in verbatim mode. If there is no "rest", then we don't
502 # add it, as the new line will be added by the processed output.
502 # add it, as the new line will be added by the processed output.
503 ret.append(rest)
503 ret.append(rest)
504
504
505 # Fetch the processed output. (This is not the submitted output.)
505 # Fetch the processed output. (This is not the submitted output.)
506 self.cout.seek(0)
506 self.cout.seek(0)
507 processed_output = self.cout.read()
507 processed_output = self.cout.read()
508 if not is_suppress and not is_semicolon:
508 if not is_suppress and not is_semicolon:
509 #
509 #
510 # In IPythonDirective.run, the elements of `ret` are eventually
510 # In IPythonDirective.run, the elements of `ret` are eventually
511 # combined such that '' entries correspond to newlines. So if
511 # combined such that '' entries correspond to newlines. So if
512 # `processed_output` is equal to '', then the adding it to `ret`
512 # `processed_output` is equal to '', then the adding it to `ret`
513 # ensures that there is a blank line between consecutive inputs
513 # ensures that there is a blank line between consecutive inputs
514 # that have no outputs, as in:
514 # that have no outputs, as in:
515 #
515 #
516 # In [1]: x = 4
516 # In [1]: x = 4
517 #
517 #
518 # In [2]: x = 5
518 # In [2]: x = 5
519 #
519 #
520 # When there is processed output, it has a '\n' at the tail end. So
520 # When there is processed output, it has a '\n' at the tail end. So
521 # adding the output to `ret` will provide the necessary spacing
521 # adding the output to `ret` will provide the necessary spacing
522 # between consecutive input/output blocks, as in:
522 # between consecutive input/output blocks, as in:
523 #
523 #
524 # In [1]: x
524 # In [1]: x
525 # Out[1]: 5
525 # Out[1]: 5
526 #
526 #
527 # In [2]: x
527 # In [2]: x
528 # Out[2]: 5
528 # Out[2]: 5
529 #
529 #
530 # When there is stdout from the input, it also has a '\n' at the
530 # When there is stdout from the input, it also has a '\n' at the
531 # tail end, and so this ensures proper spacing as well. E.g.:
531 # tail end, and so this ensures proper spacing as well. E.g.:
532 #
532 #
533 # In [1]: print x
533 # In [1]: print x
534 # 5
534 # 5
535 #
535 #
536 # In [2]: x = 5
536 # In [2]: x = 5
537 #
537 #
538 # When in verbatim mode, `processed_output` is empty (because
538 # When in verbatim mode, `processed_output` is empty (because
539 # nothing was passed to IP. Sometimes the submitted code block has
539 # nothing was passed to IP. Sometimes the submitted code block has
540 # an Out[] portion and sometimes it does not. When it does not, we
540 # an Out[] portion and sometimes it does not. When it does not, we
541 # need to ensure proper spacing, so we have to add '' to `ret`.
541 # need to ensure proper spacing, so we have to add '' to `ret`.
542 # However, if there is an Out[] in the submitted code, then we do
542 # However, if there is an Out[] in the submitted code, then we do
543 # not want to add a newline as `process_output` has stuff to add.
543 # not want to add a newline as `process_output` has stuff to add.
544 # The difficulty is that `process_input` doesn't know if
544 # The difficulty is that `process_input` doesn't know if
545 # `process_output` will be called---so it doesn't know if there is
545 # `process_output` will be called---so it doesn't know if there is
546 # Out[] in the code block. The requires that we include a hack in
546 # Out[] in the code block. The requires that we include a hack in
547 # `process_block`. See the comments there.
547 # `process_block`. See the comments there.
548 #
548 #
549 ret.append(processed_output)
549 ret.append(processed_output)
550 elif is_semicolon:
550 elif is_semicolon:
551 # Make sure there is a newline after the semicolon.
551 # Make sure there is a newline after the semicolon.
552 ret.append('')
552 ret.append('')
553
553
554 # context information
554 # context information
555 filename = "Unknown"
555 filename = "Unknown"
556 lineno = 0
556 lineno = 0
557 if self.directive.state:
557 if self.directive.state:
558 filename = self.directive.state.document.current_source
558 filename = self.directive.state.document.current_source
559 lineno = self.directive.state.document.current_line
559 lineno = self.directive.state.document.current_line
560
560
561 # Use sphinx logger for warnings
561 # Use sphinx logger for warnings
562 logger = logging.getLogger(__name__)
562 logger = logging.getLogger(__name__)
563
563
564 # output any exceptions raised during execution to stdout
564 # output any exceptions raised during execution to stdout
565 # unless :okexcept: has been specified.
565 # unless :okexcept: has been specified.
566 if not is_okexcept and (("Traceback" in processed_output) or ("SyntaxError" in processed_output)):
566 if not is_okexcept and (
567 s = '\n>>>' + ('-' * 73) + '\n'
567 ("Traceback" in processed_output) or ("SyntaxError" in processed_output)
568 ):
569 s = "\n>>>" + ("-" * 73) + "\n"
568 s += "Exception in %s at block ending on line %s\n" % (filename, lineno)
570 s += "Exception in %s at block ending on line %s\n" % (filename, lineno)
569 s += "Specify :okexcept: as an option in the ipython:: block to suppress this message\n"
571 s += "Specify :okexcept: as an option in the ipython:: block to suppress this message\n"
570 s += processed_output + "\n"
572 s += processed_output + "\n"
571 s += "<<<" + ("-" * 73)
573 s += "<<<" + ("-" * 73)
572 logger.warning(s)
574 logger.warning(s)
573 if self.warning_is_error:
575 if self.warning_is_error:
574 raise RuntimeError('Non Expected exception in `{}` line {}'.format(filename, lineno))
576 raise RuntimeError('Non Expected exception in `{}` line {}'.format(filename, lineno))
575
577
576 # output any warning raised during execution to stdout
578 # output any warning raised during execution to stdout
577 # unless :okwarning: has been specified.
579 # unless :okwarning: has been specified.
578 if not is_okwarning:
580 if not is_okwarning:
579 for w in ws:
581 for w in ws:
580 s = '\n>>>' + ('-' * 73) + '\n'
582 s = "\n>>>" + ("-" * 73) + "\n"
581 s += "Warning in %s at block ending on line %s\n" % (filename, lineno)
583 s += "Warning in %s at block ending on line %s\n" % (filename, lineno)
582 s += "Specify :okwarning: as an option in the ipython:: block to suppress this message\n"
584 s += "Specify :okwarning: as an option in the ipython:: block to suppress this message\n"
583 s += ('-' * 76) + '\n'
585 s += ("-" * 76) + "\n"
584 s += warnings.formatwarning(w.message, w.category,
586 s += warnings.formatwarning(
585 w.filename, w.lineno, w.line)
587 w.message, w.category, w.filename, w.lineno, w.line
586 s += '<<<' + ('-' * 73)
588 )
589 s += "<<<" + ("-" * 73)
587 logger.warning(s)
590 logger.warning(s)
588 if self.warning_is_error:
591 if self.warning_is_error:
589 raise RuntimeError('Non Expected warning in `{}` line {}'.format(filename, lineno))
592 raise RuntimeError('Non Expected warning in `{}` line {}'.format(filename, lineno))
590
593
591 self.clear_cout()
594 self.clear_cout()
592 return (ret, input_lines, processed_output,
595 return (ret, input_lines, processed_output,
593 is_doctest, decorator, image_file, image_directive)
596 is_doctest, decorator, image_file, image_directive)
594
597
595
598
596 def process_output(self, data, output_prompt, input_lines, output,
599 def process_output(self, data, output_prompt, input_lines, output,
597 is_doctest, decorator, image_file):
600 is_doctest, decorator, image_file):
598 """
601 """
599 Process data block for OUTPUT token.
602 Process data block for OUTPUT token.
600
603
601 """
604 """
602 # Recall: `data` is the submitted output, and `output` is the processed
605 # Recall: `data` is the submitted output, and `output` is the processed
603 # output from `input_lines`.
606 # output from `input_lines`.
604
607
605 TAB = ' ' * 4
608 TAB = ' ' * 4
606
609
607 if is_doctest and output is not None:
610 if is_doctest and output is not None:
608
611
609 found = output # This is the processed output
612 found = output # This is the processed output
610 found = found.strip()
613 found = found.strip()
611 submitted = data.strip()
614 submitted = data.strip()
612
615
613 if self.directive is None:
616 if self.directive is None:
614 source = 'Unavailable'
617 source = 'Unavailable'
615 content = 'Unavailable'
618 content = 'Unavailable'
616 else:
619 else:
617 source = self.directive.state.document.current_source
620 source = self.directive.state.document.current_source
618 content = self.directive.content
621 content = self.directive.content
619 # Add tabs and join into a single string.
622 # Add tabs and join into a single string.
620 content = '\n'.join([TAB + line for line in content])
623 content = '\n'.join([TAB + line for line in content])
621
624
622 # Make sure the output contains the output prompt.
625 # Make sure the output contains the output prompt.
623 ind = found.find(output_prompt)
626 ind = found.find(output_prompt)
624 if ind < 0:
627 if ind < 0:
625 e = ('output does not contain output prompt\n\n'
628 e = ('output does not contain output prompt\n\n'
626 'Document source: {0}\n\n'
629 'Document source: {0}\n\n'
627 'Raw content: \n{1}\n\n'
630 'Raw content: \n{1}\n\n'
628 'Input line(s):\n{TAB}{2}\n\n'
631 'Input line(s):\n{TAB}{2}\n\n'
629 'Output line(s):\n{TAB}{3}\n\n')
632 'Output line(s):\n{TAB}{3}\n\n')
630 e = e.format(source, content, '\n'.join(input_lines),
633 e = e.format(source, content, '\n'.join(input_lines),
631 repr(found), TAB=TAB)
634 repr(found), TAB=TAB)
632 raise RuntimeError(e)
635 raise RuntimeError(e)
633 found = found[len(output_prompt):].strip()
636 found = found[len(output_prompt):].strip()
634
637
635 # Handle the actual doctest comparison.
638 # Handle the actual doctest comparison.
636 if decorator.strip() == '@doctest':
639 if decorator.strip() == '@doctest':
637 # Standard doctest
640 # Standard doctest
638 if found != submitted:
641 if found != submitted:
639 e = ('doctest failure\n\n'
642 e = ('doctest failure\n\n'
640 'Document source: {0}\n\n'
643 'Document source: {0}\n\n'
641 'Raw content: \n{1}\n\n'
644 'Raw content: \n{1}\n\n'
642 'On input line(s):\n{TAB}{2}\n\n'
645 'On input line(s):\n{TAB}{2}\n\n'
643 'we found output:\n{TAB}{3}\n\n'
646 'we found output:\n{TAB}{3}\n\n'
644 'instead of the expected:\n{TAB}{4}\n\n')
647 'instead of the expected:\n{TAB}{4}\n\n')
645 e = e.format(source, content, '\n'.join(input_lines),
648 e = e.format(source, content, '\n'.join(input_lines),
646 repr(found), repr(submitted), TAB=TAB)
649 repr(found), repr(submitted), TAB=TAB)
647 raise RuntimeError(e)
650 raise RuntimeError(e)
648 else:
651 else:
649 self.custom_doctest(decorator, input_lines, found, submitted)
652 self.custom_doctest(decorator, input_lines, found, submitted)
650
653
651 # When in verbatim mode, this holds additional submitted output
654 # When in verbatim mode, this holds additional submitted output
652 # to be written in the final Sphinx output.
655 # to be written in the final Sphinx output.
653 # https://github.com/ipython/ipython/issues/5776
656 # https://github.com/ipython/ipython/issues/5776
654 out_data = []
657 out_data = []
655
658
656 is_verbatim = decorator=='@verbatim' or self.is_verbatim
659 is_verbatim = decorator=='@verbatim' or self.is_verbatim
657 if is_verbatim and data.strip():
660 if is_verbatim and data.strip():
658 # Note that `ret` in `process_block` has '' as its last element if
661 # Note that `ret` in `process_block` has '' as its last element if
659 # the code block was in verbatim mode. So if there is no submitted
662 # the code block was in verbatim mode. So if there is no submitted
660 # output, then we will have proper spacing only if we do not add
663 # output, then we will have proper spacing only if we do not add
661 # an additional '' to `out_data`. This is why we condition on
664 # an additional '' to `out_data`. This is why we condition on
662 # `and data.strip()`.
665 # `and data.strip()`.
663
666
664 # The submitted output has no output prompt. If we want the
667 # The submitted output has no output prompt. If we want the
665 # prompt and the code to appear, we need to join them now
668 # prompt and the code to appear, we need to join them now
666 # instead of adding them separately---as this would create an
669 # instead of adding them separately---as this would create an
667 # undesired newline. How we do this ultimately depends on the
670 # undesired newline. How we do this ultimately depends on the
668 # format of the output regex. I'll do what works for the default
671 # format of the output regex. I'll do what works for the default
669 # prompt for now, and we might have to adjust if it doesn't work
672 # prompt for now, and we might have to adjust if it doesn't work
670 # in other cases. Finally, the submitted output does not have
673 # in other cases. Finally, the submitted output does not have
671 # a trailing newline, so we must add it manually.
674 # a trailing newline, so we must add it manually.
672 out_data.append("{0} {1}\n".format(output_prompt, data))
675 out_data.append("{0} {1}\n".format(output_prompt, data))
673
676
674 return out_data
677 return out_data
675
678
676 def process_comment(self, data):
679 def process_comment(self, data):
677 """Process data fPblock for COMMENT token."""
680 """Process data fPblock for COMMENT token."""
678 if not self.is_suppress:
681 if not self.is_suppress:
679 return [data]
682 return [data]
680
683
681 def save_image(self, image_file):
684 def save_image(self, image_file):
682 """
685 """
683 Saves the image file to disk.
686 Saves the image file to disk.
684 """
687 """
685 self.ensure_pyplot()
688 self.ensure_pyplot()
686 command = 'plt.gcf().savefig("%s")'%image_file
689 command = 'plt.gcf().savefig("%s")'%image_file
687 #print 'SAVEFIG', command # dbg
690 #print 'SAVEFIG', command # dbg
688 self.process_input_line('bookmark ipy_thisdir', store_history=False)
691 self.process_input_line('bookmark ipy_thisdir', store_history=False)
689 self.process_input_line('cd -b ipy_savedir', store_history=False)
692 self.process_input_line('cd -b ipy_savedir', store_history=False)
690 self.process_input_line(command, store_history=False)
693 self.process_input_line(command, store_history=False)
691 self.process_input_line('cd -b ipy_thisdir', store_history=False)
694 self.process_input_line('cd -b ipy_thisdir', store_history=False)
692 self.process_input_line('bookmark -d ipy_thisdir', store_history=False)
695 self.process_input_line('bookmark -d ipy_thisdir', store_history=False)
693 self.clear_cout()
696 self.clear_cout()
694
697
695 def process_block(self, block):
698 def process_block(self, block):
696 """
699 """
697 process block from the block_parser and return a list of processed lines
700 process block from the block_parser and return a list of processed lines
698 """
701 """
699 ret = []
702 ret = []
700 output = None
703 output = None
701 input_lines = None
704 input_lines = None
702 lineno = self.IP.execution_count
705 lineno = self.IP.execution_count
703
706
704 input_prompt = self.promptin % lineno
707 input_prompt = self.promptin % lineno
705 output_prompt = self.promptout % lineno
708 output_prompt = self.promptout % lineno
706 image_file = None
709 image_file = None
707 image_directive = None
710 image_directive = None
708
711
709 found_input = False
712 found_input = False
710 for token, data in block:
713 for token, data in block:
711 if token == COMMENT:
714 if token == COMMENT:
712 out_data = self.process_comment(data)
715 out_data = self.process_comment(data)
713 elif token == INPUT:
716 elif token == INPUT:
714 found_input = True
717 found_input = True
715 (out_data, input_lines, output, is_doctest,
718 (out_data, input_lines, output, is_doctest,
716 decorator, image_file, image_directive) = \
719 decorator, image_file, image_directive) = \
717 self.process_input(data, input_prompt, lineno)
720 self.process_input(data, input_prompt, lineno)
718 elif token == OUTPUT:
721 elif token == OUTPUT:
719 if not found_input:
722 if not found_input:
720
723
721 TAB = ' ' * 4
724 TAB = ' ' * 4
722 linenumber = 0
725 linenumber = 0
723 source = 'Unavailable'
726 source = 'Unavailable'
724 content = 'Unavailable'
727 content = 'Unavailable'
725 if self.directive:
728 if self.directive:
726 linenumber = self.directive.state.document.current_line
729 linenumber = self.directive.state.document.current_line
727 source = self.directive.state.document.current_source
730 source = self.directive.state.document.current_source
728 content = self.directive.content
731 content = self.directive.content
729 # Add tabs and join into a single string.
732 # Add tabs and join into a single string.
730 content = '\n'.join([TAB + line for line in content])
733 content = '\n'.join([TAB + line for line in content])
731
734
732 e = ('\n\nInvalid block: Block contains an output prompt '
735 e = ('\n\nInvalid block: Block contains an output prompt '
733 'without an input prompt.\n\n'
736 'without an input prompt.\n\n'
734 'Document source: {0}\n\n'
737 'Document source: {0}\n\n'
735 'Content begins at line {1}: \n\n{2}\n\n'
738 'Content begins at line {1}: \n\n{2}\n\n'
736 'Problematic block within content: \n\n{TAB}{3}\n\n')
739 'Problematic block within content: \n\n{TAB}{3}\n\n')
737 e = e.format(source, linenumber, content, block, TAB=TAB)
740 e = e.format(source, linenumber, content, block, TAB=TAB)
738
741
739 # Write, rather than include in exception, since Sphinx
742 # Write, rather than include in exception, since Sphinx
740 # will truncate tracebacks.
743 # will truncate tracebacks.
741 sys.stdout.write(e)
744 sys.stdout.write(e)
742 raise RuntimeError('An invalid block was detected.')
745 raise RuntimeError('An invalid block was detected.')
743 out_data = \
746 out_data = \
744 self.process_output(data, output_prompt, input_lines,
747 self.process_output(data, output_prompt, input_lines,
745 output, is_doctest, decorator,
748 output, is_doctest, decorator,
746 image_file)
749 image_file)
747 if out_data:
750 if out_data:
748 # Then there was user submitted output in verbatim mode.
751 # Then there was user submitted output in verbatim mode.
749 # We need to remove the last element of `ret` that was
752 # We need to remove the last element of `ret` that was
750 # added in `process_input`, as it is '' and would introduce
753 # added in `process_input`, as it is '' and would introduce
751 # an undesirable newline.
754 # an undesirable newline.
752 assert(ret[-1] == '')
755 assert(ret[-1] == '')
753 del ret[-1]
756 del ret[-1]
754
757
755 if out_data:
758 if out_data:
756 ret.extend(out_data)
759 ret.extend(out_data)
757
760
758 # save the image files
761 # save the image files
759 if image_file is not None:
762 if image_file is not None:
760 self.save_image(image_file)
763 self.save_image(image_file)
761
764
762 return ret, image_directive
765 return ret, image_directive
763
766
764 def ensure_pyplot(self):
767 def ensure_pyplot(self):
765 """
768 """
766 Ensures that pyplot has been imported into the embedded IPython shell.
769 Ensures that pyplot has been imported into the embedded IPython shell.
767
770
768 Also, makes sure to set the backend appropriately if not set already.
771 Also, makes sure to set the backend appropriately if not set already.
769
772
770 """
773 """
771 # We are here if the @figure pseudo decorator was used. Thus, it's
774 # We are here if the @figure pseudo decorator was used. Thus, it's
772 # possible that we could be here even if python_mplbackend were set to
775 # possible that we could be here even if python_mplbackend were set to
773 # `None`. That's also strange and perhaps worthy of raising an
776 # `None`. That's also strange and perhaps worthy of raising an
774 # exception, but for now, we just set the backend to 'agg'.
777 # exception, but for now, we just set the backend to 'agg'.
775
778
776 if not self._pyplot_imported:
779 if not self._pyplot_imported:
777 if 'matplotlib.backends' not in sys.modules:
780 if 'matplotlib.backends' not in sys.modules:
778 # Then ipython_matplotlib was set to None but there was a
781 # Then ipython_matplotlib was set to None but there was a
779 # call to the @figure decorator (and ipython_execlines did
782 # call to the @figure decorator (and ipython_execlines did
780 # not set a backend).
783 # not set a backend).
781 #raise Exception("No backend was set, but @figure was used!")
784 #raise Exception("No backend was set, but @figure was used!")
782 import matplotlib
785 import matplotlib
783 matplotlib.use('agg')
786 matplotlib.use('agg')
784
787
785 # Always import pyplot into embedded shell.
788 # Always import pyplot into embedded shell.
786 self.process_input_line('import matplotlib.pyplot as plt',
789 self.process_input_line('import matplotlib.pyplot as plt',
787 store_history=False)
790 store_history=False)
788 self._pyplot_imported = True
791 self._pyplot_imported = True
789
792
790 def process_pure_python(self, content):
793 def process_pure_python(self, content):
791 """
794 """
792 content is a list of strings. it is unedited directive content
795 content is a list of strings. it is unedited directive content
793
796
794 This runs it line by line in the InteractiveShell, prepends
797 This runs it line by line in the InteractiveShell, prepends
795 prompts as needed capturing stderr and stdout, then returns
798 prompts as needed capturing stderr and stdout, then returns
796 the content as a list as if it were ipython code
799 the content as a list as if it were ipython code
797 """
800 """
798 output = []
801 output = []
799 savefig = False # keep up with this to clear figure
802 savefig = False # keep up with this to clear figure
800 multiline = False # to handle line continuation
803 multiline = False # to handle line continuation
801 multiline_start = None
804 multiline_start = None
802 fmtin = self.promptin
805 fmtin = self.promptin
803
806
804 ct = 0
807 ct = 0
805
808
806 for lineno, line in enumerate(content):
809 for lineno, line in enumerate(content):
807
810
808 line_stripped = line.strip()
811 line_stripped = line.strip()
809 if not len(line):
812 if not len(line):
810 output.append(line)
813 output.append(line)
811 continue
814 continue
812
815
813 # handle decorators
816 # handle decorators
814 if line_stripped.startswith('@'):
817 if line_stripped.startswith('@'):
815 output.extend([line])
818 output.extend([line])
816 if 'savefig' in line:
819 if 'savefig' in line:
817 savefig = True # and need to clear figure
820 savefig = True # and need to clear figure
818 continue
821 continue
819
822
820 # handle comments
823 # handle comments
821 if line_stripped.startswith('#'):
824 if line_stripped.startswith('#'):
822 output.extend([line])
825 output.extend([line])
823 continue
826 continue
824
827
825 # deal with lines checking for multiline
828 # deal with lines checking for multiline
826 continuation = u' %s:'% ''.join(['.']*(len(str(ct))+2))
829 continuation = u' %s:'% ''.join(['.']*(len(str(ct))+2))
827 if not multiline:
830 if not multiline:
828 modified = u"%s %s" % (fmtin % ct, line_stripped)
831 modified = u"%s %s" % (fmtin % ct, line_stripped)
829 output.append(modified)
832 output.append(modified)
830 ct += 1
833 ct += 1
831 try:
834 try:
832 ast.parse(line_stripped)
835 ast.parse(line_stripped)
833 output.append(u'')
836 output.append(u'')
834 except Exception: # on a multiline
837 except Exception: # on a multiline
835 multiline = True
838 multiline = True
836 multiline_start = lineno
839 multiline_start = lineno
837 else: # still on a multiline
840 else: # still on a multiline
838 modified = u'%s %s' % (continuation, line)
841 modified = u'%s %s' % (continuation, line)
839 output.append(modified)
842 output.append(modified)
840
843
841 # if the next line is indented, it should be part of multiline
844 # if the next line is indented, it should be part of multiline
842 if len(content) > lineno + 1:
845 if len(content) > lineno + 1:
843 nextline = content[lineno + 1]
846 nextline = content[lineno + 1]
844 if len(nextline) - len(nextline.lstrip()) > 3:
847 if len(nextline) - len(nextline.lstrip()) > 3:
845 continue
848 continue
846 try:
849 try:
847 mod = ast.parse(
850 mod = ast.parse(
848 '\n'.join(content[multiline_start:lineno+1]))
851 '\n'.join(content[multiline_start:lineno+1]))
849 if isinstance(mod.body[0], ast.FunctionDef):
852 if isinstance(mod.body[0], ast.FunctionDef):
850 # check to see if we have the whole function
853 # check to see if we have the whole function
851 for element in mod.body[0].body:
854 for element in mod.body[0].body:
852 if isinstance(element, ast.Return):
855 if isinstance(element, ast.Return):
853 multiline = False
856 multiline = False
854 else:
857 else:
855 output.append(u'')
858 output.append(u'')
856 multiline = False
859 multiline = False
857 except Exception:
860 except Exception:
858 pass
861 pass
859
862
860 if savefig: # clear figure if plotted
863 if savefig: # clear figure if plotted
861 self.ensure_pyplot()
864 self.ensure_pyplot()
862 self.process_input_line('plt.clf()', store_history=False)
865 self.process_input_line('plt.clf()', store_history=False)
863 self.clear_cout()
866 self.clear_cout()
864 savefig = False
867 savefig = False
865
868
866 return output
869 return output
867
870
868 def custom_doctest(self, decorator, input_lines, found, submitted):
871 def custom_doctest(self, decorator, input_lines, found, submitted):
869 """
872 """
870 Perform a specialized doctest.
873 Perform a specialized doctest.
871
874
872 """
875 """
873 from .custom_doctests import doctests
876 from .custom_doctests import doctests
874
877
875 args = decorator.split()
878 args = decorator.split()
876 doctest_type = args[1]
879 doctest_type = args[1]
877 if doctest_type in doctests:
880 if doctest_type in doctests:
878 doctests[doctest_type](self, args, input_lines, found, submitted)
881 doctests[doctest_type](self, args, input_lines, found, submitted)
879 else:
882 else:
880 e = "Invalid option to @doctest: {0}".format(doctest_type)
883 e = "Invalid option to @doctest: {0}".format(doctest_type)
881 raise Exception(e)
884 raise Exception(e)
882
885
883
886
884 class IPythonDirective(Directive):
887 class IPythonDirective(Directive):
885
888
886 has_content = True
889 has_content = True
887 required_arguments = 0
890 required_arguments = 0
888 optional_arguments = 4 # python, suppress, verbatim, doctest
891 optional_arguments = 4 # python, suppress, verbatim, doctest
889 final_argumuent_whitespace = True
892 final_argumuent_whitespace = True
890 option_spec = { 'python': directives.unchanged,
893 option_spec = { 'python': directives.unchanged,
891 'suppress' : directives.flag,
894 'suppress' : directives.flag,
892 'verbatim' : directives.flag,
895 'verbatim' : directives.flag,
893 'doctest' : directives.flag,
896 'doctest' : directives.flag,
894 'okexcept': directives.flag,
897 'okexcept': directives.flag,
895 'okwarning': directives.flag
898 'okwarning': directives.flag
896 }
899 }
897
900
898 shell = None
901 shell = None
899
902
900 seen_docs = set()
903 seen_docs = set()
901
904
902 def get_config_options(self):
905 def get_config_options(self):
903 # contains sphinx configuration variables
906 # contains sphinx configuration variables
904 config = self.state.document.settings.env.config
907 config = self.state.document.settings.env.config
905
908
906 # get config variables to set figure output directory
909 # get config variables to set figure output directory
907 savefig_dir = config.ipython_savefig_dir
910 savefig_dir = config.ipython_savefig_dir
908 source_dir = self.state.document.settings.env.srcdir
911 source_dir = self.state.document.settings.env.srcdir
909 savefig_dir = os.path.join(source_dir, savefig_dir)
912 savefig_dir = os.path.join(source_dir, savefig_dir)
910
913
911 # get regex and prompt stuff
914 # get regex and prompt stuff
912 rgxin = config.ipython_rgxin
915 rgxin = config.ipython_rgxin
913 rgxout = config.ipython_rgxout
916 rgxout = config.ipython_rgxout
914 warning_is_error= config.ipython_warning_is_error
917 warning_is_error= config.ipython_warning_is_error
915 promptin = config.ipython_promptin
918 promptin = config.ipython_promptin
916 promptout = config.ipython_promptout
919 promptout = config.ipython_promptout
917 mplbackend = config.ipython_mplbackend
920 mplbackend = config.ipython_mplbackend
918 exec_lines = config.ipython_execlines
921 exec_lines = config.ipython_execlines
919 hold_count = config.ipython_holdcount
922 hold_count = config.ipython_holdcount
920
923
921 return (savefig_dir, source_dir, rgxin, rgxout,
924 return (savefig_dir, source_dir, rgxin, rgxout,
922 promptin, promptout, mplbackend, exec_lines, hold_count, warning_is_error)
925 promptin, promptout, mplbackend, exec_lines, hold_count, warning_is_error)
923
926
924 def setup(self):
927 def setup(self):
925 # Get configuration values.
928 # Get configuration values.
926 (savefig_dir, source_dir, rgxin, rgxout, promptin, promptout,
929 (savefig_dir, source_dir, rgxin, rgxout, promptin, promptout,
927 mplbackend, exec_lines, hold_count, warning_is_error) = self.get_config_options()
930 mplbackend, exec_lines, hold_count, warning_is_error) = self.get_config_options()
928
931
929 try:
932 try:
930 os.makedirs(savefig_dir)
933 os.makedirs(savefig_dir)
931 except OSError as e:
934 except OSError as e:
932 if e.errno != errno.EEXIST:
935 if e.errno != errno.EEXIST:
933 raise
936 raise
934
937
935 if self.shell is None:
938 if self.shell is None:
936 # We will be here many times. However, when the
939 # We will be here many times. However, when the
937 # EmbeddedSphinxShell is created, its interactive shell member
940 # EmbeddedSphinxShell is created, its interactive shell member
938 # is the same for each instance.
941 # is the same for each instance.
939
942
940 if mplbackend and 'matplotlib.backends' not in sys.modules and use_matplotlib:
943 if mplbackend and 'matplotlib.backends' not in sys.modules and use_matplotlib:
941 import matplotlib
944 import matplotlib
942 matplotlib.use(mplbackend)
945 matplotlib.use(mplbackend)
943
946
944 # Must be called after (potentially) importing matplotlib and
947 # Must be called after (potentially) importing matplotlib and
945 # setting its backend since exec_lines might import pylab.
948 # setting its backend since exec_lines might import pylab.
946 self.shell = EmbeddedSphinxShell(exec_lines)
949 self.shell = EmbeddedSphinxShell(exec_lines)
947
950
948 # Store IPython directive to enable better error messages
951 # Store IPython directive to enable better error messages
949 self.shell.directive = self
952 self.shell.directive = self
950
953
951 # reset the execution count if we haven't processed this doc
954 # reset the execution count if we haven't processed this doc
952 #NOTE: this may be borked if there are multiple seen_doc tmp files
955 #NOTE: this may be borked if there are multiple seen_doc tmp files
953 #check time stamp?
956 #check time stamp?
954 if not self.state.document.current_source in self.seen_docs:
957 if not self.state.document.current_source in self.seen_docs:
955 self.shell.IP.history_manager.reset()
958 self.shell.IP.history_manager.reset()
956 self.shell.IP.execution_count = 1
959 self.shell.IP.execution_count = 1
957 self.seen_docs.add(self.state.document.current_source)
960 self.seen_docs.add(self.state.document.current_source)
958
961
959 # and attach to shell so we don't have to pass them around
962 # and attach to shell so we don't have to pass them around
960 self.shell.rgxin = rgxin
963 self.shell.rgxin = rgxin
961 self.shell.rgxout = rgxout
964 self.shell.rgxout = rgxout
962 self.shell.promptin = promptin
965 self.shell.promptin = promptin
963 self.shell.promptout = promptout
966 self.shell.promptout = promptout
964 self.shell.savefig_dir = savefig_dir
967 self.shell.savefig_dir = savefig_dir
965 self.shell.source_dir = source_dir
968 self.shell.source_dir = source_dir
966 self.shell.hold_count = hold_count
969 self.shell.hold_count = hold_count
967 self.shell.warning_is_error = warning_is_error
970 self.shell.warning_is_error = warning_is_error
968
971
969 # setup bookmark for saving figures directory
972 # setup bookmark for saving figures directory
970 self.shell.process_input_line('bookmark ipy_savedir %s'%savefig_dir,
973 self.shell.process_input_line('bookmark ipy_savedir %s'%savefig_dir,
971 store_history=False)
974 store_history=False)
972 self.shell.clear_cout()
975 self.shell.clear_cout()
973
976
974 return rgxin, rgxout, promptin, promptout
977 return rgxin, rgxout, promptin, promptout
975
978
976 def teardown(self):
979 def teardown(self):
977 # delete last bookmark
980 # delete last bookmark
978 self.shell.process_input_line('bookmark -d ipy_savedir',
981 self.shell.process_input_line('bookmark -d ipy_savedir',
979 store_history=False)
982 store_history=False)
980 self.shell.clear_cout()
983 self.shell.clear_cout()
981
984
982 def run(self):
985 def run(self):
983 debug = False
986 debug = False
984
987
985 #TODO, any reason block_parser can't be a method of embeddable shell
988 #TODO, any reason block_parser can't be a method of embeddable shell
986 # then we wouldn't have to carry these around
989 # then we wouldn't have to carry these around
987 rgxin, rgxout, promptin, promptout = self.setup()
990 rgxin, rgxout, promptin, promptout = self.setup()
988
991
989 options = self.options
992 options = self.options
990 self.shell.is_suppress = 'suppress' in options
993 self.shell.is_suppress = 'suppress' in options
991 self.shell.is_doctest = 'doctest' in options
994 self.shell.is_doctest = 'doctest' in options
992 self.shell.is_verbatim = 'verbatim' in options
995 self.shell.is_verbatim = 'verbatim' in options
993 self.shell.is_okexcept = 'okexcept' in options
996 self.shell.is_okexcept = 'okexcept' in options
994 self.shell.is_okwarning = 'okwarning' in options
997 self.shell.is_okwarning = 'okwarning' in options
995
998
996 # handle pure python code
999 # handle pure python code
997 if 'python' in self.arguments:
1000 if 'python' in self.arguments:
998 content = self.content
1001 content = self.content
999 self.content = self.shell.process_pure_python(content)
1002 self.content = self.shell.process_pure_python(content)
1000
1003
1001 # parts consists of all text within the ipython-block.
1004 # parts consists of all text within the ipython-block.
1002 # Each part is an input/output block.
1005 # Each part is an input/output block.
1003 parts = '\n'.join(self.content).split('\n\n')
1006 parts = '\n'.join(self.content).split('\n\n')
1004
1007
1005 lines = ['.. code-block:: ipython', '']
1008 lines = ['.. code-block:: ipython', '']
1006 figures = []
1009 figures = []
1007
1010
1008 # Use sphinx logger for warnings
1011 # Use sphinx logger for warnings
1009 logger = logging.getLogger(__name__)
1012 logger = logging.getLogger(__name__)
1010
1013
1011 for part in parts:
1014 for part in parts:
1012 block = block_parser(part, rgxin, rgxout, promptin, promptout)
1015 block = block_parser(part, rgxin, rgxout, promptin, promptout)
1013 if len(block):
1016 if len(block):
1014 rows, figure = self.shell.process_block(block)
1017 rows, figure = self.shell.process_block(block)
1015 for row in rows:
1018 for row in rows:
1016 lines.extend([' {0}'.format(line)
1019 lines.extend([' {0}'.format(line)
1017 for line in row.split('\n')])
1020 for line in row.split('\n')])
1018
1021
1019 if figure is not None:
1022 if figure is not None:
1020 figures.append(figure)
1023 figures.append(figure)
1021 else:
1024 else:
1022 message = 'Code input with no code at {}, line {}'\
1025 message = 'Code input with no code at {}, line {}'\
1023 .format(
1026 .format(
1024 self.state.document.current_source,
1027 self.state.document.current_source,
1025 self.state.document.current_line)
1028 self.state.document.current_line)
1026 if self.shell.warning_is_error:
1029 if self.shell.warning_is_error:
1027 raise RuntimeError(message)
1030 raise RuntimeError(message)
1028 else:
1031 else:
1029 logger.warning(message)
1032 logger.warning(message)
1030
1033
1031 for figure in figures:
1034 for figure in figures:
1032 lines.append('')
1035 lines.append('')
1033 lines.extend(figure.split('\n'))
1036 lines.extend(figure.split('\n'))
1034 lines.append('')
1037 lines.append('')
1035
1038
1036 if len(lines) > 2:
1039 if len(lines) > 2:
1037 if debug:
1040 if debug:
1038 print('\n'.join(lines))
1041 print('\n'.join(lines))
1039 else:
1042 else:
1040 # This has to do with input, not output. But if we comment
1043 # This has to do with input, not output. But if we comment
1041 # these lines out, then no IPython code will appear in the
1044 # these lines out, then no IPython code will appear in the
1042 # final output.
1045 # final output.
1043 self.state_machine.insert_input(
1046 self.state_machine.insert_input(
1044 lines, self.state_machine.input_lines.source(0))
1047 lines, self.state_machine.input_lines.source(0))
1045
1048
1046 # cleanup
1049 # cleanup
1047 self.teardown()
1050 self.teardown()
1048
1051
1049 return []
1052 return []
1050
1053
1051 # Enable as a proper Sphinx directive
1054 # Enable as a proper Sphinx directive
1052 def setup(app):
1055 def setup(app):
1053 setup.app = app
1056 setup.app = app
1054
1057
1055 app.add_directive('ipython', IPythonDirective)
1058 app.add_directive('ipython', IPythonDirective)
1056 app.add_config_value('ipython_savefig_dir', 'savefig', 'env')
1059 app.add_config_value('ipython_savefig_dir', 'savefig', 'env')
1057 app.add_config_value('ipython_warning_is_error', True, 'env')
1060 app.add_config_value('ipython_warning_is_error', True, 'env')
1058 app.add_config_value('ipython_rgxin',
1061 app.add_config_value('ipython_rgxin',
1059 re.compile(r'In \[(\d+)\]:\s?(.*)\s*'), 'env')
1062 re.compile(r'In \[(\d+)\]:\s?(.*)\s*'), 'env')
1060 app.add_config_value('ipython_rgxout',
1063 app.add_config_value('ipython_rgxout',
1061 re.compile(r'Out\[(\d+)\]:\s?(.*)\s*'), 'env')
1064 re.compile(r'Out\[(\d+)\]:\s?(.*)\s*'), 'env')
1062 app.add_config_value('ipython_promptin', 'In [%d]:', 'env')
1065 app.add_config_value('ipython_promptin', 'In [%d]:', 'env')
1063 app.add_config_value('ipython_promptout', 'Out[%d]:', 'env')
1066 app.add_config_value('ipython_promptout', 'Out[%d]:', 'env')
1064
1067
1065 # We could just let matplotlib pick whatever is specified as the default
1068 # We could just let matplotlib pick whatever is specified as the default
1066 # backend in the matplotlibrc file, but this would cause issues if the
1069 # backend in the matplotlibrc file, but this would cause issues if the
1067 # backend didn't work in headless environments. For this reason, 'agg'
1070 # backend didn't work in headless environments. For this reason, 'agg'
1068 # is a good default backend choice.
1071 # is a good default backend choice.
1069 app.add_config_value('ipython_mplbackend', 'agg', 'env')
1072 app.add_config_value('ipython_mplbackend', 'agg', 'env')
1070
1073
1071 # If the user sets this config value to `None`, then EmbeddedSphinxShell's
1074 # If the user sets this config value to `None`, then EmbeddedSphinxShell's
1072 # __init__ method will treat it as [].
1075 # __init__ method will treat it as [].
1073 execlines = ['import numpy as np']
1076 execlines = ['import numpy as np']
1074 if use_matplotlib:
1077 if use_matplotlib:
1075 execlines.append('import matplotlib.pyplot as plt')
1078 execlines.append('import matplotlib.pyplot as plt')
1076 app.add_config_value('ipython_execlines', execlines, 'env')
1079 app.add_config_value('ipython_execlines', execlines, 'env')
1077
1080
1078 app.add_config_value('ipython_holdcount', True, 'env')
1081 app.add_config_value('ipython_holdcount', True, 'env')
1079
1082
1080 metadata = {'parallel_read_safe': True, 'parallel_write_safe': True}
1083 metadata = {'parallel_read_safe': True, 'parallel_write_safe': True}
1081 return metadata
1084 return metadata
1082
1085
1083 # Simple smoke test, needs to be converted to a proper automatic test.
1086 # Simple smoke test, needs to be converted to a proper automatic test.
1084 def test():
1087 def test():
1085
1088
1086 examples = [
1089 examples = [
1087 r"""
1090 r"""
1088 In [9]: pwd
1091 In [9]: pwd
1089 Out[9]: '/home/jdhunter/py4science/book'
1092 Out[9]: '/home/jdhunter/py4science/book'
1090
1093
1091 In [10]: cd bookdata/
1094 In [10]: cd bookdata/
1092 /home/jdhunter/py4science/book/bookdata
1095 /home/jdhunter/py4science/book/bookdata
1093
1096
1094 In [2]: from pylab import *
1097 In [2]: from pylab import *
1095
1098
1096 In [2]: ion()
1099 In [2]: ion()
1097
1100
1098 In [3]: im = imread('stinkbug.png')
1101 In [3]: im = imread('stinkbug.png')
1099
1102
1100 @savefig mystinkbug.png width=4in
1103 @savefig mystinkbug.png width=4in
1101 In [4]: imshow(im)
1104 In [4]: imshow(im)
1102 Out[4]: <matplotlib.image.AxesImage object at 0x39ea850>
1105 Out[4]: <matplotlib.image.AxesImage object at 0x39ea850>
1103
1106
1104 """,
1107 """,
1105 r"""
1108 r"""
1106
1109
1107 In [1]: x = 'hello world'
1110 In [1]: x = 'hello world'
1108
1111
1109 # string methods can be
1112 # string methods can be
1110 # used to alter the string
1113 # used to alter the string
1111 @doctest
1114 @doctest
1112 In [2]: x.upper()
1115 In [2]: x.upper()
1113 Out[2]: 'HELLO WORLD'
1116 Out[2]: 'HELLO WORLD'
1114
1117
1115 @verbatim
1118 @verbatim
1116 In [3]: x.st<TAB>
1119 In [3]: x.st<TAB>
1117 x.startswith x.strip
1120 x.startswith x.strip
1118 """,
1121 """,
1119 r"""
1122 r"""
1120
1123
1121 In [130]: url = 'http://ichart.finance.yahoo.com/table.csv?s=CROX\
1124 In [130]: url = 'http://ichart.finance.yahoo.com/table.csv?s=CROX\
1122 .....: &d=9&e=22&f=2009&g=d&a=1&br=8&c=2006&ignore=.csv'
1125 .....: &d=9&e=22&f=2009&g=d&a=1&br=8&c=2006&ignore=.csv'
1123
1126
1124 In [131]: print url.split('&')
1127 In [131]: print url.split('&')
1125 ['http://ichart.finance.yahoo.com/table.csv?s=CROX', 'd=9', 'e=22', 'f=2009', 'g=d', 'a=1', 'b=8', 'c=2006', 'ignore=.csv']
1128 ['http://ichart.finance.yahoo.com/table.csv?s=CROX', 'd=9', 'e=22', 'f=2009', 'g=d', 'a=1', 'b=8', 'c=2006', 'ignore=.csv']
1126
1129
1127 In [60]: import urllib
1130 In [60]: import urllib
1128
1131
1129 """,
1132 """,
1130 r"""\
1133 r"""\
1131
1134
1132 In [133]: import numpy.random
1135 In [133]: import numpy.random
1133
1136
1134 @suppress
1137 @suppress
1135 In [134]: numpy.random.seed(2358)
1138 In [134]: numpy.random.seed(2358)
1136
1139
1137 @doctest
1140 @doctest
1138 In [135]: numpy.random.rand(10,2)
1141 In [135]: numpy.random.rand(10,2)
1139 Out[135]:
1142 Out[135]:
1140 array([[ 0.64524308, 0.59943846],
1143 array([[ 0.64524308, 0.59943846],
1141 [ 0.47102322, 0.8715456 ],
1144 [ 0.47102322, 0.8715456 ],
1142 [ 0.29370834, 0.74776844],
1145 [ 0.29370834, 0.74776844],
1143 [ 0.99539577, 0.1313423 ],
1146 [ 0.99539577, 0.1313423 ],
1144 [ 0.16250302, 0.21103583],
1147 [ 0.16250302, 0.21103583],
1145 [ 0.81626524, 0.1312433 ],
1148 [ 0.81626524, 0.1312433 ],
1146 [ 0.67338089, 0.72302393],
1149 [ 0.67338089, 0.72302393],
1147 [ 0.7566368 , 0.07033696],
1150 [ 0.7566368 , 0.07033696],
1148 [ 0.22591016, 0.77731835],
1151 [ 0.22591016, 0.77731835],
1149 [ 0.0072729 , 0.34273127]])
1152 [ 0.0072729 , 0.34273127]])
1150
1153
1151 """,
1154 """,
1152
1155
1153 r"""
1156 r"""
1154 In [106]: print x
1157 In [106]: print x
1155 jdh
1158 jdh
1156
1159
1157 In [109]: for i in range(10):
1160 In [109]: for i in range(10):
1158 .....: print i
1161 .....: print i
1159 .....:
1162 .....:
1160 .....:
1163 .....:
1161 0
1164 0
1162 1
1165 1
1163 2
1166 2
1164 3
1167 3
1165 4
1168 4
1166 5
1169 5
1167 6
1170 6
1168 7
1171 7
1169 8
1172 8
1170 9
1173 9
1171 """,
1174 """,
1172
1175
1173 r"""
1176 r"""
1174
1177
1175 In [144]: from pylab import *
1178 In [144]: from pylab import *
1176
1179
1177 In [145]: ion()
1180 In [145]: ion()
1178
1181
1179 # use a semicolon to suppress the output
1182 # use a semicolon to suppress the output
1180 @savefig test_hist.png width=4in
1183 @savefig test_hist.png width=4in
1181 In [151]: hist(np.random.randn(10000), 100);
1184 In [151]: hist(np.random.randn(10000), 100);
1182
1185
1183
1186
1184 @savefig test_plot.png width=4in
1187 @savefig test_plot.png width=4in
1185 In [151]: plot(np.random.randn(10000), 'o');
1188 In [151]: plot(np.random.randn(10000), 'o');
1186 """,
1189 """,
1187
1190
1188 r"""
1191 r"""
1189 # use a semicolon to suppress the output
1192 # use a semicolon to suppress the output
1190 In [151]: plt.clf()
1193 In [151]: plt.clf()
1191
1194
1192 @savefig plot_simple.png width=4in
1195 @savefig plot_simple.png width=4in
1193 In [151]: plot([1,2,3])
1196 In [151]: plot([1,2,3])
1194
1197
1195 @savefig hist_simple.png width=4in
1198 @savefig hist_simple.png width=4in
1196 In [151]: hist(np.random.randn(10000), 100);
1199 In [151]: hist(np.random.randn(10000), 100);
1197
1200
1198 """,
1201 """,
1199 r"""
1202 r"""
1200 # update the current fig
1203 # update the current fig
1201 In [151]: ylabel('number')
1204 In [151]: ylabel('number')
1202
1205
1203 In [152]: title('normal distribution')
1206 In [152]: title('normal distribution')
1204
1207
1205
1208
1206 @savefig hist_with_text.png
1209 @savefig hist_with_text.png
1207 In [153]: grid(True)
1210 In [153]: grid(True)
1208
1211
1209 @doctest float
1212 @doctest float
1210 In [154]: 0.1 + 0.2
1213 In [154]: 0.1 + 0.2
1211 Out[154]: 0.3
1214 Out[154]: 0.3
1212
1215
1213 @doctest float
1216 @doctest float
1214 In [155]: np.arange(16).reshape(4,4)
1217 In [155]: np.arange(16).reshape(4,4)
1215 Out[155]:
1218 Out[155]:
1216 array([[ 0, 1, 2, 3],
1219 array([[ 0, 1, 2, 3],
1217 [ 4, 5, 6, 7],
1220 [ 4, 5, 6, 7],
1218 [ 8, 9, 10, 11],
1221 [ 8, 9, 10, 11],
1219 [12, 13, 14, 15]])
1222 [12, 13, 14, 15]])
1220
1223
1221 In [1]: x = np.arange(16, dtype=float).reshape(4,4)
1224 In [1]: x = np.arange(16, dtype=float).reshape(4,4)
1222
1225
1223 In [2]: x[0,0] = np.inf
1226 In [2]: x[0,0] = np.inf
1224
1227
1225 In [3]: x[0,1] = np.nan
1228 In [3]: x[0,1] = np.nan
1226
1229
1227 @doctest float
1230 @doctest float
1228 In [4]: x
1231 In [4]: x
1229 Out[4]:
1232 Out[4]:
1230 array([[ inf, nan, 2., 3.],
1233 array([[ inf, nan, 2., 3.],
1231 [ 4., 5., 6., 7.],
1234 [ 4., 5., 6., 7.],
1232 [ 8., 9., 10., 11.],
1235 [ 8., 9., 10., 11.],
1233 [ 12., 13., 14., 15.]])
1236 [ 12., 13., 14., 15.]])
1234
1237
1235
1238
1236 """,
1239 """,
1237 ]
1240 ]
1238 # skip local-file depending first example:
1241 # skip local-file depending first example:
1239 examples = examples[1:]
1242 examples = examples[1:]
1240
1243
1241 #ipython_directive.DEBUG = True # dbg
1244 #ipython_directive.DEBUG = True # dbg
1242 #options = dict(suppress=True) # dbg
1245 #options = dict(suppress=True) # dbg
1243 options = {}
1246 options = {}
1244 for example in examples:
1247 for example in examples:
1245 content = example.split('\n')
1248 content = example.split('\n')
1246 IPythonDirective('debug', arguments=None, options=options,
1249 IPythonDirective('debug', arguments=None, options=options,
1247 content=content, lineno=0,
1250 content=content, lineno=0,
1248 content_offset=None, block_text=None,
1251 content_offset=None, block_text=None,
1249 state=None, state_machine=None,
1252 state=None, state_machine=None,
1250 )
1253 )
1251
1254
1252 # Run test suite as a script
1255 # Run test suite as a script
1253 if __name__=='__main__':
1256 if __name__=='__main__':
1254 if not os.path.isdir('_static'):
1257 if not os.path.isdir('_static'):
1255 os.mkdir('_static')
1258 os.mkdir('_static')
1256 test()
1259 test()
1257 print('All OK? Check figures in _static/')
1260 print('All OK? Check figures in _static/')
General Comments 0
You need to be logged in to leave comments. Login now