Show More
@@ -0,0 +1,20 b'' | |||||
|
1 | # Set this prefix to where you want to install the plugin | |||
|
2 | PREFIX=~/usr/local | |||
|
3 | PREFIX=~/tmp/local | |||
|
4 | ||||
|
5 | plugin: IPython_doctest_plugin.egg-info | |||
|
6 | ||||
|
7 | test: plugin dtexample.py | |||
|
8 | nosetests -s --with-ipdoctest --doctest-tests --doctest-extension=txt \ | |||
|
9 | dtexample.py test*.txt | |||
|
10 | ||||
|
11 | deb: plugin dtexample.py | |||
|
12 | nosetests -vs --with-ipdoctest --doctest-tests --doctest-extension=txt \ | |||
|
13 | test_combo.txt | |||
|
14 | ||||
|
15 | IPython_doctest_plugin.egg-info: ipdoctest.py setup.py | |||
|
16 | python setup.py install --prefix=$(PREFIX) | |||
|
17 | touch $@ | |||
|
18 | ||||
|
19 | clean: | |||
|
20 | rm -rf IPython_doctest_plugin.egg-info *~ *pyc build/ dist/ |
@@ -0,0 +1,39 b'' | |||||
|
1 | ======================================================= | |||
|
2 | Nose plugin with IPython and extension module support | |||
|
3 | ======================================================= | |||
|
4 | ||||
|
5 | This directory provides the key functionality for test support that IPython | |||
|
6 | needs as a nose plugin, which can be installed for use in projects other than | |||
|
7 | IPython. | |||
|
8 | ||||
|
9 | The presence of a Makefile here is mostly for development and debugging | |||
|
10 | purposes as it only provides a few shorthand commands. You can manually | |||
|
11 | install the plugin by using standard Python procedures (``setup.py install`` | |||
|
12 | with appropriate arguments). | |||
|
13 | ||||
|
14 | To install the plugin using the Makefile, edit its first line to reflect where | |||
|
15 | you'd like the installation. If you want it system-wide, you may want to edit | |||
|
16 | the install line in the plugin target to use sudo and no prefix:: | |||
|
17 | ||||
|
18 | sudo python setup.py install | |||
|
19 | ||||
|
20 | instead of the code using `--prefix` that's in there. | |||
|
21 | ||||
|
22 | Once you've set the prefix, simply build/install the plugin with:: | |||
|
23 | ||||
|
24 | make | |||
|
25 | ||||
|
26 | and run the tests with:: | |||
|
27 | ||||
|
28 | make test | |||
|
29 | ||||
|
30 | You should see output similar to:: | |||
|
31 | ||||
|
32 | maqroll[plugin]> make test | |||
|
33 | nosetests -s --with-ipdoctest --doctest-tests dtexample.py | |||
|
34 | .. | |||
|
35 | ---------------------------------------------------------------------- | |||
|
36 | Ran 2 tests in 0.016s | |||
|
37 | ||||
|
38 | OK | |||
|
39 |
@@ -0,0 +1,72 b'' | |||||
|
1 | """Simple example using doctests. | |||
|
2 | ||||
|
3 | This file just contains doctests both using plain python and IPython prompts. | |||
|
4 | All tests should be loaded by nose. | |||
|
5 | """ | |||
|
6 | ||||
|
7 | def pyfunc(): | |||
|
8 | """Some pure python tests... | |||
|
9 | ||||
|
10 | >>> pyfunc() | |||
|
11 | 'pyfunc' | |||
|
12 | ||||
|
13 | >>> import os | |||
|
14 | ||||
|
15 | >>> 2+3 | |||
|
16 | 5 | |||
|
17 | ||||
|
18 | >>> for i in range(3): | |||
|
19 | ... print i, | |||
|
20 | ... print i+1, | |||
|
21 | ... | |||
|
22 | 0 1 1 2 2 3 | |||
|
23 | """ | |||
|
24 | ||||
|
25 | return 'pyfunc' | |||
|
26 | ||||
|
27 | def ipfunc(): | |||
|
28 | """Some ipython tests... | |||
|
29 | ||||
|
30 | In [1]: import os | |||
|
31 | ||||
|
32 | In [2]: cd / | |||
|
33 | / | |||
|
34 | ||||
|
35 | In [3]: 2+3 | |||
|
36 | Out[3]: 5 | |||
|
37 | ||||
|
38 | In [26]: for i in range(3): | |||
|
39 | ....: print i, | |||
|
40 | ....: print i+1, | |||
|
41 | ....: | |||
|
42 | 0 1 1 2 2 3 | |||
|
43 | ||||
|
44 | ||||
|
45 | Examples that access the operating system work: | |||
|
46 | ||||
|
47 | In [1]: !echo hello | |||
|
48 | hello | |||
|
49 | ||||
|
50 | In [2]: !echo hello > /tmp/foo | |||
|
51 | ||||
|
52 | In [3]: !cat /tmp/foo | |||
|
53 | hello | |||
|
54 | ||||
|
55 | In [4]: rm -f /tmp/foo | |||
|
56 | ||||
|
57 | It's OK to use '_' for the last result, but do NOT try to use IPython's | |||
|
58 | numbered history of _NN outputs, since those won't exist under the | |||
|
59 | doctest environment: | |||
|
60 | ||||
|
61 | In [7]: 3+4 | |||
|
62 | Out[7]: 7 | |||
|
63 | ||||
|
64 | In [8]: _+3 | |||
|
65 | Out[8]: 10 | |||
|
66 | ||||
|
67 | In [9]: ipfunc() | |||
|
68 | Out[9]: 'ipfunc' | |||
|
69 | """ | |||
|
70 | ||||
|
71 | return 'ipfunc' | |||
|
72 |
This diff has been collapsed as it changes many lines, (587 lines changed) Show them Hide them | |||||
@@ -0,0 +1,587 b'' | |||||
|
1 | """Nose Plugin that supports IPython doctests. | |||
|
2 | ||||
|
3 | Limitations: | |||
|
4 | ||||
|
5 | - When generating examples for use as doctests, make sure that you have | |||
|
6 | pretty-printing OFF. This can be done either by starting ipython with the | |||
|
7 | flag '--nopprint', by setting pprint to 0 in your ipythonrc file, or by | |||
|
8 | interactively disabling it with %Pprint. This is required so that IPython | |||
|
9 | output matches that of normal Python, which is used by doctest for internal | |||
|
10 | execution. | |||
|
11 | ||||
|
12 | - Do not rely on specific prompt numbers for results (such as using | |||
|
13 | '_34==True', for example). For IPython tests run via an external process the | |||
|
14 | prompt numbers may be different, and IPython tests run as normal python code | |||
|
15 | won't even have these special _NN variables set at all. | |||
|
16 | ||||
|
17 | - IPython functions that produce output as a side-effect of calling a system | |||
|
18 | process (e.g. 'ls') can be doc-tested, but they must be handled in an | |||
|
19 | external IPython process. Such doctests must be tagged with: | |||
|
20 | ||||
|
21 | # ipdoctest: EXTERNAL | |||
|
22 | ||||
|
23 | so that the testing machinery handles them differently. Since these are run | |||
|
24 | via pexpect in an external process, they can't deal with exceptions or other | |||
|
25 | fancy featurs of regular doctests. You must limit such tests to simple | |||
|
26 | matching of the output. For this reason, I recommend you limit these kinds | |||
|
27 | of doctests to features that truly require a separate process, and use the | |||
|
28 | normal IPython ones (which have all the features of normal doctests) for | |||
|
29 | everything else. See the examples at the bottom of this file for a | |||
|
30 | comparison of what can be done with both types. | |||
|
31 | """ | |||
|
32 | ||||
|
33 | ||||
|
34 | #----------------------------------------------------------------------------- | |||
|
35 | # Module imports | |||
|
36 | ||||
|
37 | # From the standard library | |||
|
38 | import __builtin__ | |||
|
39 | import commands | |||
|
40 | import doctest | |||
|
41 | import inspect | |||
|
42 | import logging | |||
|
43 | import os | |||
|
44 | import re | |||
|
45 | import sys | |||
|
46 | import unittest | |||
|
47 | ||||
|
48 | from inspect import getmodule | |||
|
49 | ||||
|
50 | # Third-party modules | |||
|
51 | import nose.core | |||
|
52 | ||||
|
53 | from nose.plugins import doctests, Plugin | |||
|
54 | from nose.util import anyp, getpackage, test_address, resolve_name, tolist | |||
|
55 | ||||
|
56 | # Our own imports | |||
|
57 | #from extdoctest import ExtensionDoctest, DocTestFinder | |||
|
58 | #from dttools import DocTestFinder, DocTestCase | |||
|
59 | #----------------------------------------------------------------------------- | |||
|
60 | # Module globals and other constants | |||
|
61 | ||||
|
62 | log = logging.getLogger(__name__) | |||
|
63 | ||||
|
64 | ########################################################################### | |||
|
65 | # *** HACK *** | |||
|
66 | # We must start our own ipython object and heavily muck with it so that all the | |||
|
67 | # modifications IPython makes to system behavior don't send the doctest | |||
|
68 | # machinery into a fit. This code should be considered a gross hack, but it | |||
|
69 | # gets the job done. | |||
|
70 | ||||
|
71 | def start_ipython(): | |||
|
72 | """Start a global IPython shell, which we need for IPython-specific syntax. | |||
|
73 | """ | |||
|
74 | import IPython | |||
|
75 | ||||
|
76 | def xsys(cmd): | |||
|
77 | """Execute a command and print its output. | |||
|
78 | ||||
|
79 | This is just a convenience function to replace the IPython system call | |||
|
80 | with one that is more doctest-friendly. | |||
|
81 | """ | |||
|
82 | cmd = _ip.IP.var_expand(cmd,depth=1) | |||
|
83 | sys.stdout.write(commands.getoutput(cmd)) | |||
|
84 | sys.stdout.flush() | |||
|
85 | ||||
|
86 | # Store certain global objects that IPython modifies | |||
|
87 | _displayhook = sys.displayhook | |||
|
88 | _excepthook = sys.excepthook | |||
|
89 | _main = sys.modules.get('__main__') | |||
|
90 | ||||
|
91 | # Start IPython instance | |||
|
92 | IPython.Shell.IPShell(['--classic','--noterm_title']) | |||
|
93 | ||||
|
94 | # Deactivate the various python system hooks added by ipython for | |||
|
95 | # interactive convenience so we don't confuse the doctest system | |||
|
96 | sys.modules['__main__'] = _main | |||
|
97 | sys.displayhook = _displayhook | |||
|
98 | sys.excepthook = _excepthook | |||
|
99 | ||||
|
100 | # So that ipython magics and aliases can be doctested (they work by making | |||
|
101 | # a call into a global _ip object) | |||
|
102 | _ip = IPython.ipapi.get() | |||
|
103 | __builtin__._ip = _ip | |||
|
104 | ||||
|
105 | # Modify the IPython system call with one that uses getoutput, so that we | |||
|
106 | # can capture subcommands and print them to Python's stdout, otherwise the | |||
|
107 | # doctest machinery would miss them. | |||
|
108 | _ip.system = xsys | |||
|
109 | ||||
|
110 | # The start call MUST be made here. I'm not sure yet why it doesn't work if | |||
|
111 | # it is made later, at plugin initialization time, but in all my tests, that's | |||
|
112 | # the case. | |||
|
113 | start_ipython() | |||
|
114 | ||||
|
115 | # *** END HACK *** | |||
|
116 | ########################################################################### | |||
|
117 | ||||
|
118 | #----------------------------------------------------------------------------- | |||
|
119 | # Modified version of the one in the stdlib, that fixes a python bug (doctests | |||
|
120 | # not found in extension modules, http://bugs.python.org/issue3158) | |||
|
121 | class DocTestFinder(doctest.DocTestFinder): | |||
|
122 | ||||
|
123 | def _from_module(self, module, object): | |||
|
124 | """ | |||
|
125 | Return true if the given object is defined in the given | |||
|
126 | module. | |||
|
127 | """ | |||
|
128 | if module is None: | |||
|
129 | #print '_fm C1' # dbg | |||
|
130 | return True | |||
|
131 | elif inspect.isfunction(object): | |||
|
132 | #print '_fm C2' # dbg | |||
|
133 | return module.__dict__ is object.func_globals | |||
|
134 | elif inspect.isbuiltin(object): | |||
|
135 | #print '_fm C2-1' # dbg | |||
|
136 | return module.__name__ == object.__module__ | |||
|
137 | elif inspect.isclass(object): | |||
|
138 | #print '_fm C3' # dbg | |||
|
139 | return module.__name__ == object.__module__ | |||
|
140 | elif inspect.ismethod(object): | |||
|
141 | # This one may be a bug in cython that fails to correctly set the | |||
|
142 | # __module__ attribute of methods, but since the same error is easy | |||
|
143 | # to make by extension code writers, having this safety in place | |||
|
144 | # isn't such a bad idea | |||
|
145 | #print '_fm C3-1' # dbg | |||
|
146 | return module.__name__ == object.im_class.__module__ | |||
|
147 | elif inspect.getmodule(object) is not None: | |||
|
148 | #print '_fm C4' # dbg | |||
|
149 | #print 'C4 mod',module,'obj',object # dbg | |||
|
150 | return module is inspect.getmodule(object) | |||
|
151 | elif hasattr(object, '__module__'): | |||
|
152 | #print '_fm C5' # dbg | |||
|
153 | return module.__name__ == object.__module__ | |||
|
154 | elif isinstance(object, property): | |||
|
155 | #print '_fm C6' # dbg | |||
|
156 | return True # [XX] no way not be sure. | |||
|
157 | else: | |||
|
158 | raise ValueError("object must be a class or function") | |||
|
159 | ||||
|
160 | ||||
|
161 | ||||
|
162 | def _find(self, tests, obj, name, module, source_lines, globs, seen): | |||
|
163 | """ | |||
|
164 | Find tests for the given object and any contained objects, and | |||
|
165 | add them to `tests`. | |||
|
166 | """ | |||
|
167 | ||||
|
168 | doctest.DocTestFinder._find(self,tests, obj, name, module, | |||
|
169 | source_lines, globs, seen) | |||
|
170 | ||||
|
171 | # Below we re-run pieces of the above method with manual modifications, | |||
|
172 | # because the original code is buggy and fails to correctly identify | |||
|
173 | # doctests in extension modules. | |||
|
174 | ||||
|
175 | # Local shorthands | |||
|
176 | from inspect import isroutine, isclass, ismodule | |||
|
177 | ||||
|
178 | # Look for tests in a module's contained objects. | |||
|
179 | if inspect.ismodule(obj) and self._recurse: | |||
|
180 | for valname, val in obj.__dict__.items(): | |||
|
181 | valname1 = '%s.%s' % (name, valname) | |||
|
182 | if ( (isroutine(val) or isclass(val)) | |||
|
183 | and self._from_module(module, val) ): | |||
|
184 | ||||
|
185 | self._find(tests, val, valname1, module, source_lines, | |||
|
186 | globs, seen) | |||
|
187 | ||||
|
188 | ||||
|
189 | # Look for tests in a class's contained objects. | |||
|
190 | if inspect.isclass(obj) and self._recurse: | |||
|
191 | #print 'RECURSE into class:',obj # dbg | |||
|
192 | for valname, val in obj.__dict__.items(): | |||
|
193 | #valname1 = '%s.%s' % (name, valname) # dbg | |||
|
194 | #print 'N',name,'VN:',valname,'val:',str(val)[:77] # dbg | |||
|
195 | # Special handling for staticmethod/classmethod. | |||
|
196 | if isinstance(val, staticmethod): | |||
|
197 | val = getattr(obj, valname) | |||
|
198 | if isinstance(val, classmethod): | |||
|
199 | val = getattr(obj, valname).im_func | |||
|
200 | ||||
|
201 | # Recurse to methods, properties, and nested classes. | |||
|
202 | if ((inspect.isfunction(val) or inspect.isclass(val) or | |||
|
203 | inspect.ismethod(val) or | |||
|
204 | isinstance(val, property)) and | |||
|
205 | self._from_module(module, val)): | |||
|
206 | valname = '%s.%s' % (name, valname) | |||
|
207 | self._find(tests, val, valname, module, source_lines, | |||
|
208 | globs, seen) | |||
|
209 | ||||
|
210 | ||||
|
211 | class DocTestCase(doctests.DocTestCase): | |||
|
212 | """Proxy for DocTestCase: provides an address() method that | |||
|
213 | returns the correct address for the doctest case. Otherwise | |||
|
214 | acts as a proxy to the test case. To provide hints for address(), | |||
|
215 | an obj may also be passed -- this will be used as the test object | |||
|
216 | for purposes of determining the test address, if it is provided. | |||
|
217 | """ | |||
|
218 | ||||
|
219 | # doctests loaded via find(obj) omit the module name | |||
|
220 | # so we need to override id, __repr__ and shortDescription | |||
|
221 | # bonus: this will squash a 2.3 vs 2.4 incompatiblity | |||
|
222 | def id(self): | |||
|
223 | name = self._dt_test.name | |||
|
224 | filename = self._dt_test.filename | |||
|
225 | if filename is not None: | |||
|
226 | pk = getpackage(filename) | |||
|
227 | if pk is not None and not name.startswith(pk): | |||
|
228 | name = "%s.%s" % (pk, name) | |||
|
229 | return name | |||
|
230 | ||||
|
231 | ||||
|
232 | # Classes and functions | |||
|
233 | ||||
|
234 | def is_extension_module(filename): | |||
|
235 | """Return whether the given filename is an extension module. | |||
|
236 | ||||
|
237 | This simply checks that the extension is either .so or .pyd. | |||
|
238 | """ | |||
|
239 | return os.path.splitext(filename)[1].lower() in ('.so','.pyd') | |||
|
240 | ||||
|
241 | ||||
|
242 | # A simple subclassing of the original with a different class name, so we can | |||
|
243 | # distinguish and treat differently IPython examples from pure python ones. | |||
|
244 | class IPExample(doctest.Example): pass | |||
|
245 | ||||
|
246 | class IPExternalExample(doctest.Example): | |||
|
247 | """Doctest examples to be run in an external process.""" | |||
|
248 | ||||
|
249 | def __init__(self, source, want, exc_msg=None, lineno=0, indent=0, | |||
|
250 | options=None): | |||
|
251 | # Parent constructor | |||
|
252 | doctest.Example.__init__(self,source,want,exc_msg,lineno,indent,options) | |||
|
253 | ||||
|
254 | # An EXTRA newline is needed to prevent pexpect hangs | |||
|
255 | self.source += '\n' | |||
|
256 | ||||
|
257 | class IPDocTestParser(doctest.DocTestParser): | |||
|
258 | """ | |||
|
259 | A class used to parse strings containing doctest examples. | |||
|
260 | ||||
|
261 | Note: This is a version modified to properly recognize IPython input and | |||
|
262 | convert any IPython examples into valid Python ones. | |||
|
263 | """ | |||
|
264 | # This regular expression is used to find doctest examples in a | |||
|
265 | # string. It defines three groups: `source` is the source code | |||
|
266 | # (including leading indentation and prompts); `indent` is the | |||
|
267 | # indentation of the first (PS1) line of the source code; and | |||
|
268 | # `want` is the expected output (including leading indentation). | |||
|
269 | ||||
|
270 | # Classic Python prompts or default IPython ones | |||
|
271 | _PS1_PY = r'>>>' | |||
|
272 | _PS2_PY = r'\.\.\.' | |||
|
273 | ||||
|
274 | _PS1_IP = r'In\ \[\d+\]:' | |||
|
275 | _PS2_IP = r'\ \ \ \.\.\.+:' | |||
|
276 | ||||
|
277 | _RE_TPL = r''' | |||
|
278 | # Source consists of a PS1 line followed by zero or more PS2 lines. | |||
|
279 | (?P<source> | |||
|
280 | (?:^(?P<indent> [ ]*) (?P<ps1> %s) .*) # PS1 line | |||
|
281 | (?:\n [ ]* (?P<ps2> %s) .*)*) # PS2 lines | |||
|
282 | \n? # a newline | |||
|
283 | # Want consists of any non-blank lines that do not start with PS1. | |||
|
284 | (?P<want> (?:(?![ ]*$) # Not a blank line | |||
|
285 | (?![ ]*%s) # Not a line starting with PS1 | |||
|
286 | (?![ ]*%s) # Not a line starting with PS2 | |||
|
287 | .*$\n? # But any other line | |||
|
288 | )*) | |||
|
289 | ''' | |||
|
290 | ||||
|
291 | _EXAMPLE_RE_PY = re.compile( _RE_TPL % (_PS1_PY,_PS2_PY,_PS1_PY,_PS2_PY), | |||
|
292 | re.MULTILINE | re.VERBOSE) | |||
|
293 | ||||
|
294 | _EXAMPLE_RE_IP = re.compile( _RE_TPL % (_PS1_IP,_PS2_IP,_PS1_IP,_PS2_IP), | |||
|
295 | re.MULTILINE | re.VERBOSE) | |||
|
296 | ||||
|
297 | def ip2py(self,source): | |||
|
298 | """Convert input IPython source into valid Python.""" | |||
|
299 | out = [] | |||
|
300 | newline = out.append | |||
|
301 | for line in source.splitlines(): | |||
|
302 | #newline(_ip.IPipython.prefilter(line,True)) | |||
|
303 | newline(_ip.IP.prefilter(line,True)) | |||
|
304 | newline('') # ensure a closing newline, needed by doctest | |||
|
305 | return '\n'.join(out) | |||
|
306 | ||||
|
307 | def parse(self, string, name='<string>'): | |||
|
308 | """ | |||
|
309 | Divide the given string into examples and intervening text, | |||
|
310 | and return them as a list of alternating Examples and strings. | |||
|
311 | Line numbers for the Examples are 0-based. The optional | |||
|
312 | argument `name` is a name identifying this string, and is only | |||
|
313 | used for error messages. | |||
|
314 | """ | |||
|
315 | ||||
|
316 | #print 'Parse string:\n',string # dbg | |||
|
317 | ||||
|
318 | string = string.expandtabs() | |||
|
319 | # If all lines begin with the same indentation, then strip it. | |||
|
320 | min_indent = self._min_indent(string) | |||
|
321 | if min_indent > 0: | |||
|
322 | string = '\n'.join([l[min_indent:] for l in string.split('\n')]) | |||
|
323 | ||||
|
324 | output = [] | |||
|
325 | charno, lineno = 0, 0 | |||
|
326 | ||||
|
327 | # Whether to convert the input from ipython to python syntax | |||
|
328 | ip2py = False | |||
|
329 | # Find all doctest examples in the string. First, try them as Python | |||
|
330 | # examples, then as IPython ones | |||
|
331 | terms = list(self._EXAMPLE_RE_PY.finditer(string)) | |||
|
332 | if terms: | |||
|
333 | # Normal Python example | |||
|
334 | #print '-'*70 # dbg | |||
|
335 | #print 'PyExample, Source:\n',string # dbg | |||
|
336 | #print '-'*70 # dbg | |||
|
337 | Example = doctest.Example | |||
|
338 | else: | |||
|
339 | # It's an ipython example. Note that IPExamples are run | |||
|
340 | # in-process, so their syntax must be turned into valid python. | |||
|
341 | # IPExternalExamples are run out-of-process (via pexpect) so they | |||
|
342 | # don't need any filtering (a real ipython will be executing them). | |||
|
343 | terms = list(self._EXAMPLE_RE_IP.finditer(string)) | |||
|
344 | if re.search(r'#\s*ipdoctest:\s*EXTERNAL',string): | |||
|
345 | #print '-'*70 # dbg | |||
|
346 | #print 'IPExternalExample, Source:\n',string # dbg | |||
|
347 | #print '-'*70 # dbg | |||
|
348 | Example = IPExternalExample | |||
|
349 | else: | |||
|
350 | #print '-'*70 # dbg | |||
|
351 | #print 'IPExample, Source:\n',string # dbg | |||
|
352 | #print '-'*70 # dbg | |||
|
353 | Example = IPExample | |||
|
354 | ip2py = True | |||
|
355 | ||||
|
356 | for m in terms: | |||
|
357 | # Add the pre-example text to `output`. | |||
|
358 | output.append(string[charno:m.start()]) | |||
|
359 | # Update lineno (lines before this example) | |||
|
360 | lineno += string.count('\n', charno, m.start()) | |||
|
361 | # Extract info from the regexp match. | |||
|
362 | (source, options, want, exc_msg) = \ | |||
|
363 | self._parse_example(m, name, lineno,ip2py) | |||
|
364 | if Example is IPExternalExample: | |||
|
365 | options[doctest.NORMALIZE_WHITESPACE] = True | |||
|
366 | want += '\n' | |||
|
367 | # Create an Example, and add it to the list. | |||
|
368 | if not self._IS_BLANK_OR_COMMENT(source): | |||
|
369 | #print 'Example source:', source # dbg | |||
|
370 | output.append(Example(source, want, exc_msg, | |||
|
371 | lineno=lineno, | |||
|
372 | indent=min_indent+len(m.group('indent')), | |||
|
373 | options=options)) | |||
|
374 | # Update lineno (lines inside this example) | |||
|
375 | lineno += string.count('\n', m.start(), m.end()) | |||
|
376 | # Update charno. | |||
|
377 | charno = m.end() | |||
|
378 | # Add any remaining post-example text to `output`. | |||
|
379 | output.append(string[charno:]) | |||
|
380 | ||||
|
381 | return output | |||
|
382 | ||||
|
383 | def _parse_example(self, m, name, lineno,ip2py=False): | |||
|
384 | """ | |||
|
385 | Given a regular expression match from `_EXAMPLE_RE` (`m`), | |||
|
386 | return a pair `(source, want)`, where `source` is the matched | |||
|
387 | example's source code (with prompts and indentation stripped); | |||
|
388 | and `want` is the example's expected output (with indentation | |||
|
389 | stripped). | |||
|
390 | ||||
|
391 | `name` is the string's name, and `lineno` is the line number | |||
|
392 | where the example starts; both are used for error messages. | |||
|
393 | ||||
|
394 | Optional: | |||
|
395 | `ip2py`: if true, filter the input via IPython to convert the syntax | |||
|
396 | into valid python. | |||
|
397 | """ | |||
|
398 | ||||
|
399 | # Get the example's indentation level. | |||
|
400 | indent = len(m.group('indent')) | |||
|
401 | ||||
|
402 | # Divide source into lines; check that they're properly | |||
|
403 | # indented; and then strip their indentation & prompts. | |||
|
404 | source_lines = m.group('source').split('\n') | |||
|
405 | ||||
|
406 | # We're using variable-length input prompts | |||
|
407 | ps1 = m.group('ps1') | |||
|
408 | ps2 = m.group('ps2') | |||
|
409 | ps1_len = len(ps1) | |||
|
410 | ||||
|
411 | self._check_prompt_blank(source_lines, indent, name, lineno,ps1_len) | |||
|
412 | if ps2: | |||
|
413 | self._check_prefix(source_lines[1:], ' '*indent + ps2, name, lineno) | |||
|
414 | ||||
|
415 | source = '\n'.join([sl[indent+ps1_len+1:] for sl in source_lines]) | |||
|
416 | ||||
|
417 | if ip2py: | |||
|
418 | # Convert source input from IPython into valid Python syntax | |||
|
419 | source = self.ip2py(source) | |||
|
420 | ||||
|
421 | # Divide want into lines; check that it's properly indented; and | |||
|
422 | # then strip the indentation. Spaces before the last newline should | |||
|
423 | # be preserved, so plain rstrip() isn't good enough. | |||
|
424 | want = m.group('want') | |||
|
425 | want_lines = want.split('\n') | |||
|
426 | if len(want_lines) > 1 and re.match(r' *$', want_lines[-1]): | |||
|
427 | del want_lines[-1] # forget final newline & spaces after it | |||
|
428 | self._check_prefix(want_lines, ' '*indent, name, | |||
|
429 | lineno + len(source_lines)) | |||
|
430 | ||||
|
431 | # Remove ipython output prompt that might be present in the first line | |||
|
432 | want_lines[0] = re.sub(r'Out\[\d+\]: \s*?\n?','',want_lines[0]) | |||
|
433 | ||||
|
434 | want = '\n'.join([wl[indent:] for wl in want_lines]) | |||
|
435 | ||||
|
436 | # If `want` contains a traceback message, then extract it. | |||
|
437 | m = self._EXCEPTION_RE.match(want) | |||
|
438 | if m: | |||
|
439 | exc_msg = m.group('msg') | |||
|
440 | else: | |||
|
441 | exc_msg = None | |||
|
442 | ||||
|
443 | # Extract options from the source. | |||
|
444 | options = self._find_options(source, name, lineno) | |||
|
445 | ||||
|
446 | return source, options, want, exc_msg | |||
|
447 | ||||
|
448 | def _check_prompt_blank(self, lines, indent, name, lineno, ps1_len): | |||
|
449 | """ | |||
|
450 | Given the lines of a source string (including prompts and | |||
|
451 | leading indentation), check to make sure that every prompt is | |||
|
452 | followed by a space character. If any line is not followed by | |||
|
453 | a space character, then raise ValueError. | |||
|
454 | ||||
|
455 | Note: IPython-modified version which takes the input prompt length as a | |||
|
456 | parameter, so that prompts of variable length can be dealt with. | |||
|
457 | """ | |||
|
458 | space_idx = indent+ps1_len | |||
|
459 | min_len = space_idx+1 | |||
|
460 | for i, line in enumerate(lines): | |||
|
461 | if len(line) >= min_len and line[space_idx] != ' ': | |||
|
462 | raise ValueError('line %r of the docstring for %s ' | |||
|
463 | 'lacks blank after %s: %r' % | |||
|
464 | (lineno+i+1, name, | |||
|
465 | line[indent:space_idx], line)) | |||
|
466 | ||||
|
467 | SKIP = doctest.register_optionflag('SKIP') | |||
|
468 | ||||
|
469 | ########################################################################### | |||
|
470 | ||||
|
471 | class DocFileCase(doctest.DocFileCase): | |||
|
472 | """Overrides to provide filename | |||
|
473 | """ | |||
|
474 | def address(self): | |||
|
475 | return (self._dt_test.filename, None, None) | |||
|
476 | ||||
|
477 | ||||
|
478 | class ExtensionDoctest(doctests.Doctest): | |||
|
479 | """Nose Plugin that supports doctests in extension modules. | |||
|
480 | """ | |||
|
481 | name = 'extdoctest' # call nosetests with --with-extdoctest | |||
|
482 | enabled = True | |||
|
483 | ||||
|
484 | def options(self, parser, env=os.environ): | |||
|
485 | Plugin.options(self, parser, env) | |||
|
486 | ||||
|
487 | def configure(self, options, config): | |||
|
488 | Plugin.configure(self, options, config) | |||
|
489 | self.doctest_tests = options.doctest_tests | |||
|
490 | self.extension = tolist(options.doctestExtension) | |||
|
491 | self.finder = DocTestFinder() | |||
|
492 | self.parser = doctest.DocTestParser() | |||
|
493 | ||||
|
494 | ||||
|
495 | def loadTestsFromExtensionModule(self,filename): | |||
|
496 | bpath,mod = os.path.split(filename) | |||
|
497 | modname = os.path.splitext(mod)[0] | |||
|
498 | try: | |||
|
499 | sys.path.append(bpath) | |||
|
500 | module = __import__(modname) | |||
|
501 | tests = list(self.loadTestsFromModule(module)) | |||
|
502 | finally: | |||
|
503 | sys.path.pop() | |||
|
504 | return tests | |||
|
505 | ||||
|
506 | def loadTestsFromFile(self, filename): | |||
|
507 | if is_extension_module(filename): | |||
|
508 | for t in self.loadTestsFromExtensionModule(filename): | |||
|
509 | yield t | |||
|
510 | else: | |||
|
511 | ## for t in list(doctests.Doctest.loadTestsFromFile(self,filename)): | |||
|
512 | ## yield t | |||
|
513 | pass | |||
|
514 | ||||
|
515 | if self.extension and anyp(filename.endswith, self.extension): | |||
|
516 | #print 'lTF',filename # dbg | |||
|
517 | name = os.path.basename(filename) | |||
|
518 | dh = open(filename) | |||
|
519 | try: | |||
|
520 | doc = dh.read() | |||
|
521 | finally: | |||
|
522 | dh.close() | |||
|
523 | test = self.parser.get_doctest( | |||
|
524 | doc, globs={'__file__': filename}, name=name, | |||
|
525 | filename=filename, lineno=0) | |||
|
526 | if test.examples: | |||
|
527 | #print 'FileCase:',test.examples # dbg | |||
|
528 | yield DocFileCase(test) | |||
|
529 | else: | |||
|
530 | yield False # no tests to load | |||
|
531 | ||||
|
532 | ||||
|
533 | def wantFile(self,filename): | |||
|
534 | """Return whether the given filename should be scanned for tests. | |||
|
535 | ||||
|
536 | Modified version that accepts extension modules as valid containers for | |||
|
537 | doctests. | |||
|
538 | """ | |||
|
539 | #print 'Filename:',filename # dbg | |||
|
540 | ||||
|
541 | if is_extension_module(filename): | |||
|
542 | return True | |||
|
543 | else: | |||
|
544 | return doctests.Doctest.wantFile(self,filename) | |||
|
545 | ||||
|
546 | # NOTE: the method below is a *copy* of the one in the nose doctests | |||
|
547 | # plugin, but we have to replicate it here in order to have it resolve the | |||
|
548 | # DocTestCase (last line) to our local copy, since the nose plugin doesn't | |||
|
549 | # provide a public hook for what TestCase class to use. The alternative | |||
|
550 | # would be to monkeypatch doctest in the stdlib, but that's ugly and | |||
|
551 | # brittle, since a change in plugin load order can break it. So for now, | |||
|
552 | # we just paste this in here, inelegant as this may be. | |||
|
553 | ||||
|
554 | def loadTestsFromModule(self, module): | |||
|
555 | #print 'lTM',module # dbg | |||
|
556 | ||||
|
557 | if not self.matches(module.__name__): | |||
|
558 | log.debug("Doctest doesn't want module %s", module) | |||
|
559 | return | |||
|
560 | tests = self.finder.find(module) | |||
|
561 | if not tests: | |||
|
562 | return | |||
|
563 | tests.sort() | |||
|
564 | module_file = module.__file__ | |||
|
565 | if module_file[-4:] in ('.pyc', '.pyo'): | |||
|
566 | module_file = module_file[:-1] | |||
|
567 | for test in tests: | |||
|
568 | if not test.examples: | |||
|
569 | continue | |||
|
570 | if not test.filename: | |||
|
571 | test.filename = module_file | |||
|
572 | yield DocTestCase(test) | |||
|
573 | ||||
|
574 | class IPythonDoctest(ExtensionDoctest): | |||
|
575 | """Nose Plugin that supports doctests in extension modules. | |||
|
576 | """ | |||
|
577 | name = 'ipdoctest' # call nosetests with --with-ipdoctest | |||
|
578 | enabled = True | |||
|
579 | ||||
|
580 | def configure(self, options, config): | |||
|
581 | ||||
|
582 | Plugin.configure(self, options, config) | |||
|
583 | self.doctest_tests = options.doctest_tests | |||
|
584 | self.extension = tolist(options.doctestExtension) | |||
|
585 | self.parser = IPDocTestParser() | |||
|
586 | #self.finder = DocTestFinder(parser=IPDocTestParser()) | |||
|
587 | self.finder = DocTestFinder(parser=self.parser) |
@@ -0,0 +1,18 b'' | |||||
|
1 | #!/usr/bin/env python | |||
|
2 | """Nose-based test runner. | |||
|
3 | """ | |||
|
4 | ||||
|
5 | from nose.core import main | |||
|
6 | from nose.plugins.builtin import plugins | |||
|
7 | from nose.plugins.doctests import Doctest | |||
|
8 | ||||
|
9 | import ipdoctest | |||
|
10 | from ipdoctest import IPDocTestRunner | |||
|
11 | ||||
|
12 | if __name__ == '__main__': | |||
|
13 | print 'WARNING: this code is incomplete!' | |||
|
14 | ||||
|
15 | ||||
|
16 | pp = [x() for x in plugins] # activate all builtin plugins first | |||
|
17 | main(testRunner=IPDocTestRunner(), | |||
|
18 | plugins=pp+[ipdoctest.IPythonDoctest(),Doctest()]) |
@@ -0,0 +1,18 b'' | |||||
|
1 | #!/usr/bin/env python | |||
|
2 | """A Nose plugin to support IPython doctests. | |||
|
3 | """ | |||
|
4 | ||||
|
5 | from setuptools import setup | |||
|
6 | ||||
|
7 | setup(name='IPython doctest plugin', | |||
|
8 | version='0.1', | |||
|
9 | author='The IPython Team', | |||
|
10 | description = 'Nose plugin to load IPython-extended doctests', | |||
|
11 | license = 'LGPL', | |||
|
12 | py_modules = ['ipdoctest'], | |||
|
13 | entry_points = { | |||
|
14 | 'nose.plugins.0.10': ['ipdoctest = ipdoctest:IPythonDoctest', | |||
|
15 | 'extdoctest = ipdoctest:ExtensionDoctest', | |||
|
16 | ], | |||
|
17 | }, | |||
|
18 | ) |
@@ -0,0 +1,36 b'' | |||||
|
1 | ======================= | |||
|
2 | Combo testing example | |||
|
3 | ======================= | |||
|
4 | ||||
|
5 | This is a simple example that mixes ipython doctests:: | |||
|
6 | ||||
|
7 | In [1]: import code | |||
|
8 | ||||
|
9 | In [2]: 2**12 | |||
|
10 | Out[2]: 4096 | |||
|
11 | ||||
|
12 | with command-line example information that does *not* get executed:: | |||
|
13 | ||||
|
14 | $ mpirun -n 4 ipengine --controller-port=10000 --controller-ip=host0 | |||
|
15 | ||||
|
16 | and with literal examples of Python source code:: | |||
|
17 | ||||
|
18 | controller = dict(host='myhost', | |||
|
19 | engine_port=None, # default is 10105 | |||
|
20 | control_port=None, | |||
|
21 | ) | |||
|
22 | ||||
|
23 | # keys are hostnames, values are the number of engine on that host | |||
|
24 | engines = dict(node1=2, | |||
|
25 | node2=2, | |||
|
26 | node3=2, | |||
|
27 | node3=2, | |||
|
28 | ) | |||
|
29 | ||||
|
30 | # Force failure to detect that this test is being run. | |||
|
31 | 1/0 | |||
|
32 | ||||
|
33 | These source code examples are executed but no output is compared at all. An | |||
|
34 | error or failure is reported only if an exception is raised. | |||
|
35 | ||||
|
36 | NOTE: the execution of pure python blocks is not yet working! |
@@ -0,0 +1,24 b'' | |||||
|
1 | ===================================== | |||
|
2 | Tests in example form - pure python | |||
|
3 | ===================================== | |||
|
4 | ||||
|
5 | This file contains doctest examples embedded as code blocks, using normal | |||
|
6 | Python prompts. See the accompanying file for similar examples using IPython | |||
|
7 | prompts (you can't mix both types within one file). The following will be run | |||
|
8 | as a test:: | |||
|
9 | ||||
|
10 | >>> 1+1 | |||
|
11 | 2 | |||
|
12 | >>> print "hello" | |||
|
13 | hello | |||
|
14 | ||||
|
15 | More than one example works:: | |||
|
16 | ||||
|
17 | >>> s="Hello World" | |||
|
18 | ||||
|
19 | >>> s.upper() | |||
|
20 | 'HELLO WORLD' | |||
|
21 | ||||
|
22 | but you should note that the *entire* test file is considered to be a single | |||
|
23 | test. Individual code blocks that fail are printed separately as ``example | |||
|
24 | failures``, but the whole file is still counted and reported as one test. |
@@ -0,0 +1,30 b'' | |||||
|
1 | ================================= | |||
|
2 | Tests in example form - IPython | |||
|
3 | ================================= | |||
|
4 | ||||
|
5 | You can write text files with examples that use IPython prompts (as long as you | |||
|
6 | use the nose ipython doctest plugin), but you can not mix and match prompt | |||
|
7 | styles in a single file. That is, you either use all ``>>>`` prompts or all | |||
|
8 | IPython-style prompts. Your test suite *can* have both types, you just need to | |||
|
9 | put each type of example in a separate. Using IPython prompts, you can paste | |||
|
10 | directly from your session:: | |||
|
11 | ||||
|
12 | In [5]: s="Hello World" | |||
|
13 | ||||
|
14 | In [6]: s.upper() | |||
|
15 | Out[6]: 'HELLO WORLD' | |||
|
16 | ||||
|
17 | Another example:: | |||
|
18 | ||||
|
19 | In [8]: 1+3 | |||
|
20 | Out[8]: 4 | |||
|
21 | ||||
|
22 | Just like in IPython docstrings, you can use all IPython syntax and features:: | |||
|
23 | ||||
|
24 | In [9]: !echo "hello" | |||
|
25 | hello | |||
|
26 | ||||
|
27 | In [10]: a='hi' | |||
|
28 | ||||
|
29 | In [11]: !echo $a | |||
|
30 | hi |
General Comments 0
You need to be logged in to leave comments.
Login now