##// END OF EJS Templates
Add new '%autoreload 3' option...
Spas Kalaydzhisyki -
Show More
@@ -0,0 +1,14 b''
1 Autoreload 3 feature
2 ====================
3
4 Example: When an IPython session is ran with the 'autoreload' extension loaded,
5 you will now have the option '3' to select which means the following:
6
7 1. replicate all functionality from option 2
8 2. autoload all new funcs/classes/enums/globals from the module when they're added
9 3. autoload all newly imported funcs/classes/enums/globals from external modules
10
11 Try ``%autoreload 3`` in an IPython session after running ``%load_ext autoreload``
12
13 For more information please see unit test -
14 extensions/tests/test_autoreload.py : 'test_autoload_newly_added_objects'
@@ -1,550 +1,595 b''
1 """IPython extension to reload modules before executing user code.
1 """IPython extension to reload modules before executing user code.
2
2
3 ``autoreload`` reloads modules automatically before entering the execution of
3 ``autoreload`` reloads modules automatically before entering the execution of
4 code typed at the IPython prompt.
4 code typed at the IPython prompt.
5
5
6 This makes for example the following workflow possible:
6 This makes for example the following workflow possible:
7
7
8 .. sourcecode:: ipython
8 .. sourcecode:: ipython
9
9
10 In [1]: %load_ext autoreload
10 In [1]: %load_ext autoreload
11
11
12 In [2]: %autoreload 2
12 In [2]: %autoreload 2
13
13
14 In [3]: from foo import some_function
14 In [3]: from foo import some_function
15
15
16 In [4]: some_function()
16 In [4]: some_function()
17 Out[4]: 42
17 Out[4]: 42
18
18
19 In [5]: # open foo.py in an editor and change some_function to return 43
19 In [5]: # open foo.py in an editor and change some_function to return 43
20
20
21 In [6]: some_function()
21 In [6]: some_function()
22 Out[6]: 43
22 Out[6]: 43
23
23
24 The module was reloaded without reloading it explicitly, and the object
24 The module was reloaded without reloading it explicitly, and the object
25 imported with ``from foo import ...`` was also updated.
25 imported with ``from foo import ...`` was also updated.
26
26
27 Usage
27 Usage
28 =====
28 =====
29
29
30 The following magic commands are provided:
30 The following magic commands are provided:
31
31
32 ``%autoreload``
32 ``%autoreload``
33
33
34 Reload all modules (except those excluded by ``%aimport``)
34 Reload all modules (except those excluded by ``%aimport``)
35 automatically now.
35 automatically now.
36
36
37 ``%autoreload 0``
37 ``%autoreload 0``
38
38
39 Disable automatic reloading.
39 Disable automatic reloading.
40
40
41 ``%autoreload 1``
41 ``%autoreload 1``
42
42
43 Reload all modules imported with ``%aimport`` every time before
43 Reload all modules imported with ``%aimport`` every time before
44 executing the Python code typed.
44 executing the Python code typed.
45
45
46 ``%autoreload 2``
46 ``%autoreload 2``
47
47
48 Reload all modules (except those excluded by ``%aimport``) every
48 Reload all modules (except those excluded by ``%aimport``) every
49 time before executing the Python code typed.
49 time before executing the Python code typed.
50
50
51 ``%autoreload 3``
52
53 Reload all modules AND autoload newly added objects
54 (except those excluded by ``%aimport``)
55 every time before executing the Python code typed.
56
51 ``%aimport``
57 ``%aimport``
52
58
53 List modules which are to be automatically imported or not to be imported.
59 List modules which are to be automatically imported or not to be imported.
54
60
55 ``%aimport foo``
61 ``%aimport foo``
56
62
57 Import module 'foo' and mark it to be autoreloaded for ``%autoreload 1``
63 Import module 'foo' and mark it to be autoreloaded for ``%autoreload 1``
58
64
59 ``%aimport foo, bar``
65 ``%aimport foo, bar``
60
66
61 Import modules 'foo', 'bar' and mark them to be autoreloaded for ``%autoreload 1``
67 Import modules 'foo', 'bar' and mark them to be autoreloaded for ``%autoreload 1``
62
68
63 ``%aimport -foo``
69 ``%aimport -foo``
64
70
65 Mark module 'foo' to not be autoreloaded.
71 Mark module 'foo' to not be autoreloaded.
66
72
67 Caveats
73 Caveats
68 =======
74 =======
69
75
70 Reloading Python modules in a reliable way is in general difficult,
76 Reloading Python modules in a reliable way is in general difficult,
71 and unexpected things may occur. ``%autoreload`` tries to work around
77 and unexpected things may occur. ``%autoreload`` tries to work around
72 common pitfalls by replacing function code objects and parts of
78 common pitfalls by replacing function code objects and parts of
73 classes previously in the module with new versions. This makes the
79 classes previously in the module with new versions. This makes the
74 following things to work:
80 following things to work:
75
81
76 - Functions and classes imported via 'from xxx import foo' are upgraded
82 - Functions and classes imported via 'from xxx import foo' are upgraded
77 to new versions when 'xxx' is reloaded.
83 to new versions when 'xxx' is reloaded.
78
84
79 - Methods and properties of classes are upgraded on reload, so that
85 - Methods and properties of classes are upgraded on reload, so that
80 calling 'c.foo()' on an object 'c' created before the reload causes
86 calling 'c.foo()' on an object 'c' created before the reload causes
81 the new code for 'foo' to be executed.
87 the new code for 'foo' to be executed.
82
88
83 Some of the known remaining caveats are:
89 Some of the known remaining caveats are:
84
90
85 - Replacing code objects does not always succeed: changing a @property
91 - Replacing code objects does not always succeed: changing a @property
86 in a class to an ordinary method or a method to a member variable
92 in a class to an ordinary method or a method to a member variable
87 can cause problems (but in old objects only).
93 can cause problems (but in old objects only).
88
94
89 - Functions that are removed (eg. via monkey-patching) from a module
95 - Functions that are removed (eg. via monkey-patching) from a module
90 before it is reloaded are not upgraded.
96 before it is reloaded are not upgraded.
91
97
92 - C extension modules cannot be reloaded, and so cannot be autoreloaded.
98 - C extension modules cannot be reloaded, and so cannot be autoreloaded.
93 """
99 """
94
100
95 skip_doctest = True
101 skip_doctest = True
96
102
97 #-----------------------------------------------------------------------------
103 #-----------------------------------------------------------------------------
98 # Copyright (C) 2000 Thomas Heller
104 # Copyright (C) 2000 Thomas Heller
99 # Copyright (C) 2008 Pauli Virtanen <pav@iki.fi>
105 # Copyright (C) 2008 Pauli Virtanen <pav@iki.fi>
100 # Copyright (C) 2012 The IPython Development Team
106 # Copyright (C) 2012 The IPython Development Team
101 #
107 #
102 # Distributed under the terms of the BSD License. The full license is in
108 # Distributed under the terms of the BSD License. The full license is in
103 # the file COPYING, distributed as part of this software.
109 # the file COPYING, distributed as part of this software.
104 #-----------------------------------------------------------------------------
110 #-----------------------------------------------------------------------------
105 #
111 #
106 # This IPython module is written by Pauli Virtanen, based on the autoreload
112 # This IPython module is written by Pauli Virtanen, based on the autoreload
107 # code by Thomas Heller.
113 # code by Thomas Heller.
108
114
109 #-----------------------------------------------------------------------------
115 #-----------------------------------------------------------------------------
110 # Imports
116 # Imports
111 #-----------------------------------------------------------------------------
117 #-----------------------------------------------------------------------------
112
118
113 import os
119 import os
114 import sys
120 import sys
115 import traceback
121 import traceback
116 import types
122 import types
117 import weakref
123 import weakref
118 import gc
124 import gc
119 from importlib import import_module
125 from importlib import import_module
120 from importlib.util import source_from_cache
126 from importlib.util import source_from_cache
121 from imp import reload
127 from imp import reload
122
128
123 #------------------------------------------------------------------------------
129 #------------------------------------------------------------------------------
124 # Autoreload functionality
130 # Autoreload functionality
125 #------------------------------------------------------------------------------
131 #------------------------------------------------------------------------------
126
132
127 class ModuleReloader(object):
133 class ModuleReloader(object):
128 enabled = False
134 enabled = False
129 """Whether this reloader is enabled"""
135 """Whether this reloader is enabled"""
130
136
131 check_all = True
137 check_all = True
132 """Autoreload all modules, not just those listed in 'modules'"""
138 """Autoreload all modules, not just those listed in 'modules'"""
133
139
134 def __init__(self):
140 autoload_obj = False
141 """Autoreload all modules AND autoload all new objects"""
142
143 def __init__(self, shell=None):
135 # Modules that failed to reload: {module: mtime-on-failed-reload, ...}
144 # Modules that failed to reload: {module: mtime-on-failed-reload, ...}
136 self.failed = {}
145 self.failed = {}
137 # Modules specially marked as autoreloadable.
146 # Modules specially marked as autoreloadable.
138 self.modules = {}
147 self.modules = {}
139 # Modules specially marked as not autoreloadable.
148 # Modules specially marked as not autoreloadable.
140 self.skip_modules = {}
149 self.skip_modules = {}
141 # (module-name, name) -> weakref, for replacing old code objects
150 # (module-name, name) -> weakref, for replacing old code objects
142 self.old_objects = {}
151 self.old_objects = {}
143 # Module modification timestamps
152 # Module modification timestamps
144 self.modules_mtimes = {}
153 self.modules_mtimes = {}
154 self.shell = shell
145
155
146 # Cache module modification times
156 # Cache module modification times
147 self.check(check_all=True, do_reload=False)
157 self.check(check_all=True, do_reload=False)
148
158
149 def mark_module_skipped(self, module_name):
159 def mark_module_skipped(self, module_name):
150 """Skip reloading the named module in the future"""
160 """Skip reloading the named module in the future"""
151 try:
161 try:
152 del self.modules[module_name]
162 del self.modules[module_name]
153 except KeyError:
163 except KeyError:
154 pass
164 pass
155 self.skip_modules[module_name] = True
165 self.skip_modules[module_name] = True
156
166
157 def mark_module_reloadable(self, module_name):
167 def mark_module_reloadable(self, module_name):
158 """Reload the named module in the future (if it is imported)"""
168 """Reload the named module in the future (if it is imported)"""
159 try:
169 try:
160 del self.skip_modules[module_name]
170 del self.skip_modules[module_name]
161 except KeyError:
171 except KeyError:
162 pass
172 pass
163 self.modules[module_name] = True
173 self.modules[module_name] = True
164
174
165 def aimport_module(self, module_name):
175 def aimport_module(self, module_name):
166 """Import a module, and mark it reloadable
176 """Import a module, and mark it reloadable
167
177
168 Returns
178 Returns
169 -------
179 -------
170 top_module : module
180 top_module : module
171 The imported module if it is top-level, or the top-level
181 The imported module if it is top-level, or the top-level
172 top_name : module
182 top_name : module
173 Name of top_module
183 Name of top_module
174
184
175 """
185 """
176 self.mark_module_reloadable(module_name)
186 self.mark_module_reloadable(module_name)
177
187
178 import_module(module_name)
188 import_module(module_name)
179 top_name = module_name.split('.')[0]
189 top_name = module_name.split('.')[0]
180 top_module = sys.modules[top_name]
190 top_module = sys.modules[top_name]
181 return top_module, top_name
191 return top_module, top_name
182
192
183 def filename_and_mtime(self, module):
193 def filename_and_mtime(self, module):
184 if not hasattr(module, '__file__') or module.__file__ is None:
194 if not hasattr(module, '__file__') or module.__file__ is None:
185 return None, None
195 return None, None
186
196
187 if getattr(module, '__name__', None) in [None, '__mp_main__', '__main__']:
197 if getattr(module, '__name__', None) in [None, '__mp_main__', '__main__']:
188 # we cannot reload(__main__) or reload(__mp_main__)
198 # we cannot reload(__main__) or reload(__mp_main__)
189 return None, None
199 return None, None
190
200
191 filename = module.__file__
201 filename = module.__file__
192 path, ext = os.path.splitext(filename)
202 path, ext = os.path.splitext(filename)
193
203
194 if ext.lower() == '.py':
204 if ext.lower() == '.py':
195 py_filename = filename
205 py_filename = filename
196 else:
206 else:
197 try:
207 try:
198 py_filename = source_from_cache(filename)
208 py_filename = source_from_cache(filename)
199 except ValueError:
209 except ValueError:
200 return None, None
210 return None, None
201
211
202 try:
212 try:
203 pymtime = os.stat(py_filename).st_mtime
213 pymtime = os.stat(py_filename).st_mtime
204 except OSError:
214 except OSError:
205 return None, None
215 return None, None
206
216
207 return py_filename, pymtime
217 return py_filename, pymtime
208
218
209 def check(self, check_all=False, do_reload=True):
219 def check(self, check_all=False, do_reload=True):
210 """Check whether some modules need to be reloaded."""
220 """Check whether some modules need to be reloaded."""
211
221
212 if not self.enabled and not check_all:
222 if not self.enabled and not check_all:
213 return
223 return
214
224
215 if check_all or self.check_all:
225 if check_all or self.check_all:
216 modules = list(sys.modules.keys())
226 modules = list(sys.modules.keys())
217 else:
227 else:
218 modules = list(self.modules.keys())
228 modules = list(self.modules.keys())
219
229
220 for modname in modules:
230 for modname in modules:
221 m = sys.modules.get(modname, None)
231 m = sys.modules.get(modname, None)
222
232
223 if modname in self.skip_modules:
233 if modname in self.skip_modules:
224 continue
234 continue
225
235
226 py_filename, pymtime = self.filename_and_mtime(m)
236 py_filename, pymtime = self.filename_and_mtime(m)
227 if py_filename is None:
237 if py_filename is None:
228 continue
238 continue
229
239
230 try:
240 try:
231 if pymtime <= self.modules_mtimes[modname]:
241 if pymtime <= self.modules_mtimes[modname]:
232 continue
242 continue
233 except KeyError:
243 except KeyError:
234 self.modules_mtimes[modname] = pymtime
244 self.modules_mtimes[modname] = pymtime
235 continue
245 continue
236 else:
246 else:
237 if self.failed.get(py_filename, None) == pymtime:
247 if self.failed.get(py_filename, None) == pymtime:
238 continue
248 continue
239
249
240 self.modules_mtimes[modname] = pymtime
250 self.modules_mtimes[modname] = pymtime
241
251
242 # If we've reached this point, we should try to reload the module
252 # If we've reached this point, we should try to reload the module
243 if do_reload:
253 if do_reload:
244 try:
254 try:
255 if self.autoload_obj:
256 superreload(m, reload, self.old_objects, self.shell)
257 else:
245 superreload(m, reload, self.old_objects)
258 superreload(m, reload, self.old_objects)
246 if py_filename in self.failed:
259 if py_filename in self.failed:
247 del self.failed[py_filename]
260 del self.failed[py_filename]
248 except:
261 except:
249 print("[autoreload of %s failed: %s]" % (
262 print("[autoreload of %s failed: %s]" % (
250 modname, traceback.format_exc(10)), file=sys.stderr)
263 modname, traceback.format_exc(10)), file=sys.stderr)
251 self.failed[py_filename] = pymtime
264 self.failed[py_filename] = pymtime
252
265
253 #------------------------------------------------------------------------------
266 #------------------------------------------------------------------------------
254 # superreload
267 # superreload
255 #------------------------------------------------------------------------------
268 #------------------------------------------------------------------------------
256
269
257
270
258 func_attrs = ['__code__', '__defaults__', '__doc__',
271 func_attrs = ['__code__', '__defaults__', '__doc__',
259 '__closure__', '__globals__', '__dict__']
272 '__closure__', '__globals__', '__dict__']
260
273
261
274
262 def update_function(old, new):
275 def update_function(old, new):
263 """Upgrade the code object of a function"""
276 """Upgrade the code object of a function"""
264 for name in func_attrs:
277 for name in func_attrs:
265 try:
278 try:
266 setattr(old, name, getattr(new, name))
279 setattr(old, name, getattr(new, name))
267 except (AttributeError, TypeError):
280 except (AttributeError, TypeError):
268 pass
281 pass
269
282
270
283
271 def update_instances(old, new):
284 def update_instances(old, new):
272 """Use garbage collector to find all instances that refer to the old
285 """Use garbage collector to find all instances that refer to the old
273 class definition and update their __class__ to point to the new class
286 class definition and update their __class__ to point to the new class
274 definition"""
287 definition"""
275
288
276 refs = gc.get_referrers(old)
289 refs = gc.get_referrers(old)
277
290
278 for ref in refs:
291 for ref in refs:
279 if type(ref) is old:
292 if type(ref) is old:
280 ref.__class__ = new
293 ref.__class__ = new
281
294
282
295
283 def update_class(old, new):
296 def update_class(old, new):
284 """Replace stuff in the __dict__ of a class, and upgrade
297 """Replace stuff in the __dict__ of a class, and upgrade
285 method code objects, and add new methods, if any"""
298 method code objects, and add new methods, if any"""
286 for key in list(old.__dict__.keys()):
299 for key in list(old.__dict__.keys()):
287 old_obj = getattr(old, key)
300 old_obj = getattr(old, key)
288 try:
301 try:
289 new_obj = getattr(new, key)
302 new_obj = getattr(new, key)
290 # explicitly checking that comparison returns True to handle
303 # explicitly checking that comparison returns True to handle
291 # cases where `==` doesn't return a boolean.
304 # cases where `==` doesn't return a boolean.
292 if (old_obj == new_obj) is True:
305 if (old_obj == new_obj) is True:
293 continue
306 continue
294 except AttributeError:
307 except AttributeError:
295 # obsolete attribute: remove it
308 # obsolete attribute: remove it
296 try:
309 try:
297 delattr(old, key)
310 delattr(old, key)
298 except (AttributeError, TypeError):
311 except (AttributeError, TypeError):
299 pass
312 pass
300 continue
313 continue
301
314
302 if update_generic(old_obj, new_obj): continue
315 if update_generic(old_obj, new_obj): continue
303
316
304 try:
317 try:
305 setattr(old, key, getattr(new, key))
318 setattr(old, key, getattr(new, key))
306 except (AttributeError, TypeError):
319 except (AttributeError, TypeError):
307 pass # skip non-writable attributes
320 pass # skip non-writable attributes
308
321
309 for key in list(new.__dict__.keys()):
322 for key in list(new.__dict__.keys()):
310 if key not in list(old.__dict__.keys()):
323 if key not in list(old.__dict__.keys()):
311 try:
324 try:
312 setattr(old, key, getattr(new, key))
325 setattr(old, key, getattr(new, key))
313 except (AttributeError, TypeError):
326 except (AttributeError, TypeError):
314 pass # skip non-writable attributes
327 pass # skip non-writable attributes
315
328
316 # update all instances of class
329 # update all instances of class
317 update_instances(old, new)
330 update_instances(old, new)
318
331
319
332
320 def update_property(old, new):
333 def update_property(old, new):
321 """Replace get/set/del functions of a property"""
334 """Replace get/set/del functions of a property"""
322 update_generic(old.fdel, new.fdel)
335 update_generic(old.fdel, new.fdel)
323 update_generic(old.fget, new.fget)
336 update_generic(old.fget, new.fget)
324 update_generic(old.fset, new.fset)
337 update_generic(old.fset, new.fset)
325
338
326
339
327 def isinstance2(a, b, typ):
340 def isinstance2(a, b, typ):
328 return isinstance(a, typ) and isinstance(b, typ)
341 return isinstance(a, typ) and isinstance(b, typ)
329
342
330
343
331 UPDATE_RULES = [
344 UPDATE_RULES = [
332 (lambda a, b: isinstance2(a, b, type),
345 (lambda a, b: isinstance2(a, b, type),
333 update_class),
346 update_class),
334 (lambda a, b: isinstance2(a, b, types.FunctionType),
347 (lambda a, b: isinstance2(a, b, types.FunctionType),
335 update_function),
348 update_function),
336 (lambda a, b: isinstance2(a, b, property),
349 (lambda a, b: isinstance2(a, b, property),
337 update_property),
350 update_property),
338 ]
351 ]
339 UPDATE_RULES.extend([(lambda a, b: isinstance2(a, b, types.MethodType),
352 UPDATE_RULES.extend([(lambda a, b: isinstance2(a, b, types.MethodType),
340 lambda a, b: update_function(a.__func__, b.__func__)),
353 lambda a, b: update_function(a.__func__, b.__func__)),
341 ])
354 ])
342
355
343
356
344 def update_generic(a, b):
357 def update_generic(a, b):
345 for type_check, update in UPDATE_RULES:
358 for type_check, update in UPDATE_RULES:
346 if type_check(a, b):
359 if type_check(a, b):
347 update(a, b)
360 update(a, b)
348 return True
361 return True
349 return False
362 return False
350
363
351
364
352 class StrongRef(object):
365 class StrongRef(object):
353 def __init__(self, obj):
366 def __init__(self, obj):
354 self.obj = obj
367 self.obj = obj
355 def __call__(self):
368 def __call__(self):
356 return self.obj
369 return self.obj
357
370
358
371
359 def superreload(module, reload=reload, old_objects=None):
372 def append_obj(module, d, name, obj, autoload=False):
373 not_in_mod = not hasattr(obj, '__module__') or obj.__module__ != module.__name__
374 if autoload:
375 # check needed for module global built-ins (int, str, dict,..)
376 if name.startswith('__') and not_in_mod:
377 return False
378 else:
379 if not_in_mod:
380 return False
381
382 key = (module.__name__, name)
383 try:
384 d.setdefault(key, []).append(weakref.ref(obj))
385 except TypeError:
386 pass
387 return True
388
389
390 def superreload(module, reload=reload, old_objects=None, shell=None):
360 """Enhanced version of the builtin reload function.
391 """Enhanced version of the builtin reload function.
361
392
362 superreload remembers objects previously in the module, and
393 superreload remembers objects previously in the module, and
363
394
364 - upgrades the class dictionary of every old class in the module
395 - upgrades the class dictionary of every old class in the module
365 - upgrades the code object of every old function and method
396 - upgrades the code object of every old function and method
366 - clears the module's namespace before reloading
397 - clears the module's namespace before reloading
367
398
368 """
399 """
369 if old_objects is None:
400 if old_objects is None:
370 old_objects = {}
401 old_objects = {}
371
402
372 # collect old objects in the module
403 # collect old objects in the module
373 for name, obj in list(module.__dict__.items()):
404 for name, obj in list(module.__dict__.items()):
374 if not hasattr(obj, '__module__') or obj.__module__ != module.__name__:
405 if not append_obj(module, old_objects, name, obj):
375 continue
406 continue
376 key = (module.__name__, name)
407 key = (module.__name__, name)
377 try:
408 try:
378 old_objects.setdefault(key, []).append(weakref.ref(obj))
409 old_objects.setdefault(key, []).append(weakref.ref(obj))
379 except TypeError:
410 except TypeError:
380 pass
411 pass
381
412
382 # reload module
413 # reload module
383 try:
414 try:
384 # clear namespace first from old cruft
415 # clear namespace first from old cruft
385 old_dict = module.__dict__.copy()
416 old_dict = module.__dict__.copy()
386 old_name = module.__name__
417 old_name = module.__name__
387 module.__dict__.clear()
418 module.__dict__.clear()
388 module.__dict__['__name__'] = old_name
419 module.__dict__['__name__'] = old_name
389 module.__dict__['__loader__'] = old_dict['__loader__']
420 module.__dict__['__loader__'] = old_dict['__loader__']
390 except (TypeError, AttributeError, KeyError):
421 except (TypeError, AttributeError, KeyError):
391 pass
422 pass
392
423
393 try:
424 try:
394 module = reload(module)
425 module = reload(module)
395 except:
426 except:
396 # restore module dictionary on failed reload
427 # restore module dictionary on failed reload
397 module.__dict__.update(old_dict)
428 module.__dict__.update(old_dict)
398 raise
429 raise
399
430
400 # iterate over all objects and update functions & classes
431 # iterate over all objects and update functions & classes
401 for name, new_obj in list(module.__dict__.items()):
432 for name, new_obj in list(module.__dict__.items()):
402 key = (module.__name__, name)
433 key = (module.__name__, name)
403 if key not in old_objects: continue
434 if key not in old_objects:
435 # here 'shell' acts both as a flag and as an output var
436 if (
437 shell is None or
438 name == 'Enum' or
439 not append_obj(module, old_objects, name, new_obj, True)
440 ):
441 continue
442 shell.user_ns[name] = new_obj
404
443
405 new_refs = []
444 new_refs = []
406 for old_ref in old_objects[key]:
445 for old_ref in old_objects[key]:
407 old_obj = old_ref()
446 old_obj = old_ref()
408 if old_obj is None: continue
447 if old_obj is None: continue
409 new_refs.append(old_ref)
448 new_refs.append(old_ref)
410 update_generic(old_obj, new_obj)
449 update_generic(old_obj, new_obj)
411
450
412 if new_refs:
451 if new_refs:
413 old_objects[key] = new_refs
452 old_objects[key] = new_refs
414 else:
453 else:
415 del old_objects[key]
454 del old_objects[key]
416
455
417 return module
456 return module
418
457
419 #------------------------------------------------------------------------------
458 #------------------------------------------------------------------------------
420 # IPython connectivity
459 # IPython connectivity
421 #------------------------------------------------------------------------------
460 #------------------------------------------------------------------------------
422
461
423 from IPython.core.magic import Magics, magics_class, line_magic
462 from IPython.core.magic import Magics, magics_class, line_magic
424
463
425 @magics_class
464 @magics_class
426 class AutoreloadMagics(Magics):
465 class AutoreloadMagics(Magics):
427 def __init__(self, *a, **kw):
466 def __init__(self, *a, **kw):
428 super(AutoreloadMagics, self).__init__(*a, **kw)
467 super(AutoreloadMagics, self).__init__(*a, **kw)
429 self._reloader = ModuleReloader()
468 self._reloader = ModuleReloader(self.shell)
430 self._reloader.check_all = False
469 self._reloader.check_all = False
470 self._reloader.autoload_obj = False
431 self.loaded_modules = set(sys.modules)
471 self.loaded_modules = set(sys.modules)
432
472
433 @line_magic
473 @line_magic
434 def autoreload(self, parameter_s=''):
474 def autoreload(self, parameter_s=''):
435 r"""%autoreload => Reload modules automatically
475 r"""%autoreload => Reload modules automatically
436
476
437 %autoreload
477 %autoreload
438 Reload all modules (except those excluded by %aimport) automatically
478 Reload all modules (except those excluded by %aimport) automatically
439 now.
479 now.
440
480
441 %autoreload 0
481 %autoreload 0
442 Disable automatic reloading.
482 Disable automatic reloading.
443
483
444 %autoreload 1
484 %autoreload 1
445 Reload all modules imported with %aimport every time before executing
485 Reload all modules imported with %aimport every time before executing
446 the Python code typed.
486 the Python code typed.
447
487
448 %autoreload 2
488 %autoreload 2
449 Reload all modules (except those excluded by %aimport) every time
489 Reload all modules (except those excluded by %aimport) every time
450 before executing the Python code typed.
490 before executing the Python code typed.
451
491
452 Reloading Python modules in a reliable way is in general
492 Reloading Python modules in a reliable way is in general
453 difficult, and unexpected things may occur. %autoreload tries to
493 difficult, and unexpected things may occur. %autoreload tries to
454 work around common pitfalls by replacing function code objects and
494 work around common pitfalls by replacing function code objects and
455 parts of classes previously in the module with new versions. This
495 parts of classes previously in the module with new versions. This
456 makes the following things to work:
496 makes the following things to work:
457
497
458 - Functions and classes imported via 'from xxx import foo' are upgraded
498 - Functions and classes imported via 'from xxx import foo' are upgraded
459 to new versions when 'xxx' is reloaded.
499 to new versions when 'xxx' is reloaded.
460
500
461 - Methods and properties of classes are upgraded on reload, so that
501 - Methods and properties of classes are upgraded on reload, so that
462 calling 'c.foo()' on an object 'c' created before the reload causes
502 calling 'c.foo()' on an object 'c' created before the reload causes
463 the new code for 'foo' to be executed.
503 the new code for 'foo' to be executed.
464
504
465 Some of the known remaining caveats are:
505 Some of the known remaining caveats are:
466
506
467 - Replacing code objects does not always succeed: changing a @property
507 - Replacing code objects does not always succeed: changing a @property
468 in a class to an ordinary method or a method to a member variable
508 in a class to an ordinary method or a method to a member variable
469 can cause problems (but in old objects only).
509 can cause problems (but in old objects only).
470
510
471 - Functions that are removed (eg. via monkey-patching) from a module
511 - Functions that are removed (eg. via monkey-patching) from a module
472 before it is reloaded are not upgraded.
512 before it is reloaded are not upgraded.
473
513
474 - C extension modules cannot be reloaded, and so cannot be
514 - C extension modules cannot be reloaded, and so cannot be
475 autoreloaded.
515 autoreloaded.
476
516
477 """
517 """
478 if parameter_s == '':
518 if parameter_s == '':
479 self._reloader.check(True)
519 self._reloader.check(True)
480 elif parameter_s == '0':
520 elif parameter_s == '0':
481 self._reloader.enabled = False
521 self._reloader.enabled = False
482 elif parameter_s == '1':
522 elif parameter_s == '1':
483 self._reloader.check_all = False
523 self._reloader.check_all = False
484 self._reloader.enabled = True
524 self._reloader.enabled = True
485 elif parameter_s == '2':
525 elif parameter_s == '2':
486 self._reloader.check_all = True
526 self._reloader.check_all = True
487 self._reloader.enabled = True
527 self._reloader.enabled = True
528 self._reloader.enabled = True
529 elif parameter_s == '3':
530 self._reloader.check_all = True
531 self._reloader.enabled = True
532 self._reloader.autoload_obj = True
488
533
489 @line_magic
534 @line_magic
490 def aimport(self, parameter_s='', stream=None):
535 def aimport(self, parameter_s='', stream=None):
491 """%aimport => Import modules for automatic reloading.
536 """%aimport => Import modules for automatic reloading.
492
537
493 %aimport
538 %aimport
494 List modules to automatically import and not to import.
539 List modules to automatically import and not to import.
495
540
496 %aimport foo
541 %aimport foo
497 Import module 'foo' and mark it to be autoreloaded for %autoreload 1
542 Import module 'foo' and mark it to be autoreloaded for %autoreload 1
498
543
499 %aimport foo, bar
544 %aimport foo, bar
500 Import modules 'foo', 'bar' and mark them to be autoreloaded for %autoreload 1
545 Import modules 'foo', 'bar' and mark them to be autoreloaded for %autoreload 1
501
546
502 %aimport -foo
547 %aimport -foo
503 Mark module 'foo' to not be autoreloaded for %autoreload 1
548 Mark module 'foo' to not be autoreloaded for %autoreload 1
504 """
549 """
505 modname = parameter_s
550 modname = parameter_s
506 if not modname:
551 if not modname:
507 to_reload = sorted(self._reloader.modules.keys())
552 to_reload = sorted(self._reloader.modules.keys())
508 to_skip = sorted(self._reloader.skip_modules.keys())
553 to_skip = sorted(self._reloader.skip_modules.keys())
509 if stream is None:
554 if stream is None:
510 stream = sys.stdout
555 stream = sys.stdout
511 if self._reloader.check_all:
556 if self._reloader.check_all:
512 stream.write("Modules to reload:\nall-except-skipped\n")
557 stream.write("Modules to reload:\nall-except-skipped\n")
513 else:
558 else:
514 stream.write("Modules to reload:\n%s\n" % ' '.join(to_reload))
559 stream.write("Modules to reload:\n%s\n" % ' '.join(to_reload))
515 stream.write("\nModules to skip:\n%s\n" % ' '.join(to_skip))
560 stream.write("\nModules to skip:\n%s\n" % ' '.join(to_skip))
516 elif modname.startswith('-'):
561 elif modname.startswith('-'):
517 modname = modname[1:]
562 modname = modname[1:]
518 self._reloader.mark_module_skipped(modname)
563 self._reloader.mark_module_skipped(modname)
519 else:
564 else:
520 for _module in ([_.strip() for _ in modname.split(',')]):
565 for _module in ([_.strip() for _ in modname.split(',')]):
521 top_module, top_name = self._reloader.aimport_module(_module)
566 top_module, top_name = self._reloader.aimport_module(_module)
522
567
523 # Inject module to user namespace
568 # Inject module to user namespace
524 self.shell.push({top_name: top_module})
569 self.shell.push({top_name: top_module})
525
570
526 def pre_run_cell(self):
571 def pre_run_cell(self):
527 if self._reloader.enabled:
572 if self._reloader.enabled:
528 try:
573 try:
529 self._reloader.check()
574 self._reloader.check()
530 except:
575 except:
531 pass
576 pass
532
577
533 def post_execute_hook(self):
578 def post_execute_hook(self):
534 """Cache the modification times of any modules imported in this execution
579 """Cache the modification times of any modules imported in this execution
535 """
580 """
536 newly_loaded_modules = set(sys.modules) - self.loaded_modules
581 newly_loaded_modules = set(sys.modules) - self.loaded_modules
537 for modname in newly_loaded_modules:
582 for modname in newly_loaded_modules:
538 _, pymtime = self._reloader.filename_and_mtime(sys.modules[modname])
583 _, pymtime = self._reloader.filename_and_mtime(sys.modules[modname])
539 if pymtime is not None:
584 if pymtime is not None:
540 self._reloader.modules_mtimes[modname] = pymtime
585 self._reloader.modules_mtimes[modname] = pymtime
541
586
542 self.loaded_modules.update(newly_loaded_modules)
587 self.loaded_modules.update(newly_loaded_modules)
543
588
544
589
545 def load_ipython_extension(ip):
590 def load_ipython_extension(ip):
546 """Load the extension in IPython."""
591 """Load the extension in IPython."""
547 auto_reload = AutoreloadMagics(ip)
592 auto_reload = AutoreloadMagics(ip)
548 ip.register_magics(auto_reload)
593 ip.register_magics(auto_reload)
549 ip.events.register('pre_run_cell', auto_reload.pre_run_cell)
594 ip.events.register('pre_run_cell', auto_reload.pre_run_cell)
550 ip.events.register('post_execute', auto_reload.post_execute_hook)
595 ip.events.register('post_execute', auto_reload.post_execute_hook)
@@ -1,447 +1,530 b''
1 """Tests for autoreload extension.
1 """Tests for autoreload extension.
2 """
2 """
3 #-----------------------------------------------------------------------------
3 #-----------------------------------------------------------------------------
4 # Copyright (c) 2012 IPython Development Team.
4 # Copyright (c) 2012 IPython Development Team.
5 #
5 #
6 # Distributed under the terms of the Modified BSD License.
6 # Distributed under the terms of the Modified BSD License.
7 #
7 #
8 # The full license is in the file COPYING.txt, distributed with this software.
8 # The full license is in the file COPYING.txt, distributed with this software.
9 #-----------------------------------------------------------------------------
9 #-----------------------------------------------------------------------------
10
10
11 #-----------------------------------------------------------------------------
11 #-----------------------------------------------------------------------------
12 # Imports
12 # Imports
13 #-----------------------------------------------------------------------------
13 #-----------------------------------------------------------------------------
14
14
15 import os
15 import os
16 import sys
16 import sys
17 import tempfile
17 import tempfile
18 import textwrap
18 import textwrap
19 import shutil
19 import shutil
20 import random
20 import random
21 import time
21 import time
22 from io import StringIO
22 from io import StringIO
23
23
24 import nose.tools as nt
24 import nose.tools as nt
25 import IPython.testing.tools as tt
25 import IPython.testing.tools as tt
26
26
27 from unittest import TestCase
27 from unittest import TestCase
28
28
29 from IPython.extensions.autoreload import AutoreloadMagics
29 from IPython.extensions.autoreload import AutoreloadMagics
30 from IPython.core.events import EventManager, pre_run_cell
30 from IPython.core.events import EventManager, pre_run_cell
31
31
32 #-----------------------------------------------------------------------------
32 #-----------------------------------------------------------------------------
33 # Test fixture
33 # Test fixture
34 #-----------------------------------------------------------------------------
34 #-----------------------------------------------------------------------------
35
35
36 noop = lambda *a, **kw: None
36 noop = lambda *a, **kw: None
37
37
38 class FakeShell:
38 class FakeShell:
39
39
40 def __init__(self):
40 def __init__(self):
41 self.ns = {}
41 self.ns = {}
42 self.user_ns = self.ns
42 self.user_ns = self.ns
43 self.user_ns_hidden = {}
43 self.user_ns_hidden = {}
44 self.events = EventManager(self, {'pre_run_cell', pre_run_cell})
44 self.events = EventManager(self, {'pre_run_cell', pre_run_cell})
45 self.auto_magics = AutoreloadMagics(shell=self)
45 self.auto_magics = AutoreloadMagics(shell=self)
46 self.events.register('pre_run_cell', self.auto_magics.pre_run_cell)
46 self.events.register('pre_run_cell', self.auto_magics.pre_run_cell)
47
47
48 register_magics = set_hook = noop
48 register_magics = set_hook = noop
49
49
50 def run_code(self, code):
50 def run_code(self, code):
51 self.events.trigger('pre_run_cell')
51 self.events.trigger('pre_run_cell')
52 exec(code, self.user_ns)
52 exec(code, self.user_ns)
53 self.auto_magics.post_execute_hook()
53 self.auto_magics.post_execute_hook()
54
54
55 def push(self, items):
55 def push(self, items):
56 self.ns.update(items)
56 self.ns.update(items)
57
57
58 def magic_autoreload(self, parameter):
58 def magic_autoreload(self, parameter):
59 self.auto_magics.autoreload(parameter)
59 self.auto_magics.autoreload(parameter)
60
60
61 def magic_aimport(self, parameter, stream=None):
61 def magic_aimport(self, parameter, stream=None):
62 self.auto_magics.aimport(parameter, stream=stream)
62 self.auto_magics.aimport(parameter, stream=stream)
63 self.auto_magics.post_execute_hook()
63 self.auto_magics.post_execute_hook()
64
64
65
65
66 class Fixture(TestCase):
66 class Fixture(TestCase):
67 """Fixture for creating test module files"""
67 """Fixture for creating test module files"""
68
68
69 test_dir = None
69 test_dir = None
70 old_sys_path = None
70 old_sys_path = None
71 filename_chars = "abcdefghijklmopqrstuvwxyz0123456789"
71 filename_chars = "abcdefghijklmopqrstuvwxyz0123456789"
72
72
73 def setUp(self):
73 def setUp(self):
74 self.test_dir = tempfile.mkdtemp()
74 self.test_dir = tempfile.mkdtemp()
75 self.old_sys_path = list(sys.path)
75 self.old_sys_path = list(sys.path)
76 sys.path.insert(0, self.test_dir)
76 sys.path.insert(0, self.test_dir)
77 self.shell = FakeShell()
77 self.shell = FakeShell()
78
78
79 def tearDown(self):
79 def tearDown(self):
80 shutil.rmtree(self.test_dir)
80 shutil.rmtree(self.test_dir)
81 sys.path = self.old_sys_path
81 sys.path = self.old_sys_path
82
82
83 self.test_dir = None
83 self.test_dir = None
84 self.old_sys_path = None
84 self.old_sys_path = None
85 self.shell = None
85 self.shell = None
86
86
87 def get_module(self):
87 def get_module(self):
88 module_name = "tmpmod_" + "".join(random.sample(self.filename_chars,20))
88 module_name = "tmpmod_" + "".join(random.sample(self.filename_chars,20))
89 if module_name in sys.modules:
89 if module_name in sys.modules:
90 del sys.modules[module_name]
90 del sys.modules[module_name]
91 file_name = os.path.join(self.test_dir, module_name + ".py")
91 file_name = os.path.join(self.test_dir, module_name + ".py")
92 return module_name, file_name
92 return module_name, file_name
93
93
94 def write_file(self, filename, content):
94 def write_file(self, filename, content):
95 """
95 """
96 Write a file, and force a timestamp difference of at least one second
96 Write a file, and force a timestamp difference of at least one second
97
97
98 Notes
98 Notes
99 -----
99 -----
100 Python's .pyc files record the timestamp of their compilation
100 Python's .pyc files record the timestamp of their compilation
101 with a time resolution of one second.
101 with a time resolution of one second.
102
102
103 Therefore, we need to force a timestamp difference between .py
103 Therefore, we need to force a timestamp difference between .py
104 and .pyc, without having the .py file be timestamped in the
104 and .pyc, without having the .py file be timestamped in the
105 future, and without changing the timestamp of the .pyc file
105 future, and without changing the timestamp of the .pyc file
106 (because that is stored in the file). The only reliable way
106 (because that is stored in the file). The only reliable way
107 to achieve this seems to be to sleep.
107 to achieve this seems to be to sleep.
108 """
108 """
109 content = textwrap.dedent(content)
109 content = textwrap.dedent(content)
110 # Sleep one second + eps
110 # Sleep one second + eps
111 time.sleep(1.05)
111 time.sleep(1.05)
112
112
113 # Write
113 # Write
114 with open(filename, 'w') as f:
114 with open(filename, 'w') as f:
115 f.write(content)
115 f.write(content)
116
116
117 def new_module(self, code):
117 def new_module(self, code):
118 code = textwrap.dedent(code)
118 code = textwrap.dedent(code)
119 mod_name, mod_fn = self.get_module()
119 mod_name, mod_fn = self.get_module()
120 with open(mod_fn, 'w') as f:
120 with open(mod_fn, 'w') as f:
121 f.write(code)
121 f.write(code)
122 return mod_name, mod_fn
122 return mod_name, mod_fn
123
123
124 #-----------------------------------------------------------------------------
124 #-----------------------------------------------------------------------------
125 # Test automatic reloading
125 # Test automatic reloading
126 #-----------------------------------------------------------------------------
126 #-----------------------------------------------------------------------------
127
127
128 def pickle_get_current_class(obj):
128 def pickle_get_current_class(obj):
129 """
129 """
130 Original issue comes from pickle; hence the name.
130 Original issue comes from pickle; hence the name.
131 """
131 """
132 name = obj.__class__.__name__
132 name = obj.__class__.__name__
133 module_name = getattr(obj, "__module__", None)
133 module_name = getattr(obj, "__module__", None)
134 obj2 = sys.modules[module_name]
134 obj2 = sys.modules[module_name]
135 for subpath in name.split("."):
135 for subpath in name.split("."):
136 obj2 = getattr(obj2, subpath)
136 obj2 = getattr(obj2, subpath)
137 return obj2
137 return obj2
138
138
139 class TestAutoreload(Fixture):
139 class TestAutoreload(Fixture):
140
140
141 def test_reload_enums(self):
141 def test_reload_enums(self):
142 mod_name, mod_fn = self.new_module(textwrap.dedent("""
142 mod_name, mod_fn = self.new_module(textwrap.dedent("""
143 from enum import Enum
143 from enum import Enum
144 class MyEnum(Enum):
144 class MyEnum(Enum):
145 A = 'A'
145 A = 'A'
146 B = 'B'
146 B = 'B'
147 """))
147 """))
148 self.shell.magic_autoreload("2")
148 self.shell.magic_autoreload("2")
149 self.shell.magic_aimport(mod_name)
149 self.shell.magic_aimport(mod_name)
150 self.write_file(mod_fn, textwrap.dedent("""
150 self.write_file(mod_fn, textwrap.dedent("""
151 from enum import Enum
151 from enum import Enum
152 class MyEnum(Enum):
152 class MyEnum(Enum):
153 A = 'A'
153 A = 'A'
154 B = 'B'
154 B = 'B'
155 C = 'C'
155 C = 'C'
156 """))
156 """))
157 with tt.AssertNotPrints(('[autoreload of %s failed:' % mod_name), channel='stderr'):
157 with tt.AssertNotPrints(('[autoreload of %s failed:' % mod_name), channel='stderr'):
158 self.shell.run_code("pass") # trigger another reload
158 self.shell.run_code("pass") # trigger another reload
159
159
160 def test_reload_class_type(self):
160 def test_reload_class_type(self):
161 self.shell.magic_autoreload("2")
161 self.shell.magic_autoreload("2")
162 mod_name, mod_fn = self.new_module(
162 mod_name, mod_fn = self.new_module(
163 """
163 """
164 class Test():
164 class Test():
165 def meth(self):
165 def meth(self):
166 return "old"
166 return "old"
167 """
167 """
168 )
168 )
169 assert "test" not in self.shell.ns
169 assert "test" not in self.shell.ns
170 assert "result" not in self.shell.ns
170 assert "result" not in self.shell.ns
171
171
172 self.shell.run_code("from %s import Test" % mod_name)
172 self.shell.run_code("from %s import Test" % mod_name)
173 self.shell.run_code("test = Test()")
173 self.shell.run_code("test = Test()")
174
174
175 self.write_file(
175 self.write_file(
176 mod_fn,
176 mod_fn,
177 """
177 """
178 class Test():
178 class Test():
179 def meth(self):
179 def meth(self):
180 return "new"
180 return "new"
181 """,
181 """,
182 )
182 )
183
183
184 test_object = self.shell.ns["test"]
184 test_object = self.shell.ns["test"]
185
185
186 # important to trigger autoreload logic !
186 # important to trigger autoreload logic !
187 self.shell.run_code("pass")
187 self.shell.run_code("pass")
188
188
189 test_class = pickle_get_current_class(test_object)
189 test_class = pickle_get_current_class(test_object)
190 assert isinstance(test_object, test_class)
190 assert isinstance(test_object, test_class)
191
191
192 # extra check.
192 # extra check.
193 self.shell.run_code("import pickle")
193 self.shell.run_code("import pickle")
194 self.shell.run_code("p = pickle.dumps(test)")
194 self.shell.run_code("p = pickle.dumps(test)")
195
195
196 def test_reload_class_attributes(self):
196 def test_reload_class_attributes(self):
197 self.shell.magic_autoreload("2")
197 self.shell.magic_autoreload("2")
198 mod_name, mod_fn = self.new_module(textwrap.dedent("""
198 mod_name, mod_fn = self.new_module(textwrap.dedent("""
199 class MyClass:
199 class MyClass:
200
200
201 def __init__(self, a=10):
201 def __init__(self, a=10):
202 self.a = a
202 self.a = a
203 self.b = 22
203 self.b = 22
204 # self.toto = 33
204 # self.toto = 33
205
205
206 def square(self):
206 def square(self):
207 print('compute square')
207 print('compute square')
208 return self.a*self.a
208 return self.a*self.a
209 """
209 """
210 )
210 )
211 )
211 )
212 self.shell.run_code("from %s import MyClass" % mod_name)
212 self.shell.run_code("from %s import MyClass" % mod_name)
213 self.shell.run_code("first = MyClass(5)")
213 self.shell.run_code("first = MyClass(5)")
214 self.shell.run_code("first.square()")
214 self.shell.run_code("first.square()")
215 with nt.assert_raises(AttributeError):
215 with nt.assert_raises(AttributeError):
216 self.shell.run_code("first.cube()")
216 self.shell.run_code("first.cube()")
217 with nt.assert_raises(AttributeError):
217 with nt.assert_raises(AttributeError):
218 self.shell.run_code("first.power(5)")
218 self.shell.run_code("first.power(5)")
219 self.shell.run_code("first.b")
219 self.shell.run_code("first.b")
220 with nt.assert_raises(AttributeError):
220 with nt.assert_raises(AttributeError):
221 self.shell.run_code("first.toto")
221 self.shell.run_code("first.toto")
222
222
223 # remove square, add power
223 # remove square, add power
224
224
225 self.write_file(
225 self.write_file(
226 mod_fn,
226 mod_fn,
227 textwrap.dedent(
227 textwrap.dedent(
228 """
228 """
229 class MyClass:
229 class MyClass:
230
230
231 def __init__(self, a=10):
231 def __init__(self, a=10):
232 self.a = a
232 self.a = a
233 self.b = 11
233 self.b = 11
234
234
235 def power(self, p):
235 def power(self, p):
236 print('compute power '+str(p))
236 print('compute power '+str(p))
237 return self.a**p
237 return self.a**p
238 """
238 """
239 ),
239 ),
240 )
240 )
241
241
242 self.shell.run_code("second = MyClass(5)")
242 self.shell.run_code("second = MyClass(5)")
243
243
244 for object_name in {'first', 'second'}:
244 for object_name in {'first', 'second'}:
245 self.shell.run_code("{object_name}.power(5)".format(object_name=object_name))
245 self.shell.run_code("{object_name}.power(5)".format(object_name=object_name))
246 with nt.assert_raises(AttributeError):
246 with nt.assert_raises(AttributeError):
247 self.shell.run_code("{object_name}.cube()".format(object_name=object_name))
247 self.shell.run_code("{object_name}.cube()".format(object_name=object_name))
248 with nt.assert_raises(AttributeError):
248 with nt.assert_raises(AttributeError):
249 self.shell.run_code("{object_name}.square()".format(object_name=object_name))
249 self.shell.run_code("{object_name}.square()".format(object_name=object_name))
250 self.shell.run_code("{object_name}.b".format(object_name=object_name))
250 self.shell.run_code("{object_name}.b".format(object_name=object_name))
251 self.shell.run_code("{object_name}.a".format(object_name=object_name))
251 self.shell.run_code("{object_name}.a".format(object_name=object_name))
252 with nt.assert_raises(AttributeError):
252 with nt.assert_raises(AttributeError):
253 self.shell.run_code("{object_name}.toto".format(object_name=object_name))
253 self.shell.run_code("{object_name}.toto".format(object_name=object_name))
254
254
255 def test_autoload_newly_added_objects(self):
256 self.shell.magic_autoreload("3")
257 mod_code = """
258 def func1(): pass
259 """
260 mod_name, mod_fn = self.new_module(textwrap.dedent(mod_code))
261 self.shell.run_code(f"from {mod_name} import *")
262 self.shell.run_code("func1()")
263 with nt.assert_raises(NameError):
264 self.shell.run_code('func2()')
265 with nt.assert_raises(NameError):
266 self.shell.run_code('t = Test()')
267 with nt.assert_raises(NameError):
268 self.shell.run_code('number')
269
270 # ----------- TEST NEW OBJ LOADED --------------------------
271
272 new_code = """
273 def func1(): pass
274 def func2(): pass
275 class Test: pass
276 number = 0
277 from enum import Enum
278 class TestEnum(Enum):
279 A = 'a'
280 """
281 self.write_file(mod_fn, textwrap.dedent(new_code))
282
283 # test function now exists in shell's namespace namespace
284 self.shell.run_code("func2()")
285 # test function now exists in module's dict
286 self.shell.run_code(f"import sys; sys.modules['{mod_name}'].func2()")
287 # test class now exists
288 self.shell.run_code("t = Test()")
289 # test global built-in var now exists
290 self.shell.run_code('number')
291 # test the enumerations gets loaded succesfully
292 self.shell.run_code("TestEnum.A")
293
294 # ----------- TEST NEW OBJ CAN BE CHANGED --------------------
295
296 new_code = """
297 def func1(): return 'changed'
298 def func2(): return 'changed'
299 class Test:
300 def new_func(self):
301 return 'changed'
302 number = 1
303 from enum import Enum
304 class TestEnum(Enum):
305 A = 'a'
306 B = 'added'
307 """
308 self.write_file(mod_fn, textwrap.dedent(new_code))
309 self.shell.run_code("assert func1() == 'changed'")
310 self.shell.run_code("assert func2() == 'changed'")
311 self.shell.run_code("t = Test(); assert t.new_func() == 'changed'")
312 self.shell.run_code("assert number == 1")
313 self.shell.run_code("assert TestEnum.B.value == 'added'")
314
315 # ----------- TEST IMPORT FROM MODULE --------------------------
316
317 new_mod_code = '''
318 from enum import Enum
319 class Ext(Enum):
320 A = 'ext'
321 def ext_func():
322 return 'ext'
323 class ExtTest:
324 def meth(self):
325 return 'ext'
326 ext_int = 2
327 '''
328 new_mod_name, new_mod_fn = self.new_module(textwrap.dedent(new_mod_code))
329 current_mod_code = f'''
330 from {new_mod_name} import *
331 '''
332 self.write_file(mod_fn, textwrap.dedent(current_mod_code))
333 self.shell.run_code("assert Ext.A.value == 'ext'")
334 self.shell.run_code("assert ext_func() == 'ext'")
335 self.shell.run_code("t = ExtTest(); assert t.meth() == 'ext'")
336 self.shell.run_code("assert ext_int == 2")
337
255 def _check_smoketest(self, use_aimport=True):
338 def _check_smoketest(self, use_aimport=True):
256 """
339 """
257 Functional test for the automatic reloader using either
340 Functional test for the automatic reloader using either
258 '%autoreload 1' or '%autoreload 2'
341 '%autoreload 1' or '%autoreload 2'
259 """
342 """
260
343
261 mod_name, mod_fn = self.new_module("""
344 mod_name, mod_fn = self.new_module("""
262 x = 9
345 x = 9
263
346
264 z = 123 # this item will be deleted
347 z = 123 # this item will be deleted
265
348
266 def foo(y):
349 def foo(y):
267 return y + 3
350 return y + 3
268
351
269 class Baz(object):
352 class Baz(object):
270 def __init__(self, x):
353 def __init__(self, x):
271 self.x = x
354 self.x = x
272 def bar(self, y):
355 def bar(self, y):
273 return self.x + y
356 return self.x + y
274 @property
357 @property
275 def quux(self):
358 def quux(self):
276 return 42
359 return 42
277 def zzz(self):
360 def zzz(self):
278 '''This method will be deleted below'''
361 '''This method will be deleted below'''
279 return 99
362 return 99
280
363
281 class Bar: # old-style class: weakref doesn't work for it on Python < 2.7
364 class Bar: # old-style class: weakref doesn't work for it on Python < 2.7
282 def foo(self):
365 def foo(self):
283 return 1
366 return 1
284 """)
367 """)
285
368
286 #
369 #
287 # Import module, and mark for reloading
370 # Import module, and mark for reloading
288 #
371 #
289 if use_aimport:
372 if use_aimport:
290 self.shell.magic_autoreload("1")
373 self.shell.magic_autoreload("1")
291 self.shell.magic_aimport(mod_name)
374 self.shell.magic_aimport(mod_name)
292 stream = StringIO()
375 stream = StringIO()
293 self.shell.magic_aimport("", stream=stream)
376 self.shell.magic_aimport("", stream=stream)
294 nt.assert_in(("Modules to reload:\n%s" % mod_name), stream.getvalue())
377 nt.assert_in(("Modules to reload:\n%s" % mod_name), stream.getvalue())
295
378
296 with nt.assert_raises(ImportError):
379 with nt.assert_raises(ImportError):
297 self.shell.magic_aimport("tmpmod_as318989e89ds")
380 self.shell.magic_aimport("tmpmod_as318989e89ds")
298 else:
381 else:
299 self.shell.magic_autoreload("2")
382 self.shell.magic_autoreload("2")
300 self.shell.run_code("import %s" % mod_name)
383 self.shell.run_code("import %s" % mod_name)
301 stream = StringIO()
384 stream = StringIO()
302 self.shell.magic_aimport("", stream=stream)
385 self.shell.magic_aimport("", stream=stream)
303 nt.assert_true("Modules to reload:\nall-except-skipped" in
386 nt.assert_true("Modules to reload:\nall-except-skipped" in
304 stream.getvalue())
387 stream.getvalue())
305 nt.assert_in(mod_name, self.shell.ns)
388 nt.assert_in(mod_name, self.shell.ns)
306
389
307 mod = sys.modules[mod_name]
390 mod = sys.modules[mod_name]
308
391
309 #
392 #
310 # Test module contents
393 # Test module contents
311 #
394 #
312 old_foo = mod.foo
395 old_foo = mod.foo
313 old_obj = mod.Baz(9)
396 old_obj = mod.Baz(9)
314 old_obj2 = mod.Bar()
397 old_obj2 = mod.Bar()
315
398
316 def check_module_contents():
399 def check_module_contents():
317 nt.assert_equal(mod.x, 9)
400 nt.assert_equal(mod.x, 9)
318 nt.assert_equal(mod.z, 123)
401 nt.assert_equal(mod.z, 123)
319
402
320 nt.assert_equal(old_foo(0), 3)
403 nt.assert_equal(old_foo(0), 3)
321 nt.assert_equal(mod.foo(0), 3)
404 nt.assert_equal(mod.foo(0), 3)
322
405
323 obj = mod.Baz(9)
406 obj = mod.Baz(9)
324 nt.assert_equal(old_obj.bar(1), 10)
407 nt.assert_equal(old_obj.bar(1), 10)
325 nt.assert_equal(obj.bar(1), 10)
408 nt.assert_equal(obj.bar(1), 10)
326 nt.assert_equal(obj.quux, 42)
409 nt.assert_equal(obj.quux, 42)
327 nt.assert_equal(obj.zzz(), 99)
410 nt.assert_equal(obj.zzz(), 99)
328
411
329 obj2 = mod.Bar()
412 obj2 = mod.Bar()
330 nt.assert_equal(old_obj2.foo(), 1)
413 nt.assert_equal(old_obj2.foo(), 1)
331 nt.assert_equal(obj2.foo(), 1)
414 nt.assert_equal(obj2.foo(), 1)
332
415
333 check_module_contents()
416 check_module_contents()
334
417
335 #
418 #
336 # Simulate a failed reload: no reload should occur and exactly
419 # Simulate a failed reload: no reload should occur and exactly
337 # one error message should be printed
420 # one error message should be printed
338 #
421 #
339 self.write_file(mod_fn, """
422 self.write_file(mod_fn, """
340 a syntax error
423 a syntax error
341 """)
424 """)
342
425
343 with tt.AssertPrints(('[autoreload of %s failed:' % mod_name), channel='stderr'):
426 with tt.AssertPrints(('[autoreload of %s failed:' % mod_name), channel='stderr'):
344 self.shell.run_code("pass") # trigger reload
427 self.shell.run_code("pass") # trigger reload
345 with tt.AssertNotPrints(('[autoreload of %s failed:' % mod_name), channel='stderr'):
428 with tt.AssertNotPrints(('[autoreload of %s failed:' % mod_name), channel='stderr'):
346 self.shell.run_code("pass") # trigger another reload
429 self.shell.run_code("pass") # trigger another reload
347 check_module_contents()
430 check_module_contents()
348
431
349 #
432 #
350 # Rewrite module (this time reload should succeed)
433 # Rewrite module (this time reload should succeed)
351 #
434 #
352 self.write_file(mod_fn, """
435 self.write_file(mod_fn, """
353 x = 10
436 x = 10
354
437
355 def foo(y):
438 def foo(y):
356 return y + 4
439 return y + 4
357
440
358 class Baz(object):
441 class Baz(object):
359 def __init__(self, x):
442 def __init__(self, x):
360 self.x = x
443 self.x = x
361 def bar(self, y):
444 def bar(self, y):
362 return self.x + y + 1
445 return self.x + y + 1
363 @property
446 @property
364 def quux(self):
447 def quux(self):
365 return 43
448 return 43
366
449
367 class Bar: # old-style class
450 class Bar: # old-style class
368 def foo(self):
451 def foo(self):
369 return 2
452 return 2
370 """)
453 """)
371
454
372 def check_module_contents():
455 def check_module_contents():
373 nt.assert_equal(mod.x, 10)
456 nt.assert_equal(mod.x, 10)
374 nt.assert_false(hasattr(mod, 'z'))
457 nt.assert_false(hasattr(mod, 'z'))
375
458
376 nt.assert_equal(old_foo(0), 4) # superreload magic!
459 nt.assert_equal(old_foo(0), 4) # superreload magic!
377 nt.assert_equal(mod.foo(0), 4)
460 nt.assert_equal(mod.foo(0), 4)
378
461
379 obj = mod.Baz(9)
462 obj = mod.Baz(9)
380 nt.assert_equal(old_obj.bar(1), 11) # superreload magic!
463 nt.assert_equal(old_obj.bar(1), 11) # superreload magic!
381 nt.assert_equal(obj.bar(1), 11)
464 nt.assert_equal(obj.bar(1), 11)
382
465
383 nt.assert_equal(old_obj.quux, 43)
466 nt.assert_equal(old_obj.quux, 43)
384 nt.assert_equal(obj.quux, 43)
467 nt.assert_equal(obj.quux, 43)
385
468
386 nt.assert_false(hasattr(old_obj, 'zzz'))
469 nt.assert_false(hasattr(old_obj, 'zzz'))
387 nt.assert_false(hasattr(obj, 'zzz'))
470 nt.assert_false(hasattr(obj, 'zzz'))
388
471
389 obj2 = mod.Bar()
472 obj2 = mod.Bar()
390 nt.assert_equal(old_obj2.foo(), 2)
473 nt.assert_equal(old_obj2.foo(), 2)
391 nt.assert_equal(obj2.foo(), 2)
474 nt.assert_equal(obj2.foo(), 2)
392
475
393 self.shell.run_code("pass") # trigger reload
476 self.shell.run_code("pass") # trigger reload
394 check_module_contents()
477 check_module_contents()
395
478
396 #
479 #
397 # Another failure case: deleted file (shouldn't reload)
480 # Another failure case: deleted file (shouldn't reload)
398 #
481 #
399 os.unlink(mod_fn)
482 os.unlink(mod_fn)
400
483
401 self.shell.run_code("pass") # trigger reload
484 self.shell.run_code("pass") # trigger reload
402 check_module_contents()
485 check_module_contents()
403
486
404 #
487 #
405 # Disable autoreload and rewrite module: no reload should occur
488 # Disable autoreload and rewrite module: no reload should occur
406 #
489 #
407 if use_aimport:
490 if use_aimport:
408 self.shell.magic_aimport("-" + mod_name)
491 self.shell.magic_aimport("-" + mod_name)
409 stream = StringIO()
492 stream = StringIO()
410 self.shell.magic_aimport("", stream=stream)
493 self.shell.magic_aimport("", stream=stream)
411 nt.assert_true(("Modules to skip:\n%s" % mod_name) in
494 nt.assert_true(("Modules to skip:\n%s" % mod_name) in
412 stream.getvalue())
495 stream.getvalue())
413
496
414 # This should succeed, although no such module exists
497 # This should succeed, although no such module exists
415 self.shell.magic_aimport("-tmpmod_as318989e89ds")
498 self.shell.magic_aimport("-tmpmod_as318989e89ds")
416 else:
499 else:
417 self.shell.magic_autoreload("0")
500 self.shell.magic_autoreload("0")
418
501
419 self.write_file(mod_fn, """
502 self.write_file(mod_fn, """
420 x = -99
503 x = -99
421 """)
504 """)
422
505
423 self.shell.run_code("pass") # trigger reload
506 self.shell.run_code("pass") # trigger reload
424 self.shell.run_code("pass")
507 self.shell.run_code("pass")
425 check_module_contents()
508 check_module_contents()
426
509
427 #
510 #
428 # Re-enable autoreload: reload should now occur
511 # Re-enable autoreload: reload should now occur
429 #
512 #
430 if use_aimport:
513 if use_aimport:
431 self.shell.magic_aimport(mod_name)
514 self.shell.magic_aimport(mod_name)
432 else:
515 else:
433 self.shell.magic_autoreload("")
516 self.shell.magic_autoreload("")
434
517
435 self.shell.run_code("pass") # trigger reload
518 self.shell.run_code("pass") # trigger reload
436 nt.assert_equal(mod.x, -99)
519 nt.assert_equal(mod.x, -99)
437
520
438 def test_smoketest_aimport(self):
521 def test_smoketest_aimport(self):
439 self._check_smoketest(use_aimport=True)
522 self._check_smoketest(use_aimport=True)
440
523
441 def test_smoketest_autoreload(self):
524 def test_smoketest_autoreload(self):
442 self._check_smoketest(use_aimport=False)
525 self._check_smoketest(use_aimport=False)
443
526
444
527
445
528
446
529
447
530
General Comments 0
You need to be logged in to leave comments. Login now