##// END OF EJS Templates
Merge branch 'main' into shaperilio/autoreload-verbosity
Emilio Graff -
r27880:51eb040f merge
parent child Browse files
Show More

The requested changes are too big and content was truncated. Show full diff

1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
@@ -1,39 +1,39 b''
1 1 # This workflow will install Python dependencies, run tests and lint with a variety of Python versions
2 2 # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
3 3
4 4 name: Python package
5 5
6 6 permissions:
7 7 contents: read
8 8
9 9 on:
10 10 push:
11 11 branches: [ main, 7.x ]
12 12 pull_request:
13 13 branches: [ main, 7.x ]
14 14
15 15 jobs:
16 16 formatting:
17 17
18 18 runs-on: ubuntu-latest
19 19 timeout-minutes: 5
20 20 steps:
21 21 - uses: actions/checkout@v3
22 22 with:
23 23 fetch-depth: 0
24 24 - name: Set up Python
25 25 uses: actions/setup-python@v4
26 26 with:
27 27 python-version: 3.x
28 28 - name: Install dependencies
29 29 run: |
30 30 python -m pip install --upgrade pip
31 pip install darker black==21.12b0
31 pip install darker==1.5.1 black==22.10.0
32 32 - name: Lint with darker
33 33 run: |
34 34 darker -r 60625f241f298b5039cb2debc365db38aa7bb522 --check --diff . || (
35 35 echo "Changes need auto-formatting. Run:"
36 36 echo " darker -r 60625f241f298b5039cb2debc365db38aa7bb522"
37 37 echo "then commit and push changes to fix."
38 38 exit 1
39 39 )
@@ -1,40 +1,39 b''
1 1 MANIFEST
2 2 build
3 3 dist
4 4 _build
5 5 docs/man/*.gz
6 6 docs/source/api/generated
7 7 docs/source/config/options
8 8 docs/source/config/shortcuts/*.csv
9 9 docs/source/savefig
10 10 docs/source/interactive/magics-generated.txt
11 11 docs/gh-pages
12 12 jupyter_notebook/notebook/static/mathjax
13 13 jupyter_notebook/static/style/*.map
14 14 *.py[co]
15 15 __pycache__
16 16 *.egg-info
17 17 *~
18 18 *.bak
19 19 .ipynb_checkpoints
20 20 .tox
21 21 .DS_Store
22 22 \#*#
23 23 .#*
24 24 .cache
25 25 .coverage
26 26 *.swp
27 .vscode
28 27 .pytest_cache
29 28 .python-version
30 29 venv*/
31 30 .mypy_cache/
32 31
33 32 # jetbrains ide stuff
34 33 *.iml
35 34 .idea/
36 35
37 36 # vscode ide stuff
38 37 *.code-workspace
39 38 .history
40 39 .vscode
@@ -1,196 +1,199 b''
1 1 """Compiler tools with improved interactive support.
2 2
3 3 Provides compilation machinery similar to codeop, but with caching support so
4 4 we can provide interactive tracebacks.
5 5
6 6 Authors
7 7 -------
8 8 * Robert Kern
9 9 * Fernando Perez
10 10 * Thomas Kluyver
11 11 """
12 12
13 13 # Note: though it might be more natural to name this module 'compiler', that
14 14 # name is in the stdlib and name collisions with the stdlib tend to produce
15 15 # weird problems (often with third-party tools).
16 16
17 17 #-----------------------------------------------------------------------------
18 18 # Copyright (C) 2010-2011 The IPython Development Team.
19 19 #
20 20 # Distributed under the terms of the BSD License.
21 21 #
22 22 # The full license is in the file COPYING.txt, distributed with this software.
23 23 #-----------------------------------------------------------------------------
24 24
25 25 #-----------------------------------------------------------------------------
26 26 # Imports
27 27 #-----------------------------------------------------------------------------
28 28
29 29 # Stdlib imports
30 30 import __future__
31 31 from ast import PyCF_ONLY_AST
32 32 import codeop
33 33 import functools
34 34 import hashlib
35 35 import linecache
36 36 import operator
37 37 import time
38 38 from contextlib import contextmanager
39 39
40 40 #-----------------------------------------------------------------------------
41 41 # Constants
42 42 #-----------------------------------------------------------------------------
43 43
44 44 # Roughly equal to PyCF_MASK | PyCF_MASK_OBSOLETE as defined in pythonrun.h,
45 45 # this is used as a bitmask to extract future-related code flags.
46 46 PyCF_MASK = functools.reduce(operator.or_,
47 47 (getattr(__future__, fname).compiler_flag
48 48 for fname in __future__.all_feature_names))
49 49
50 50 #-----------------------------------------------------------------------------
51 51 # Local utilities
52 52 #-----------------------------------------------------------------------------
53 53
54 54 def code_name(code, number=0):
55 55 """ Compute a (probably) unique name for code for caching.
56 56
57 57 This now expects code to be unicode.
58 58 """
59 59 hash_digest = hashlib.sha1(code.encode("utf-8")).hexdigest()
60 60 # Include the number and 12 characters of the hash in the name. It's
61 61 # pretty much impossible that in a single session we'll have collisions
62 62 # even with truncated hashes, and the full one makes tracebacks too long
63 63 return '<ipython-input-{0}-{1}>'.format(number, hash_digest[:12])
64 64
65 65 #-----------------------------------------------------------------------------
66 66 # Classes and functions
67 67 #-----------------------------------------------------------------------------
68 68
69 69 class CachingCompiler(codeop.Compile):
70 70 """A compiler that caches code compiled from interactive statements.
71 71 """
72 72
73 73 def __init__(self):
74 74 codeop.Compile.__init__(self)
75 75
76 # This is ugly, but it must be done this way to allow multiple
77 # simultaneous ipython instances to coexist. Since Python itself
78 # directly accesses the data structures in the linecache module, and
79 # the cache therein is global, we must work with that data structure.
80 # We must hold a reference to the original checkcache routine and call
81 # that in our own check_cache() below, but the special IPython cache
82 # must also be shared by all IPython instances. If we were to hold
83 # separate caches (one in each CachingCompiler instance), any call made
84 # by Python itself to linecache.checkcache() would obliterate the
85 # cached data from the other IPython instances.
86 if not hasattr(linecache, '_ipython_cache'):
87 linecache._ipython_cache = {}
88 if not hasattr(linecache, '_checkcache_ori'):
89 linecache._checkcache_ori = linecache.checkcache
90 # Now, we must monkeypatch the linecache directly so that parts of the
91 # stdlib that call it outside our control go through our codepath
92 # (otherwise we'd lose our tracebacks).
93 linecache.checkcache = check_linecache_ipython
94
95 76 # Caching a dictionary { filename: execution_count } for nicely
96 77 # rendered tracebacks. The filename corresponds to the filename
97 78 # argument used for the builtins.compile function.
98 79 self._filename_map = {}
99 80
100 81 def ast_parse(self, source, filename='<unknown>', symbol='exec'):
101 82 """Parse code to an AST with the current compiler flags active.
102 83
103 84 Arguments are exactly the same as ast.parse (in the standard library),
104 85 and are passed to the built-in compile function."""
105 86 return compile(source, filename, symbol, self.flags | PyCF_ONLY_AST, 1)
106 87
107 88 def reset_compiler_flags(self):
108 89 """Reset compiler flags to default state."""
109 90 # This value is copied from codeop.Compile.__init__, so if that ever
110 91 # changes, it will need to be updated.
111 92 self.flags = codeop.PyCF_DONT_IMPLY_DEDENT
112 93
113 94 @property
114 95 def compiler_flags(self):
115 96 """Flags currently active in the compilation process.
116 97 """
117 98 return self.flags
118 99
119 100 def get_code_name(self, raw_code, transformed_code, number):
120 101 """Compute filename given the code, and the cell number.
121 102
122 103 Parameters
123 104 ----------
124 105 raw_code : str
125 106 The raw cell code.
126 107 transformed_code : str
127 108 The executable Python source code to cache and compile.
128 109 number : int
129 110 A number which forms part of the code's name. Used for the execution
130 111 counter.
131 112
132 113 Returns
133 114 -------
134 115 The computed filename.
135 116 """
136 117 return code_name(transformed_code, number)
137 118
138 119 def cache(self, transformed_code, number=0, raw_code=None):
139 120 """Make a name for a block of code, and cache the code.
140 121
141 122 Parameters
142 123 ----------
143 124 transformed_code : str
144 125 The executable Python source code to cache and compile.
145 126 number : int
146 127 A number which forms part of the code's name. Used for the execution
147 128 counter.
148 129 raw_code : str
149 130 The raw code before transformation, if None, set to `transformed_code`.
150 131
151 132 Returns
152 133 -------
153 134 The name of the cached code (as a string). Pass this as the filename
154 135 argument to compilation, so that tracebacks are correctly hooked up.
155 136 """
156 137 if raw_code is None:
157 138 raw_code = transformed_code
158 139
159 140 name = self.get_code_name(raw_code, transformed_code, number)
160 141
161 142 # Save the execution count
162 143 self._filename_map[name] = number
163 144
145 # Since Python 2.5, setting mtime to `None` means the lines will
146 # never be removed by `linecache.checkcache`. This means all the
147 # monkeypatching has *never* been necessary, since this code was
148 # only added in 2010, at which point IPython had already stopped
149 # supporting Python 2.4.
150 #
151 # Note that `linecache.clearcache` and `linecache.updatecache` may
152 # still remove our code from the cache, but those show explicit
153 # intent, and we should not try to interfere. Normally the former
154 # is never called except when out of memory, and the latter is only
155 # called for lines *not* in the cache.
164 156 entry = (
165 157 len(transformed_code),
166 time.time(),
158 None,
167 159 [line + "\n" for line in transformed_code.splitlines()],
168 160 name,
169 161 )
170 162 linecache.cache[name] = entry
171 linecache._ipython_cache[name] = entry
172 163 return name
173 164
174 165 @contextmanager
175 166 def extra_flags(self, flags):
176 167 ## bits that we'll set to 1
177 168 turn_on_bits = ~self.flags & flags
178 169
179 170
180 171 self.flags = self.flags | flags
181 172 try:
182 173 yield
183 174 finally:
184 175 # turn off only the bits we turned on so that something like
185 176 # __future__ that set flags stays.
186 177 self.flags &= ~turn_on_bits
187 178
188 179
189 180 def check_linecache_ipython(*args):
190 """Call linecache.checkcache() safely protecting our cached values.
181 """Deprecated since IPython 8.6. Call linecache.checkcache() directly.
182
183 It was already not necessary to call this function directly. If no
184 CachingCompiler had been created, this function would fail badly. If
185 an instance had been created, this function would've been monkeypatched
186 into place.
187
188 As of IPython 8.6, the monkeypatching has gone away entirely. But there
189 were still internal callers of this function, so maybe external callers
190 also existed?
191 191 """
192 # First call the original checkcache as intended
193 linecache._checkcache_ori(*args)
194 # Then, update back the cache with our data, so that tracebacks related
195 # to our compiled codes can be produced.
196 linecache.cache.update(linecache._ipython_cache)
192 import warnings
193
194 warnings.warn(
195 "Deprecated Since IPython 8.6, Just call linecache.checkcache() directly.",
196 DeprecationWarning,
197 stacklevel=2,
198 )
199 linecache.checkcache()
This diff has been collapsed as it changes many lines, (973 lines changed) Show them Hide them
@@ -1,2276 +1,2957 b''
1 1 """Completion for IPython.
2 2
3 3 This module started as fork of the rlcompleter module in the Python standard
4 4 library. The original enhancements made to rlcompleter have been sent
5 5 upstream and were accepted as of Python 2.3,
6 6
7 7 This module now support a wide variety of completion mechanism both available
8 8 for normal classic Python code, as well as completer for IPython specific
9 9 Syntax like magics.
10 10
11 11 Latex and Unicode completion
12 12 ============================
13 13
14 14 IPython and compatible frontends not only can complete your code, but can help
15 15 you to input a wide range of characters. In particular we allow you to insert
16 16 a unicode character using the tab completion mechanism.
17 17
18 18 Forward latex/unicode completion
19 19 --------------------------------
20 20
21 21 Forward completion allows you to easily type a unicode character using its latex
22 22 name, or unicode long description. To do so type a backslash follow by the
23 23 relevant name and press tab:
24 24
25 25
26 26 Using latex completion:
27 27
28 28 .. code::
29 29
30 30 \\alpha<tab>
31 31 α
32 32
33 33 or using unicode completion:
34 34
35 35
36 36 .. code::
37 37
38 38 \\GREEK SMALL LETTER ALPHA<tab>
39 39 α
40 40
41 41
42 42 Only valid Python identifiers will complete. Combining characters (like arrow or
43 43 dots) are also available, unlike latex they need to be put after the their
44 44 counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
45 45
46 46 Some browsers are known to display combining characters incorrectly.
47 47
48 48 Backward latex completion
49 49 -------------------------
50 50
51 51 It is sometime challenging to know how to type a character, if you are using
52 52 IPython, or any compatible frontend you can prepend backslash to the character
53 53 and press ``<tab>`` to expand it to its latex form.
54 54
55 55 .. code::
56 56
57 57 \\α<tab>
58 58 \\alpha
59 59
60 60
61 61 Both forward and backward completions can be deactivated by setting the
62 62 ``Completer.backslash_combining_completions`` option to ``False``.
63 63
64 64
65 65 Experimental
66 66 ============
67 67
68 68 Starting with IPython 6.0, this module can make use of the Jedi library to
69 69 generate completions both using static analysis of the code, and dynamically
70 70 inspecting multiple namespaces. Jedi is an autocompletion and static analysis
71 71 for Python. The APIs attached to this new mechanism is unstable and will
72 72 raise unless use in an :any:`provisionalcompleter` context manager.
73 73
74 74 You will find that the following are experimental:
75 75
76 76 - :any:`provisionalcompleter`
77 77 - :any:`IPCompleter.completions`
78 78 - :any:`Completion`
79 79 - :any:`rectify_completions`
80 80
81 81 .. note::
82 82
83 83 better name for :any:`rectify_completions` ?
84 84
85 85 We welcome any feedback on these new API, and we also encourage you to try this
86 86 module in debug mode (start IPython with ``--Completer.debug=True``) in order
87 87 to have extra logging information if :any:`jedi` is crashing, or if current
88 88 IPython completer pending deprecations are returning results not yet handled
89 89 by :any:`jedi`
90 90
91 91 Using Jedi for tab completion allow snippets like the following to work without
92 92 having to execute any code:
93 93
94 94 >>> myvar = ['hello', 42]
95 95 ... myvar[1].bi<tab>
96 96
97 97 Tab completion will be able to infer that ``myvar[1]`` is a real number without
98 98 executing any code unlike the previously available ``IPCompleter.greedy``
99 99 option.
100 100
101 101 Be sure to update :any:`jedi` to the latest stable version or to try the
102 102 current development version to get better completions.
103
104 Matchers
105 ========
106
107 All completions routines are implemented using unified *Matchers* API.
108 The matchers API is provisional and subject to change without notice.
109
110 The built-in matchers include:
111
112 - :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
113 - :any:`IPCompleter.magic_matcher`: completions for magics,
114 - :any:`IPCompleter.unicode_name_matcher`,
115 :any:`IPCompleter.fwd_unicode_matcher`
116 and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
117 - :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
118 - :any:`IPCompleter.file_matcher`: paths to files and directories,
119 - :any:`IPCompleter.python_func_kw_matcher` - function keywords,
120 - :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
121 - ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
122 - :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
123 implementation in :any:`InteractiveShell` which uses IPython hooks system
124 (`complete_command`) with string dispatch (including regular expressions).
125 Differently to other matchers, ``custom_completer_matcher`` will not suppress
126 Jedi results to match behaviour in earlier IPython versions.
127
128 Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
129
130 Matcher API
131 -----------
132
133 Simplifying some details, the ``Matcher`` interface can described as
134
135 .. code-block::
136
137 MatcherAPIv1 = Callable[[str], list[str]]
138 MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
139
140 Matcher = MatcherAPIv1 | MatcherAPIv2
141
142 The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
143 and remains supported as a simplest way for generating completions. This is also
144 currently the only API supported by the IPython hooks system `complete_command`.
145
146 To distinguish between matcher versions ``matcher_api_version`` attribute is used.
147 More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
148 and requires a literal ``2`` for v2 Matchers.
149
150 Once the API stabilises future versions may relax the requirement for specifying
151 ``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
152 please do not rely on the presence of ``matcher_api_version`` for any purposes.
153
154 Suppression of competing matchers
155 ---------------------------------
156
157 By default results from all matchers are combined, in the order determined by
158 their priority. Matchers can request to suppress results from subsequent
159 matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
160
161 When multiple matchers simultaneously request surpression, the results from of
162 the matcher with higher priority will be returned.
163
164 Sometimes it is desirable to suppress most but not all other matchers;
165 this can be achieved by adding a list of identifiers of matchers which
166 should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
167
168 The suppression behaviour can is user-configurable via
169 :any:`IPCompleter.suppress_competing_matchers`.
103 170 """
104 171
105 172
106 173 # Copyright (c) IPython Development Team.
107 174 # Distributed under the terms of the Modified BSD License.
108 175 #
109 176 # Some of this code originated from rlcompleter in the Python standard library
110 177 # Copyright (C) 2001 Python Software Foundation, www.python.org
111 178
112
179 from __future__ import annotations
113 180 import builtins as builtin_mod
114 181 import glob
115 182 import inspect
116 183 import itertools
117 184 import keyword
118 185 import os
119 186 import re
120 187 import string
121 188 import sys
122 189 import time
123 190 import unicodedata
124 191 import uuid
125 192 import warnings
126 193 from contextlib import contextmanager
194 from dataclasses import dataclass
195 from functools import cached_property, partial
127 196 from importlib import import_module
128 197 from types import SimpleNamespace
129 from typing import Iterable, Iterator, List, Tuple, Union, Any, Sequence, Dict, NamedTuple, Pattern, Optional
198 from typing import (
199 Iterable,
200 Iterator,
201 List,
202 Tuple,
203 Union,
204 Any,
205 Sequence,
206 Dict,
207 NamedTuple,
208 Pattern,
209 Optional,
210 TYPE_CHECKING,
211 Set,
212 Literal,
213 )
130 214
131 215 from IPython.core.error import TryNext
132 216 from IPython.core.inputtransformer2 import ESC_MAGIC
133 217 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
134 218 from IPython.core.oinspect import InspectColors
135 219 from IPython.testing.skipdoctest import skip_doctest
136 220 from IPython.utils import generics
221 from IPython.utils.decorators import sphinx_options
137 222 from IPython.utils.dir2 import dir2, get_real_method
223 from IPython.utils.docs import GENERATING_DOCUMENTATION
138 224 from IPython.utils.path import ensure_dir_exists
139 225 from IPython.utils.process import arg_split
140 from traitlets import Bool, Enum, Int, List as ListTrait, Unicode, default, observe
226 from traitlets import (
227 Bool,
228 Enum,
229 Int,
230 List as ListTrait,
231 Unicode,
232 Dict as DictTrait,
233 Union as UnionTrait,
234 default,
235 observe,
236 )
141 237 from traitlets.config.configurable import Configurable
142 238
143 239 import __main__
144 240
145 241 # skip module docstests
146 242 __skip_doctest__ = True
147 243
244
148 245 try:
149 246 import jedi
150 247 jedi.settings.case_insensitive_completion = False
151 248 import jedi.api.helpers
152 249 import jedi.api.classes
153 250 JEDI_INSTALLED = True
154 251 except ImportError:
155 252 JEDI_INSTALLED = False
156 #-----------------------------------------------------------------------------
253
254
255 if TYPE_CHECKING or GENERATING_DOCUMENTATION:
256 from typing import cast
257 from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias
258 else:
259
260 def cast(obj, type_):
261 """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
262 return obj
263
264 # do not require on runtime
265 NotRequired = Tuple # requires Python >=3.11
266 TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
267 Protocol = object # requires Python >=3.8
268 TypeAlias = Any # requires Python >=3.10
269 if GENERATING_DOCUMENTATION:
270 from typing import TypedDict
271
272 # -----------------------------------------------------------------------------
157 273 # Globals
158 274 #-----------------------------------------------------------------------------
159 275
160 276 # ranges where we have most of the valid unicode names. We could be more finer
161 277 # grained but is it worth it for performance While unicode have character in the
162 278 # range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
163 279 # write this). With below range we cover them all, with a density of ~67%
164 280 # biggest next gap we consider only adds up about 1% density and there are 600
165 281 # gaps that would need hard coding.
166 282 _UNICODE_RANGES = [(32, 0x3134b), (0xe0001, 0xe01f0)]
167 283
168 284 # Public API
169 __all__ = ['Completer','IPCompleter']
285 __all__ = ["Completer", "IPCompleter"]
170 286
171 287 if sys.platform == 'win32':
172 288 PROTECTABLES = ' '
173 289 else:
174 290 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
175 291
176 292 # Protect against returning an enormous number of completions which the frontend
177 293 # may have trouble processing.
178 294 MATCHES_LIMIT = 500
179 295
296 # Completion type reported when no type can be inferred.
297 _UNKNOWN_TYPE = "<unknown>"
180 298
181 299 class ProvisionalCompleterWarning(FutureWarning):
182 300 """
183 301 Exception raise by an experimental feature in this module.
184 302
185 303 Wrap code in :any:`provisionalcompleter` context manager if you
186 304 are certain you want to use an unstable feature.
187 305 """
188 306 pass
189 307
190 308 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
191 309
192 310
193 311 @skip_doctest
194 312 @contextmanager
195 313 def provisionalcompleter(action='ignore'):
196 314 """
197 315 This context manager has to be used in any place where unstable completer
198 316 behavior and API may be called.
199 317
200 318 >>> with provisionalcompleter():
201 319 ... completer.do_experimental_things() # works
202 320
203 321 >>> completer.do_experimental_things() # raises.
204 322
205 323 .. note::
206 324
207 325 Unstable
208 326
209 327 By using this context manager you agree that the API in use may change
210 328 without warning, and that you won't complain if they do so.
211 329
212 330 You also understand that, if the API is not to your liking, you should report
213 331 a bug to explain your use case upstream.
214 332
215 333 We'll be happy to get your feedback, feature requests, and improvements on
216 334 any of the unstable APIs!
217 335 """
218 336 with warnings.catch_warnings():
219 337 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
220 338 yield
221 339
222 340
223 341 def has_open_quotes(s):
224 342 """Return whether a string has open quotes.
225 343
226 344 This simply counts whether the number of quote characters of either type in
227 345 the string is odd.
228 346
229 347 Returns
230 348 -------
231 349 If there is an open quote, the quote character is returned. Else, return
232 350 False.
233 351 """
234 352 # We check " first, then ', so complex cases with nested quotes will get
235 353 # the " to take precedence.
236 354 if s.count('"') % 2:
237 355 return '"'
238 356 elif s.count("'") % 2:
239 357 return "'"
240 358 else:
241 359 return False
242 360
243 361
244 362 def protect_filename(s, protectables=PROTECTABLES):
245 363 """Escape a string to protect certain characters."""
246 364 if set(s) & set(protectables):
247 365 if sys.platform == "win32":
248 366 return '"' + s + '"'
249 367 else:
250 368 return "".join(("\\" + c if c in protectables else c) for c in s)
251 369 else:
252 370 return s
253 371
254 372
255 373 def expand_user(path:str) -> Tuple[str, bool, str]:
256 374 """Expand ``~``-style usernames in strings.
257 375
258 376 This is similar to :func:`os.path.expanduser`, but it computes and returns
259 377 extra information that will be useful if the input was being used in
260 378 computing completions, and you wish to return the completions with the
261 379 original '~' instead of its expanded value.
262 380
263 381 Parameters
264 382 ----------
265 383 path : str
266 384 String to be expanded. If no ~ is present, the output is the same as the
267 385 input.
268 386
269 387 Returns
270 388 -------
271 389 newpath : str
272 390 Result of ~ expansion in the input path.
273 391 tilde_expand : bool
274 392 Whether any expansion was performed or not.
275 393 tilde_val : str
276 394 The value that ~ was replaced with.
277 395 """
278 396 # Default values
279 397 tilde_expand = False
280 398 tilde_val = ''
281 399 newpath = path
282 400
283 401 if path.startswith('~'):
284 402 tilde_expand = True
285 403 rest = len(path)-1
286 404 newpath = os.path.expanduser(path)
287 405 if rest:
288 406 tilde_val = newpath[:-rest]
289 407 else:
290 408 tilde_val = newpath
291 409
292 410 return newpath, tilde_expand, tilde_val
293 411
294 412
295 413 def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
296 414 """Does the opposite of expand_user, with its outputs.
297 415 """
298 416 if tilde_expand:
299 417 return path.replace(tilde_val, '~')
300 418 else:
301 419 return path
302 420
303 421
304 422 def completions_sorting_key(word):
305 423 """key for sorting completions
306 424
307 425 This does several things:
308 426
309 427 - Demote any completions starting with underscores to the end
310 428 - Insert any %magic and %%cellmagic completions in the alphabetical order
311 429 by their name
312 430 """
313 431 prio1, prio2 = 0, 0
314 432
315 433 if word.startswith('__'):
316 434 prio1 = 2
317 435 elif word.startswith('_'):
318 436 prio1 = 1
319 437
320 438 if word.endswith('='):
321 439 prio1 = -1
322 440
323 441 if word.startswith('%%'):
324 442 # If there's another % in there, this is something else, so leave it alone
325 443 if not "%" in word[2:]:
326 444 word = word[2:]
327 445 prio2 = 2
328 446 elif word.startswith('%'):
329 447 if not "%" in word[1:]:
330 448 word = word[1:]
331 449 prio2 = 1
332 450
333 451 return prio1, word, prio2
334 452
335 453
336 454 class _FakeJediCompletion:
337 455 """
338 456 This is a workaround to communicate to the UI that Jedi has crashed and to
339 457 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
340 458
341 459 Added in IPython 6.0 so should likely be removed for 7.0
342 460
343 461 """
344 462
345 463 def __init__(self, name):
346 464
347 465 self.name = name
348 466 self.complete = name
349 467 self.type = 'crashed'
350 468 self.name_with_symbols = name
351 469 self.signature = ''
352 470 self._origin = 'fake'
353 471
354 472 def __repr__(self):
355 473 return '<Fake completion object jedi has crashed>'
356 474
357 475
476 _JediCompletionLike = Union[jedi.api.Completion, _FakeJediCompletion]
477
478
358 479 class Completion:
359 480 """
360 Completion object used and return by IPython completers.
481 Completion object used and returned by IPython completers.
361 482
362 483 .. warning::
363 484
364 485 Unstable
365 486
366 487 This function is unstable, API may change without warning.
367 488 It will also raise unless use in proper context manager.
368 489
369 490 This act as a middle ground :any:`Completion` object between the
370 491 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
371 492 object. While Jedi need a lot of information about evaluator and how the
372 493 code should be ran/inspected, PromptToolkit (and other frontend) mostly
373 494 need user facing information.
374 495
375 496 - Which range should be replaced replaced by what.
376 497 - Some metadata (like completion type), or meta information to displayed to
377 498 the use user.
378 499
379 500 For debugging purpose we can also store the origin of the completion (``jedi``,
380 501 ``IPython.python_matches``, ``IPython.magics_matches``...).
381 502 """
382 503
383 504 __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
384 505
385 506 def __init__(self, start: int, end: int, text: str, *, type: str=None, _origin='', signature='') -> None:
386 507 warnings.warn("``Completion`` is a provisional API (as of IPython 6.0). "
387 508 "It may change without warnings. "
388 509 "Use in corresponding context manager.",
389 510 category=ProvisionalCompleterWarning, stacklevel=2)
390 511
391 512 self.start = start
392 513 self.end = end
393 514 self.text = text
394 515 self.type = type
395 516 self.signature = signature
396 517 self._origin = _origin
397 518
398 519 def __repr__(self):
399 520 return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
400 521 (self.start, self.end, self.text, self.type or '?', self.signature or '?')
401 522
402 523 def __eq__(self, other)->Bool:
403 524 """
404 525 Equality and hash do not hash the type (as some completer may not be
405 526 able to infer the type), but are use to (partially) de-duplicate
406 527 completion.
407 528
408 529 Completely de-duplicating completion is a bit tricker that just
409 530 comparing as it depends on surrounding text, which Completions are not
410 531 aware of.
411 532 """
412 533 return self.start == other.start and \
413 534 self.end == other.end and \
414 535 self.text == other.text
415 536
416 537 def __hash__(self):
417 538 return hash((self.start, self.end, self.text))
418 539
419 540
541 class SimpleCompletion:
542 """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
543
544 .. warning::
545
546 Provisional
547
548 This class is used to describe the currently supported attributes of
549 simple completion items, and any additional implementation details
550 should not be relied on. Additional attributes may be included in
551 future versions, and meaning of text disambiguated from the current
552 dual meaning of "text to insert" and "text to used as a label".
553 """
554
555 __slots__ = ["text", "type"]
556
557 def __init__(self, text: str, *, type: str = None):
558 self.text = text
559 self.type = type
560
561 def __repr__(self):
562 return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
563
564
565 class _MatcherResultBase(TypedDict):
566 """Definition of dictionary to be returned by new-style Matcher (API v2)."""
567
568 #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
569 matched_fragment: NotRequired[str]
570
571 #: Whether to suppress results from all other matchers (True), some
572 #: matchers (set of identifiers) or none (False); default is False.
573 suppress: NotRequired[Union[bool, Set[str]]]
574
575 #: Identifiers of matchers which should NOT be suppressed when this matcher
576 #: requests to suppress all other matchers; defaults to an empty set.
577 do_not_suppress: NotRequired[Set[str]]
578
579 #: Are completions already ordered and should be left as-is? default is False.
580 ordered: NotRequired[bool]
581
582
583 @sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
584 class SimpleMatcherResult(_MatcherResultBase, TypedDict):
585 """Result of new-style completion matcher."""
586
587 # note: TypedDict is added again to the inheritance chain
588 # in order to get __orig_bases__ for documentation
589
590 #: List of candidate completions
591 completions: Sequence[SimpleCompletion]
592
593
594 class _JediMatcherResult(_MatcherResultBase):
595 """Matching result returned by Jedi (will be processed differently)"""
596
597 #: list of candidate completions
598 completions: Iterable[_JediCompletionLike]
599
600
601 @dataclass
602 class CompletionContext:
603 """Completion context provided as an argument to matchers in the Matcher API v2."""
604
605 # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
606 # which was not explicitly visible as an argument of the matcher, making any refactor
607 # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
608 # from the completer, and make substituting them in sub-classes easier.
609
610 #: Relevant fragment of code directly preceding the cursor.
611 #: The extraction of token is implemented via splitter heuristic
612 #: (following readline behaviour for legacy reasons), which is user configurable
613 #: (by switching the greedy mode).
614 token: str
615
616 #: The full available content of the editor or buffer
617 full_text: str
618
619 #: Cursor position in the line (the same for ``full_text`` and ``text``).
620 cursor_position: int
621
622 #: Cursor line in ``full_text``.
623 cursor_line: int
624
625 #: The maximum number of completions that will be used downstream.
626 #: Matchers can use this information to abort early.
627 #: The built-in Jedi matcher is currently excepted from this limit.
628 # If not given, return all possible completions.
629 limit: Optional[int]
630
631 @cached_property
632 def text_until_cursor(self) -> str:
633 return self.line_with_cursor[: self.cursor_position]
634
635 @cached_property
636 def line_with_cursor(self) -> str:
637 return self.full_text.split("\n")[self.cursor_line]
638
639
640 #: Matcher results for API v2.
641 MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
642
643
644 class _MatcherAPIv1Base(Protocol):
645 def __call__(self, text: str) -> list[str]:
646 """Call signature."""
647
648
649 class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
650 #: API version
651 matcher_api_version: Optional[Literal[1]]
652
653 def __call__(self, text: str) -> list[str]:
654 """Call signature."""
655
656
657 #: Protocol describing Matcher API v1.
658 MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
659
660
661 class MatcherAPIv2(Protocol):
662 """Protocol describing Matcher API v2."""
663
664 #: API version
665 matcher_api_version: Literal[2] = 2
666
667 def __call__(self, context: CompletionContext) -> MatcherResult:
668 """Call signature."""
669
670
671 Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
672
673
674 def completion_matcher(
675 *, priority: float = None, identifier: str = None, api_version: int = 1
676 ):
677 """Adds attributes describing the matcher.
678
679 Parameters
680 ----------
681 priority : Optional[float]
682 The priority of the matcher, determines the order of execution of matchers.
683 Higher priority means that the matcher will be executed first. Defaults to 0.
684 identifier : Optional[str]
685 identifier of the matcher allowing users to modify the behaviour via traitlets,
686 and also used to for debugging (will be passed as ``origin`` with the completions).
687 Defaults to matcher function ``__qualname__``.
688 api_version: Optional[int]
689 version of the Matcher API used by this matcher.
690 Currently supported values are 1 and 2.
691 Defaults to 1.
692 """
693
694 def wrapper(func: Matcher):
695 func.matcher_priority = priority or 0
696 func.matcher_identifier = identifier or func.__qualname__
697 func.matcher_api_version = api_version
698 if TYPE_CHECKING:
699 if api_version == 1:
700 func = cast(func, MatcherAPIv1)
701 elif api_version == 2:
702 func = cast(func, MatcherAPIv2)
703 return func
704
705 return wrapper
706
707
708 def _get_matcher_priority(matcher: Matcher):
709 return getattr(matcher, "matcher_priority", 0)
710
711
712 def _get_matcher_id(matcher: Matcher):
713 return getattr(matcher, "matcher_identifier", matcher.__qualname__)
714
715
716 def _get_matcher_api_version(matcher):
717 return getattr(matcher, "matcher_api_version", 1)
718
719
720 context_matcher = partial(completion_matcher, api_version=2)
721
722
420 723 _IC = Iterable[Completion]
421 724
422 725
423 726 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
424 727 """
425 728 Deduplicate a set of completions.
426 729
427 730 .. warning::
428 731
429 732 Unstable
430 733
431 734 This function is unstable, API may change without warning.
432 735
433 736 Parameters
434 737 ----------
435 738 text : str
436 739 text that should be completed.
437 740 completions : Iterator[Completion]
438 741 iterator over the completions to deduplicate
439 742
440 743 Yields
441 744 ------
442 745 `Completions` objects
443 746 Completions coming from multiple sources, may be different but end up having
444 747 the same effect when applied to ``text``. If this is the case, this will
445 748 consider completions as equal and only emit the first encountered.
446 749 Not folded in `completions()` yet for debugging purpose, and to detect when
447 750 the IPython completer does return things that Jedi does not, but should be
448 751 at some point.
449 752 """
450 753 completions = list(completions)
451 754 if not completions:
452 755 return
453 756
454 757 new_start = min(c.start for c in completions)
455 758 new_end = max(c.end for c in completions)
456 759
457 760 seen = set()
458 761 for c in completions:
459 762 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
460 763 if new_text not in seen:
461 764 yield c
462 765 seen.add(new_text)
463 766
464 767
465 768 def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
466 769 """
467 770 Rectify a set of completions to all have the same ``start`` and ``end``
468 771
469 772 .. warning::
470 773
471 774 Unstable
472 775
473 776 This function is unstable, API may change without warning.
474 777 It will also raise unless use in proper context manager.
475 778
476 779 Parameters
477 780 ----------
478 781 text : str
479 782 text that should be completed.
480 783 completions : Iterator[Completion]
481 784 iterator over the completions to rectify
482 785 _debug : bool
483 786 Log failed completion
484 787
485 788 Notes
486 789 -----
487 790 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
488 791 the Jupyter Protocol requires them to behave like so. This will readjust
489 792 the completion to have the same ``start`` and ``end`` by padding both
490 793 extremities with surrounding text.
491 794
492 795 During stabilisation should support a ``_debug`` option to log which
493 796 completion are return by the IPython completer and not found in Jedi in
494 797 order to make upstream bug report.
495 798 """
496 799 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
497 800 "It may change without warnings. "
498 801 "Use in corresponding context manager.",
499 802 category=ProvisionalCompleterWarning, stacklevel=2)
500 803
501 804 completions = list(completions)
502 805 if not completions:
503 806 return
504 807 starts = (c.start for c in completions)
505 808 ends = (c.end for c in completions)
506 809
507 810 new_start = min(starts)
508 811 new_end = max(ends)
509 812
510 813 seen_jedi = set()
511 814 seen_python_matches = set()
512 815 for c in completions:
513 816 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
514 817 if c._origin == 'jedi':
515 818 seen_jedi.add(new_text)
516 819 elif c._origin == 'IPCompleter.python_matches':
517 820 seen_python_matches.add(new_text)
518 821 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
519 822 diff = seen_python_matches.difference(seen_jedi)
520 823 if diff and _debug:
521 824 print('IPython.python matches have extras:', diff)
522 825
523 826
524 827 if sys.platform == 'win32':
525 828 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
526 829 else:
527 830 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
528 831
529 832 GREEDY_DELIMS = ' =\r\n'
530 833
531 834
532 835 class CompletionSplitter(object):
533 836 """An object to split an input line in a manner similar to readline.
534 837
535 838 By having our own implementation, we can expose readline-like completion in
536 839 a uniform manner to all frontends. This object only needs to be given the
537 840 line of text to be split and the cursor position on said line, and it
538 841 returns the 'word' to be completed on at the cursor after splitting the
539 842 entire line.
540 843
541 844 What characters are used as splitting delimiters can be controlled by
542 845 setting the ``delims`` attribute (this is a property that internally
543 846 automatically builds the necessary regular expression)"""
544 847
545 848 # Private interface
546 849
547 850 # A string of delimiter characters. The default value makes sense for
548 851 # IPython's most typical usage patterns.
549 852 _delims = DELIMS
550 853
551 854 # The expression (a normal string) to be compiled into a regular expression
552 855 # for actual splitting. We store it as an attribute mostly for ease of
553 856 # debugging, since this type of code can be so tricky to debug.
554 857 _delim_expr = None
555 858
556 859 # The regular expression that does the actual splitting
557 860 _delim_re = None
558 861
559 862 def __init__(self, delims=None):
560 863 delims = CompletionSplitter._delims if delims is None else delims
561 864 self.delims = delims
562 865
563 866 @property
564 867 def delims(self):
565 868 """Return the string of delimiter characters."""
566 869 return self._delims
567 870
568 871 @delims.setter
569 872 def delims(self, delims):
570 873 """Set the delimiters for line splitting."""
571 874 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
572 875 self._delim_re = re.compile(expr)
573 876 self._delims = delims
574 877 self._delim_expr = expr
575 878
576 879 def split_line(self, line, cursor_pos=None):
577 880 """Split a line of text with a cursor at the given position.
578 881 """
579 882 l = line if cursor_pos is None else line[:cursor_pos]
580 883 return self._delim_re.split(l)[-1]
581 884
582 885
583 886
584 887 class Completer(Configurable):
585 888
586 889 greedy = Bool(False,
587 890 help="""Activate greedy completion
588 891 PENDING DEPRECATION. this is now mostly taken care of with Jedi.
589 892
590 893 This will enable completion on elements of lists, results of function calls, etc.,
591 894 but can be unsafe because the code is actually evaluated on TAB.
592 895 """,
593 896 ).tag(config=True)
594 897
595 898 use_jedi = Bool(default_value=JEDI_INSTALLED,
596 899 help="Experimental: Use Jedi to generate autocompletions. "
597 900 "Default to True if jedi is installed.").tag(config=True)
598 901
599 902 jedi_compute_type_timeout = Int(default_value=400,
600 903 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
601 904 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
602 905 performance by preventing jedi to build its cache.
603 906 """).tag(config=True)
604 907
605 908 debug = Bool(default_value=False,
606 909 help='Enable debug for the Completer. Mostly print extra '
607 910 'information for experimental jedi integration.')\
608 911 .tag(config=True)
609 912
610 913 backslash_combining_completions = Bool(True,
611 914 help="Enable unicode completions, e.g. \\alpha<tab> . "
612 915 "Includes completion of latex commands, unicode names, and expanding "
613 916 "unicode characters back to latex commands.").tag(config=True)
614 917
615 918 def __init__(self, namespace=None, global_namespace=None, **kwargs):
616 919 """Create a new completer for the command line.
617 920
618 921 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
619 922
620 923 If unspecified, the default namespace where completions are performed
621 924 is __main__ (technically, __main__.__dict__). Namespaces should be
622 925 given as dictionaries.
623 926
624 927 An optional second namespace can be given. This allows the completer
625 928 to handle cases where both the local and global scopes need to be
626 929 distinguished.
627 930 """
628 931
629 932 # Don't bind to namespace quite yet, but flag whether the user wants a
630 933 # specific namespace or to use __main__.__dict__. This will allow us
631 934 # to bind to __main__.__dict__ at completion time, not now.
632 935 if namespace is None:
633 936 self.use_main_ns = True
634 937 else:
635 938 self.use_main_ns = False
636 939 self.namespace = namespace
637 940
638 941 # The global namespace, if given, can be bound directly
639 942 if global_namespace is None:
640 943 self.global_namespace = {}
641 944 else:
642 945 self.global_namespace = global_namespace
643 946
644 947 self.custom_matchers = []
645 948
646 949 super(Completer, self).__init__(**kwargs)
647 950
648 951 def complete(self, text, state):
649 952 """Return the next possible completion for 'text'.
650 953
651 954 This is called successively with state == 0, 1, 2, ... until it
652 955 returns None. The completion should begin with 'text'.
653 956
654 957 """
655 958 if self.use_main_ns:
656 959 self.namespace = __main__.__dict__
657 960
658 961 if state == 0:
659 962 if "." in text:
660 963 self.matches = self.attr_matches(text)
661 964 else:
662 965 self.matches = self.global_matches(text)
663 966 try:
664 967 return self.matches[state]
665 968 except IndexError:
666 969 return None
667 970
668 971 def global_matches(self, text):
669 972 """Compute matches when text is a simple name.
670 973
671 974 Return a list of all keywords, built-in functions and names currently
672 975 defined in self.namespace or self.global_namespace that match.
673 976
674 977 """
675 978 matches = []
676 979 match_append = matches.append
677 980 n = len(text)
678 981 for lst in [
679 982 keyword.kwlist,
680 983 builtin_mod.__dict__.keys(),
681 984 list(self.namespace.keys()),
682 985 list(self.global_namespace.keys()),
683 986 ]:
684 987 for word in lst:
685 988 if word[:n] == text and word != "__builtins__":
686 989 match_append(word)
687 990
688 991 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
689 992 for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
690 993 shortened = {
691 994 "_".join([sub[0] for sub in word.split("_")]): word
692 995 for word in lst
693 996 if snake_case_re.match(word)
694 997 }
695 998 for word in shortened.keys():
696 999 if word[:n] == text and word != "__builtins__":
697 1000 match_append(shortened[word])
698 1001 return matches
699 1002
700 1003 def attr_matches(self, text):
701 1004 """Compute matches when text contains a dot.
702 1005
703 1006 Assuming the text is of the form NAME.NAME....[NAME], and is
704 1007 evaluatable in self.namespace or self.global_namespace, it will be
705 1008 evaluated and its attributes (as revealed by dir()) are used as
706 1009 possible completions. (For class instances, class members are
707 1010 also considered.)
708 1011
709 1012 WARNING: this can still invoke arbitrary C code, if an object
710 1013 with a __getattr__ hook is evaluated.
711 1014
712 1015 """
713 1016
714 1017 # Another option, seems to work great. Catches things like ''.<tab>
715 1018 m = re.match(r"(\S+(\.\w+)*)\.(\w*)$", text)
716 1019
717 1020 if m:
718 1021 expr, attr = m.group(1, 3)
719 1022 elif self.greedy:
720 1023 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
721 1024 if not m2:
722 1025 return []
723 1026 expr, attr = m2.group(1,2)
724 1027 else:
725 1028 return []
726 1029
727 1030 try:
728 1031 obj = eval(expr, self.namespace)
729 1032 except:
730 1033 try:
731 1034 obj = eval(expr, self.global_namespace)
732 1035 except:
733 1036 return []
734 1037
735 1038 if self.limit_to__all__ and hasattr(obj, '__all__'):
736 1039 words = get__all__entries(obj)
737 1040 else:
738 1041 words = dir2(obj)
739 1042
740 1043 try:
741 1044 words = generics.complete_object(obj, words)
742 1045 except TryNext:
743 1046 pass
744 1047 except AssertionError:
745 1048 raise
746 1049 except Exception:
747 1050 # Silence errors from completion function
748 1051 #raise # dbg
749 1052 pass
750 1053 # Build match list to return
751 1054 n = len(attr)
752 1055 return [u"%s.%s" % (expr, w) for w in words if w[:n] == attr ]
753 1056
754 1057
755 1058 def get__all__entries(obj):
756 1059 """returns the strings in the __all__ attribute"""
757 1060 try:
758 1061 words = getattr(obj, '__all__')
759 1062 except:
760 1063 return []
761 1064
762 1065 return [w for w in words if isinstance(w, str)]
763 1066
764 1067
765 1068 def match_dict_keys(keys: List[Union[str, bytes, Tuple[Union[str, bytes]]]], prefix: str, delims: str,
766 1069 extra_prefix: Optional[Tuple[str, bytes]]=None) -> Tuple[str, int, List[str]]:
767 1070 """Used by dict_key_matches, matching the prefix to a list of keys
768 1071
769 1072 Parameters
770 1073 ----------
771 1074 keys
772 1075 list of keys in dictionary currently being completed.
773 1076 prefix
774 1077 Part of the text already typed by the user. E.g. `mydict[b'fo`
775 1078 delims
776 1079 String of delimiters to consider when finding the current key.
777 1080 extra_prefix : optional
778 1081 Part of the text already typed in multi-key index cases. E.g. for
779 1082 `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
780 1083
781 1084 Returns
782 1085 -------
783 1086 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
784 1087 ``quote`` being the quote that need to be used to close current string.
785 1088 ``token_start`` the position where the replacement should start occurring,
786 1089 ``matches`` a list of replacement/completion
787 1090
788 1091 """
789 1092 prefix_tuple = extra_prefix if extra_prefix else ()
790 1093 Nprefix = len(prefix_tuple)
791 1094 def filter_prefix_tuple(key):
792 1095 # Reject too short keys
793 1096 if len(key) <= Nprefix:
794 1097 return False
795 1098 # Reject keys with non str/bytes in it
796 1099 for k in key:
797 1100 if not isinstance(k, (str, bytes)):
798 1101 return False
799 1102 # Reject keys that do not match the prefix
800 1103 for k, pt in zip(key, prefix_tuple):
801 1104 if k != pt:
802 1105 return False
803 1106 # All checks passed!
804 1107 return True
805 1108
806 1109 filtered_keys:List[Union[str,bytes]] = []
807 1110 def _add_to_filtered_keys(key):
808 1111 if isinstance(key, (str, bytes)):
809 1112 filtered_keys.append(key)
810 1113
811 1114 for k in keys:
812 1115 if isinstance(k, tuple):
813 1116 if filter_prefix_tuple(k):
814 1117 _add_to_filtered_keys(k[Nprefix])
815 1118 else:
816 1119 _add_to_filtered_keys(k)
817 1120
818 1121 if not prefix:
819 1122 return '', 0, [repr(k) for k in filtered_keys]
820 1123 quote_match = re.search('["\']', prefix)
821 1124 assert quote_match is not None # silence mypy
822 1125 quote = quote_match.group()
823 1126 try:
824 1127 prefix_str = eval(prefix + quote, {})
825 1128 except Exception:
826 1129 return '', 0, []
827 1130
828 1131 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
829 1132 token_match = re.search(pattern, prefix, re.UNICODE)
830 1133 assert token_match is not None # silence mypy
831 1134 token_start = token_match.start()
832 1135 token_prefix = token_match.group()
833 1136
834 1137 matched:List[str] = []
835 1138 for key in filtered_keys:
836 1139 try:
837 1140 if not key.startswith(prefix_str):
838 1141 continue
839 1142 except (AttributeError, TypeError, UnicodeError):
840 1143 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
841 1144 continue
842 1145
843 1146 # reformat remainder of key to begin with prefix
844 1147 rem = key[len(prefix_str):]
845 1148 # force repr wrapped in '
846 1149 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
847 1150 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
848 1151 if quote == '"':
849 1152 # The entered prefix is quoted with ",
850 1153 # but the match is quoted with '.
851 1154 # A contained " hence needs escaping for comparison:
852 1155 rem_repr = rem_repr.replace('"', '\\"')
853 1156
854 1157 # then reinsert prefix from start of token
855 1158 matched.append('%s%s' % (token_prefix, rem_repr))
856 1159 return quote, token_start, matched
857 1160
858 1161
859 1162 def cursor_to_position(text:str, line:int, column:int)->int:
860 1163 """
861 1164 Convert the (line,column) position of the cursor in text to an offset in a
862 1165 string.
863 1166
864 1167 Parameters
865 1168 ----------
866 1169 text : str
867 1170 The text in which to calculate the cursor offset
868 1171 line : int
869 1172 Line of the cursor; 0-indexed
870 1173 column : int
871 1174 Column of the cursor 0-indexed
872 1175
873 1176 Returns
874 1177 -------
875 1178 Position of the cursor in ``text``, 0-indexed.
876 1179
877 1180 See Also
878 1181 --------
879 1182 position_to_cursor : reciprocal of this function
880 1183
881 1184 """
882 1185 lines = text.split('\n')
883 1186 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
884 1187
885 1188 return sum(len(l) + 1 for l in lines[:line]) + column
886 1189
887 1190 def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
888 1191 """
889 1192 Convert the position of the cursor in text (0 indexed) to a line
890 1193 number(0-indexed) and a column number (0-indexed) pair
891 1194
892 1195 Position should be a valid position in ``text``.
893 1196
894 1197 Parameters
895 1198 ----------
896 1199 text : str
897 1200 The text in which to calculate the cursor offset
898 1201 offset : int
899 1202 Position of the cursor in ``text``, 0-indexed.
900 1203
901 1204 Returns
902 1205 -------
903 1206 (line, column) : (int, int)
904 1207 Line of the cursor; 0-indexed, column of the cursor 0-indexed
905 1208
906 1209 See Also
907 1210 --------
908 1211 cursor_to_position : reciprocal of this function
909 1212
910 1213 """
911 1214
912 1215 assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
913 1216
914 1217 before = text[:offset]
915 1218 blines = before.split('\n') # ! splitnes trim trailing \n
916 1219 line = before.count('\n')
917 1220 col = len(blines[-1])
918 1221 return line, col
919 1222
920 1223
921 1224 def _safe_isinstance(obj, module, class_name):
922 1225 """Checks if obj is an instance of module.class_name if loaded
923 1226 """
924 1227 return (module in sys.modules and
925 1228 isinstance(obj, getattr(import_module(module), class_name)))
926 1229
927 def back_unicode_name_matches(text:str) -> Tuple[str, Sequence[str]]:
1230
1231 @context_matcher()
1232 def back_unicode_name_matcher(context: CompletionContext):
1233 """Match Unicode characters back to Unicode name
1234
1235 Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1236 """
1237 fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1238 return _convert_matcher_v1_result_to_v2(
1239 matches, type="unicode", fragment=fragment, suppress_if_matches=True
1240 )
1241
1242
1243 def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
928 1244 """Match Unicode characters back to Unicode name
929 1245
930 1246 This does ``☃`` -> ``\\snowman``
931 1247
932 1248 Note that snowman is not a valid python3 combining character but will be expanded.
933 1249 Though it will not recombine back to the snowman character by the completion machinery.
934 1250
935 1251 This will not either back-complete standard sequences like \\n, \\b ...
936 1252
1253 .. deprecated:: 8.6
1254 You can use :meth:`back_unicode_name_matcher` instead.
1255
937 1256 Returns
938 1257 =======
939 1258
940 1259 Return a tuple with two elements:
941 1260
942 1261 - The Unicode character that was matched (preceded with a backslash), or
943 1262 empty string,
944 1263 - a sequence (of 1), name for the match Unicode character, preceded by
945 1264 backslash, or empty if no match.
946
947 1265 """
948 1266 if len(text)<2:
949 1267 return '', ()
950 1268 maybe_slash = text[-2]
951 1269 if maybe_slash != '\\':
952 1270 return '', ()
953 1271
954 1272 char = text[-1]
955 1273 # no expand on quote for completion in strings.
956 1274 # nor backcomplete standard ascii keys
957 1275 if char in string.ascii_letters or char in ('"',"'"):
958 1276 return '', ()
959 1277 try :
960 1278 unic = unicodedata.name(char)
961 1279 return '\\'+char,('\\'+unic,)
962 1280 except KeyError:
963 1281 pass
964 1282 return '', ()
965 1283
966 def back_latex_name_matches(text:str) -> Tuple[str, Sequence[str]] :
1284
1285 @context_matcher()
1286 def back_latex_name_matcher(context: CompletionContext):
1287 """Match latex characters back to unicode name
1288
1289 Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1290 """
1291 fragment, matches = back_latex_name_matches(context.text_until_cursor)
1292 return _convert_matcher_v1_result_to_v2(
1293 matches, type="latex", fragment=fragment, suppress_if_matches=True
1294 )
1295
1296
1297 def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
967 1298 """Match latex characters back to unicode name
968 1299
969 1300 This does ``\\ℵ`` -> ``\\aleph``
970 1301
1302 .. deprecated:: 8.6
1303 You can use :meth:`back_latex_name_matcher` instead.
971 1304 """
972 1305 if len(text)<2:
973 1306 return '', ()
974 1307 maybe_slash = text[-2]
975 1308 if maybe_slash != '\\':
976 1309 return '', ()
977 1310
978 1311
979 1312 char = text[-1]
980 1313 # no expand on quote for completion in strings.
981 1314 # nor backcomplete standard ascii keys
982 1315 if char in string.ascii_letters or char in ('"',"'"):
983 1316 return '', ()
984 1317 try :
985 1318 latex = reverse_latex_symbol[char]
986 1319 # '\\' replace the \ as well
987 1320 return '\\'+char,[latex]
988 1321 except KeyError:
989 1322 pass
990 1323 return '', ()
991 1324
992 1325
993 1326 def _formatparamchildren(parameter) -> str:
994 1327 """
995 1328 Get parameter name and value from Jedi Private API
996 1329
997 1330 Jedi does not expose a simple way to get `param=value` from its API.
998 1331
999 1332 Parameters
1000 1333 ----------
1001 1334 parameter
1002 1335 Jedi's function `Param`
1003 1336
1004 1337 Returns
1005 1338 -------
1006 1339 A string like 'a', 'b=1', '*args', '**kwargs'
1007 1340
1008 1341 """
1009 1342 description = parameter.description
1010 1343 if not description.startswith('param '):
1011 1344 raise ValueError('Jedi function parameter description have change format.'
1012 1345 'Expected "param ...", found %r".' % description)
1013 1346 return description[6:]
1014 1347
1015 1348 def _make_signature(completion)-> str:
1016 1349 """
1017 1350 Make the signature from a jedi completion
1018 1351
1019 1352 Parameters
1020 1353 ----------
1021 1354 completion : jedi.Completion
1022 1355 object does not complete a function type
1023 1356
1024 1357 Returns
1025 1358 -------
1026 1359 a string consisting of the function signature, with the parenthesis but
1027 1360 without the function name. example:
1028 1361 `(a, *args, b=1, **kwargs)`
1029 1362
1030 1363 """
1031 1364
1032 1365 # it looks like this might work on jedi 0.17
1033 1366 if hasattr(completion, 'get_signatures'):
1034 1367 signatures = completion.get_signatures()
1035 1368 if not signatures:
1036 1369 return '(?)'
1037 1370
1038 1371 c0 = completion.get_signatures()[0]
1039 1372 return '('+c0.to_string().split('(', maxsplit=1)[1]
1040 1373
1041 1374 return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1042 1375 for p in signature.defined_names()) if f])
1043 1376
1044 1377
1045 class _CompleteResult(NamedTuple):
1046 matched_text : str
1047 matches: Sequence[str]
1048 matches_origin: Sequence[str]
1049 jedi_matches: Any
1378 _CompleteResult = Dict[str, MatcherResult]
1379
1380
1381 def _convert_matcher_v1_result_to_v2(
1382 matches: Sequence[str],
1383 type: str,
1384 fragment: str = None,
1385 suppress_if_matches: bool = False,
1386 ) -> SimpleMatcherResult:
1387 """Utility to help with transition"""
1388 result = {
1389 "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1390 "suppress": (True if matches else False) if suppress_if_matches else False,
1391 }
1392 if fragment is not None:
1393 result["matched_fragment"] = fragment
1394 return result
1050 1395
1051 1396
1052 1397 class IPCompleter(Completer):
1053 1398 """Extension of the completer class with IPython-specific features"""
1054 1399
1055 1400 __dict_key_regexps: Optional[Dict[bool,Pattern]] = None
1056 1401
1057 1402 @observe('greedy')
1058 1403 def _greedy_changed(self, change):
1059 1404 """update the splitter and readline delims when greedy is changed"""
1060 1405 if change['new']:
1061 1406 self.splitter.delims = GREEDY_DELIMS
1062 1407 else:
1063 1408 self.splitter.delims = DELIMS
1064 1409
1065 dict_keys_only = Bool(False,
1066 help="""Whether to show dict key matches only""")
1410 dict_keys_only = Bool(
1411 False,
1412 help="""
1413 Whether to show dict key matches only.
1414
1415 (disables all matchers except for `IPCompleter.dict_key_matcher`).
1416 """,
1417 )
1418
1419 suppress_competing_matchers = UnionTrait(
1420 [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1421 default_value=None,
1422 help="""
1423 Whether to suppress completions from other *Matchers*.
1424
1425 When set to ``None`` (default) the matchers will attempt to auto-detect
1426 whether suppression of other matchers is desirable. For example, at
1427 the beginning of a line followed by `%` we expect a magic completion
1428 to be the only applicable option, and after ``my_dict['`` we usually
1429 expect a completion with an existing dictionary key.
1430
1431 If you want to disable this heuristic and see completions from all matchers,
1432 set ``IPCompleter.suppress_competing_matchers = False``.
1433 To disable the heuristic for specific matchers provide a dictionary mapping:
1434 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1435
1436 Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1437 completions to the set of matchers with the highest priority;
1438 this is equivalent to ``IPCompleter.merge_completions`` and
1439 can be beneficial for performance, but will sometimes omit relevant
1440 candidates from matchers further down the priority list.
1441 """,
1442 ).tag(config=True)
1067 1443
1068 merge_completions = Bool(True,
1444 merge_completions = Bool(
1445 True,
1069 1446 help="""Whether to merge completion results into a single list
1070 1447
1071 1448 If False, only the completion results from the first non-empty
1072 1449 completer will be returned.
1073 """
1450
1451 As of version 8.6.0, setting the value to ``False`` is an alias for:
1452 ``IPCompleter.suppress_competing_matchers = True.``.
1453 """,
1454 ).tag(config=True)
1455
1456 disable_matchers = ListTrait(
1457 Unicode(), help="""List of matchers to disable."""
1074 1458 ).tag(config=True)
1075 omit__names = Enum((0,1,2), default_value=2,
1459
1460 omit__names = Enum(
1461 (0, 1, 2),
1462 default_value=2,
1076 1463 help="""Instruct the completer to omit private method names
1077 1464
1078 1465 Specifically, when completing on ``object.<tab>``.
1079 1466
1080 1467 When 2 [default]: all names that start with '_' will be excluded.
1081 1468
1082 1469 When 1: all 'magic' names (``__foo__``) will be excluded.
1083 1470
1084 1471 When 0: nothing will be excluded.
1085 1472 """
1086 1473 ).tag(config=True)
1087 1474 limit_to__all__ = Bool(False,
1088 1475 help="""
1089 1476 DEPRECATED as of version 5.0.
1090 1477
1091 1478 Instruct the completer to use __all__ for the completion
1092 1479
1093 1480 Specifically, when completing on ``object.<tab>``.
1094 1481
1095 1482 When True: only those names in obj.__all__ will be included.
1096 1483
1097 1484 When False [default]: the __all__ attribute is ignored
1098 1485 """,
1099 1486 ).tag(config=True)
1100 1487
1101 1488 profile_completions = Bool(
1102 1489 default_value=False,
1103 1490 help="If True, emit profiling data for completion subsystem using cProfile."
1104 1491 ).tag(config=True)
1105 1492
1106 1493 profiler_output_dir = Unicode(
1107 1494 default_value=".completion_profiles",
1108 1495 help="Template for path at which to output profile data for completions."
1109 1496 ).tag(config=True)
1110 1497
1111 1498 @observe('limit_to__all__')
1112 1499 def _limit_to_all_changed(self, change):
1113 1500 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1114 1501 'value has been deprecated since IPython 5.0, will be made to have '
1115 1502 'no effects and then removed in future version of IPython.',
1116 1503 UserWarning)
1117 1504
1118 1505 def __init__(
1119 1506 self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1120 1507 ):
1121 1508 """IPCompleter() -> completer
1122 1509
1123 1510 Return a completer object.
1124 1511
1125 1512 Parameters
1126 1513 ----------
1127 1514 shell
1128 1515 a pointer to the ipython shell itself. This is needed
1129 1516 because this completer knows about magic functions, and those can
1130 1517 only be accessed via the ipython instance.
1131 1518 namespace : dict, optional
1132 1519 an optional dict where completions are performed.
1133 1520 global_namespace : dict, optional
1134 1521 secondary optional dict for completions, to
1135 1522 handle cases (such as IPython embedded inside functions) where
1136 1523 both Python scopes are visible.
1137 1524 config : Config
1138 1525 traitlet's config object
1139 1526 **kwargs
1140 1527 passed to super class unmodified.
1141 1528 """
1142 1529
1143 1530 self.magic_escape = ESC_MAGIC
1144 1531 self.splitter = CompletionSplitter()
1145 1532
1146 1533 # _greedy_changed() depends on splitter and readline being defined:
1147 1534 super().__init__(
1148 1535 namespace=namespace,
1149 1536 global_namespace=global_namespace,
1150 1537 config=config,
1151 **kwargs
1538 **kwargs,
1152 1539 )
1153 1540
1154 1541 # List where completion matches will be stored
1155 1542 self.matches = []
1156 1543 self.shell = shell
1157 1544 # Regexp to split filenames with spaces in them
1158 1545 self.space_name_re = re.compile(r'([^\\] )')
1159 1546 # Hold a local ref. to glob.glob for speed
1160 1547 self.glob = glob.glob
1161 1548
1162 1549 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1163 1550 # buffers, to avoid completion problems.
1164 1551 term = os.environ.get('TERM','xterm')
1165 1552 self.dumb_terminal = term in ['dumb','emacs']
1166 1553
1167 1554 # Special handling of backslashes needed in win32 platforms
1168 1555 if sys.platform == "win32":
1169 1556 self.clean_glob = self._clean_glob_win32
1170 1557 else:
1171 1558 self.clean_glob = self._clean_glob
1172 1559
1173 1560 #regexp to parse docstring for function signature
1174 1561 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1175 1562 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1176 1563 #use this if positional argument name is also needed
1177 1564 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1178 1565
1179 1566 self.magic_arg_matchers = [
1180 self.magic_config_matches,
1181 self.magic_color_matches,
1567 self.magic_config_matcher,
1568 self.magic_color_matcher,
1182 1569 ]
1183 1570
1184 1571 # This is set externally by InteractiveShell
1185 1572 self.custom_completers = None
1186 1573
1187 1574 # This is a list of names of unicode characters that can be completed
1188 1575 # into their corresponding unicode value. The list is large, so we
1189 1576 # lazily initialize it on first use. Consuming code should access this
1190 1577 # attribute through the `@unicode_names` property.
1191 1578 self._unicode_names = None
1192 1579
1580 self._backslash_combining_matchers = [
1581 self.latex_name_matcher,
1582 self.unicode_name_matcher,
1583 back_latex_name_matcher,
1584 back_unicode_name_matcher,
1585 self.fwd_unicode_matcher,
1586 ]
1587
1588 if not self.backslash_combining_completions:
1589 for matcher in self._backslash_combining_matchers:
1590 self.disable_matchers.append(matcher.matcher_identifier)
1591
1592 if not self.merge_completions:
1593 self.suppress_competing_matchers = True
1594
1193 1595 @property
1194 def matchers(self) -> List[Any]:
1596 def matchers(self) -> List[Matcher]:
1195 1597 """All active matcher routines for completion"""
1196 1598 if self.dict_keys_only:
1197 return [self.dict_key_matches]
1599 return [self.dict_key_matcher]
1198 1600
1199 1601 if self.use_jedi:
1200 1602 return [
1201 1603 *self.custom_matchers,
1202 self.dict_key_matches,
1203 self.file_matches,
1204 self.magic_matches,
1604 *self._backslash_combining_matchers,
1605 *self.magic_arg_matchers,
1606 self.custom_completer_matcher,
1607 self.magic_matcher,
1608 self._jedi_matcher,
1609 self.dict_key_matcher,
1610 self.file_matcher,
1205 1611 ]
1206 1612 else:
1207 1613 return [
1208 1614 *self.custom_matchers,
1209 self.dict_key_matches,
1615 *self._backslash_combining_matchers,
1616 *self.magic_arg_matchers,
1617 self.custom_completer_matcher,
1618 self.dict_key_matcher,
1619 # TODO: convert python_matches to v2 API
1620 self.magic_matcher,
1210 1621 self.python_matches,
1211 self.file_matches,
1212 self.magic_matches,
1213 self.python_func_kw_matches,
1622 self.file_matcher,
1623 self.python_func_kw_matcher,
1214 1624 ]
1215 1625
1216 1626 def all_completions(self, text:str) -> List[str]:
1217 1627 """
1218 1628 Wrapper around the completion methods for the benefit of emacs.
1219 1629 """
1220 1630 prefix = text.rpartition('.')[0]
1221 1631 with provisionalcompleter():
1222 1632 return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
1223 1633 for c in self.completions(text, len(text))]
1224 1634
1225 1635 return self.complete(text)[1]
1226 1636
1227 1637 def _clean_glob(self, text:str):
1228 1638 return self.glob("%s*" % text)
1229 1639
1230 1640 def _clean_glob_win32(self, text:str):
1231 1641 return [f.replace("\\","/")
1232 1642 for f in self.glob("%s*" % text)]
1233 1643
1234 def file_matches(self, text:str)->List[str]:
1644 @context_matcher()
1645 def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
1646 """Same as :any:`file_matches`, but adopted to new Matcher API."""
1647 matches = self.file_matches(context.token)
1648 # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
1649 # starts with `/home/`, `C:\`, etc)
1650 return _convert_matcher_v1_result_to_v2(matches, type="path")
1651
1652 def file_matches(self, text: str) -> List[str]:
1235 1653 """Match filenames, expanding ~USER type strings.
1236 1654
1237 1655 Most of the seemingly convoluted logic in this completer is an
1238 1656 attempt to handle filenames with spaces in them. And yet it's not
1239 1657 quite perfect, because Python's readline doesn't expose all of the
1240 1658 GNU readline details needed for this to be done correctly.
1241 1659
1242 1660 For a filename with a space in it, the printed completions will be
1243 1661 only the parts after what's already been typed (instead of the
1244 1662 full completions, as is normally done). I don't think with the
1245 1663 current (as of Python 2.3) Python readline it's possible to do
1246 better."""
1664 better.
1665
1666 .. deprecated:: 8.6
1667 You can use :meth:`file_matcher` instead.
1668 """
1247 1669
1248 1670 # chars that require escaping with backslash - i.e. chars
1249 1671 # that readline treats incorrectly as delimiters, but we
1250 1672 # don't want to treat as delimiters in filename matching
1251 1673 # when escaped with backslash
1252 1674 if text.startswith('!'):
1253 1675 text = text[1:]
1254 1676 text_prefix = u'!'
1255 1677 else:
1256 1678 text_prefix = u''
1257 1679
1258 1680 text_until_cursor = self.text_until_cursor
1259 1681 # track strings with open quotes
1260 1682 open_quotes = has_open_quotes(text_until_cursor)
1261 1683
1262 1684 if '(' in text_until_cursor or '[' in text_until_cursor:
1263 1685 lsplit = text
1264 1686 else:
1265 1687 try:
1266 1688 # arg_split ~ shlex.split, but with unicode bugs fixed by us
1267 1689 lsplit = arg_split(text_until_cursor)[-1]
1268 1690 except ValueError:
1269 1691 # typically an unmatched ", or backslash without escaped char.
1270 1692 if open_quotes:
1271 1693 lsplit = text_until_cursor.split(open_quotes)[-1]
1272 1694 else:
1273 1695 return []
1274 1696 except IndexError:
1275 1697 # tab pressed on empty line
1276 1698 lsplit = ""
1277 1699
1278 1700 if not open_quotes and lsplit != protect_filename(lsplit):
1279 1701 # if protectables are found, do matching on the whole escaped name
1280 1702 has_protectables = True
1281 1703 text0,text = text,lsplit
1282 1704 else:
1283 1705 has_protectables = False
1284 1706 text = os.path.expanduser(text)
1285 1707
1286 1708 if text == "":
1287 1709 return [text_prefix + protect_filename(f) for f in self.glob("*")]
1288 1710
1289 1711 # Compute the matches from the filesystem
1290 1712 if sys.platform == 'win32':
1291 1713 m0 = self.clean_glob(text)
1292 1714 else:
1293 1715 m0 = self.clean_glob(text.replace('\\', ''))
1294 1716
1295 1717 if has_protectables:
1296 1718 # If we had protectables, we need to revert our changes to the
1297 1719 # beginning of filename so that we don't double-write the part
1298 1720 # of the filename we have so far
1299 1721 len_lsplit = len(lsplit)
1300 1722 matches = [text_prefix + text0 +
1301 1723 protect_filename(f[len_lsplit:]) for f in m0]
1302 1724 else:
1303 1725 if open_quotes:
1304 1726 # if we have a string with an open quote, we don't need to
1305 1727 # protect the names beyond the quote (and we _shouldn't_, as
1306 1728 # it would cause bugs when the filesystem call is made).
1307 1729 matches = m0 if sys.platform == "win32" else\
1308 1730 [protect_filename(f, open_quotes) for f in m0]
1309 1731 else:
1310 1732 matches = [text_prefix +
1311 1733 protect_filename(f) for f in m0]
1312 1734
1313 1735 # Mark directories in input list by appending '/' to their names.
1314 1736 return [x+'/' if os.path.isdir(x) else x for x in matches]
1315 1737
1316 def magic_matches(self, text:str):
1317 """Match magics"""
1738 @context_matcher()
1739 def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
1740 """Match magics."""
1741 text = context.token
1742 matches = self.magic_matches(text)
1743 result = _convert_matcher_v1_result_to_v2(matches, type="magic")
1744 is_magic_prefix = len(text) > 0 and text[0] == "%"
1745 result["suppress"] = is_magic_prefix and bool(result["completions"])
1746 return result
1747
1748 def magic_matches(self, text: str):
1749 """Match magics.
1750
1751 .. deprecated:: 8.6
1752 You can use :meth:`magic_matcher` instead.
1753 """
1318 1754 # Get all shell magics now rather than statically, so magics loaded at
1319 1755 # runtime show up too.
1320 1756 lsm = self.shell.magics_manager.lsmagic()
1321 1757 line_magics = lsm['line']
1322 1758 cell_magics = lsm['cell']
1323 1759 pre = self.magic_escape
1324 1760 pre2 = pre+pre
1325 1761
1326 1762 explicit_magic = text.startswith(pre)
1327 1763
1328 1764 # Completion logic:
1329 1765 # - user gives %%: only do cell magics
1330 1766 # - user gives %: do both line and cell magics
1331 1767 # - no prefix: do both
1332 1768 # In other words, line magics are skipped if the user gives %% explicitly
1333 1769 #
1334 1770 # We also exclude magics that match any currently visible names:
1335 1771 # https://github.com/ipython/ipython/issues/4877, unless the user has
1336 1772 # typed a %:
1337 1773 # https://github.com/ipython/ipython/issues/10754
1338 1774 bare_text = text.lstrip(pre)
1339 1775 global_matches = self.global_matches(bare_text)
1340 1776 if not explicit_magic:
1341 1777 def matches(magic):
1342 1778 """
1343 1779 Filter magics, in particular remove magics that match
1344 1780 a name present in global namespace.
1345 1781 """
1346 1782 return ( magic.startswith(bare_text) and
1347 1783 magic not in global_matches )
1348 1784 else:
1349 1785 def matches(magic):
1350 1786 return magic.startswith(bare_text)
1351 1787
1352 1788 comp = [ pre2+m for m in cell_magics if matches(m)]
1353 1789 if not text.startswith(pre2):
1354 1790 comp += [ pre+m for m in line_magics if matches(m)]
1355 1791
1356 1792 return comp
1357 1793
1358 def magic_config_matches(self, text:str) -> List[str]:
1359 """ Match class names and attributes for %config magic """
1794 @context_matcher()
1795 def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
1796 """Match class names and attributes for %config magic."""
1797 # NOTE: uses `line_buffer` equivalent for compatibility
1798 matches = self.magic_config_matches(context.line_with_cursor)
1799 return _convert_matcher_v1_result_to_v2(matches, type="param")
1800
1801 def magic_config_matches(self, text: str) -> List[str]:
1802 """Match class names and attributes for %config magic.
1803
1804 .. deprecated:: 8.6
1805 You can use :meth:`magic_config_matcher` instead.
1806 """
1360 1807 texts = text.strip().split()
1361 1808
1362 1809 if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
1363 1810 # get all configuration classes
1364 1811 classes = sorted(set([ c for c in self.shell.configurables
1365 1812 if c.__class__.class_traits(config=True)
1366 1813 ]), key=lambda x: x.__class__.__name__)
1367 1814 classnames = [ c.__class__.__name__ for c in classes ]
1368 1815
1369 1816 # return all classnames if config or %config is given
1370 1817 if len(texts) == 1:
1371 1818 return classnames
1372 1819
1373 1820 # match classname
1374 1821 classname_texts = texts[1].split('.')
1375 1822 classname = classname_texts[0]
1376 1823 classname_matches = [ c for c in classnames
1377 1824 if c.startswith(classname) ]
1378 1825
1379 1826 # return matched classes or the matched class with attributes
1380 1827 if texts[1].find('.') < 0:
1381 1828 return classname_matches
1382 1829 elif len(classname_matches) == 1 and \
1383 1830 classname_matches[0] == classname:
1384 1831 cls = classes[classnames.index(classname)].__class__
1385 1832 help = cls.class_get_help()
1386 1833 # strip leading '--' from cl-args:
1387 1834 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
1388 1835 return [ attr.split('=')[0]
1389 1836 for attr in help.strip().splitlines()
1390 1837 if attr.startswith(texts[1]) ]
1391 1838 return []
1392 1839
1393 def magic_color_matches(self, text:str) -> List[str] :
1394 """ Match color schemes for %colors magic"""
1840 @context_matcher()
1841 def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
1842 """Match color schemes for %colors magic."""
1843 # NOTE: uses `line_buffer` equivalent for compatibility
1844 matches = self.magic_color_matches(context.line_with_cursor)
1845 return _convert_matcher_v1_result_to_v2(matches, type="param")
1846
1847 def magic_color_matches(self, text: str) -> List[str]:
1848 """Match color schemes for %colors magic.
1849
1850 .. deprecated:: 8.6
1851 You can use :meth:`magic_color_matcher` instead.
1852 """
1395 1853 texts = text.split()
1396 1854 if text.endswith(' '):
1397 1855 # .split() strips off the trailing whitespace. Add '' back
1398 1856 # so that: '%colors ' -> ['%colors', '']
1399 1857 texts.append('')
1400 1858
1401 1859 if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
1402 1860 prefix = texts[1]
1403 1861 return [ color for color in InspectColors.keys()
1404 1862 if color.startswith(prefix) ]
1405 1863 return []
1406 1864
1407 def _jedi_matches(self, cursor_column:int, cursor_line:int, text:str) -> Iterable[Any]:
1865 @context_matcher(identifier="IPCompleter.jedi_matcher")
1866 def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
1867 matches = self._jedi_matches(
1868 cursor_column=context.cursor_position,
1869 cursor_line=context.cursor_line,
1870 text=context.full_text,
1871 )
1872 return {
1873 "completions": matches,
1874 # static analysis should not suppress other matchers
1875 "suppress": False,
1876 }
1877
1878 def _jedi_matches(
1879 self, cursor_column: int, cursor_line: int, text: str
1880 ) -> Iterable[_JediCompletionLike]:
1408 1881 """
1409 Return a list of :any:`jedi.api.Completions` object from a ``text`` and
1882 Return a list of :any:`jedi.api.Completion`s object from a ``text`` and
1410 1883 cursor position.
1411 1884
1412 1885 Parameters
1413 1886 ----------
1414 1887 cursor_column : int
1415 1888 column position of the cursor in ``text``, 0-indexed.
1416 1889 cursor_line : int
1417 1890 line position of the cursor in ``text``, 0-indexed
1418 1891 text : str
1419 1892 text to complete
1420 1893
1421 1894 Notes
1422 1895 -----
1423 1896 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
1424 1897 object containing a string with the Jedi debug information attached.
1898
1899 .. deprecated:: 8.6
1900 You can use :meth:`_jedi_matcher` instead.
1425 1901 """
1426 1902 namespaces = [self.namespace]
1427 1903 if self.global_namespace is not None:
1428 1904 namespaces.append(self.global_namespace)
1429 1905
1430 1906 completion_filter = lambda x:x
1431 1907 offset = cursor_to_position(text, cursor_line, cursor_column)
1432 1908 # filter output if we are completing for object members
1433 1909 if offset:
1434 1910 pre = text[offset-1]
1435 1911 if pre == '.':
1436 1912 if self.omit__names == 2:
1437 1913 completion_filter = lambda c:not c.name.startswith('_')
1438 1914 elif self.omit__names == 1:
1439 1915 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
1440 1916 elif self.omit__names == 0:
1441 1917 completion_filter = lambda x:x
1442 1918 else:
1443 1919 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
1444 1920
1445 1921 interpreter = jedi.Interpreter(text[:offset], namespaces)
1446 1922 try_jedi = True
1447 1923
1448 1924 try:
1449 1925 # find the first token in the current tree -- if it is a ' or " then we are in a string
1450 1926 completing_string = False
1451 1927 try:
1452 1928 first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
1453 1929 except StopIteration:
1454 1930 pass
1455 1931 else:
1456 1932 # note the value may be ', ", or it may also be ''' or """, or
1457 1933 # in some cases, """what/you/typed..., but all of these are
1458 1934 # strings.
1459 1935 completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
1460 1936
1461 1937 # if we are in a string jedi is likely not the right candidate for
1462 1938 # now. Skip it.
1463 1939 try_jedi = not completing_string
1464 1940 except Exception as e:
1465 1941 # many of things can go wrong, we are using private API just don't crash.
1466 1942 if self.debug:
1467 1943 print("Error detecting if completing a non-finished string :", e, '|')
1468 1944
1469 1945 if not try_jedi:
1470 1946 return []
1471 1947 try:
1472 1948 return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
1473 1949 except Exception as e:
1474 1950 if self.debug:
1475 1951 return [_FakeJediCompletion('Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""' % (e))]
1476 1952 else:
1477 1953 return []
1478 1954
1479 1955 def python_matches(self, text:str)->List[str]:
1480 1956 """Match attributes or global python names"""
1481 1957 if "." in text:
1482 1958 try:
1483 1959 matches = self.attr_matches(text)
1484 1960 if text.endswith('.') and self.omit__names:
1485 1961 if self.omit__names == 1:
1486 1962 # true if txt is _not_ a __ name, false otherwise:
1487 1963 no__name = (lambda txt:
1488 1964 re.match(r'.*\.__.*?__',txt) is None)
1489 1965 else:
1490 1966 # true if txt is _not_ a _ name, false otherwise:
1491 1967 no__name = (lambda txt:
1492 1968 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
1493 1969 matches = filter(no__name, matches)
1494 1970 except NameError:
1495 1971 # catches <undefined attributes>.<tab>
1496 1972 matches = []
1497 1973 else:
1498 1974 matches = self.global_matches(text)
1499 1975 return matches
1500 1976
1501 1977 def _default_arguments_from_docstring(self, doc):
1502 1978 """Parse the first line of docstring for call signature.
1503 1979
1504 1980 Docstring should be of the form 'min(iterable[, key=func])\n'.
1505 1981 It can also parse cython docstring of the form
1506 1982 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
1507 1983 """
1508 1984 if doc is None:
1509 1985 return []
1510 1986
1511 1987 #care only the firstline
1512 1988 line = doc.lstrip().splitlines()[0]
1513 1989
1514 1990 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1515 1991 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
1516 1992 sig = self.docstring_sig_re.search(line)
1517 1993 if sig is None:
1518 1994 return []
1519 1995 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
1520 1996 sig = sig.groups()[0].split(',')
1521 1997 ret = []
1522 1998 for s in sig:
1523 1999 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1524 2000 ret += self.docstring_kwd_re.findall(s)
1525 2001 return ret
1526 2002
1527 2003 def _default_arguments(self, obj):
1528 2004 """Return the list of default arguments of obj if it is callable,
1529 2005 or empty list otherwise."""
1530 2006 call_obj = obj
1531 2007 ret = []
1532 2008 if inspect.isbuiltin(obj):
1533 2009 pass
1534 2010 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
1535 2011 if inspect.isclass(obj):
1536 2012 #for cython embedsignature=True the constructor docstring
1537 2013 #belongs to the object itself not __init__
1538 2014 ret += self._default_arguments_from_docstring(
1539 2015 getattr(obj, '__doc__', ''))
1540 2016 # for classes, check for __init__,__new__
1541 2017 call_obj = (getattr(obj, '__init__', None) or
1542 2018 getattr(obj, '__new__', None))
1543 2019 # for all others, check if they are __call__able
1544 2020 elif hasattr(obj, '__call__'):
1545 2021 call_obj = obj.__call__
1546 2022 ret += self._default_arguments_from_docstring(
1547 2023 getattr(call_obj, '__doc__', ''))
1548 2024
1549 2025 _keeps = (inspect.Parameter.KEYWORD_ONLY,
1550 2026 inspect.Parameter.POSITIONAL_OR_KEYWORD)
1551 2027
1552 2028 try:
1553 2029 sig = inspect.signature(obj)
1554 2030 ret.extend(k for k, v in sig.parameters.items() if
1555 2031 v.kind in _keeps)
1556 2032 except ValueError:
1557 2033 pass
1558 2034
1559 2035 return list(set(ret))
1560 2036
2037 @context_matcher()
2038 def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2039 """Match named parameters (kwargs) of the last open function."""
2040 matches = self.python_func_kw_matches(context.token)
2041 return _convert_matcher_v1_result_to_v2(matches, type="param")
2042
1561 2043 def python_func_kw_matches(self, text):
1562 """Match named parameters (kwargs) of the last open function"""
2044 """Match named parameters (kwargs) of the last open function.
2045
2046 .. deprecated:: 8.6
2047 You can use :meth:`python_func_kw_matcher` instead.
2048 """
1563 2049
1564 2050 if "." in text: # a parameter cannot be dotted
1565 2051 return []
1566 2052 try: regexp = self.__funcParamsRegex
1567 2053 except AttributeError:
1568 2054 regexp = self.__funcParamsRegex = re.compile(r'''
1569 2055 '.*?(?<!\\)' | # single quoted strings or
1570 2056 ".*?(?<!\\)" | # double quoted strings or
1571 2057 \w+ | # identifier
1572 2058 \S # other characters
1573 2059 ''', re.VERBOSE | re.DOTALL)
1574 2060 # 1. find the nearest identifier that comes before an unclosed
1575 2061 # parenthesis before the cursor
1576 2062 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
1577 2063 tokens = regexp.findall(self.text_until_cursor)
1578 2064 iterTokens = reversed(tokens); openPar = 0
1579 2065
1580 2066 for token in iterTokens:
1581 2067 if token == ')':
1582 2068 openPar -= 1
1583 2069 elif token == '(':
1584 2070 openPar += 1
1585 2071 if openPar > 0:
1586 2072 # found the last unclosed parenthesis
1587 2073 break
1588 2074 else:
1589 2075 return []
1590 2076 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
1591 2077 ids = []
1592 2078 isId = re.compile(r'\w+$').match
1593 2079
1594 2080 while True:
1595 2081 try:
1596 2082 ids.append(next(iterTokens))
1597 2083 if not isId(ids[-1]):
1598 2084 ids.pop(); break
1599 2085 if not next(iterTokens) == '.':
1600 2086 break
1601 2087 except StopIteration:
1602 2088 break
1603 2089
1604 2090 # Find all named arguments already assigned to, as to avoid suggesting
1605 2091 # them again
1606 2092 usedNamedArgs = set()
1607 2093 par_level = -1
1608 2094 for token, next_token in zip(tokens, tokens[1:]):
1609 2095 if token == '(':
1610 2096 par_level += 1
1611 2097 elif token == ')':
1612 2098 par_level -= 1
1613 2099
1614 2100 if par_level != 0:
1615 2101 continue
1616 2102
1617 2103 if next_token != '=':
1618 2104 continue
1619 2105
1620 2106 usedNamedArgs.add(token)
1621 2107
1622 2108 argMatches = []
1623 2109 try:
1624 2110 callableObj = '.'.join(ids[::-1])
1625 2111 namedArgs = self._default_arguments(eval(callableObj,
1626 2112 self.namespace))
1627 2113
1628 2114 # Remove used named arguments from the list, no need to show twice
1629 2115 for namedArg in set(namedArgs) - usedNamedArgs:
1630 2116 if namedArg.startswith(text):
1631 2117 argMatches.append("%s=" %namedArg)
1632 2118 except:
1633 2119 pass
1634 2120
1635 2121 return argMatches
1636 2122
1637 2123 @staticmethod
1638 2124 def _get_keys(obj: Any) -> List[Any]:
1639 2125 # Objects can define their own completions by defining an
1640 2126 # _ipy_key_completions_() method.
1641 2127 method = get_real_method(obj, '_ipython_key_completions_')
1642 2128 if method is not None:
1643 2129 return method()
1644 2130
1645 2131 # Special case some common in-memory dict-like types
1646 2132 if isinstance(obj, dict) or\
1647 2133 _safe_isinstance(obj, 'pandas', 'DataFrame'):
1648 2134 try:
1649 2135 return list(obj.keys())
1650 2136 except Exception:
1651 2137 return []
1652 2138 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
1653 2139 _safe_isinstance(obj, 'numpy', 'void'):
1654 2140 return obj.dtype.names or []
1655 2141 return []
1656 2142
1657 def dict_key_matches(self, text:str) -> List[str]:
1658 "Match string keys in a dictionary, after e.g. 'foo[' "
2143 @context_matcher()
2144 def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2145 """Match string keys in a dictionary, after e.g. ``foo[``."""
2146 matches = self.dict_key_matches(context.token)
2147 return _convert_matcher_v1_result_to_v2(
2148 matches, type="dict key", suppress_if_matches=True
2149 )
2150
2151 def dict_key_matches(self, text: str) -> List[str]:
2152 """Match string keys in a dictionary, after e.g. ``foo[``.
1659 2153
2154 .. deprecated:: 8.6
2155 You can use :meth:`dict_key_matcher` instead.
2156 """
1660 2157
1661 2158 if self.__dict_key_regexps is not None:
1662 2159 regexps = self.__dict_key_regexps
1663 2160 else:
1664 2161 dict_key_re_fmt = r'''(?x)
1665 2162 ( # match dict-referring expression wrt greedy setting
1666 2163 %s
1667 2164 )
1668 2165 \[ # open bracket
1669 2166 \s* # and optional whitespace
1670 2167 # Capture any number of str-like objects (e.g. "a", "b", 'c')
1671 2168 ((?:[uUbB]? # string prefix (r not handled)
1672 2169 (?:
1673 2170 '(?:[^']|(?<!\\)\\')*'
1674 2171 |
1675 2172 "(?:[^"]|(?<!\\)\\")*"
1676 2173 )
1677 2174 \s*,\s*
1678 2175 )*)
1679 2176 ([uUbB]? # string prefix (r not handled)
1680 2177 (?: # unclosed string
1681 2178 '(?:[^']|(?<!\\)\\')*
1682 2179 |
1683 2180 "(?:[^"]|(?<!\\)\\")*
1684 2181 )
1685 2182 )?
1686 2183 $
1687 2184 '''
1688 2185 regexps = self.__dict_key_regexps = {
1689 2186 False: re.compile(dict_key_re_fmt % r'''
1690 2187 # identifiers separated by .
1691 2188 (?!\d)\w+
1692 2189 (?:\.(?!\d)\w+)*
1693 2190 '''),
1694 2191 True: re.compile(dict_key_re_fmt % '''
1695 2192 .+
1696 2193 ''')
1697 2194 }
1698 2195
1699 2196 match = regexps[self.greedy].search(self.text_until_cursor)
1700 2197
1701 2198 if match is None:
1702 2199 return []
1703 2200
1704 2201 expr, prefix0, prefix = match.groups()
1705 2202 try:
1706 2203 obj = eval(expr, self.namespace)
1707 2204 except Exception:
1708 2205 try:
1709 2206 obj = eval(expr, self.global_namespace)
1710 2207 except Exception:
1711 2208 return []
1712 2209
1713 2210 keys = self._get_keys(obj)
1714 2211 if not keys:
1715 2212 return keys
1716 2213
1717 2214 extra_prefix = eval(prefix0) if prefix0 != '' else None
1718 2215
1719 2216 closing_quote, token_offset, matches = match_dict_keys(keys, prefix, self.splitter.delims, extra_prefix=extra_prefix)
1720 2217 if not matches:
1721 2218 return matches
1722 2219
1723 2220 # get the cursor position of
1724 2221 # - the text being completed
1725 2222 # - the start of the key text
1726 2223 # - the start of the completion
1727 2224 text_start = len(self.text_until_cursor) - len(text)
1728 2225 if prefix:
1729 2226 key_start = match.start(3)
1730 2227 completion_start = key_start + token_offset
1731 2228 else:
1732 2229 key_start = completion_start = match.end()
1733 2230
1734 2231 # grab the leading prefix, to make sure all completions start with `text`
1735 2232 if text_start > key_start:
1736 2233 leading = ''
1737 2234 else:
1738 2235 leading = text[text_start:completion_start]
1739 2236
1740 2237 # the index of the `[` character
1741 2238 bracket_idx = match.end(1)
1742 2239
1743 2240 # append closing quote and bracket as appropriate
1744 2241 # this is *not* appropriate if the opening quote or bracket is outside
1745 2242 # the text given to this method
1746 2243 suf = ''
1747 2244 continuation = self.line_buffer[len(self.text_until_cursor):]
1748 2245 if key_start > text_start and closing_quote:
1749 2246 # quotes were opened inside text, maybe close them
1750 2247 if continuation.startswith(closing_quote):
1751 2248 continuation = continuation[len(closing_quote):]
1752 2249 else:
1753 2250 suf += closing_quote
1754 2251 if bracket_idx > text_start:
1755 2252 # brackets were opened inside text, maybe close them
1756 2253 if not continuation.startswith(']'):
1757 2254 suf += ']'
1758 2255
1759 2256 return [leading + k + suf for k in matches]
1760 2257
2258 @context_matcher()
2259 def unicode_name_matcher(self, context: CompletionContext):
2260 """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2261 fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2262 return _convert_matcher_v1_result_to_v2(
2263 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2264 )
2265
1761 2266 @staticmethod
1762 def unicode_name_matches(text:str) -> Tuple[str, List[str]] :
2267 def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
1763 2268 """Match Latex-like syntax for unicode characters base
1764 2269 on the name of the character.
1765 2270
1766 2271 This does ``\\GREEK SMALL LETTER ETA`` -> ``η``
1767 2272
1768 2273 Works only on valid python 3 identifier, or on combining characters that
1769 2274 will combine to form a valid identifier.
1770 2275 """
1771 2276 slashpos = text.rfind('\\')
1772 2277 if slashpos > -1:
1773 2278 s = text[slashpos+1:]
1774 2279 try :
1775 2280 unic = unicodedata.lookup(s)
1776 2281 # allow combining chars
1777 2282 if ('a'+unic).isidentifier():
1778 2283 return '\\'+s,[unic]
1779 2284 except KeyError:
1780 2285 pass
1781 2286 return '', []
1782 2287
2288 @context_matcher()
2289 def latex_name_matcher(self, context: CompletionContext):
2290 """Match Latex syntax for unicode characters.
1783 2291
1784 def latex_matches(self, text:str) -> Tuple[str, Sequence[str]]:
2292 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``α``
2293 """
2294 fragment, matches = self.latex_matches(context.text_until_cursor)
2295 return _convert_matcher_v1_result_to_v2(
2296 matches, type="latex", fragment=fragment, suppress_if_matches=True
2297 )
2298
2299 def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
1785 2300 """Match Latex syntax for unicode characters.
1786 2301
1787 2302 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``α``
2303
2304 .. deprecated:: 8.6
2305 You can use :meth:`latex_name_matcher` instead.
1788 2306 """
1789 2307 slashpos = text.rfind('\\')
1790 2308 if slashpos > -1:
1791 2309 s = text[slashpos:]
1792 2310 if s in latex_symbols:
1793 2311 # Try to complete a full latex symbol to unicode
1794 2312 # \\alpha -> α
1795 2313 return s, [latex_symbols[s]]
1796 2314 else:
1797 2315 # If a user has partially typed a latex symbol, give them
1798 2316 # a full list of options \al -> [\aleph, \alpha]
1799 2317 matches = [k for k in latex_symbols if k.startswith(s)]
1800 2318 if matches:
1801 2319 return s, matches
1802 2320 return '', ()
1803 2321
2322 @context_matcher()
2323 def custom_completer_matcher(self, context):
2324 """Dispatch custom completer.
2325
2326 If a match is found, suppresses all other matchers except for Jedi.
2327 """
2328 matches = self.dispatch_custom_completer(context.token) or []
2329 result = _convert_matcher_v1_result_to_v2(
2330 matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2331 )
2332 result["ordered"] = True
2333 result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2334 return result
2335
1804 2336 def dispatch_custom_completer(self, text):
2337 """
2338 .. deprecated:: 8.6
2339 You can use :meth:`custom_completer_matcher` instead.
2340 """
1805 2341 if not self.custom_completers:
1806 2342 return
1807 2343
1808 2344 line = self.line_buffer
1809 2345 if not line.strip():
1810 2346 return None
1811 2347
1812 2348 # Create a little structure to pass all the relevant information about
1813 2349 # the current completion to any custom completer.
1814 2350 event = SimpleNamespace()
1815 2351 event.line = line
1816 2352 event.symbol = text
1817 2353 cmd = line.split(None,1)[0]
1818 2354 event.command = cmd
1819 2355 event.text_until_cursor = self.text_until_cursor
1820 2356
1821 2357 # for foo etc, try also to find completer for %foo
1822 2358 if not cmd.startswith(self.magic_escape):
1823 2359 try_magic = self.custom_completers.s_matches(
1824 2360 self.magic_escape + cmd)
1825 2361 else:
1826 2362 try_magic = []
1827 2363
1828 2364 for c in itertools.chain(self.custom_completers.s_matches(cmd),
1829 2365 try_magic,
1830 2366 self.custom_completers.flat_matches(self.text_until_cursor)):
1831 2367 try:
1832 2368 res = c(event)
1833 2369 if res:
1834 2370 # first, try case sensitive match
1835 2371 withcase = [r for r in res if r.startswith(text)]
1836 2372 if withcase:
1837 2373 return withcase
1838 2374 # if none, then case insensitive ones are ok too
1839 2375 text_low = text.lower()
1840 2376 return [r for r in res if r.lower().startswith(text_low)]
1841 2377 except TryNext:
1842 2378 pass
1843 2379 except KeyboardInterrupt:
1844 2380 """
1845 2381 If custom completer take too long,
1846 2382 let keyboard interrupt abort and return nothing.
1847 2383 """
1848 2384 break
1849 2385
1850 2386 return None
1851 2387
1852 2388 def completions(self, text: str, offset: int)->Iterator[Completion]:
1853 2389 """
1854 2390 Returns an iterator over the possible completions
1855 2391
1856 2392 .. warning::
1857 2393
1858 2394 Unstable
1859 2395
1860 2396 This function is unstable, API may change without warning.
1861 2397 It will also raise unless use in proper context manager.
1862 2398
1863 2399 Parameters
1864 2400 ----------
1865 2401 text : str
1866 2402 Full text of the current input, multi line string.
1867 2403 offset : int
1868 2404 Integer representing the position of the cursor in ``text``. Offset
1869 2405 is 0-based indexed.
1870 2406
1871 2407 Yields
1872 2408 ------
1873 2409 Completion
1874 2410
1875 2411 Notes
1876 2412 -----
1877 2413 The cursor on a text can either be seen as being "in between"
1878 2414 characters or "On" a character depending on the interface visible to
1879 2415 the user. For consistency the cursor being on "in between" characters X
1880 2416 and Y is equivalent to the cursor being "on" character Y, that is to say
1881 2417 the character the cursor is on is considered as being after the cursor.
1882 2418
1883 2419 Combining characters may span more that one position in the
1884 2420 text.
1885 2421
1886 2422 .. note::
1887 2423
1888 2424 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
1889 2425 fake Completion token to distinguish completion returned by Jedi
1890 2426 and usual IPython completion.
1891 2427
1892 2428 .. note::
1893 2429
1894 2430 Completions are not completely deduplicated yet. If identical
1895 2431 completions are coming from different sources this function does not
1896 2432 ensure that each completion object will only be present once.
1897 2433 """
1898 2434 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
1899 2435 "It may change without warnings. "
1900 2436 "Use in corresponding context manager.",
1901 2437 category=ProvisionalCompleterWarning, stacklevel=2)
1902 2438
1903 2439 seen = set()
1904 2440 profiler:Optional[cProfile.Profile]
1905 2441 try:
1906 2442 if self.profile_completions:
1907 2443 import cProfile
1908 2444 profiler = cProfile.Profile()
1909 2445 profiler.enable()
1910 2446 else:
1911 2447 profiler = None
1912 2448
1913 2449 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
1914 2450 if c and (c in seen):
1915 2451 continue
1916 2452 yield c
1917 2453 seen.add(c)
1918 2454 except KeyboardInterrupt:
1919 2455 """if completions take too long and users send keyboard interrupt,
1920 2456 do not crash and return ASAP. """
1921 2457 pass
1922 2458 finally:
1923 2459 if profiler is not None:
1924 2460 profiler.disable()
1925 2461 ensure_dir_exists(self.profiler_output_dir)
1926 2462 output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
1927 2463 print("Writing profiler output to", output_path)
1928 2464 profiler.dump_stats(output_path)
1929 2465
1930 2466 def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
1931 2467 """
1932 2468 Core completion module.Same signature as :any:`completions`, with the
1933 2469 extra `timeout` parameter (in seconds).
1934 2470
1935 2471 Computing jedi's completion ``.type`` can be quite expensive (it is a
1936 2472 lazy property) and can require some warm-up, more warm up than just
1937 2473 computing the ``name`` of a completion. The warm-up can be :
1938 2474
1939 2475 - Long warm-up the first time a module is encountered after
1940 2476 install/update: actually build parse/inference tree.
1941 2477
1942 2478 - first time the module is encountered in a session: load tree from
1943 2479 disk.
1944 2480
1945 2481 We don't want to block completions for tens of seconds so we give the
1946 2482 completer a "budget" of ``_timeout`` seconds per invocation to compute
1947 2483 completions types, the completions that have not yet been computed will
1948 2484 be marked as "unknown" an will have a chance to be computed next round
1949 2485 are things get cached.
1950 2486
1951 2487 Keep in mind that Jedi is not the only thing treating the completion so
1952 2488 keep the timeout short-ish as if we take more than 0.3 second we still
1953 2489 have lots of processing to do.
1954 2490
1955 2491 """
1956 2492 deadline = time.monotonic() + _timeout
1957 2493
1958
1959 2494 before = full_text[:offset]
1960 2495 cursor_line, cursor_column = position_to_cursor(full_text, offset)
1961 2496
1962 matched_text, matches, matches_origin, jedi_matches = self._complete(
1963 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column)
2497 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2498
2499 results = self._complete(
2500 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
2501 )
2502 non_jedi_results: Dict[str, SimpleMatcherResult] = {
2503 identifier: result
2504 for identifier, result in results.items()
2505 if identifier != jedi_matcher_id
2506 }
2507
2508 jedi_matches = (
2509 cast(results[jedi_matcher_id], _JediMatcherResult)["completions"]
2510 if jedi_matcher_id in results
2511 else ()
2512 )
1964 2513
1965 2514 iter_jm = iter(jedi_matches)
1966 2515 if _timeout:
1967 2516 for jm in iter_jm:
1968 2517 try:
1969 2518 type_ = jm.type
1970 2519 except Exception:
1971 2520 if self.debug:
1972 2521 print("Error in Jedi getting type of ", jm)
1973 2522 type_ = None
1974 2523 delta = len(jm.name_with_symbols) - len(jm.complete)
1975 2524 if type_ == 'function':
1976 2525 signature = _make_signature(jm)
1977 2526 else:
1978 2527 signature = ''
1979 2528 yield Completion(start=offset - delta,
1980 2529 end=offset,
1981 2530 text=jm.name_with_symbols,
1982 2531 type=type_,
1983 2532 signature=signature,
1984 2533 _origin='jedi')
1985 2534
1986 2535 if time.monotonic() > deadline:
1987 2536 break
1988 2537
1989 2538 for jm in iter_jm:
1990 2539 delta = len(jm.name_with_symbols) - len(jm.complete)
1991 yield Completion(start=offset - delta,
1992 end=offset,
1993 text=jm.name_with_symbols,
1994 type='<unknown>', # don't compute type for speed
1995 _origin='jedi',
1996 signature='')
1997
1998
1999 start_offset = before.rfind(matched_text)
2540 yield Completion(
2541 start=offset - delta,
2542 end=offset,
2543 text=jm.name_with_symbols,
2544 type=_UNKNOWN_TYPE, # don't compute type for speed
2545 _origin="jedi",
2546 signature="",
2547 )
2000 2548
2001 2549 # TODO:
2002 2550 # Suppress this, right now just for debug.
2003 if jedi_matches and matches and self.debug:
2004 yield Completion(start=start_offset, end=offset, text='--jedi/ipython--',
2005 _origin='debug', type='none', signature='')
2551 if jedi_matches and non_jedi_results and self.debug:
2552 some_start_offset = before.rfind(
2553 next(iter(non_jedi_results.values()))["matched_fragment"]
2554 )
2555 yield Completion(
2556 start=some_start_offset,
2557 end=offset,
2558 text="--jedi/ipython--",
2559 _origin="debug",
2560 type="none",
2561 signature="",
2562 )
2006 2563
2007 # I'm unsure if this is always true, so let's assert and see if it
2008 # crash
2009 assert before.endswith(matched_text)
2010 for m, t in zip(matches, matches_origin):
2011 yield Completion(start=start_offset, end=offset, text=m, _origin=t, signature='', type='<unknown>')
2564 ordered = []
2565 sortable = []
2566
2567 for origin, result in non_jedi_results.items():
2568 matched_text = result["matched_fragment"]
2569 start_offset = before.rfind(matched_text)
2570 is_ordered = result.get("ordered", False)
2571 container = ordered if is_ordered else sortable
2572
2573 # I'm unsure if this is always true, so let's assert and see if it
2574 # crash
2575 assert before.endswith(matched_text)
2576
2577 for simple_completion in result["completions"]:
2578 completion = Completion(
2579 start=start_offset,
2580 end=offset,
2581 text=simple_completion.text,
2582 _origin=origin,
2583 signature="",
2584 type=simple_completion.type or _UNKNOWN_TYPE,
2585 )
2586 container.append(completion)
2012 2587
2588 yield from list(self._deduplicate(ordered + self._sort(sortable)))[
2589 :MATCHES_LIMIT
2590 ]
2013 2591
2014 2592 def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
2015 2593 """Find completions for the given text and line context.
2016 2594
2017 2595 Note that both the text and the line_buffer are optional, but at least
2018 2596 one of them must be given.
2019 2597
2020 2598 Parameters
2021 2599 ----------
2022 2600 text : string, optional
2023 2601 Text to perform the completion on. If not given, the line buffer
2024 2602 is split using the instance's CompletionSplitter object.
2025 2603 line_buffer : string, optional
2026 2604 If not given, the completer attempts to obtain the current line
2027 2605 buffer via readline. This keyword allows clients which are
2028 2606 requesting for text completions in non-readline contexts to inform
2029 2607 the completer of the entire text.
2030 2608 cursor_pos : int, optional
2031 2609 Index of the cursor in the full line buffer. Should be provided by
2032 2610 remote frontends where kernel has no access to frontend state.
2033 2611
2034 2612 Returns
2035 2613 -------
2036 2614 Tuple of two items:
2037 2615 text : str
2038 2616 Text that was actually used in the completion.
2039 2617 matches : list
2040 2618 A list of completion matches.
2041 2619
2042 2620 Notes
2043 2621 -----
2044 2622 This API is likely to be deprecated and replaced by
2045 2623 :any:`IPCompleter.completions` in the future.
2046 2624
2047 2625 """
2048 2626 warnings.warn('`Completer.complete` is pending deprecation since '
2049 2627 'IPython 6.0 and will be replaced by `Completer.completions`.',
2050 2628 PendingDeprecationWarning)
2051 2629 # potential todo, FOLD the 3rd throw away argument of _complete
2052 2630 # into the first 2 one.
2053 return self._complete(line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0)[:2]
2631 # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
2632 # TODO: should we deprecate now, or does it stay?
2633
2634 results = self._complete(
2635 line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
2636 )
2637
2638 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2639
2640 return self._arrange_and_extract(
2641 results,
2642 # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
2643 skip_matchers={jedi_matcher_id},
2644 # this API does not support different start/end positions (fragments of token).
2645 abort_if_offset_changes=True,
2646 )
2647
2648 def _arrange_and_extract(
2649 self,
2650 results: Dict[str, MatcherResult],
2651 skip_matchers: Set[str],
2652 abort_if_offset_changes: bool,
2653 ):
2654
2655 sortable = []
2656 ordered = []
2657 most_recent_fragment = None
2658 for identifier, result in results.items():
2659 if identifier in skip_matchers:
2660 continue
2661 if not result["completions"]:
2662 continue
2663 if not most_recent_fragment:
2664 most_recent_fragment = result["matched_fragment"]
2665 if (
2666 abort_if_offset_changes
2667 and result["matched_fragment"] != most_recent_fragment
2668 ):
2669 break
2670 if result.get("ordered", False):
2671 ordered.extend(result["completions"])
2672 else:
2673 sortable.extend(result["completions"])
2674
2675 if not most_recent_fragment:
2676 most_recent_fragment = "" # to satisfy typechecker (and just in case)
2677
2678 return most_recent_fragment, [
2679 m.text for m in self._deduplicate(ordered + self._sort(sortable))
2680 ]
2054 2681
2055 2682 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
2056 2683 full_text=None) -> _CompleteResult:
2057 2684 """
2058 2685 Like complete but can also returns raw jedi completions as well as the
2059 2686 origin of the completion text. This could (and should) be made much
2060 2687 cleaner but that will be simpler once we drop the old (and stateful)
2061 2688 :any:`complete` API.
2062 2689
2063 2690 With current provisional API, cursor_pos act both (depending on the
2064 2691 caller) as the offset in the ``text`` or ``line_buffer``, or as the
2065 2692 ``column`` when passing multiline strings this could/should be renamed
2066 2693 but would add extra noise.
2067 2694
2068 2695 Parameters
2069 2696 ----------
2070 2697 cursor_line
2071 2698 Index of the line the cursor is on. 0 indexed.
2072 2699 cursor_pos
2073 2700 Position of the cursor in the current line/line_buffer/text. 0
2074 2701 indexed.
2075 2702 line_buffer : optional, str
2076 2703 The current line the cursor is in, this is mostly due to legacy
2077 2704 reason that readline could only give a us the single current line.
2078 2705 Prefer `full_text`.
2079 2706 text : str
2080 2707 The current "token" the cursor is in, mostly also for historical
2081 2708 reasons. as the completer would trigger only after the current line
2082 2709 was parsed.
2083 2710 full_text : str
2084 2711 Full text of the current cell.
2085 2712
2086 2713 Returns
2087 2714 -------
2088 A tuple of N elements which are (likely):
2089 matched_text: ? the text that the complete matched
2090 matches: list of completions ?
2091 matches_origin: ? list same length as matches, and where each completion came from
2092 jedi_matches: list of Jedi matches, have it's own structure.
2715 An ordered dictionary where keys are identifiers of completion
2716 matchers and values are ``MatcherResult``s.
2093 2717 """
2094 2718
2095
2096 2719 # if the cursor position isn't given, the only sane assumption we can
2097 2720 # make is that it's at the end of the line (the common case)
2098 2721 if cursor_pos is None:
2099 2722 cursor_pos = len(line_buffer) if text is None else len(text)
2100 2723
2101 2724 if self.use_main_ns:
2102 2725 self.namespace = __main__.__dict__
2103 2726
2104 2727 # if text is either None or an empty string, rely on the line buffer
2105 2728 if (not line_buffer) and full_text:
2106 2729 line_buffer = full_text.split('\n')[cursor_line]
2107 if not text: # issue #11508: check line_buffer before calling split_line
2108 text = self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ''
2109
2110 if self.backslash_combining_completions:
2111 # allow deactivation of these on windows.
2112 base_text = text if not line_buffer else line_buffer[:cursor_pos]
2113
2114 for meth in (self.latex_matches,
2115 self.unicode_name_matches,
2116 back_latex_name_matches,
2117 back_unicode_name_matches,
2118 self.fwd_unicode_match):
2119 name_text, name_matches = meth(base_text)
2120 if name_text:
2121 return _CompleteResult(name_text, name_matches[:MATCHES_LIMIT], \
2122 [meth.__qualname__]*min(len(name_matches), MATCHES_LIMIT), ())
2123
2730 if not text: # issue #11508: check line_buffer before calling split_line
2731 text = (
2732 self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
2733 )
2124 2734
2125 2735 # If no line buffer is given, assume the input text is all there was
2126 2736 if line_buffer is None:
2127 2737 line_buffer = text
2128 2738
2739 # deprecated - do not use `line_buffer` in new code.
2129 2740 self.line_buffer = line_buffer
2130 2741 self.text_until_cursor = self.line_buffer[:cursor_pos]
2131 2742
2132 # Do magic arg matches
2133 for matcher in self.magic_arg_matchers:
2134 matches = list(matcher(line_buffer))[:MATCHES_LIMIT]
2135 if matches:
2136 origins = [matcher.__qualname__] * len(matches)
2137 return _CompleteResult(text, matches, origins, ())
2743 if not full_text:
2744 full_text = line_buffer
2745
2746 context = CompletionContext(
2747 full_text=full_text,
2748 cursor_position=cursor_pos,
2749 cursor_line=cursor_line,
2750 token=text,
2751 limit=MATCHES_LIMIT,
2752 )
2138 2753
2139 2754 # Start with a clean slate of completions
2140 matches = []
2755 results = {}
2141 2756
2142 # FIXME: we should extend our api to return a dict with completions for
2143 # different types of objects. The rlcomplete() method could then
2144 # simply collapse the dict into a list for readline, but we'd have
2145 # richer completion semantics in other environments.
2146 is_magic_prefix = len(text) > 0 and text[0] == "%"
2147 completions: Iterable[Any] = []
2148 if self.use_jedi and not is_magic_prefix:
2149 if not full_text:
2150 full_text = line_buffer
2151 completions = self._jedi_matches(
2152 cursor_pos, cursor_line, full_text)
2153
2154 if self.merge_completions:
2155 matches = []
2156 for matcher in self.matchers:
2157 try:
2158 matches.extend([(m, matcher.__qualname__)
2159 for m in matcher(text)])
2160 except:
2161 # Show the ugly traceback if the matcher causes an
2162 # exception, but do NOT crash the kernel!
2163 sys.excepthook(*sys.exc_info())
2164 else:
2165 for matcher in self.matchers:
2166 matches = [(m, matcher.__qualname__)
2167 for m in matcher(text)]
2168 if matches:
2169 break
2170
2171 seen = set()
2172 filtered_matches = set()
2173 for m in matches:
2174 t, c = m
2175 if t not in seen:
2176 filtered_matches.add(m)
2177 seen.add(t)
2757 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2178 2758
2179 _filtered_matches = sorted(filtered_matches, key=lambda x: completions_sorting_key(x[0]))
2759 suppressed_matchers = set()
2180 2760
2181 custom_res = [(m, 'custom') for m in self.dispatch_custom_completer(text) or []]
2182
2183 _filtered_matches = custom_res or _filtered_matches
2184
2185 _filtered_matches = _filtered_matches[:MATCHES_LIMIT]
2186 _matches = [m[0] for m in _filtered_matches]
2187 origins = [m[1] for m in _filtered_matches]
2761 matchers = {
2762 _get_matcher_id(matcher): matcher
2763 for matcher in sorted(
2764 self.matchers, key=_get_matcher_priority, reverse=True
2765 )
2766 }
2188 2767
2189 self.matches = _matches
2768 for matcher_id, matcher in matchers.items():
2769 api_version = _get_matcher_api_version(matcher)
2770 matcher_id = _get_matcher_id(matcher)
2190 2771
2191 return _CompleteResult(text, _matches, origins, completions)
2192
2193 def fwd_unicode_match(self, text:str) -> Tuple[str, Sequence[str]]:
2772 if matcher_id in self.disable_matchers:
2773 continue
2774
2775 if matcher_id in results:
2776 warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
2777
2778 if matcher_id in suppressed_matchers:
2779 continue
2780
2781 try:
2782 if api_version == 1:
2783 result = _convert_matcher_v1_result_to_v2(
2784 matcher(text), type=_UNKNOWN_TYPE
2785 )
2786 elif api_version == 2:
2787 result = cast(matcher, MatcherAPIv2)(context)
2788 else:
2789 raise ValueError(f"Unsupported API version {api_version}")
2790 except:
2791 # Show the ugly traceback if the matcher causes an
2792 # exception, but do NOT crash the kernel!
2793 sys.excepthook(*sys.exc_info())
2794 continue
2795
2796 # set default value for matched fragment if suffix was not selected.
2797 result["matched_fragment"] = result.get("matched_fragment", context.token)
2798
2799 if not suppressed_matchers:
2800 suppression_recommended = result.get("suppress", False)
2801
2802 suppression_config = (
2803 self.suppress_competing_matchers.get(matcher_id, None)
2804 if isinstance(self.suppress_competing_matchers, dict)
2805 else self.suppress_competing_matchers
2806 )
2807 should_suppress = (
2808 (suppression_config is True)
2809 or (suppression_recommended and (suppression_config is not False))
2810 ) and len(result["completions"])
2811
2812 if should_suppress:
2813 suppression_exceptions = result.get("do_not_suppress", set())
2814 try:
2815 to_suppress = set(suppression_recommended)
2816 except TypeError:
2817 to_suppress = set(matchers)
2818 suppressed_matchers = to_suppress - suppression_exceptions
2819
2820 new_results = {}
2821 for previous_matcher_id, previous_result in results.items():
2822 if previous_matcher_id not in suppressed_matchers:
2823 new_results[previous_matcher_id] = previous_result
2824 results = new_results
2825
2826 results[matcher_id] = result
2827
2828 _, matches = self._arrange_and_extract(
2829 results,
2830 # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
2831 # if it was omission, we can remove the filtering step, otherwise remove this comment.
2832 skip_matchers={jedi_matcher_id},
2833 abort_if_offset_changes=False,
2834 )
2835
2836 # populate legacy stateful API
2837 self.matches = matches
2838
2839 return results
2840
2841 @staticmethod
2842 def _deduplicate(
2843 matches: Sequence[SimpleCompletion],
2844 ) -> Iterable[SimpleCompletion]:
2845 filtered_matches = {}
2846 for match in matches:
2847 text = match.text
2848 if (
2849 text not in filtered_matches
2850 or filtered_matches[text].type == _UNKNOWN_TYPE
2851 ):
2852 filtered_matches[text] = match
2853
2854 return filtered_matches.values()
2855
2856 @staticmethod
2857 def _sort(matches: Sequence[SimpleCompletion]):
2858 return sorted(matches, key=lambda x: completions_sorting_key(x.text))
2859
2860 @context_matcher()
2861 def fwd_unicode_matcher(self, context: CompletionContext):
2862 """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
2863 # TODO: use `context.limit` to terminate early once we matched the maximum
2864 # number that will be used downstream; can be added as an optional to
2865 # `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
2866 fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
2867 return _convert_matcher_v1_result_to_v2(
2868 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2869 )
2870
2871 def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
2194 2872 """
2195 2873 Forward match a string starting with a backslash with a list of
2196 2874 potential Unicode completions.
2197 2875
2198 Will compute list list of Unicode character names on first call and cache it.
2876 Will compute list of Unicode character names on first call and cache it.
2877
2878 .. deprecated:: 8.6
2879 You can use :meth:`fwd_unicode_matcher` instead.
2199 2880
2200 2881 Returns
2201 2882 -------
2202 2883 At tuple with:
2203 2884 - matched text (empty if no matches)
2204 2885 - list of potential completions, empty tuple otherwise)
2205 2886 """
2206 2887 # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
2207 2888 # We could do a faster match using a Trie.
2208 2889
2209 2890 # Using pygtrie the following seem to work:
2210 2891
2211 2892 # s = PrefixSet()
2212 2893
2213 2894 # for c in range(0,0x10FFFF + 1):
2214 2895 # try:
2215 2896 # s.add(unicodedata.name(chr(c)))
2216 2897 # except ValueError:
2217 2898 # pass
2218 2899 # [''.join(k) for k in s.iter(prefix)]
2219 2900
2220 2901 # But need to be timed and adds an extra dependency.
2221 2902
2222 2903 slashpos = text.rfind('\\')
2223 2904 # if text starts with slash
2224 2905 if slashpos > -1:
2225 2906 # PERF: It's important that we don't access self._unicode_names
2226 2907 # until we're inside this if-block. _unicode_names is lazily
2227 2908 # initialized, and it takes a user-noticeable amount of time to
2228 2909 # initialize it, so we don't want to initialize it unless we're
2229 2910 # actually going to use it.
2230 2911 s = text[slashpos + 1 :]
2231 2912 sup = s.upper()
2232 2913 candidates = [x for x in self.unicode_names if x.startswith(sup)]
2233 2914 if candidates:
2234 2915 return s, candidates
2235 2916 candidates = [x for x in self.unicode_names if sup in x]
2236 2917 if candidates:
2237 2918 return s, candidates
2238 2919 splitsup = sup.split(" ")
2239 2920 candidates = [
2240 2921 x for x in self.unicode_names if all(u in x for u in splitsup)
2241 2922 ]
2242 2923 if candidates:
2243 2924 return s, candidates
2244 2925
2245 2926 return "", ()
2246 2927
2247 2928 # if text does not start with slash
2248 2929 else:
2249 2930 return '', ()
2250 2931
2251 2932 @property
2252 2933 def unicode_names(self) -> List[str]:
2253 2934 """List of names of unicode code points that can be completed.
2254 2935
2255 2936 The list is lazily initialized on first access.
2256 2937 """
2257 2938 if self._unicode_names is None:
2258 2939 names = []
2259 2940 for c in range(0,0x10FFFF + 1):
2260 2941 try:
2261 2942 names.append(unicodedata.name(chr(c)))
2262 2943 except ValueError:
2263 2944 pass
2264 2945 self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
2265 2946
2266 2947 return self._unicode_names
2267 2948
2268 2949 def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
2269 2950 names = []
2270 2951 for start,stop in ranges:
2271 2952 for c in range(start, stop) :
2272 2953 try:
2273 2954 names.append(unicodedata.name(chr(c)))
2274 2955 except ValueError:
2275 2956 pass
2276 2957 return names
@@ -1,1277 +1,1290 b''
1 1 # -*- coding: utf-8 -*-
2 2 """Top-level display functions for displaying object in different formats."""
3 3
4 4 # Copyright (c) IPython Development Team.
5 5 # Distributed under the terms of the Modified BSD License.
6 6
7 7
8 8 from binascii import b2a_base64, hexlify
9 9 import html
10 10 import json
11 11 import mimetypes
12 12 import os
13 13 import struct
14 14 import warnings
15 15 from copy import deepcopy
16 16 from os.path import splitext
17 17 from pathlib import Path, PurePath
18 18
19 19 from IPython.utils.py3compat import cast_unicode
20 20 from IPython.testing.skipdoctest import skip_doctest
21 21 from . import display_functions
22 22
23 23
24 24 __all__ = ['display_pretty', 'display_html', 'display_markdown',
25 25 'display_svg', 'display_png', 'display_jpeg', 'display_latex', 'display_json',
26 26 'display_javascript', 'display_pdf', 'DisplayObject', 'TextDisplayObject',
27 27 'Pretty', 'HTML', 'Markdown', 'Math', 'Latex', 'SVG', 'ProgressBar', 'JSON',
28 28 'GeoJSON', 'Javascript', 'Image', 'set_matplotlib_formats',
29 29 'set_matplotlib_close',
30 30 'Video']
31 31
32 32 _deprecated_names = ["display", "clear_output", "publish_display_data", "update_display", "DisplayHandle"]
33 33
34 34 __all__ = __all__ + _deprecated_names
35 35
36 36
37 37 # ----- warn to import from IPython.display -----
38 38
39 39 from warnings import warn
40 40
41 41
42 42 def __getattr__(name):
43 43 if name in _deprecated_names:
44 44 warn(f"Importing {name} from IPython.core.display is deprecated since IPython 7.14, please import from IPython display", DeprecationWarning, stacklevel=2)
45 45 return getattr(display_functions, name)
46 46
47 47 if name in globals().keys():
48 48 return globals()[name]
49 49 else:
50 50 raise AttributeError(f"module {__name__} has no attribute {name}")
51 51
52 52
53 53 #-----------------------------------------------------------------------------
54 54 # utility functions
55 55 #-----------------------------------------------------------------------------
56 56
57 57 def _safe_exists(path):
58 58 """Check path, but don't let exceptions raise"""
59 59 try:
60 60 return os.path.exists(path)
61 61 except Exception:
62 62 return False
63 63
64 64
65 65 def _display_mimetype(mimetype, objs, raw=False, metadata=None):
66 66 """internal implementation of all display_foo methods
67 67
68 68 Parameters
69 69 ----------
70 70 mimetype : str
71 71 The mimetype to be published (e.g. 'image/png')
72 72 *objs : object
73 73 The Python objects to display, or if raw=True raw text data to
74 74 display.
75 75 raw : bool
76 76 Are the data objects raw data or Python objects that need to be
77 77 formatted before display? [default: False]
78 78 metadata : dict (optional)
79 79 Metadata to be associated with the specific mimetype output.
80 80 """
81 81 if metadata:
82 82 metadata = {mimetype: metadata}
83 83 if raw:
84 84 # turn list of pngdata into list of { 'image/png': pngdata }
85 85 objs = [ {mimetype: obj} for obj in objs ]
86 86 display_functions.display(*objs, raw=raw, metadata=metadata, include=[mimetype])
87 87
88 88 #-----------------------------------------------------------------------------
89 89 # Main functions
90 90 #-----------------------------------------------------------------------------
91 91
92 92
93 93 def display_pretty(*objs, **kwargs):
94 94 """Display the pretty (default) representation of an object.
95 95
96 96 Parameters
97 97 ----------
98 98 *objs : object
99 99 The Python objects to display, or if raw=True raw text data to
100 100 display.
101 101 raw : bool
102 102 Are the data objects raw data or Python objects that need to be
103 103 formatted before display? [default: False]
104 104 metadata : dict (optional)
105 105 Metadata to be associated with the specific mimetype output.
106 106 """
107 107 _display_mimetype('text/plain', objs, **kwargs)
108 108
109 109
110 110 def display_html(*objs, **kwargs):
111 111 """Display the HTML representation of an object.
112 112
113 113 Note: If raw=False and the object does not have a HTML
114 114 representation, no HTML will be shown.
115 115
116 116 Parameters
117 117 ----------
118 118 *objs : object
119 119 The Python objects to display, or if raw=True raw HTML data to
120 120 display.
121 121 raw : bool
122 122 Are the data objects raw data or Python objects that need to be
123 123 formatted before display? [default: False]
124 124 metadata : dict (optional)
125 125 Metadata to be associated with the specific mimetype output.
126 126 """
127 127 _display_mimetype('text/html', objs, **kwargs)
128 128
129 129
130 130 def display_markdown(*objs, **kwargs):
131 131 """Displays the Markdown representation of an object.
132 132
133 133 Parameters
134 134 ----------
135 135 *objs : object
136 136 The Python objects to display, or if raw=True raw markdown data to
137 137 display.
138 138 raw : bool
139 139 Are the data objects raw data or Python objects that need to be
140 140 formatted before display? [default: False]
141 141 metadata : dict (optional)
142 142 Metadata to be associated with the specific mimetype output.
143 143 """
144 144
145 145 _display_mimetype('text/markdown', objs, **kwargs)
146 146
147 147
148 148 def display_svg(*objs, **kwargs):
149 149 """Display the SVG representation of an object.
150 150
151 151 Parameters
152 152 ----------
153 153 *objs : object
154 154 The Python objects to display, or if raw=True raw svg data to
155 155 display.
156 156 raw : bool
157 157 Are the data objects raw data or Python objects that need to be
158 158 formatted before display? [default: False]
159 159 metadata : dict (optional)
160 160 Metadata to be associated with the specific mimetype output.
161 161 """
162 162 _display_mimetype('image/svg+xml', objs, **kwargs)
163 163
164 164
165 165 def display_png(*objs, **kwargs):
166 166 """Display the PNG representation of an object.
167 167
168 168 Parameters
169 169 ----------
170 170 *objs : object
171 171 The Python objects to display, or if raw=True raw png data to
172 172 display.
173 173 raw : bool
174 174 Are the data objects raw data or Python objects that need to be
175 175 formatted before display? [default: False]
176 176 metadata : dict (optional)
177 177 Metadata to be associated with the specific mimetype output.
178 178 """
179 179 _display_mimetype('image/png', objs, **kwargs)
180 180
181 181
182 182 def display_jpeg(*objs, **kwargs):
183 183 """Display the JPEG representation of an object.
184 184
185 185 Parameters
186 186 ----------
187 187 *objs : object
188 188 The Python objects to display, or if raw=True raw JPEG data to
189 189 display.
190 190 raw : bool
191 191 Are the data objects raw data or Python objects that need to be
192 192 formatted before display? [default: False]
193 193 metadata : dict (optional)
194 194 Metadata to be associated with the specific mimetype output.
195 195 """
196 196 _display_mimetype('image/jpeg', objs, **kwargs)
197 197
198 198
199 199 def display_latex(*objs, **kwargs):
200 200 """Display the LaTeX representation of an object.
201 201
202 202 Parameters
203 203 ----------
204 204 *objs : object
205 205 The Python objects to display, or if raw=True raw latex data to
206 206 display.
207 207 raw : bool
208 208 Are the data objects raw data or Python objects that need to be
209 209 formatted before display? [default: False]
210 210 metadata : dict (optional)
211 211 Metadata to be associated with the specific mimetype output.
212 212 """
213 213 _display_mimetype('text/latex', objs, **kwargs)
214 214
215 215
216 216 def display_json(*objs, **kwargs):
217 217 """Display the JSON representation of an object.
218 218
219 219 Note that not many frontends support displaying JSON.
220 220
221 221 Parameters
222 222 ----------
223 223 *objs : object
224 224 The Python objects to display, or if raw=True raw json data to
225 225 display.
226 226 raw : bool
227 227 Are the data objects raw data or Python objects that need to be
228 228 formatted before display? [default: False]
229 229 metadata : dict (optional)
230 230 Metadata to be associated with the specific mimetype output.
231 231 """
232 232 _display_mimetype('application/json', objs, **kwargs)
233 233
234 234
235 235 def display_javascript(*objs, **kwargs):
236 236 """Display the Javascript representation of an object.
237 237
238 238 Parameters
239 239 ----------
240 240 *objs : object
241 241 The Python objects to display, or if raw=True raw javascript data to
242 242 display.
243 243 raw : bool
244 244 Are the data objects raw data or Python objects that need to be
245 245 formatted before display? [default: False]
246 246 metadata : dict (optional)
247 247 Metadata to be associated with the specific mimetype output.
248 248 """
249 249 _display_mimetype('application/javascript', objs, **kwargs)
250 250
251 251
252 252 def display_pdf(*objs, **kwargs):
253 253 """Display the PDF representation of an object.
254 254
255 255 Parameters
256 256 ----------
257 257 *objs : object
258 258 The Python objects to display, or if raw=True raw javascript data to
259 259 display.
260 260 raw : bool
261 261 Are the data objects raw data or Python objects that need to be
262 262 formatted before display? [default: False]
263 263 metadata : dict (optional)
264 264 Metadata to be associated with the specific mimetype output.
265 265 """
266 266 _display_mimetype('application/pdf', objs, **kwargs)
267 267
268 268
269 269 #-----------------------------------------------------------------------------
270 270 # Smart classes
271 271 #-----------------------------------------------------------------------------
272 272
273 273
274 274 class DisplayObject(object):
275 275 """An object that wraps data to be displayed."""
276 276
277 277 _read_flags = 'r'
278 278 _show_mem_addr = False
279 279 metadata = None
280 280
281 281 def __init__(self, data=None, url=None, filename=None, metadata=None):
282 282 """Create a display object given raw data.
283 283
284 284 When this object is returned by an expression or passed to the
285 285 display function, it will result in the data being displayed
286 286 in the frontend. The MIME type of the data should match the
287 287 subclasses used, so the Png subclass should be used for 'image/png'
288 288 data. If the data is a URL, the data will first be downloaded
289 289 and then displayed. If
290 290
291 291 Parameters
292 292 ----------
293 293 data : unicode, str or bytes
294 294 The raw data or a URL or file to load the data from
295 295 url : unicode
296 296 A URL to download the data from.
297 297 filename : unicode
298 298 Path to a local file to load the data from.
299 299 metadata : dict
300 300 Dict of metadata associated to be the object when displayed
301 301 """
302 302 if isinstance(data, (Path, PurePath)):
303 303 data = str(data)
304 304
305 305 if data is not None and isinstance(data, str):
306 306 if data.startswith('http') and url is None:
307 307 url = data
308 308 filename = None
309 309 data = None
310 310 elif _safe_exists(data) and filename is None:
311 311 url = None
312 312 filename = data
313 313 data = None
314 314
315 315 self.url = url
316 316 self.filename = filename
317 317 # because of @data.setter methods in
318 318 # subclasses ensure url and filename are set
319 319 # before assigning to self.data
320 320 self.data = data
321 321
322 322 if metadata is not None:
323 323 self.metadata = metadata
324 324 elif self.metadata is None:
325 325 self.metadata = {}
326 326
327 327 self.reload()
328 328 self._check_data()
329 329
330 330 def __repr__(self):
331 331 if not self._show_mem_addr:
332 332 cls = self.__class__
333 333 r = "<%s.%s object>" % (cls.__module__, cls.__name__)
334 334 else:
335 335 r = super(DisplayObject, self).__repr__()
336 336 return r
337 337
338 338 def _check_data(self):
339 339 """Override in subclasses if there's something to check."""
340 340 pass
341 341
342 342 def _data_and_metadata(self):
343 343 """shortcut for returning metadata with shape information, if defined"""
344 344 if self.metadata:
345 345 return self.data, deepcopy(self.metadata)
346 346 else:
347 347 return self.data
348 348
349 349 def reload(self):
350 350 """Reload the raw data from file or URL."""
351 351 if self.filename is not None:
352 352 encoding = None if "b" in self._read_flags else "utf-8"
353 353 with open(self.filename, self._read_flags, encoding=encoding) as f:
354 354 self.data = f.read()
355 355 elif self.url is not None:
356 356 # Deferred import
357 357 from urllib.request import urlopen
358 358 response = urlopen(self.url)
359 359 data = response.read()
360 360 # extract encoding from header, if there is one:
361 361 encoding = None
362 362 if 'content-type' in response.headers:
363 363 for sub in response.headers['content-type'].split(';'):
364 364 sub = sub.strip()
365 365 if sub.startswith('charset'):
366 366 encoding = sub.split('=')[-1].strip()
367 367 break
368 368 if 'content-encoding' in response.headers:
369 369 # TODO: do deflate?
370 370 if 'gzip' in response.headers['content-encoding']:
371 371 import gzip
372 372 from io import BytesIO
373 373
374 374 # assume utf-8 if encoding is not specified
375 375 with gzip.open(
376 376 BytesIO(data), "rt", encoding=encoding or "utf-8"
377 377 ) as fp:
378 378 encoding = None
379 379 data = fp.read()
380 380
381 381 # decode data, if an encoding was specified
382 382 # We only touch self.data once since
383 383 # subclasses such as SVG have @data.setter methods
384 384 # that transform self.data into ... well svg.
385 385 if encoding:
386 386 self.data = data.decode(encoding, 'replace')
387 387 else:
388 388 self.data = data
389 389
390 390
391 391 class TextDisplayObject(DisplayObject):
392 """Validate that display data is text"""
392 """Create a text display object given raw data.
393
394 Parameters
395 ----------
396 data : str or unicode
397 The raw data or a URL or file to load the data from.
398 url : unicode
399 A URL to download the data from.
400 filename : unicode
401 Path to a local file to load the data from.
402 metadata : dict
403 Dict of metadata associated to be the object when displayed
404 """
393 405 def _check_data(self):
394 406 if self.data is not None and not isinstance(self.data, str):
395 407 raise TypeError("%s expects text, not %r" % (self.__class__.__name__, self.data))
396 408
397 409 class Pretty(TextDisplayObject):
398 410
399 411 def _repr_pretty_(self, pp, cycle):
400 412 return pp.text(self.data)
401 413
402 414
403 415 class HTML(TextDisplayObject):
404 416
405 417 def __init__(self, data=None, url=None, filename=None, metadata=None):
406 418 def warn():
407 419 if not data:
408 420 return False
409 421
410 422 #
411 423 # Avoid calling lower() on the entire data, because it could be a
412 424 # long string and we're only interested in its beginning and end.
413 425 #
414 426 prefix = data[:10].lower()
415 427 suffix = data[-10:].lower()
416 428 return prefix.startswith("<iframe ") and suffix.endswith("</iframe>")
417 429
418 430 if warn():
419 431 warnings.warn("Consider using IPython.display.IFrame instead")
420 432 super(HTML, self).__init__(data=data, url=url, filename=filename, metadata=metadata)
421 433
422 434 def _repr_html_(self):
423 435 return self._data_and_metadata()
424 436
425 437 def __html__(self):
426 438 """
427 439 This method exists to inform other HTML-using modules (e.g. Markupsafe,
428 440 htmltag, etc) that this object is HTML and does not need things like
429 441 special characters (<>&) escaped.
430 442 """
431 443 return self._repr_html_()
432 444
433 445
434 446 class Markdown(TextDisplayObject):
435 447
436 448 def _repr_markdown_(self):
437 449 return self._data_and_metadata()
438 450
439 451
440 452 class Math(TextDisplayObject):
441 453
442 454 def _repr_latex_(self):
443 455 s = r"$\displaystyle %s$" % self.data.strip('$')
444 456 if self.metadata:
445 457 return s, deepcopy(self.metadata)
446 458 else:
447 459 return s
448 460
449 461
450 462 class Latex(TextDisplayObject):
451 463
452 464 def _repr_latex_(self):
453 465 return self._data_and_metadata()
454 466
455 467
456 468 class SVG(DisplayObject):
457 469 """Embed an SVG into the display.
458 470
459 471 Note if you just want to view a svg image via a URL use `:class:Image` with
460 472 a url=URL keyword argument.
461 473 """
462 474
463 475 _read_flags = 'rb'
464 476 # wrap data in a property, which extracts the <svg> tag, discarding
465 477 # document headers
466 478 _data = None
467 479
468 480 @property
469 481 def data(self):
470 482 return self._data
471 483
472 484 @data.setter
473 485 def data(self, svg):
474 486 if svg is None:
475 487 self._data = None
476 488 return
477 489 # parse into dom object
478 490 from xml.dom import minidom
479 491 x = minidom.parseString(svg)
480 492 # get svg tag (should be 1)
481 493 found_svg = x.getElementsByTagName('svg')
482 494 if found_svg:
483 495 svg = found_svg[0].toxml()
484 496 else:
485 497 # fallback on the input, trust the user
486 498 # but this is probably an error.
487 499 pass
488 500 svg = cast_unicode(svg)
489 501 self._data = svg
490 502
491 503 def _repr_svg_(self):
492 504 return self._data_and_metadata()
493 505
494 506 class ProgressBar(DisplayObject):
495 507 """Progressbar supports displaying a progressbar like element
496 508 """
497 509 def __init__(self, total):
498 510 """Creates a new progressbar
499 511
500 512 Parameters
501 513 ----------
502 514 total : int
503 515 maximum size of the progressbar
504 516 """
505 517 self.total = total
506 518 self._progress = 0
507 519 self.html_width = '60ex'
508 520 self.text_width = 60
509 521 self._display_id = hexlify(os.urandom(8)).decode('ascii')
510 522
511 523 def __repr__(self):
512 524 fraction = self.progress / self.total
513 525 filled = '=' * int(fraction * self.text_width)
514 526 rest = ' ' * (self.text_width - len(filled))
515 527 return '[{}{}] {}/{}'.format(
516 528 filled, rest,
517 529 self.progress, self.total,
518 530 )
519 531
520 532 def _repr_html_(self):
521 533 return "<progress style='width:{}' max='{}' value='{}'></progress>".format(
522 534 self.html_width, self.total, self.progress)
523 535
524 536 def display(self):
525 537 display_functions.display(self, display_id=self._display_id)
526 538
527 539 def update(self):
528 540 display_functions.display(self, display_id=self._display_id, update=True)
529 541
530 542 @property
531 543 def progress(self):
532 544 return self._progress
533 545
534 546 @progress.setter
535 547 def progress(self, value):
536 548 self._progress = value
537 549 self.update()
538 550
539 551 def __iter__(self):
540 552 self.display()
541 553 self._progress = -1 # First iteration is 0
542 554 return self
543 555
544 556 def __next__(self):
545 557 """Returns current value and increments display by one."""
546 558 self.progress += 1
547 559 if self.progress < self.total:
548 560 return self.progress
549 561 else:
550 562 raise StopIteration()
551 563
552 564 class JSON(DisplayObject):
553 565 """JSON expects a JSON-able dict or list
554 566
555 567 not an already-serialized JSON string.
556 568
557 569 Scalar types (None, number, string) are not allowed, only dict or list containers.
558 570 """
559 571 # wrap data in a property, which warns about passing already-serialized JSON
560 572 _data = None
561 573 def __init__(self, data=None, url=None, filename=None, expanded=False, metadata=None, root='root', **kwargs):
562 574 """Create a JSON display object given raw data.
563 575
564 576 Parameters
565 577 ----------
566 578 data : dict or list
567 579 JSON data to display. Not an already-serialized JSON string.
568 580 Scalar types (None, number, string) are not allowed, only dict
569 581 or list containers.
570 582 url : unicode
571 583 A URL to download the data from.
572 584 filename : unicode
573 585 Path to a local file to load the data from.
574 586 expanded : boolean
575 587 Metadata to control whether a JSON display component is expanded.
576 588 metadata : dict
577 589 Specify extra metadata to attach to the json display object.
578 590 root : str
579 591 The name of the root element of the JSON tree
580 592 """
581 593 self.metadata = {
582 594 'expanded': expanded,
583 595 'root': root,
584 596 }
585 597 if metadata:
586 598 self.metadata.update(metadata)
587 599 if kwargs:
588 600 self.metadata.update(kwargs)
589 601 super(JSON, self).__init__(data=data, url=url, filename=filename)
590 602
591 603 def _check_data(self):
592 604 if self.data is not None and not isinstance(self.data, (dict, list)):
593 605 raise TypeError("%s expects JSONable dict or list, not %r" % (self.__class__.__name__, self.data))
594 606
595 607 @property
596 608 def data(self):
597 609 return self._data
598 610
599 611 @data.setter
600 612 def data(self, data):
601 613 if isinstance(data, (Path, PurePath)):
602 614 data = str(data)
603 615
604 616 if isinstance(data, str):
605 617 if self.filename is None and self.url is None:
606 618 warnings.warn("JSON expects JSONable dict or list, not JSON strings")
607 619 data = json.loads(data)
608 620 self._data = data
609 621
610 622 def _data_and_metadata(self):
611 623 return self.data, self.metadata
612 624
613 625 def _repr_json_(self):
614 626 return self._data_and_metadata()
615 627
628
616 629 _css_t = """var link = document.createElement("link");
617 link.ref = "stylesheet";
630 link.rel = "stylesheet";
618 631 link.type = "text/css";
619 632 link.href = "%s";
620 633 document.head.appendChild(link);
621 634 """
622 635
623 636 _lib_t1 = """new Promise(function(resolve, reject) {
624 637 var script = document.createElement("script");
625 638 script.onload = resolve;
626 639 script.onerror = reject;
627 640 script.src = "%s";
628 641 document.head.appendChild(script);
629 642 }).then(() => {
630 643 """
631 644
632 645 _lib_t2 = """
633 646 });"""
634 647
635 648 class GeoJSON(JSON):
636 649 """GeoJSON expects JSON-able dict
637 650
638 651 not an already-serialized JSON string.
639 652
640 653 Scalar types (None, number, string) are not allowed, only dict containers.
641 654 """
642 655
643 656 def __init__(self, *args, **kwargs):
644 657 """Create a GeoJSON display object given raw data.
645 658
646 659 Parameters
647 660 ----------
648 661 data : dict or list
649 662 VegaLite data. Not an already-serialized JSON string.
650 663 Scalar types (None, number, string) are not allowed, only dict
651 664 or list containers.
652 665 url_template : string
653 666 Leaflet TileLayer URL template: http://leafletjs.com/reference.html#url-template
654 667 layer_options : dict
655 668 Leaflet TileLayer options: http://leafletjs.com/reference.html#tilelayer-options
656 669 url : unicode
657 670 A URL to download the data from.
658 671 filename : unicode
659 672 Path to a local file to load the data from.
660 673 metadata : dict
661 674 Specify extra metadata to attach to the json display object.
662 675
663 676 Examples
664 677 --------
665 678 The following will display an interactive map of Mars with a point of
666 679 interest on frontend that do support GeoJSON display.
667 680
668 681 >>> from IPython.display import GeoJSON
669 682
670 683 >>> GeoJSON(data={
671 684 ... "type": "Feature",
672 685 ... "geometry": {
673 686 ... "type": "Point",
674 687 ... "coordinates": [-81.327, 296.038]
675 688 ... }
676 689 ... },
677 690 ... url_template="http://s3-eu-west-1.amazonaws.com/whereonmars.cartodb.net/{basemap_id}/{z}/{x}/{y}.png",
678 691 ... layer_options={
679 692 ... "basemap_id": "celestia_mars-shaded-16k_global",
680 693 ... "attribution" : "Celestia/praesepe",
681 694 ... "minZoom" : 0,
682 695 ... "maxZoom" : 18,
683 696 ... })
684 697 <IPython.core.display.GeoJSON object>
685 698
686 699 In the terminal IPython, you will only see the text representation of
687 700 the GeoJSON object.
688 701
689 702 """
690 703
691 704 super(GeoJSON, self).__init__(*args, **kwargs)
692 705
693 706
694 707 def _ipython_display_(self):
695 708 bundle = {
696 709 'application/geo+json': self.data,
697 710 'text/plain': '<IPython.display.GeoJSON object>'
698 711 }
699 712 metadata = {
700 713 'application/geo+json': self.metadata
701 714 }
702 715 display_functions.display(bundle, metadata=metadata, raw=True)
703 716
704 717 class Javascript(TextDisplayObject):
705 718
706 719 def __init__(self, data=None, url=None, filename=None, lib=None, css=None):
707 720 """Create a Javascript display object given raw data.
708 721
709 722 When this object is returned by an expression or passed to the
710 723 display function, it will result in the data being displayed
711 724 in the frontend. If the data is a URL, the data will first be
712 725 downloaded and then displayed.
713 726
714 727 In the Notebook, the containing element will be available as `element`,
715 728 and jQuery will be available. Content appended to `element` will be
716 729 visible in the output area.
717 730
718 731 Parameters
719 732 ----------
720 733 data : unicode, str or bytes
721 734 The Javascript source code or a URL to download it from.
722 735 url : unicode
723 736 A URL to download the data from.
724 737 filename : unicode
725 738 Path to a local file to load the data from.
726 739 lib : list or str
727 740 A sequence of Javascript library URLs to load asynchronously before
728 741 running the source code. The full URLs of the libraries should
729 742 be given. A single Javascript library URL can also be given as a
730 743 string.
731 744 css : list or str
732 745 A sequence of css files to load before running the source code.
733 746 The full URLs of the css files should be given. A single css URL
734 747 can also be given as a string.
735 748 """
736 749 if isinstance(lib, str):
737 750 lib = [lib]
738 751 elif lib is None:
739 752 lib = []
740 753 if isinstance(css, str):
741 754 css = [css]
742 755 elif css is None:
743 756 css = []
744 757 if not isinstance(lib, (list,tuple)):
745 758 raise TypeError('expected sequence, got: %r' % lib)
746 759 if not isinstance(css, (list,tuple)):
747 760 raise TypeError('expected sequence, got: %r' % css)
748 761 self.lib = lib
749 762 self.css = css
750 763 super(Javascript, self).__init__(data=data, url=url, filename=filename)
751 764
752 765 def _repr_javascript_(self):
753 766 r = ''
754 767 for c in self.css:
755 768 r += _css_t % c
756 769 for l in self.lib:
757 770 r += _lib_t1 % l
758 771 r += self.data
759 772 r += _lib_t2*len(self.lib)
760 773 return r
761 774
762 775 # constants for identifying png/jpeg data
763 776 _PNG = b'\x89PNG\r\n\x1a\n'
764 777 _JPEG = b'\xff\xd8'
765 778
766 779 def _pngxy(data):
767 780 """read the (width, height) from a PNG header"""
768 781 ihdr = data.index(b'IHDR')
769 782 # next 8 bytes are width/height
770 783 return struct.unpack('>ii', data[ihdr+4:ihdr+12])
771 784
772 785 def _jpegxy(data):
773 786 """read the (width, height) from a JPEG header"""
774 787 # adapted from http://www.64lines.com/jpeg-width-height
775 788
776 789 idx = 4
777 790 while True:
778 791 block_size = struct.unpack('>H', data[idx:idx+2])[0]
779 792 idx = idx + block_size
780 793 if data[idx:idx+2] == b'\xFF\xC0':
781 794 # found Start of Frame
782 795 iSOF = idx
783 796 break
784 797 else:
785 798 # read another block
786 799 idx += 2
787 800
788 801 h, w = struct.unpack('>HH', data[iSOF+5:iSOF+9])
789 802 return w, h
790 803
791 804 def _gifxy(data):
792 805 """read the (width, height) from a GIF header"""
793 806 return struct.unpack('<HH', data[6:10])
794 807
795 808
796 809 class Image(DisplayObject):
797 810
798 811 _read_flags = 'rb'
799 812 _FMT_JPEG = u'jpeg'
800 813 _FMT_PNG = u'png'
801 814 _FMT_GIF = u'gif'
802 815 _ACCEPTABLE_EMBEDDINGS = [_FMT_JPEG, _FMT_PNG, _FMT_GIF]
803 816 _MIMETYPES = {
804 817 _FMT_PNG: 'image/png',
805 818 _FMT_JPEG: 'image/jpeg',
806 819 _FMT_GIF: 'image/gif',
807 820 }
808 821
809 822 def __init__(
810 823 self,
811 824 data=None,
812 825 url=None,
813 826 filename=None,
814 827 format=None,
815 828 embed=None,
816 829 width=None,
817 830 height=None,
818 831 retina=False,
819 832 unconfined=False,
820 833 metadata=None,
821 834 alt=None,
822 835 ):
823 836 """Create a PNG/JPEG/GIF image object given raw data.
824 837
825 838 When this object is returned by an input cell or passed to the
826 839 display function, it will result in the image being displayed
827 840 in the frontend.
828 841
829 842 Parameters
830 843 ----------
831 844 data : unicode, str or bytes
832 845 The raw image data or a URL or filename to load the data from.
833 846 This always results in embedded image data.
834 847
835 848 url : unicode
836 849 A URL to download the data from. If you specify `url=`,
837 850 the image data will not be embedded unless you also specify `embed=True`.
838 851
839 852 filename : unicode
840 853 Path to a local file to load the data from.
841 854 Images from a file are always embedded.
842 855
843 856 format : unicode
844 857 The format of the image data (png/jpeg/jpg/gif). If a filename or URL is given
845 858 for format will be inferred from the filename extension.
846 859
847 860 embed : bool
848 861 Should the image data be embedded using a data URI (True) or be
849 862 loaded using an <img> tag. Set this to True if you want the image
850 863 to be viewable later with no internet connection in the notebook.
851 864
852 865 Default is `True`, unless the keyword argument `url` is set, then
853 866 default value is `False`.
854 867
855 868 Note that QtConsole is not able to display images if `embed` is set to `False`
856 869
857 870 width : int
858 871 Width in pixels to which to constrain the image in html
859 872
860 873 height : int
861 874 Height in pixels to which to constrain the image in html
862 875
863 876 retina : bool
864 877 Automatically set the width and height to half of the measured
865 878 width and height.
866 879 This only works for embedded images because it reads the width/height
867 880 from image data.
868 881 For non-embedded images, you can just set the desired display width
869 882 and height directly.
870 883
871 884 unconfined : bool
872 885 Set unconfined=True to disable max-width confinement of the image.
873 886
874 887 metadata : dict
875 888 Specify extra metadata to attach to the image.
876 889
877 890 alt : unicode
878 891 Alternative text for the image, for use by screen readers.
879 892
880 893 Examples
881 894 --------
882 895 embedded image data, works in qtconsole and notebook
883 896 when passed positionally, the first arg can be any of raw image data,
884 897 a URL, or a filename from which to load image data.
885 898 The result is always embedding image data for inline images.
886 899
887 900 >>> Image('https://www.google.fr/images/srpr/logo3w.png') # doctest: +SKIP
888 901 <IPython.core.display.Image object>
889 902
890 903 >>> Image('/path/to/image.jpg')
891 904 <IPython.core.display.Image object>
892 905
893 906 >>> Image(b'RAW_PNG_DATA...')
894 907 <IPython.core.display.Image object>
895 908
896 909 Specifying Image(url=...) does not embed the image data,
897 910 it only generates ``<img>`` tag with a link to the source.
898 911 This will not work in the qtconsole or offline.
899 912
900 913 >>> Image(url='https://www.google.fr/images/srpr/logo3w.png')
901 914 <IPython.core.display.Image object>
902 915
903 916 """
904 917 if isinstance(data, (Path, PurePath)):
905 918 data = str(data)
906 919
907 920 if filename is not None:
908 921 ext = self._find_ext(filename)
909 922 elif url is not None:
910 923 ext = self._find_ext(url)
911 924 elif data is None:
912 925 raise ValueError("No image data found. Expecting filename, url, or data.")
913 926 elif isinstance(data, str) and (
914 927 data.startswith('http') or _safe_exists(data)
915 928 ):
916 929 ext = self._find_ext(data)
917 930 else:
918 931 ext = None
919 932
920 933 if format is None:
921 934 if ext is not None:
922 935 if ext == u'jpg' or ext == u'jpeg':
923 936 format = self._FMT_JPEG
924 937 elif ext == u'png':
925 938 format = self._FMT_PNG
926 939 elif ext == u'gif':
927 940 format = self._FMT_GIF
928 941 else:
929 942 format = ext.lower()
930 943 elif isinstance(data, bytes):
931 944 # infer image type from image data header,
932 945 # only if format has not been specified.
933 946 if data[:2] == _JPEG:
934 947 format = self._FMT_JPEG
935 948
936 949 # failed to detect format, default png
937 950 if format is None:
938 951 format = self._FMT_PNG
939 952
940 953 if format.lower() == 'jpg':
941 954 # jpg->jpeg
942 955 format = self._FMT_JPEG
943 956
944 957 self.format = format.lower()
945 958 self.embed = embed if embed is not None else (url is None)
946 959
947 960 if self.embed and self.format not in self._ACCEPTABLE_EMBEDDINGS:
948 961 raise ValueError("Cannot embed the '%s' image format" % (self.format))
949 962 if self.embed:
950 963 self._mimetype = self._MIMETYPES.get(self.format)
951 964
952 965 self.width = width
953 966 self.height = height
954 967 self.retina = retina
955 968 self.unconfined = unconfined
956 969 self.alt = alt
957 970 super(Image, self).__init__(data=data, url=url, filename=filename,
958 971 metadata=metadata)
959 972
960 973 if self.width is None and self.metadata.get('width', {}):
961 974 self.width = metadata['width']
962 975
963 976 if self.height is None and self.metadata.get('height', {}):
964 977 self.height = metadata['height']
965 978
966 979 if self.alt is None and self.metadata.get("alt", {}):
967 980 self.alt = metadata["alt"]
968 981
969 982 if retina:
970 983 self._retina_shape()
971 984
972 985
973 986 def _retina_shape(self):
974 987 """load pixel-doubled width and height from image data"""
975 988 if not self.embed:
976 989 return
977 990 if self.format == self._FMT_PNG:
978 991 w, h = _pngxy(self.data)
979 992 elif self.format == self._FMT_JPEG:
980 993 w, h = _jpegxy(self.data)
981 994 elif self.format == self._FMT_GIF:
982 995 w, h = _gifxy(self.data)
983 996 else:
984 997 # retina only supports png
985 998 return
986 999 self.width = w // 2
987 1000 self.height = h // 2
988 1001
989 1002 def reload(self):
990 1003 """Reload the raw data from file or URL."""
991 1004 if self.embed:
992 1005 super(Image,self).reload()
993 1006 if self.retina:
994 1007 self._retina_shape()
995 1008
996 1009 def _repr_html_(self):
997 1010 if not self.embed:
998 1011 width = height = klass = alt = ""
999 1012 if self.width:
1000 1013 width = ' width="%d"' % self.width
1001 1014 if self.height:
1002 1015 height = ' height="%d"' % self.height
1003 1016 if self.unconfined:
1004 1017 klass = ' class="unconfined"'
1005 1018 if self.alt:
1006 1019 alt = ' alt="%s"' % html.escape(self.alt)
1007 1020 return '<img src="{url}"{width}{height}{klass}{alt}/>'.format(
1008 1021 url=self.url,
1009 1022 width=width,
1010 1023 height=height,
1011 1024 klass=klass,
1012 1025 alt=alt,
1013 1026 )
1014 1027
1015 1028 def _repr_mimebundle_(self, include=None, exclude=None):
1016 1029 """Return the image as a mimebundle
1017 1030
1018 1031 Any new mimetype support should be implemented here.
1019 1032 """
1020 1033 if self.embed:
1021 1034 mimetype = self._mimetype
1022 1035 data, metadata = self._data_and_metadata(always_both=True)
1023 1036 if metadata:
1024 1037 metadata = {mimetype: metadata}
1025 1038 return {mimetype: data}, metadata
1026 1039 else:
1027 1040 return {'text/html': self._repr_html_()}
1028 1041
1029 1042 def _data_and_metadata(self, always_both=False):
1030 1043 """shortcut for returning metadata with shape information, if defined"""
1031 1044 try:
1032 1045 b64_data = b2a_base64(self.data).decode('ascii')
1033 1046 except TypeError as e:
1034 1047 raise FileNotFoundError(
1035 1048 "No such file or directory: '%s'" % (self.data)) from e
1036 1049 md = {}
1037 1050 if self.metadata:
1038 1051 md.update(self.metadata)
1039 1052 if self.width:
1040 1053 md['width'] = self.width
1041 1054 if self.height:
1042 1055 md['height'] = self.height
1043 1056 if self.unconfined:
1044 1057 md['unconfined'] = self.unconfined
1045 1058 if self.alt:
1046 1059 md["alt"] = self.alt
1047 1060 if md or always_both:
1048 1061 return b64_data, md
1049 1062 else:
1050 1063 return b64_data
1051 1064
1052 1065 def _repr_png_(self):
1053 1066 if self.embed and self.format == self._FMT_PNG:
1054 1067 return self._data_and_metadata()
1055 1068
1056 1069 def _repr_jpeg_(self):
1057 1070 if self.embed and self.format == self._FMT_JPEG:
1058 1071 return self._data_and_metadata()
1059 1072
1060 1073 def _find_ext(self, s):
1061 1074 base, ext = splitext(s)
1062 1075
1063 1076 if not ext:
1064 1077 return base
1065 1078
1066 1079 # `splitext` includes leading period, so we skip it
1067 1080 return ext[1:].lower()
1068 1081
1069 1082
1070 1083 class Video(DisplayObject):
1071 1084
1072 1085 def __init__(self, data=None, url=None, filename=None, embed=False,
1073 1086 mimetype=None, width=None, height=None, html_attributes="controls"):
1074 1087 """Create a video object given raw data or an URL.
1075 1088
1076 1089 When this object is returned by an input cell or passed to the
1077 1090 display function, it will result in the video being displayed
1078 1091 in the frontend.
1079 1092
1080 1093 Parameters
1081 1094 ----------
1082 1095 data : unicode, str or bytes
1083 1096 The raw video data or a URL or filename to load the data from.
1084 1097 Raw data will require passing ``embed=True``.
1085 1098
1086 1099 url : unicode
1087 1100 A URL for the video. If you specify ``url=``,
1088 1101 the image data will not be embedded.
1089 1102
1090 1103 filename : unicode
1091 1104 Path to a local file containing the video.
1092 1105 Will be interpreted as a local URL unless ``embed=True``.
1093 1106
1094 1107 embed : bool
1095 1108 Should the video be embedded using a data URI (True) or be
1096 1109 loaded using a <video> tag (False).
1097 1110
1098 1111 Since videos are large, embedding them should be avoided, if possible.
1099 1112 You must confirm embedding as your intention by passing ``embed=True``.
1100 1113
1101 1114 Local files can be displayed with URLs without embedding the content, via::
1102 1115
1103 1116 Video('./video.mp4')
1104 1117
1105 1118 mimetype : unicode
1106 1119 Specify the mimetype for embedded videos.
1107 1120 Default will be guessed from file extension, if available.
1108 1121
1109 1122 width : int
1110 1123 Width in pixels to which to constrain the video in HTML.
1111 1124 If not supplied, defaults to the width of the video.
1112 1125
1113 1126 height : int
1114 1127 Height in pixels to which to constrain the video in html.
1115 1128 If not supplied, defaults to the height of the video.
1116 1129
1117 1130 html_attributes : str
1118 1131 Attributes for the HTML ``<video>`` block.
1119 1132 Default: ``"controls"`` to get video controls.
1120 1133 Other examples: ``"controls muted"`` for muted video with controls,
1121 1134 ``"loop autoplay"`` for looping autoplaying video without controls.
1122 1135
1123 1136 Examples
1124 1137 --------
1125 1138 ::
1126 1139
1127 1140 Video('https://archive.org/download/Sita_Sings_the_Blues/Sita_Sings_the_Blues_small.mp4')
1128 1141 Video('path/to/video.mp4')
1129 1142 Video('path/to/video.mp4', embed=True)
1130 1143 Video('path/to/video.mp4', embed=True, html_attributes="controls muted autoplay")
1131 1144 Video(b'raw-videodata', embed=True)
1132 1145 """
1133 1146 if isinstance(data, (Path, PurePath)):
1134 1147 data = str(data)
1135 1148
1136 1149 if url is None and isinstance(data, str) and data.startswith(('http:', 'https:')):
1137 1150 url = data
1138 1151 data = None
1139 1152 elif data is not None and os.path.exists(data):
1140 1153 filename = data
1141 1154 data = None
1142 1155
1143 1156 if data and not embed:
1144 1157 msg = ''.join([
1145 1158 "To embed videos, you must pass embed=True ",
1146 1159 "(this may make your notebook files huge)\n",
1147 1160 "Consider passing Video(url='...')",
1148 1161 ])
1149 1162 raise ValueError(msg)
1150 1163
1151 1164 self.mimetype = mimetype
1152 1165 self.embed = embed
1153 1166 self.width = width
1154 1167 self.height = height
1155 1168 self.html_attributes = html_attributes
1156 1169 super(Video, self).__init__(data=data, url=url, filename=filename)
1157 1170
1158 1171 def _repr_html_(self):
1159 1172 width = height = ''
1160 1173 if self.width:
1161 1174 width = ' width="%d"' % self.width
1162 1175 if self.height:
1163 1176 height = ' height="%d"' % self.height
1164 1177
1165 1178 # External URLs and potentially local files are not embedded into the
1166 1179 # notebook output.
1167 1180 if not self.embed:
1168 1181 url = self.url if self.url is not None else self.filename
1169 1182 output = """<video src="{0}" {1} {2} {3}>
1170 1183 Your browser does not support the <code>video</code> element.
1171 1184 </video>""".format(url, self.html_attributes, width, height)
1172 1185 return output
1173 1186
1174 1187 # Embedded videos are base64-encoded.
1175 1188 mimetype = self.mimetype
1176 1189 if self.filename is not None:
1177 1190 if not mimetype:
1178 1191 mimetype, _ = mimetypes.guess_type(self.filename)
1179 1192
1180 1193 with open(self.filename, 'rb') as f:
1181 1194 video = f.read()
1182 1195 else:
1183 1196 video = self.data
1184 1197 if isinstance(video, str):
1185 1198 # unicode input is already b64-encoded
1186 1199 b64_video = video
1187 1200 else:
1188 1201 b64_video = b2a_base64(video).decode('ascii').rstrip()
1189 1202
1190 1203 output = """<video {0} {1} {2}>
1191 1204 <source src="data:{3};base64,{4}" type="{3}">
1192 1205 Your browser does not support the video tag.
1193 1206 </video>""".format(self.html_attributes, width, height, mimetype, b64_video)
1194 1207 return output
1195 1208
1196 1209 def reload(self):
1197 1210 # TODO
1198 1211 pass
1199 1212
1200 1213
1201 1214 @skip_doctest
1202 1215 def set_matplotlib_formats(*formats, **kwargs):
1203 1216 """
1204 1217 .. deprecated:: 7.23
1205 1218
1206 1219 use `matplotlib_inline.backend_inline.set_matplotlib_formats()`
1207 1220
1208 1221 Select figure formats for the inline backend. Optionally pass quality for JPEG.
1209 1222
1210 1223 For example, this enables PNG and JPEG output with a JPEG quality of 90%::
1211 1224
1212 1225 In [1]: set_matplotlib_formats('png', 'jpeg', quality=90)
1213 1226
1214 1227 To set this in your config files use the following::
1215 1228
1216 1229 c.InlineBackend.figure_formats = {'png', 'jpeg'}
1217 1230 c.InlineBackend.print_figure_kwargs.update({'quality' : 90})
1218 1231
1219 1232 Parameters
1220 1233 ----------
1221 1234 *formats : strs
1222 1235 One or more figure formats to enable: 'png', 'retina', 'jpeg', 'svg', 'pdf'.
1223 1236 **kwargs
1224 1237 Keyword args will be relayed to ``figure.canvas.print_figure``.
1225 1238 """
1226 1239 warnings.warn(
1227 1240 "`set_matplotlib_formats` is deprecated since IPython 7.23, directly "
1228 1241 "use `matplotlib_inline.backend_inline.set_matplotlib_formats()`",
1229 1242 DeprecationWarning,
1230 1243 stacklevel=2,
1231 1244 )
1232 1245
1233 1246 from matplotlib_inline.backend_inline import (
1234 1247 set_matplotlib_formats as set_matplotlib_formats_orig,
1235 1248 )
1236 1249
1237 1250 set_matplotlib_formats_orig(*formats, **kwargs)
1238 1251
1239 1252 @skip_doctest
1240 1253 def set_matplotlib_close(close=True):
1241 1254 """
1242 1255 .. deprecated:: 7.23
1243 1256
1244 1257 use `matplotlib_inline.backend_inline.set_matplotlib_close()`
1245 1258
1246 1259 Set whether the inline backend closes all figures automatically or not.
1247 1260
1248 1261 By default, the inline backend used in the IPython Notebook will close all
1249 1262 matplotlib figures automatically after each cell is run. This means that
1250 1263 plots in different cells won't interfere. Sometimes, you may want to make
1251 1264 a plot in one cell and then refine it in later cells. This can be accomplished
1252 1265 by::
1253 1266
1254 1267 In [1]: set_matplotlib_close(False)
1255 1268
1256 1269 To set this in your config files use the following::
1257 1270
1258 1271 c.InlineBackend.close_figures = False
1259 1272
1260 1273 Parameters
1261 1274 ----------
1262 1275 close : bool
1263 1276 Should all matplotlib figures be automatically closed after each cell is
1264 1277 run?
1265 1278 """
1266 1279 warnings.warn(
1267 1280 "`set_matplotlib_close` is deprecated since IPython 7.23, directly "
1268 1281 "use `matplotlib_inline.backend_inline.set_matplotlib_close()`",
1269 1282 DeprecationWarning,
1270 1283 stacklevel=2,
1271 1284 )
1272 1285
1273 1286 from matplotlib_inline.backend_inline import (
1274 1287 set_matplotlib_close as set_matplotlib_close_orig,
1275 1288 )
1276 1289
1277 1290 set_matplotlib_close_orig(close)
@@ -1,167 +1,151 b''
1 1 # encoding: utf-8
2 2 """A class for managing IPython extensions."""
3 3
4 4 # Copyright (c) IPython Development Team.
5 5 # Distributed under the terms of the Modified BSD License.
6 6
7 7 import os
8 8 import os.path
9 9 import sys
10 10 from importlib import import_module, reload
11 11
12 12 from traitlets.config.configurable import Configurable
13 13 from IPython.utils.path import ensure_dir_exists, compress_user
14 14 from IPython.utils.decorators import undoc
15 15 from traitlets import Instance
16 16
17 17
18 18 #-----------------------------------------------------------------------------
19 19 # Main class
20 20 #-----------------------------------------------------------------------------
21 21
22 22 BUILTINS_EXTS = {"storemagic": False, "autoreload": False}
23 23
24 24
25 25 class ExtensionManager(Configurable):
26 26 """A class to manage IPython extensions.
27 27
28 28 An IPython extension is an importable Python module that has
29 29 a function with the signature::
30 30
31 31 def load_ipython_extension(ipython):
32 32 # Do things with ipython
33 33
34 34 This function is called after your extension is imported and the
35 35 currently active :class:`InteractiveShell` instance is passed as
36 36 the only argument. You can do anything you want with IPython at
37 37 that point, including defining new magic and aliases, adding new
38 38 components, etc.
39 39
40 40 You can also optionally define an :func:`unload_ipython_extension(ipython)`
41 41 function, which will be called if the user unloads or reloads the extension.
42 42 The extension manager will only call :func:`load_ipython_extension` again
43 43 if the extension is reloaded.
44 44
45 45 You can put your extension modules anywhere you want, as long as
46 46 they can be imported by Python's standard import mechanism. However,
47 47 to make it easy to write extensions, you can also put your extensions
48 48 in ``os.path.join(self.ipython_dir, 'extensions')``. This directory
49 49 is added to ``sys.path`` automatically.
50 50 """
51 51
52 52 shell = Instance('IPython.core.interactiveshell.InteractiveShellABC', allow_none=True)
53 53
54 54 def __init__(self, shell=None, **kwargs):
55 55 super(ExtensionManager, self).__init__(shell=shell, **kwargs)
56 56 self.shell.observe(
57 57 self._on_ipython_dir_changed, names=('ipython_dir',)
58 58 )
59 59 self.loaded = set()
60 60
61 61 @property
62 62 def ipython_extension_dir(self):
63 63 return os.path.join(self.shell.ipython_dir, u'extensions')
64 64
65 65 def _on_ipython_dir_changed(self, change):
66 66 ensure_dir_exists(self.ipython_extension_dir)
67 67
68 68 def load_extension(self, module_str: str):
69 69 """Load an IPython extension by its module name.
70 70
71 71 Returns the string "already loaded" if the extension is already loaded,
72 72 "no load function" if the module doesn't have a load_ipython_extension
73 73 function, or None if it succeeded.
74 74 """
75 75 try:
76 76 return self._load_extension(module_str)
77 77 except ModuleNotFoundError:
78 78 if module_str in BUILTINS_EXTS:
79 79 BUILTINS_EXTS[module_str] = True
80 80 return self._load_extension("IPython.extensions." + module_str)
81 81 raise
82 82
83 83 def _load_extension(self, module_str: str):
84 84 if module_str in self.loaded:
85 85 return "already loaded"
86 86
87 87 from IPython.utils.syspathcontext import prepended_to_syspath
88 88
89 89 with self.shell.builtin_trap:
90 90 if module_str not in sys.modules:
91 with prepended_to_syspath(self.ipython_extension_dir):
92 mod = import_module(module_str)
93 if mod.__file__.startswith(self.ipython_extension_dir):
94 print(("Loading extensions from {dir} is deprecated. "
95 "We recommend managing extensions like any "
96 "other Python packages, in site-packages.").format(
97 dir=compress_user(self.ipython_extension_dir)))
91 mod = import_module(module_str)
98 92 mod = sys.modules[module_str]
99 93 if self._call_load_ipython_extension(mod):
100 94 self.loaded.add(module_str)
101 95 else:
102 96 return "no load function"
103 97
104 98 def unload_extension(self, module_str: str):
105 99 """Unload an IPython extension by its module name.
106 100
107 101 This function looks up the extension's name in ``sys.modules`` and
108 102 simply calls ``mod.unload_ipython_extension(self)``.
109 103
110 104 Returns the string "no unload function" if the extension doesn't define
111 105 a function to unload itself, "not loaded" if the extension isn't loaded,
112 106 otherwise None.
113 107 """
114 108 if BUILTINS_EXTS.get(module_str, False) is True:
115 109 module_str = "IPython.extensions." + module_str
116 110 if module_str not in self.loaded:
117 111 return "not loaded"
118 112
119 113 if module_str in sys.modules:
120 114 mod = sys.modules[module_str]
121 115 if self._call_unload_ipython_extension(mod):
122 116 self.loaded.discard(module_str)
123 117 else:
124 118 return "no unload function"
125 119
126 120 def reload_extension(self, module_str: str):
127 121 """Reload an IPython extension by calling reload.
128 122
129 123 If the module has not been loaded before,
130 124 :meth:`InteractiveShell.load_extension` is called. Otherwise
131 125 :func:`reload` is called and then the :func:`load_ipython_extension`
132 126 function of the module, if it exists is called.
133 127 """
134 128 from IPython.utils.syspathcontext import prepended_to_syspath
135 129
136 130 if BUILTINS_EXTS.get(module_str, False) is True:
137 131 module_str = "IPython.extensions." + module_str
138 132
139 133 if (module_str in self.loaded) and (module_str in sys.modules):
140 134 self.unload_extension(module_str)
141 135 mod = sys.modules[module_str]
142 136 with prepended_to_syspath(self.ipython_extension_dir):
143 137 reload(mod)
144 138 if self._call_load_ipython_extension(mod):
145 139 self.loaded.add(module_str)
146 140 else:
147 141 self.load_extension(module_str)
148 142
149 143 def _call_load_ipython_extension(self, mod):
150 144 if hasattr(mod, 'load_ipython_extension'):
151 145 mod.load_ipython_extension(self.shell)
152 146 return True
153 147
154 148 def _call_unload_ipython_extension(self, mod):
155 149 if hasattr(mod, 'unload_ipython_extension'):
156 150 mod.unload_ipython_extension(self.shell)
157 151 return True
158
159 @undoc
160 def install_extension(self, url, filename=None):
161 """
162 Deprecated.
163 """
164 # Ensure the extension directory exists
165 raise DeprecationWarning(
166 '`install_extension` and the `install_ext` magic have been deprecated since IPython 4.0'
167 'Use pip or other package managers to manage ipython extensions.')
@@ -1,787 +1,797 b''
1 1 """Input transformer machinery to support IPython special syntax.
2 2
3 3 This includes the machinery to recognise and transform ``%magic`` commands,
4 4 ``!system`` commands, ``help?`` querying, prompt stripping, and so forth.
5 5
6 6 Added: IPython 7.0. Replaces inputsplitter and inputtransformer which were
7 7 deprecated in 7.0.
8 8 """
9 9
10 10 # Copyright (c) IPython Development Team.
11 11 # Distributed under the terms of the Modified BSD License.
12 12
13 13 import ast
14 14 from codeop import CommandCompiler, Compile
15 15 import re
16 16 import tokenize
17 17 from typing import List, Tuple, Optional, Any
18 18 import warnings
19 19
20 20 _indent_re = re.compile(r'^[ \t]+')
21 21
22 22 def leading_empty_lines(lines):
23 23 """Remove leading empty lines
24 24
25 25 If the leading lines are empty or contain only whitespace, they will be
26 26 removed.
27 27 """
28 28 if not lines:
29 29 return lines
30 30 for i, line in enumerate(lines):
31 31 if line and not line.isspace():
32 32 return lines[i:]
33 33 return lines
34 34
35 35 def leading_indent(lines):
36 36 """Remove leading indentation.
37 37
38 38 If the first line starts with a spaces or tabs, the same whitespace will be
39 39 removed from each following line in the cell.
40 40 """
41 41 if not lines:
42 42 return lines
43 43 m = _indent_re.match(lines[0])
44 44 if not m:
45 45 return lines
46 46 space = m.group(0)
47 47 n = len(space)
48 48 return [l[n:] if l.startswith(space) else l
49 49 for l in lines]
50 50
51 51 class PromptStripper:
52 52 """Remove matching input prompts from a block of input.
53 53
54 54 Parameters
55 55 ----------
56 56 prompt_re : regular expression
57 57 A regular expression matching any input prompt (including continuation,
58 58 e.g. ``...``)
59 59 initial_re : regular expression, optional
60 60 A regular expression matching only the initial prompt, but not continuation.
61 61 If no initial expression is given, prompt_re will be used everywhere.
62 62 Used mainly for plain Python prompts (``>>>``), where the continuation prompt
63 63 ``...`` is a valid Python expression in Python 3, so shouldn't be stripped.
64 64
65 65 Notes
66 66 -----
67 67
68 68 If initial_re and prompt_re differ,
69 69 only initial_re will be tested against the first line.
70 70 If any prompt is found on the first two lines,
71 71 prompts will be stripped from the rest of the block.
72 72 """
73 73 def __init__(self, prompt_re, initial_re=None):
74 74 self.prompt_re = prompt_re
75 75 self.initial_re = initial_re or prompt_re
76 76
77 77 def _strip(self, lines):
78 78 return [self.prompt_re.sub('', l, count=1) for l in lines]
79 79
80 80 def __call__(self, lines):
81 81 if not lines:
82 82 return lines
83 83 if self.initial_re.match(lines[0]) or \
84 84 (len(lines) > 1 and self.prompt_re.match(lines[1])):
85 85 return self._strip(lines)
86 86 return lines
87 87
88 88 classic_prompt = PromptStripper(
89 89 prompt_re=re.compile(r'^(>>>|\.\.\.)( |$)'),
90 90 initial_re=re.compile(r'^>>>( |$)')
91 91 )
92 92
93 93 ipython_prompt = PromptStripper(
94 94 re.compile(
95 95 r"""
96 96 ^( # Match from the beginning of a line, either:
97 97
98 98 # 1. First-line prompt:
99 99 ((\[nav\]|\[ins\])?\ )? # Vi editing mode prompt, if it's there
100 100 In\ # The 'In' of the prompt, with a space
101 101 \[\d+\]: # Command index, as displayed in the prompt
102 102 \ # With a mandatory trailing space
103 103
104 104 | # ... or ...
105 105
106 106 # 2. The three dots of the multiline prompt
107 107 \s* # All leading whitespace characters
108 108 \.{3,}: # The three (or more) dots
109 109 \ ? # With an optional trailing space
110 110
111 111 )
112 112 """,
113 113 re.VERBOSE,
114 114 )
115 115 )
116 116
117 117
118 118 def cell_magic(lines):
119 119 if not lines or not lines[0].startswith('%%'):
120 120 return lines
121 121 if re.match(r'%%\w+\?', lines[0]):
122 122 # This case will be handled by help_end
123 123 return lines
124 124 magic_name, _, first_line = lines[0][2:].rstrip().partition(' ')
125 125 body = ''.join(lines[1:])
126 126 return ['get_ipython().run_cell_magic(%r, %r, %r)\n'
127 127 % (magic_name, first_line, body)]
128 128
129 129
130 130 def _find_assign_op(token_line) -> Optional[int]:
131 131 """Get the index of the first assignment in the line ('=' not inside brackets)
132 132
133 133 Note: We don't try to support multiple special assignment (a = b = %foo)
134 134 """
135 135 paren_level = 0
136 136 for i, ti in enumerate(token_line):
137 137 s = ti.string
138 138 if s == '=' and paren_level == 0:
139 139 return i
140 140 if s in {'(','[','{'}:
141 141 paren_level += 1
142 142 elif s in {')', ']', '}'}:
143 143 if paren_level > 0:
144 144 paren_level -= 1
145 145 return None
146 146
147 147 def find_end_of_continued_line(lines, start_line: int):
148 148 """Find the last line of a line explicitly extended using backslashes.
149 149
150 150 Uses 0-indexed line numbers.
151 151 """
152 152 end_line = start_line
153 153 while lines[end_line].endswith('\\\n'):
154 154 end_line += 1
155 155 if end_line >= len(lines):
156 156 break
157 157 return end_line
158 158
159 159 def assemble_continued_line(lines, start: Tuple[int, int], end_line: int):
160 160 r"""Assemble a single line from multiple continued line pieces
161 161
162 162 Continued lines are lines ending in ``\``, and the line following the last
163 163 ``\`` in the block.
164 164
165 165 For example, this code continues over multiple lines::
166 166
167 167 if (assign_ix is not None) \
168 168 and (len(line) >= assign_ix + 2) \
169 169 and (line[assign_ix+1].string == '%') \
170 170 and (line[assign_ix+2].type == tokenize.NAME):
171 171
172 172 This statement contains four continued line pieces.
173 173 Assembling these pieces into a single line would give::
174 174
175 175 if (assign_ix is not None) and (len(line) >= assign_ix + 2) and (line[...
176 176
177 177 This uses 0-indexed line numbers. *start* is (lineno, colno).
178 178
179 179 Used to allow ``%magic`` and ``!system`` commands to be continued over
180 180 multiple lines.
181 181 """
182 182 parts = [lines[start[0]][start[1]:]] + lines[start[0]+1:end_line+1]
183 183 return ' '.join([p.rstrip()[:-1] for p in parts[:-1]] # Strip backslash+newline
184 184 + [parts[-1].rstrip()]) # Strip newline from last line
185 185
186 186 class TokenTransformBase:
187 187 """Base class for transformations which examine tokens.
188 188
189 189 Special syntax should not be transformed when it occurs inside strings or
190 190 comments. This is hard to reliably avoid with regexes. The solution is to
191 191 tokenise the code as Python, and recognise the special syntax in the tokens.
192 192
193 193 IPython's special syntax is not valid Python syntax, so tokenising may go
194 194 wrong after the special syntax starts. These classes therefore find and
195 195 transform *one* instance of special syntax at a time into regular Python
196 196 syntax. After each transformation, tokens are regenerated to find the next
197 197 piece of special syntax.
198 198
199 199 Subclasses need to implement one class method (find)
200 200 and one regular method (transform).
201 201
202 202 The priority attribute can select which transformation to apply if multiple
203 203 transformers match in the same place. Lower numbers have higher priority.
204 204 This allows "%magic?" to be turned into a help call rather than a magic call.
205 205 """
206 206 # Lower numbers -> higher priority (for matches in the same location)
207 207 priority = 10
208 208
209 209 def sortby(self):
210 210 return self.start_line, self.start_col, self.priority
211 211
212 212 def __init__(self, start):
213 213 self.start_line = start[0] - 1 # Shift from 1-index to 0-index
214 214 self.start_col = start[1]
215 215
216 216 @classmethod
217 217 def find(cls, tokens_by_line):
218 218 """Find one instance of special syntax in the provided tokens.
219 219
220 220 Tokens are grouped into logical lines for convenience,
221 221 so it is easy to e.g. look at the first token of each line.
222 222 *tokens_by_line* is a list of lists of tokenize.TokenInfo objects.
223 223
224 224 This should return an instance of its class, pointing to the start
225 225 position it has found, or None if it found no match.
226 226 """
227 227 raise NotImplementedError
228 228
229 229 def transform(self, lines: List[str]):
230 230 """Transform one instance of special syntax found by ``find()``
231 231
232 232 Takes a list of strings representing physical lines,
233 233 returns a similar list of transformed lines.
234 234 """
235 235 raise NotImplementedError
236 236
237 237 class MagicAssign(TokenTransformBase):
238 238 """Transformer for assignments from magics (a = %foo)"""
239 239 @classmethod
240 240 def find(cls, tokens_by_line):
241 241 """Find the first magic assignment (a = %foo) in the cell.
242 242 """
243 243 for line in tokens_by_line:
244 244 assign_ix = _find_assign_op(line)
245 245 if (assign_ix is not None) \
246 246 and (len(line) >= assign_ix + 2) \
247 247 and (line[assign_ix+1].string == '%') \
248 248 and (line[assign_ix+2].type == tokenize.NAME):
249 249 return cls(line[assign_ix+1].start)
250 250
251 251 def transform(self, lines: List[str]):
252 252 """Transform a magic assignment found by the ``find()`` classmethod.
253 253 """
254 254 start_line, start_col = self.start_line, self.start_col
255 255 lhs = lines[start_line][:start_col]
256 256 end_line = find_end_of_continued_line(lines, start_line)
257 257 rhs = assemble_continued_line(lines, (start_line, start_col), end_line)
258 258 assert rhs.startswith('%'), rhs
259 259 magic_name, _, args = rhs[1:].partition(' ')
260 260
261 261 lines_before = lines[:start_line]
262 262 call = "get_ipython().run_line_magic({!r}, {!r})".format(magic_name, args)
263 263 new_line = lhs + call + '\n'
264 264 lines_after = lines[end_line+1:]
265 265
266 266 return lines_before + [new_line] + lines_after
267 267
268 268
269 269 class SystemAssign(TokenTransformBase):
270 270 """Transformer for assignments from system commands (a = !foo)"""
271 271 @classmethod
272 272 def find(cls, tokens_by_line):
273 273 """Find the first system assignment (a = !foo) in the cell.
274 274 """
275 275 for line in tokens_by_line:
276 276 assign_ix = _find_assign_op(line)
277 277 if (assign_ix is not None) \
278 278 and not line[assign_ix].line.strip().startswith('=') \
279 279 and (len(line) >= assign_ix + 2) \
280 280 and (line[assign_ix + 1].type == tokenize.ERRORTOKEN):
281 281 ix = assign_ix + 1
282 282
283 283 while ix < len(line) and line[ix].type == tokenize.ERRORTOKEN:
284 284 if line[ix].string == '!':
285 285 return cls(line[ix].start)
286 286 elif not line[ix].string.isspace():
287 287 break
288 288 ix += 1
289 289
290 290 def transform(self, lines: List[str]):
291 291 """Transform a system assignment found by the ``find()`` classmethod.
292 292 """
293 293 start_line, start_col = self.start_line, self.start_col
294 294
295 295 lhs = lines[start_line][:start_col]
296 296 end_line = find_end_of_continued_line(lines, start_line)
297 297 rhs = assemble_continued_line(lines, (start_line, start_col), end_line)
298 298 assert rhs.startswith('!'), rhs
299 299 cmd = rhs[1:]
300 300
301 301 lines_before = lines[:start_line]
302 302 call = "get_ipython().getoutput({!r})".format(cmd)
303 303 new_line = lhs + call + '\n'
304 304 lines_after = lines[end_line + 1:]
305 305
306 306 return lines_before + [new_line] + lines_after
307 307
308 308 # The escape sequences that define the syntax transformations IPython will
309 309 # apply to user input. These can NOT be just changed here: many regular
310 310 # expressions and other parts of the code may use their hardcoded values, and
311 311 # for all intents and purposes they constitute the 'IPython syntax', so they
312 312 # should be considered fixed.
313 313
314 314 ESC_SHELL = '!' # Send line to underlying system shell
315 315 ESC_SH_CAP = '!!' # Send line to system shell and capture output
316 316 ESC_HELP = '?' # Find information about object
317 317 ESC_HELP2 = '??' # Find extra-detailed information about object
318 318 ESC_MAGIC = '%' # Call magic function
319 319 ESC_MAGIC2 = '%%' # Call cell-magic function
320 320 ESC_QUOTE = ',' # Split args on whitespace, quote each as string and call
321 321 ESC_QUOTE2 = ';' # Quote all args as a single string, call
322 322 ESC_PAREN = '/' # Call first argument with rest of line as arguments
323 323
324 324 ESCAPE_SINGLES = {'!', '?', '%', ',', ';', '/'}
325 325 ESCAPE_DOUBLES = {'!!', '??'} # %% (cell magic) is handled separately
326 326
327 327 def _make_help_call(target, esc):
328 328 """Prepares a pinfo(2)/psearch call from a target name and the escape
329 329 (i.e. ? or ??)"""
330 330 method = 'pinfo2' if esc == '??' \
331 331 else 'psearch' if '*' in target \
332 332 else 'pinfo'
333 333 arg = " ".join([method, target])
334 334 #Prepare arguments for get_ipython().run_line_magic(magic_name, magic_args)
335 335 t_magic_name, _, t_magic_arg_s = arg.partition(' ')
336 336 t_magic_name = t_magic_name.lstrip(ESC_MAGIC)
337 337 return "get_ipython().run_line_magic(%r, %r)" % (t_magic_name, t_magic_arg_s)
338 338
339 339
340 340 def _tr_help(content):
341 341 """Translate lines escaped with: ?
342 342
343 343 A naked help line should fire the intro help screen (shell.show_usage())
344 344 """
345 345 if not content:
346 346 return 'get_ipython().show_usage()'
347 347
348 348 return _make_help_call(content, '?')
349 349
350 350 def _tr_help2(content):
351 351 """Translate lines escaped with: ??
352 352
353 353 A naked help line should fire the intro help screen (shell.show_usage())
354 354 """
355 355 if not content:
356 356 return 'get_ipython().show_usage()'
357 357
358 358 return _make_help_call(content, '??')
359 359
360 360 def _tr_magic(content):
361 361 "Translate lines escaped with a percent sign: %"
362 362 name, _, args = content.partition(' ')
363 363 return 'get_ipython().run_line_magic(%r, %r)' % (name, args)
364 364
365 365 def _tr_quote(content):
366 366 "Translate lines escaped with a comma: ,"
367 367 name, _, args = content.partition(' ')
368 368 return '%s("%s")' % (name, '", "'.join(args.split()) )
369 369
370 370 def _tr_quote2(content):
371 371 "Translate lines escaped with a semicolon: ;"
372 372 name, _, args = content.partition(' ')
373 373 return '%s("%s")' % (name, args)
374 374
375 375 def _tr_paren(content):
376 376 "Translate lines escaped with a slash: /"
377 377 name, _, args = content.partition(' ')
378 378 return '%s(%s)' % (name, ", ".join(args.split()))
379 379
380 380 tr = { ESC_SHELL : 'get_ipython().system({!r})'.format,
381 381 ESC_SH_CAP : 'get_ipython().getoutput({!r})'.format,
382 382 ESC_HELP : _tr_help,
383 383 ESC_HELP2 : _tr_help2,
384 384 ESC_MAGIC : _tr_magic,
385 385 ESC_QUOTE : _tr_quote,
386 386 ESC_QUOTE2 : _tr_quote2,
387 387 ESC_PAREN : _tr_paren }
388 388
389 389 class EscapedCommand(TokenTransformBase):
390 390 """Transformer for escaped commands like %foo, !foo, or /foo"""
391 391 @classmethod
392 392 def find(cls, tokens_by_line):
393 393 """Find the first escaped command (%foo, !foo, etc.) in the cell.
394 394 """
395 395 for line in tokens_by_line:
396 396 if not line:
397 397 continue
398 398 ix = 0
399 399 ll = len(line)
400 400 while ll > ix and line[ix].type in {tokenize.INDENT, tokenize.DEDENT}:
401 401 ix += 1
402 402 if ix >= ll:
403 403 continue
404 404 if line[ix].string in ESCAPE_SINGLES:
405 405 return cls(line[ix].start)
406 406
407 407 def transform(self, lines):
408 408 """Transform an escaped line found by the ``find()`` classmethod.
409 409 """
410 410 start_line, start_col = self.start_line, self.start_col
411 411
412 412 indent = lines[start_line][:start_col]
413 413 end_line = find_end_of_continued_line(lines, start_line)
414 414 line = assemble_continued_line(lines, (start_line, start_col), end_line)
415 415
416 416 if len(line) > 1 and line[:2] in ESCAPE_DOUBLES:
417 417 escape, content = line[:2], line[2:]
418 418 else:
419 419 escape, content = line[:1], line[1:]
420 420
421 421 if escape in tr:
422 422 call = tr[escape](content)
423 423 else:
424 424 call = ''
425 425
426 426 lines_before = lines[:start_line]
427 427 new_line = indent + call + '\n'
428 428 lines_after = lines[end_line + 1:]
429 429
430 430 return lines_before + [new_line] + lines_after
431 431
432 _help_end_re = re.compile(r"""(%{0,2}
433 (?!\d)[\w*]+ # Variable name
434 (\.(?!\d)[\w*]+)* # .etc.etc
435 )
436 (\?\??)$ # ? or ??
437 """,
438 re.VERBOSE)
432
433 _help_end_re = re.compile(
434 r"""(%{0,2}
435 (?!\d)[\w*]+ # Variable name
436 (\.(?!\d)[\w*]+|\[-?[0-9]+\])* # .etc.etc or [0], we only support literal integers.
437 )
438 (\?\??)$ # ? or ??
439 """,
440 re.VERBOSE,
441 )
442
439 443
440 444 class HelpEnd(TokenTransformBase):
441 445 """Transformer for help syntax: obj? and obj??"""
442 446 # This needs to be higher priority (lower number) than EscapedCommand so
443 447 # that inspecting magics (%foo?) works.
444 448 priority = 5
445 449
446 450 def __init__(self, start, q_locn):
447 451 super().__init__(start)
448 452 self.q_line = q_locn[0] - 1 # Shift from 1-indexed to 0-indexed
449 453 self.q_col = q_locn[1]
450 454
451 455 @classmethod
452 456 def find(cls, tokens_by_line):
453 457 """Find the first help command (foo?) in the cell.
454 458 """
455 459 for line in tokens_by_line:
456 460 # Last token is NEWLINE; look at last but one
457 461 if len(line) > 2 and line[-2].string == '?':
458 462 # Find the first token that's not INDENT/DEDENT
459 463 ix = 0
460 464 while line[ix].type in {tokenize.INDENT, tokenize.DEDENT}:
461 465 ix += 1
462 466 return cls(line[ix].start, line[-2].start)
463 467
464 468 def transform(self, lines):
465 469 """Transform a help command found by the ``find()`` classmethod.
466 470 """
467 piece = ''.join(lines[self.start_line:self.q_line+1])
468 indent, content = piece[:self.start_col], piece[self.start_col:]
469 lines_before = lines[:self.start_line]
470 lines_after = lines[self.q_line + 1:]
471
472 piece = "".join(lines[self.start_line : self.q_line + 1])
473 indent, content = piece[: self.start_col], piece[self.start_col :]
474 lines_before = lines[: self.start_line]
475 lines_after = lines[self.q_line + 1 :]
471 476
472 477 m = _help_end_re.search(content)
473 478 if not m:
474 479 raise SyntaxError(content)
475 480 assert m is not None, content
476 481 target = m.group(1)
477 482 esc = m.group(3)
478 483
479 484
480 485 call = _make_help_call(target, esc)
481 486 new_line = indent + call + '\n'
482 487
483 488 return lines_before + [new_line] + lines_after
484 489
485 490 def make_tokens_by_line(lines:List[str]):
486 491 """Tokenize a series of lines and group tokens by line.
487 492
488 493 The tokens for a multiline Python string or expression are grouped as one
489 494 line. All lines except the last lines should keep their line ending ('\\n',
490 495 '\\r\\n') for this to properly work. Use `.splitlines(keeplineending=True)`
491 496 for example when passing block of text to this function.
492 497
493 498 """
494 499 # NL tokens are used inside multiline expressions, but also after blank
495 500 # lines or comments. This is intentional - see https://bugs.python.org/issue17061
496 501 # We want to group the former case together but split the latter, so we
497 502 # track parentheses level, similar to the internals of tokenize.
498 503
499 504 # reexported from token on 3.7+
500 505 NEWLINE, NL = tokenize.NEWLINE, tokenize.NL # type: ignore
501 506 tokens_by_line: List[List[Any]] = [[]]
502 507 if len(lines) > 1 and not lines[0].endswith(("\n", "\r", "\r\n", "\x0b", "\x0c")):
503 508 warnings.warn(
504 509 "`make_tokens_by_line` received a list of lines which do not have lineending markers ('\\n', '\\r', '\\r\\n', '\\x0b', '\\x0c'), behavior will be unspecified",
505 510 stacklevel=2,
506 511 )
507 512 parenlev = 0
508 513 try:
509 514 for token in tokenize.generate_tokens(iter(lines).__next__):
510 515 tokens_by_line[-1].append(token)
511 516 if (token.type == NEWLINE) \
512 517 or ((token.type == NL) and (parenlev <= 0)):
513 518 tokens_by_line.append([])
514 519 elif token.string in {'(', '[', '{'}:
515 520 parenlev += 1
516 521 elif token.string in {')', ']', '}'}:
517 522 if parenlev > 0:
518 523 parenlev -= 1
519 524 except tokenize.TokenError:
520 525 # Input ended in a multiline string or expression. That's OK for us.
521 526 pass
522 527
523 528
524 529 if not tokens_by_line[-1]:
525 530 tokens_by_line.pop()
526 531
527 532
528 533 return tokens_by_line
529 534
530 535
531 536 def has_sunken_brackets(tokens: List[tokenize.TokenInfo]):
532 537 """Check if the depth of brackets in the list of tokens drops below 0"""
533 538 parenlev = 0
534 539 for token in tokens:
535 540 if token.string in {"(", "[", "{"}:
536 541 parenlev += 1
537 542 elif token.string in {")", "]", "}"}:
538 543 parenlev -= 1
539 544 if parenlev < 0:
540 545 return True
541 546 return False
542 547
543 548
544 549 def show_linewise_tokens(s: str):
545 550 """For investigation and debugging"""
546 if not s.endswith('\n'):
547 s += '\n'
551 warnings.warn(
552 "show_linewise_tokens is deprecated since IPython 8.6",
553 DeprecationWarning,
554 stacklevel=2,
555 )
556 if not s.endswith("\n"):
557 s += "\n"
548 558 lines = s.splitlines(keepends=True)
549 559 for line in make_tokens_by_line(lines):
550 560 print("Line -------")
551 561 for tokinfo in line:
552 562 print(" ", tokinfo)
553 563
554 564 # Arbitrary limit to prevent getting stuck in infinite loops
555 565 TRANSFORM_LOOP_LIMIT = 500
556 566
557 567 class TransformerManager:
558 568 """Applies various transformations to a cell or code block.
559 569
560 570 The key methods for external use are ``transform_cell()``
561 571 and ``check_complete()``.
562 572 """
563 573 def __init__(self):
564 574 self.cleanup_transforms = [
565 575 leading_empty_lines,
566 576 leading_indent,
567 577 classic_prompt,
568 578 ipython_prompt,
569 579 ]
570 580 self.line_transforms = [
571 581 cell_magic,
572 582 ]
573 583 self.token_transformers = [
574 584 MagicAssign,
575 585 SystemAssign,
576 586 EscapedCommand,
577 587 HelpEnd,
578 588 ]
579 589
580 590 def do_one_token_transform(self, lines):
581 591 """Find and run the transform earliest in the code.
582 592
583 593 Returns (changed, lines).
584 594
585 595 This method is called repeatedly until changed is False, indicating
586 596 that all available transformations are complete.
587 597
588 598 The tokens following IPython special syntax might not be valid, so
589 599 the transformed code is retokenised every time to identify the next
590 600 piece of special syntax. Hopefully long code cells are mostly valid
591 601 Python, not using lots of IPython special syntax, so this shouldn't be
592 602 a performance issue.
593 603 """
594 604 tokens_by_line = make_tokens_by_line(lines)
595 605 candidates = []
596 606 for transformer_cls in self.token_transformers:
597 607 transformer = transformer_cls.find(tokens_by_line)
598 608 if transformer:
599 609 candidates.append(transformer)
600 610
601 611 if not candidates:
602 612 # Nothing to transform
603 613 return False, lines
604 614 ordered_transformers = sorted(candidates, key=TokenTransformBase.sortby)
605 615 for transformer in ordered_transformers:
606 616 try:
607 617 return True, transformer.transform(lines)
608 618 except SyntaxError:
609 619 pass
610 620 return False, lines
611 621
612 622 def do_token_transforms(self, lines):
613 623 for _ in range(TRANSFORM_LOOP_LIMIT):
614 624 changed, lines = self.do_one_token_transform(lines)
615 625 if not changed:
616 626 return lines
617 627
618 628 raise RuntimeError("Input transformation still changing after "
619 629 "%d iterations. Aborting." % TRANSFORM_LOOP_LIMIT)
620 630
621 631 def transform_cell(self, cell: str) -> str:
622 632 """Transforms a cell of input code"""
623 633 if not cell.endswith('\n'):
624 634 cell += '\n' # Ensure the cell has a trailing newline
625 635 lines = cell.splitlines(keepends=True)
626 636 for transform in self.cleanup_transforms + self.line_transforms:
627 637 lines = transform(lines)
628 638
629 639 lines = self.do_token_transforms(lines)
630 640 return ''.join(lines)
631 641
632 642 def check_complete(self, cell: str):
633 643 """Return whether a block of code is ready to execute, or should be continued
634 644
635 645 Parameters
636 646 ----------
637 647 cell : string
638 648 Python input code, which can be multiline.
639 649
640 650 Returns
641 651 -------
642 652 status : str
643 653 One of 'complete', 'incomplete', or 'invalid' if source is not a
644 654 prefix of valid code.
645 655 indent_spaces : int or None
646 656 The number of spaces by which to indent the next line of code. If
647 657 status is not 'incomplete', this is None.
648 658 """
649 659 # Remember if the lines ends in a new line.
650 660 ends_with_newline = False
651 661 for character in reversed(cell):
652 662 if character == '\n':
653 663 ends_with_newline = True
654 664 break
655 665 elif character.strip():
656 666 break
657 667 else:
658 668 continue
659 669
660 670 if not ends_with_newline:
661 671 # Append an newline for consistent tokenization
662 672 # See https://bugs.python.org/issue33899
663 673 cell += '\n'
664 674
665 675 lines = cell.splitlines(keepends=True)
666 676
667 677 if not lines:
668 678 return 'complete', None
669 679
670 680 if lines[-1].endswith('\\'):
671 681 # Explicit backslash continuation
672 682 return 'incomplete', find_last_indent(lines)
673 683
674 684 try:
675 685 for transform in self.cleanup_transforms:
676 686 if not getattr(transform, 'has_side_effects', False):
677 687 lines = transform(lines)
678 688 except SyntaxError:
679 689 return 'invalid', None
680 690
681 691 if lines[0].startswith('%%'):
682 692 # Special case for cell magics - completion marked by blank line
683 693 if lines[-1].strip():
684 694 return 'incomplete', find_last_indent(lines)
685 695 else:
686 696 return 'complete', None
687 697
688 698 try:
689 699 for transform in self.line_transforms:
690 700 if not getattr(transform, 'has_side_effects', False):
691 701 lines = transform(lines)
692 702 lines = self.do_token_transforms(lines)
693 703 except SyntaxError:
694 704 return 'invalid', None
695 705
696 706 tokens_by_line = make_tokens_by_line(lines)
697 707
698 708 # Bail if we got one line and there are more closing parentheses than
699 709 # the opening ones
700 710 if (
701 711 len(lines) == 1
702 712 and tokens_by_line
703 713 and has_sunken_brackets(tokens_by_line[0])
704 714 ):
705 715 return "invalid", None
706 716
707 717 if not tokens_by_line:
708 718 return 'incomplete', find_last_indent(lines)
709 719
710 720 if tokens_by_line[-1][-1].type != tokenize.ENDMARKER:
711 721 # We're in a multiline string or expression
712 722 return 'incomplete', find_last_indent(lines)
713 723
714 724 newline_types = {tokenize.NEWLINE, tokenize.COMMENT, tokenize.ENDMARKER} # type: ignore
715 725
716 726 # Pop the last line which only contains DEDENTs and ENDMARKER
717 727 last_token_line = None
718 728 if {t.type for t in tokens_by_line[-1]} in [
719 729 {tokenize.DEDENT, tokenize.ENDMARKER},
720 730 {tokenize.ENDMARKER}
721 731 ] and len(tokens_by_line) > 1:
722 732 last_token_line = tokens_by_line.pop()
723 733
724 734 while tokens_by_line[-1] and tokens_by_line[-1][-1].type in newline_types:
725 735 tokens_by_line[-1].pop()
726 736
727 737 if not tokens_by_line[-1]:
728 738 return 'incomplete', find_last_indent(lines)
729 739
730 740 if tokens_by_line[-1][-1].string == ':':
731 741 # The last line starts a block (e.g. 'if foo:')
732 742 ix = 0
733 743 while tokens_by_line[-1][ix].type in {tokenize.INDENT, tokenize.DEDENT}:
734 744 ix += 1
735 745
736 746 indent = tokens_by_line[-1][ix].start[1]
737 747 return 'incomplete', indent + 4
738 748
739 749 if tokens_by_line[-1][0].line.endswith('\\'):
740 750 return 'incomplete', None
741 751
742 752 # At this point, our checks think the code is complete (or invalid).
743 753 # We'll use codeop.compile_command to check this with the real parser
744 754 try:
745 755 with warnings.catch_warnings():
746 756 warnings.simplefilter('error', SyntaxWarning)
747 757 res = compile_command(''.join(lines), symbol='exec')
748 758 except (SyntaxError, OverflowError, ValueError, TypeError,
749 759 MemoryError, SyntaxWarning):
750 760 return 'invalid', None
751 761 else:
752 762 if res is None:
753 763 return 'incomplete', find_last_indent(lines)
754 764
755 765 if last_token_line and last_token_line[0].type == tokenize.DEDENT:
756 766 if ends_with_newline:
757 767 return 'complete', None
758 768 return 'incomplete', find_last_indent(lines)
759 769
760 770 # If there's a blank line at the end, assume we're ready to execute
761 771 if not lines[-1].strip():
762 772 return 'complete', None
763 773
764 774 return 'complete', None
765 775
766 776
767 777 def find_last_indent(lines):
768 778 m = _indent_re.match(lines[-1])
769 779 if not m:
770 780 return 0
771 781 return len(m.group(0).replace('\t', ' '*4))
772 782
773 783
774 784 class MaybeAsyncCompile(Compile):
775 785 def __init__(self, extra_flags=0):
776 786 super().__init__()
777 787 self.flags |= extra_flags
778 788
779 789
780 790 class MaybeAsyncCommandCompiler(CommandCompiler):
781 791 def __init__(self, extra_flags=0):
782 792 self.compiler = MaybeAsyncCompile(extra_flags=extra_flags)
783 793
784 794
785 795 _extra_flags = ast.PyCF_ALLOW_TOP_LEVEL_AWAIT
786 796
787 797 compile_command = MaybeAsyncCommandCompiler(extra_flags=_extra_flags)
@@ -1,3780 +1,3835 b''
1 1 # -*- coding: utf-8 -*-
2 2 """Main IPython class."""
3 3
4 4 #-----------------------------------------------------------------------------
5 5 # Copyright (C) 2001 Janko Hauser <jhauser@zscout.de>
6 6 # Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu>
7 7 # Copyright (C) 2008-2011 The IPython Development Team
8 8 #
9 9 # Distributed under the terms of the BSD License. The full license is in
10 10 # the file COPYING, distributed as part of this software.
11 11 #-----------------------------------------------------------------------------
12 12
13 13
14 14 import abc
15 15 import ast
16 16 import atexit
17 17 import bdb
18 18 import builtins as builtin_mod
19 19 import functools
20 20 import inspect
21 21 import os
22 22 import re
23 23 import runpy
24 24 import subprocess
25 25 import sys
26 26 import tempfile
27 27 import traceback
28 28 import types
29 29 import warnings
30 30 from ast import stmt
31 31 from io import open as io_open
32 32 from logging import error
33 33 from pathlib import Path
34 34 from typing import Callable
35 35 from typing import List as ListType
36 36 from typing import Optional, Tuple
37 37 from warnings import warn
38 38
39 39 from pickleshare import PickleShareDB
40 40 from tempfile import TemporaryDirectory
41 41 from traitlets import (
42 42 Any,
43 43 Bool,
44 44 CaselessStrEnum,
45 45 Dict,
46 46 Enum,
47 47 Instance,
48 48 Integer,
49 49 List,
50 50 Type,
51 51 Unicode,
52 52 default,
53 53 observe,
54 54 validate,
55 55 )
56 56 from traitlets.config.configurable import SingletonConfigurable
57 57 from traitlets.utils.importstring import import_item
58 58
59 59 import IPython.core.hooks
60 60 from IPython.core import magic, oinspect, page, prefilter, ultratb
61 61 from IPython.core.alias import Alias, AliasManager
62 62 from IPython.core.autocall import ExitAutocall
63 63 from IPython.core.builtin_trap import BuiltinTrap
64 from IPython.core.compilerop import CachingCompiler, check_linecache_ipython
64 from IPython.core.compilerop import CachingCompiler
65 65 from IPython.core.debugger import InterruptiblePdb
66 66 from IPython.core.display_trap import DisplayTrap
67 67 from IPython.core.displayhook import DisplayHook
68 68 from IPython.core.displaypub import DisplayPublisher
69 69 from IPython.core.error import InputRejected, UsageError
70 70 from IPython.core.events import EventManager, available_events
71 71 from IPython.core.extensions import ExtensionManager
72 72 from IPython.core.formatters import DisplayFormatter
73 73 from IPython.core.history import HistoryManager
74 74 from IPython.core.inputtransformer2 import ESC_MAGIC, ESC_MAGIC2
75 75 from IPython.core.logger import Logger
76 76 from IPython.core.macro import Macro
77 77 from IPython.core.payload import PayloadManager
78 78 from IPython.core.prefilter import PrefilterManager
79 79 from IPython.core.profiledir import ProfileDir
80 80 from IPython.core.usage import default_banner
81 81 from IPython.display import display
82 82 from IPython.paths import get_ipython_dir
83 83 from IPython.testing.skipdoctest import skip_doctest
84 84 from IPython.utils import PyColorize, io, openpy, py3compat
85 85 from IPython.utils.decorators import undoc
86 86 from IPython.utils.io import ask_yes_no
87 87 from IPython.utils.ipstruct import Struct
88 88 from IPython.utils.path import ensure_dir_exists, get_home_dir, get_py_filename
89 89 from IPython.utils.process import getoutput, system
90 90 from IPython.utils.strdispatch import StrDispatch
91 91 from IPython.utils.syspathcontext import prepended_to_syspath
92 92 from IPython.utils.text import DollarFormatter, LSString, SList, format_screen
93 93
94 94 sphinxify: Optional[Callable]
95 95
96 96 try:
97 97 import docrepr.sphinxify as sphx
98 98
99 99 def sphinxify(oinfo):
100 100 wrapped_docstring = sphx.wrap_main_docstring(oinfo)
101 101
102 102 def sphinxify_docstring(docstring):
103 103 with TemporaryDirectory() as dirname:
104 104 return {
105 105 "text/html": sphx.sphinxify(wrapped_docstring, dirname),
106 106 "text/plain": docstring,
107 107 }
108 108
109 109 return sphinxify_docstring
110 110 except ImportError:
111 111 sphinxify = None
112 112
113 113
114 114 class ProvisionalWarning(DeprecationWarning):
115 115 """
116 116 Warning class for unstable features
117 117 """
118 118 pass
119 119
120 120 from ast import Module
121 121
122 122 _assign_nodes = (ast.AugAssign, ast.AnnAssign, ast.Assign)
123 123 _single_targets_nodes = (ast.AugAssign, ast.AnnAssign)
124 124
125 125 #-----------------------------------------------------------------------------
126 126 # Await Helpers
127 127 #-----------------------------------------------------------------------------
128 128
129 129 # we still need to run things using the asyncio eventloop, but there is no
130 130 # async integration
131 131 from .async_helpers import (
132 132 _asyncio_runner,
133 133 _curio_runner,
134 134 _pseudo_sync_runner,
135 135 _should_be_async,
136 136 _trio_runner,
137 137 )
138 138
139 139 #-----------------------------------------------------------------------------
140 140 # Globals
141 141 #-----------------------------------------------------------------------------
142 142
143 143 # compiled regexps for autoindent management
144 144 dedent_re = re.compile(r'^\s+raise|^\s+return|^\s+pass')
145 145
146 146 #-----------------------------------------------------------------------------
147 147 # Utilities
148 148 #-----------------------------------------------------------------------------
149 149
150
151 def is_integer_string(s: str):
152 """
153 Variant of "str.isnumeric()" that allow negative values and other ints.
154 """
155 try:
156 int(s)
157 return True
158 except ValueError:
159 return False
160 raise ValueError("Unexpected error")
161
162
150 163 @undoc
151 164 def softspace(file, newvalue):
152 165 """Copied from code.py, to remove the dependency"""
153 166
154 167 oldvalue = 0
155 168 try:
156 169 oldvalue = file.softspace
157 170 except AttributeError:
158 171 pass
159 172 try:
160 173 file.softspace = newvalue
161 174 except (AttributeError, TypeError):
162 175 # "attribute-less object" or "read-only attributes"
163 176 pass
164 177 return oldvalue
165 178
166 179 @undoc
167 180 def no_op(*a, **kw):
168 181 pass
169 182
170 183
171 184 class SpaceInInput(Exception): pass
172 185
173 186
174 187 class SeparateUnicode(Unicode):
175 188 r"""A Unicode subclass to validate separate_in, separate_out, etc.
176 189
177 190 This is a Unicode based trait that converts '0'->'' and ``'\\n'->'\n'``.
178 191 """
179 192
180 193 def validate(self, obj, value):
181 194 if value == '0': value = ''
182 195 value = value.replace('\\n','\n')
183 196 return super(SeparateUnicode, self).validate(obj, value)
184 197
185 198
186 199 @undoc
187 200 class DummyMod(object):
188 201 """A dummy module used for IPython's interactive module when
189 202 a namespace must be assigned to the module's __dict__."""
190 203 __spec__ = None
191 204
192 205
193 206 class ExecutionInfo(object):
194 207 """The arguments used for a call to :meth:`InteractiveShell.run_cell`
195 208
196 209 Stores information about what is going to happen.
197 210 """
198 211 raw_cell = None
199 212 store_history = False
200 213 silent = False
201 214 shell_futures = True
202 215 cell_id = None
203 216
204 217 def __init__(self, raw_cell, store_history, silent, shell_futures, cell_id):
205 218 self.raw_cell = raw_cell
206 219 self.store_history = store_history
207 220 self.silent = silent
208 221 self.shell_futures = shell_futures
209 222 self.cell_id = cell_id
210 223
211 224 def __repr__(self):
212 225 name = self.__class__.__qualname__
213 226 raw_cell = (
214 227 (self.raw_cell[:50] + "..") if len(self.raw_cell) > 50 else self.raw_cell
215 228 )
216 return '<%s object at %x, raw_cell="%s" store_history=%s silent=%s shell_futures=%s cell_id=%s>' % (
217 name,
218 id(self),
219 raw_cell,
220 self.store_history,
221 self.silent,
222 self.shell_futures,
223 self.cell_id,
229 return (
230 '<%s object at %x, raw_cell="%s" store_history=%s silent=%s shell_futures=%s cell_id=%s>'
231 % (
232 name,
233 id(self),
234 raw_cell,
235 self.store_history,
236 self.silent,
237 self.shell_futures,
238 self.cell_id,
239 )
224 240 )
225 241
226 242
227 243 class ExecutionResult(object):
228 244 """The result of a call to :meth:`InteractiveShell.run_cell`
229 245
230 246 Stores information about what took place.
231 247 """
232 248 execution_count = None
233 249 error_before_exec = None
234 250 error_in_exec: Optional[BaseException] = None
235 251 info = None
236 252 result = None
237 253
238 254 def __init__(self, info):
239 255 self.info = info
240 256
241 257 @property
242 258 def success(self):
243 259 return (self.error_before_exec is None) and (self.error_in_exec is None)
244 260
245 261 def raise_error(self):
246 262 """Reraises error if `success` is `False`, otherwise does nothing"""
247 263 if self.error_before_exec is not None:
248 264 raise self.error_before_exec
249 265 if self.error_in_exec is not None:
250 266 raise self.error_in_exec
251 267
252 268 def __repr__(self):
253 269 name = self.__class__.__qualname__
254 270 return '<%s object at %x, execution_count=%s error_before_exec=%s error_in_exec=%s info=%s result=%s>' %\
255 271 (name, id(self), self.execution_count, self.error_before_exec, self.error_in_exec, repr(self.info), repr(self.result))
256 272
273 @functools.wraps(io_open)
274 def _modified_open(file, *args, **kwargs):
275 if file in {0, 1, 2}:
276 raise ValueError(
277 f"IPython won't let you open fd={file} by default "
278 "as it is likely to crash IPython. If you know what you are doing, "
279 "you can use builtins' open."
280 )
281
282 return io_open(file, *args, **kwargs)
257 283
258 284 class InteractiveShell(SingletonConfigurable):
259 285 """An enhanced, interactive shell for Python."""
260 286
261 287 _instance = None
262 288
263 289 ast_transformers = List([], help=
264 290 """
265 291 A list of ast.NodeTransformer subclass instances, which will be applied
266 292 to user input before code is run.
267 293 """
268 294 ).tag(config=True)
269 295
270 296 autocall = Enum((0,1,2), default_value=0, help=
271 297 """
272 298 Make IPython automatically call any callable object even if you didn't
273 299 type explicit parentheses. For example, 'str 43' becomes 'str(43)'
274 300 automatically. The value can be '0' to disable the feature, '1' for
275 301 'smart' autocall, where it is not applied if there are no more
276 302 arguments on the line, and '2' for 'full' autocall, where all callable
277 303 objects are automatically called (even if no arguments are present).
278 304 """
279 305 ).tag(config=True)
280 306
281 307 autoindent = Bool(True, help=
282 308 """
283 309 Autoindent IPython code entered interactively.
284 310 """
285 311 ).tag(config=True)
286 312
287 313 autoawait = Bool(True, help=
288 314 """
289 315 Automatically run await statement in the top level repl.
290 316 """
291 317 ).tag(config=True)
292 318
293 319 loop_runner_map ={
294 320 'asyncio':(_asyncio_runner, True),
295 321 'curio':(_curio_runner, True),
296 322 'trio':(_trio_runner, True),
297 323 'sync': (_pseudo_sync_runner, False)
298 324 }
299 325
300 326 loop_runner = Any(default_value="IPython.core.interactiveshell._asyncio_runner",
301 327 allow_none=True,
302 328 help="""Select the loop runner that will be used to execute top-level asynchronous code"""
303 329 ).tag(config=True)
304 330
305 331 @default('loop_runner')
306 332 def _default_loop_runner(self):
307 333 return import_item("IPython.core.interactiveshell._asyncio_runner")
308 334
309 335 @validate('loop_runner')
310 336 def _import_runner(self, proposal):
311 337 if isinstance(proposal.value, str):
312 338 if proposal.value in self.loop_runner_map:
313 339 runner, autoawait = self.loop_runner_map[proposal.value]
314 340 self.autoawait = autoawait
315 341 return runner
316 342 runner = import_item(proposal.value)
317 343 if not callable(runner):
318 344 raise ValueError('loop_runner must be callable')
319 345 return runner
320 346 if not callable(proposal.value):
321 347 raise ValueError('loop_runner must be callable')
322 348 return proposal.value
323 349
324 350 automagic = Bool(True, help=
325 351 """
326 352 Enable magic commands to be called without the leading %.
327 353 """
328 354 ).tag(config=True)
329 355
330 356 banner1 = Unicode(default_banner,
331 357 help="""The part of the banner to be printed before the profile"""
332 358 ).tag(config=True)
333 359 banner2 = Unicode('',
334 360 help="""The part of the banner to be printed after the profile"""
335 361 ).tag(config=True)
336 362
337 363 cache_size = Integer(1000, help=
338 364 """
339 365 Set the size of the output cache. The default is 1000, you can
340 366 change it permanently in your config file. Setting it to 0 completely
341 367 disables the caching system, and the minimum value accepted is 3 (if
342 368 you provide a value less than 3, it is reset to 0 and a warning is
343 369 issued). This limit is defined because otherwise you'll spend more
344 370 time re-flushing a too small cache than working
345 371 """
346 372 ).tag(config=True)
347 373 color_info = Bool(True, help=
348 374 """
349 375 Use colors for displaying information about objects. Because this
350 376 information is passed through a pager (like 'less'), and some pagers
351 377 get confused with color codes, this capability can be turned off.
352 378 """
353 379 ).tag(config=True)
354 380 colors = CaselessStrEnum(('Neutral', 'NoColor','LightBG','Linux'),
355 381 default_value='Neutral',
356 382 help="Set the color scheme (NoColor, Neutral, Linux, or LightBG)."
357 383 ).tag(config=True)
358 384 debug = Bool(False).tag(config=True)
359 385 disable_failing_post_execute = Bool(False,
360 386 help="Don't call post-execute functions that have failed in the past."
361 387 ).tag(config=True)
362 388 display_formatter = Instance(DisplayFormatter, allow_none=True)
363 389 displayhook_class = Type(DisplayHook)
364 390 display_pub_class = Type(DisplayPublisher)
365 391 compiler_class = Type(CachingCompiler)
366 392
367 393 sphinxify_docstring = Bool(False, help=
368 394 """
369 395 Enables rich html representation of docstrings. (This requires the
370 396 docrepr module).
371 397 """).tag(config=True)
372 398
373 399 @observe("sphinxify_docstring")
374 400 def _sphinxify_docstring_changed(self, change):
375 401 if change['new']:
376 402 warn("`sphinxify_docstring` is provisional since IPython 5.0 and might change in future versions." , ProvisionalWarning)
377 403
378 404 enable_html_pager = Bool(False, help=
379 405 """
380 406 (Provisional API) enables html representation in mime bundles sent
381 407 to pagers.
382 408 """).tag(config=True)
383 409
384 410 @observe("enable_html_pager")
385 411 def _enable_html_pager_changed(self, change):
386 412 if change['new']:
387 413 warn("`enable_html_pager` is provisional since IPython 5.0 and might change in future versions.", ProvisionalWarning)
388 414
389 415 data_pub_class = None
390 416
391 417 exit_now = Bool(False)
392 418 exiter = Instance(ExitAutocall)
393 419 @default('exiter')
394 420 def _exiter_default(self):
395 421 return ExitAutocall(self)
396 422 # Monotonically increasing execution counter
397 423 execution_count = Integer(1)
398 424 filename = Unicode("<ipython console>")
399 425 ipython_dir= Unicode('').tag(config=True) # Set to get_ipython_dir() in __init__
400 426
401 427 # Used to transform cells before running them, and check whether code is complete
402 428 input_transformer_manager = Instance('IPython.core.inputtransformer2.TransformerManager',
403 429 ())
404 430
405 431 @property
406 432 def input_transformers_cleanup(self):
407 433 return self.input_transformer_manager.cleanup_transforms
408 434
409 435 input_transformers_post = List([],
410 436 help="A list of string input transformers, to be applied after IPython's "
411 437 "own input transformations."
412 438 )
413 439
414 440 @property
415 441 def input_splitter(self):
416 442 """Make this available for backward compatibility (pre-7.0 release) with existing code.
417 443
418 444 For example, ipykernel ipykernel currently uses
419 445 `shell.input_splitter.check_complete`
420 446 """
421 447 from warnings import warn
422 448 warn("`input_splitter` is deprecated since IPython 7.0, prefer `input_transformer_manager`.",
423 449 DeprecationWarning, stacklevel=2
424 450 )
425 451 return self.input_transformer_manager
426 452
427 453 logstart = Bool(False, help=
428 454 """
429 455 Start logging to the default log file in overwrite mode.
430 456 Use `logappend` to specify a log file to **append** logs to.
431 457 """
432 458 ).tag(config=True)
433 459 logfile = Unicode('', help=
434 460 """
435 461 The name of the logfile to use.
436 462 """
437 463 ).tag(config=True)
438 464 logappend = Unicode('', help=
439 465 """
440 466 Start logging to the given file in append mode.
441 467 Use `logfile` to specify a log file to **overwrite** logs to.
442 468 """
443 469 ).tag(config=True)
444 470 object_info_string_level = Enum((0,1,2), default_value=0,
445 471 ).tag(config=True)
446 472 pdb = Bool(False, help=
447 473 """
448 474 Automatically call the pdb debugger after every exception.
449 475 """
450 476 ).tag(config=True)
451 477 display_page = Bool(False,
452 478 help="""If True, anything that would be passed to the pager
453 479 will be displayed as regular output instead."""
454 480 ).tag(config=True)
455 481
456 482
457 483 show_rewritten_input = Bool(True,
458 484 help="Show rewritten input, e.g. for autocall."
459 485 ).tag(config=True)
460 486
461 487 quiet = Bool(False).tag(config=True)
462 488
463 489 history_length = Integer(10000,
464 490 help='Total length of command history'
465 491 ).tag(config=True)
466 492
467 493 history_load_length = Integer(1000, help=
468 494 """
469 495 The number of saved history entries to be loaded
470 496 into the history buffer at startup.
471 497 """
472 498 ).tag(config=True)
473 499
474 500 ast_node_interactivity = Enum(['all', 'last', 'last_expr', 'none', 'last_expr_or_assign'],
475 501 default_value='last_expr',
476 502 help="""
477 503 'all', 'last', 'last_expr' or 'none', 'last_expr_or_assign' specifying
478 504 which nodes should be run interactively (displaying output from expressions).
479 505 """
480 506 ).tag(config=True)
481 507
482 508 warn_venv = Bool(
483 509 True,
484 510 help="Warn if running in a virtual environment with no IPython installed (so IPython from the global environment is used).",
485 511 ).tag(config=True)
486 512
487 513 # TODO: this part of prompt management should be moved to the frontends.
488 514 # Use custom TraitTypes that convert '0'->'' and '\\n'->'\n'
489 515 separate_in = SeparateUnicode('\n').tag(config=True)
490 516 separate_out = SeparateUnicode('').tag(config=True)
491 517 separate_out2 = SeparateUnicode('').tag(config=True)
492 518 wildcards_case_sensitive = Bool(True).tag(config=True)
493 519 xmode = CaselessStrEnum(('Context', 'Plain', 'Verbose', 'Minimal'),
494 520 default_value='Context',
495 521 help="Switch modes for the IPython exception handlers."
496 522 ).tag(config=True)
497 523
498 524 # Subcomponents of InteractiveShell
499 525 alias_manager = Instance('IPython.core.alias.AliasManager', allow_none=True)
500 526 prefilter_manager = Instance('IPython.core.prefilter.PrefilterManager', allow_none=True)
501 527 builtin_trap = Instance('IPython.core.builtin_trap.BuiltinTrap', allow_none=True)
502 528 display_trap = Instance('IPython.core.display_trap.DisplayTrap', allow_none=True)
503 529 extension_manager = Instance('IPython.core.extensions.ExtensionManager', allow_none=True)
504 530 payload_manager = Instance('IPython.core.payload.PayloadManager', allow_none=True)
505 531 history_manager = Instance('IPython.core.history.HistoryAccessorBase', allow_none=True)
506 532 magics_manager = Instance('IPython.core.magic.MagicsManager', allow_none=True)
507 533
508 534 profile_dir = Instance('IPython.core.application.ProfileDir', allow_none=True)
509 535 @property
510 536 def profile(self):
511 537 if self.profile_dir is not None:
512 538 name = os.path.basename(self.profile_dir.location)
513 539 return name.replace('profile_','')
514 540
515 541
516 542 # Private interface
517 543 _post_execute = Dict()
518 544
519 545 # Tracks any GUI loop loaded for pylab
520 546 pylab_gui_select = None
521 547
522 548 last_execution_succeeded = Bool(True, help='Did last executed command succeeded')
523 549
524 550 last_execution_result = Instance('IPython.core.interactiveshell.ExecutionResult', help='Result of executing the last command', allow_none=True)
525 551
526 552 def __init__(self, ipython_dir=None, profile_dir=None,
527 553 user_module=None, user_ns=None,
528 554 custom_exceptions=((), None), **kwargs):
529 555 # This is where traits with a config_key argument are updated
530 556 # from the values on config.
531 557 super(InteractiveShell, self).__init__(**kwargs)
532 558 if 'PromptManager' in self.config:
533 559 warn('As of IPython 5.0 `PromptManager` config will have no effect'
534 560 ' and has been replaced by TerminalInteractiveShell.prompts_class')
535 561 self.configurables = [self]
536 562
537 563 # These are relatively independent and stateless
538 564 self.init_ipython_dir(ipython_dir)
539 565 self.init_profile_dir(profile_dir)
540 566 self.init_instance_attrs()
541 567 self.init_environment()
542 568
543 569 # Check if we're in a virtualenv, and set up sys.path.
544 570 self.init_virtualenv()
545 571
546 572 # Create namespaces (user_ns, user_global_ns, etc.)
547 573 self.init_create_namespaces(user_module, user_ns)
548 574 # This has to be done after init_create_namespaces because it uses
549 575 # something in self.user_ns, but before init_sys_modules, which
550 576 # is the first thing to modify sys.
551 577 # TODO: When we override sys.stdout and sys.stderr before this class
552 578 # is created, we are saving the overridden ones here. Not sure if this
553 579 # is what we want to do.
554 580 self.save_sys_module_state()
555 581 self.init_sys_modules()
556 582
557 583 # While we're trying to have each part of the code directly access what
558 584 # it needs without keeping redundant references to objects, we have too
559 585 # much legacy code that expects ip.db to exist.
560 586 self.db = PickleShareDB(os.path.join(self.profile_dir.location, 'db'))
561 587
562 588 self.init_history()
563 589 self.init_encoding()
564 590 self.init_prefilter()
565 591
566 592 self.init_syntax_highlighting()
567 593 self.init_hooks()
568 594 self.init_events()
569 595 self.init_pushd_popd_magic()
570 596 self.init_user_ns()
571 597 self.init_logger()
572 598 self.init_builtins()
573 599
574 600 # The following was in post_config_initialization
575 601 self.init_inspector()
576 602 self.raw_input_original = input
577 603 self.init_completer()
578 604 # TODO: init_io() needs to happen before init_traceback handlers
579 605 # because the traceback handlers hardcode the stdout/stderr streams.
580 606 # This logic in in debugger.Pdb and should eventually be changed.
581 607 self.init_io()
582 608 self.init_traceback_handlers(custom_exceptions)
583 609 self.init_prompts()
584 610 self.init_display_formatter()
585 611 self.init_display_pub()
586 612 self.init_data_pub()
587 613 self.init_displayhook()
588 614 self.init_magics()
589 615 self.init_alias()
590 616 self.init_logstart()
591 617 self.init_pdb()
592 618 self.init_extension_manager()
593 619 self.init_payload()
594 620 self.events.trigger('shell_initialized', self)
595 621 atexit.register(self.atexit_operations)
596 622
597 623 # The trio runner is used for running Trio in the foreground thread. It
598 624 # is different from `_trio_runner(async_fn)` in `async_helpers.py`
599 625 # which calls `trio.run()` for every cell. This runner runs all cells
600 626 # inside a single Trio event loop. If used, it is set from
601 627 # `ipykernel.kernelapp`.
602 628 self.trio_runner = None
603 629
604 630 def get_ipython(self):
605 631 """Return the currently running IPython instance."""
606 632 return self
607 633
608 634 #-------------------------------------------------------------------------
609 635 # Trait changed handlers
610 636 #-------------------------------------------------------------------------
611 637 @observe('ipython_dir')
612 638 def _ipython_dir_changed(self, change):
613 639 ensure_dir_exists(change['new'])
614 640
615 641 def set_autoindent(self,value=None):
616 642 """Set the autoindent flag.
617 643
618 644 If called with no arguments, it acts as a toggle."""
619 645 if value is None:
620 646 self.autoindent = not self.autoindent
621 647 else:
622 648 self.autoindent = value
623 649
624 650 def set_trio_runner(self, tr):
625 651 self.trio_runner = tr
626 652
627 653 #-------------------------------------------------------------------------
628 654 # init_* methods called by __init__
629 655 #-------------------------------------------------------------------------
630 656
631 657 def init_ipython_dir(self, ipython_dir):
632 658 if ipython_dir is not None:
633 659 self.ipython_dir = ipython_dir
634 660 return
635 661
636 662 self.ipython_dir = get_ipython_dir()
637 663
638 664 def init_profile_dir(self, profile_dir):
639 665 if profile_dir is not None:
640 666 self.profile_dir = profile_dir
641 667 return
642 668 self.profile_dir = ProfileDir.create_profile_dir_by_name(
643 669 self.ipython_dir, "default"
644 670 )
645 671
646 672 def init_instance_attrs(self):
647 673 self.more = False
648 674
649 675 # command compiler
650 676 self.compile = self.compiler_class()
651 677
652 678 # Make an empty namespace, which extension writers can rely on both
653 679 # existing and NEVER being used by ipython itself. This gives them a
654 680 # convenient location for storing additional information and state
655 681 # their extensions may require, without fear of collisions with other
656 682 # ipython names that may develop later.
657 683 self.meta = Struct()
658 684
659 685 # Temporary files used for various purposes. Deleted at exit.
660 686 # The files here are stored with Path from Pathlib
661 687 self.tempfiles = []
662 688 self.tempdirs = []
663 689
664 690 # keep track of where we started running (mainly for crash post-mortem)
665 691 # This is not being used anywhere currently.
666 692 self.starting_dir = os.getcwd()
667 693
668 694 # Indentation management
669 695 self.indent_current_nsp = 0
670 696
671 697 # Dict to track post-execution functions that have been registered
672 698 self._post_execute = {}
673 699
674 700 def init_environment(self):
675 701 """Any changes we need to make to the user's environment."""
676 702 pass
677 703
678 704 def init_encoding(self):
679 705 # Get system encoding at startup time. Certain terminals (like Emacs
680 706 # under Win32 have it set to None, and we need to have a known valid
681 707 # encoding to use in the raw_input() method
682 708 try:
683 709 self.stdin_encoding = sys.stdin.encoding or 'ascii'
684 710 except AttributeError:
685 711 self.stdin_encoding = 'ascii'
686 712
687 713
688 714 @observe('colors')
689 715 def init_syntax_highlighting(self, changes=None):
690 716 # Python source parser/formatter for syntax highlighting
691 717 pyformat = PyColorize.Parser(style=self.colors, parent=self).format
692 718 self.pycolorize = lambda src: pyformat(src,'str')
693 719
694 720 def refresh_style(self):
695 721 # No-op here, used in subclass
696 722 pass
697 723
698 724 def init_pushd_popd_magic(self):
699 725 # for pushd/popd management
700 726 self.home_dir = get_home_dir()
701 727
702 728 self.dir_stack = []
703 729
704 730 def init_logger(self):
705 731 self.logger = Logger(self.home_dir, logfname='ipython_log.py',
706 732 logmode='rotate')
707 733
708 734 def init_logstart(self):
709 735 """Initialize logging in case it was requested at the command line.
710 736 """
711 737 if self.logappend:
712 738 self.magic('logstart %s append' % self.logappend)
713 739 elif self.logfile:
714 740 self.magic('logstart %s' % self.logfile)
715 741 elif self.logstart:
716 742 self.magic('logstart')
717 743
718 744
719 745 def init_builtins(self):
720 746 # A single, static flag that we set to True. Its presence indicates
721 747 # that an IPython shell has been created, and we make no attempts at
722 748 # removing on exit or representing the existence of more than one
723 749 # IPython at a time.
724 750 builtin_mod.__dict__['__IPYTHON__'] = True
725 751 builtin_mod.__dict__['display'] = display
726 752
727 753 self.builtin_trap = BuiltinTrap(shell=self)
728 754
729 755 @observe('colors')
730 756 def init_inspector(self, changes=None):
731 757 # Object inspector
732 758 self.inspector = oinspect.Inspector(oinspect.InspectColors,
733 759 PyColorize.ANSICodeColors,
734 760 self.colors,
735 761 self.object_info_string_level)
736 762
737 763 def init_io(self):
738 764 # implemented in subclasses, TerminalInteractiveShell does call
739 765 # colorama.init().
740 766 pass
741 767
742 768 def init_prompts(self):
743 769 # Set system prompts, so that scripts can decide if they are running
744 770 # interactively.
745 771 sys.ps1 = 'In : '
746 772 sys.ps2 = '...: '
747 773 sys.ps3 = 'Out: '
748 774
749 775 def init_display_formatter(self):
750 776 self.display_formatter = DisplayFormatter(parent=self)
751 777 self.configurables.append(self.display_formatter)
752 778
753 779 def init_display_pub(self):
754 780 self.display_pub = self.display_pub_class(parent=self, shell=self)
755 781 self.configurables.append(self.display_pub)
756 782
757 783 def init_data_pub(self):
758 784 if not self.data_pub_class:
759 785 self.data_pub = None
760 786 return
761 787 self.data_pub = self.data_pub_class(parent=self)
762 788 self.configurables.append(self.data_pub)
763 789
764 790 def init_displayhook(self):
765 791 # Initialize displayhook, set in/out prompts and printing system
766 792 self.displayhook = self.displayhook_class(
767 793 parent=self,
768 794 shell=self,
769 795 cache_size=self.cache_size,
770 796 )
771 797 self.configurables.append(self.displayhook)
772 798 # This is a context manager that installs/revmoes the displayhook at
773 799 # the appropriate time.
774 800 self.display_trap = DisplayTrap(hook=self.displayhook)
775 801
776 802 @staticmethod
777 803 def get_path_links(p: Path):
778 804 """Gets path links including all symlinks
779 805
780 806 Examples
781 807 --------
782 808 In [1]: from IPython.core.interactiveshell import InteractiveShell
783 809
784 810 In [2]: import sys, pathlib
785 811
786 812 In [3]: paths = InteractiveShell.get_path_links(pathlib.Path(sys.executable))
787 813
788 814 In [4]: len(paths) == len(set(paths))
789 815 Out[4]: True
790 816
791 817 In [5]: bool(paths)
792 818 Out[5]: True
793 819 """
794 820 paths = [p]
795 821 while p.is_symlink():
796 822 new_path = Path(os.readlink(p))
797 823 if not new_path.is_absolute():
798 824 new_path = p.parent / new_path
799 825 p = new_path
800 826 paths.append(p)
801 827 return paths
802 828
803 829 def init_virtualenv(self):
804 830 """Add the current virtualenv to sys.path so the user can import modules from it.
805 831 This isn't perfect: it doesn't use the Python interpreter with which the
806 832 virtualenv was built, and it ignores the --no-site-packages option. A
807 833 warning will appear suggesting the user installs IPython in the
808 834 virtualenv, but for many cases, it probably works well enough.
809 835
810 836 Adapted from code snippets online.
811 837
812 838 http://blog.ufsoft.org/2009/1/29/ipython-and-virtualenv
813 839 """
814 840 if 'VIRTUAL_ENV' not in os.environ:
815 841 # Not in a virtualenv
816 842 return
817 843 elif os.environ["VIRTUAL_ENV"] == "":
818 844 warn("Virtual env path set to '', please check if this is intended.")
819 845 return
820 846
821 847 p = Path(sys.executable)
822 848 p_venv = Path(os.environ["VIRTUAL_ENV"])
823 849
824 850 # fallback venv detection:
825 851 # stdlib venv may symlink sys.executable, so we can't use realpath.
826 852 # but others can symlink *to* the venv Python, so we can't just use sys.executable.
827 853 # So we just check every item in the symlink tree (generally <= 3)
828 854 paths = self.get_path_links(p)
829 855
830 856 # In Cygwin paths like "c:\..." and '\cygdrive\c\...' are possible
831 857 if p_venv.parts[1] == "cygdrive":
832 858 drive_name = p_venv.parts[2]
833 859 p_venv = (drive_name + ":/") / Path(*p_venv.parts[3:])
834 860
835 861 if any(p_venv == p.parents[1] for p in paths):
836 862 # Our exe is inside or has access to the virtualenv, don't need to do anything.
837 863 return
838 864
839 865 if sys.platform == "win32":
840 866 virtual_env = str(Path(os.environ["VIRTUAL_ENV"], "Lib", "site-packages"))
841 867 else:
842 868 virtual_env_path = Path(
843 869 os.environ["VIRTUAL_ENV"], "lib", "python{}.{}", "site-packages"
844 870 )
845 871 p_ver = sys.version_info[:2]
846 872
847 873 # Predict version from py[thon]-x.x in the $VIRTUAL_ENV
848 874 re_m = re.search(r"\bpy(?:thon)?([23])\.(\d+)\b", os.environ["VIRTUAL_ENV"])
849 875 if re_m:
850 876 predicted_path = Path(str(virtual_env_path).format(*re_m.groups()))
851 877 if predicted_path.exists():
852 878 p_ver = re_m.groups()
853 879
854 880 virtual_env = str(virtual_env_path).format(*p_ver)
855 881 if self.warn_venv:
856 882 warn(
857 883 "Attempting to work in a virtualenv. If you encounter problems, "
858 884 "please install IPython inside the virtualenv."
859 885 )
860 886 import site
861 887 sys.path.insert(0, virtual_env)
862 888 site.addsitedir(virtual_env)
863 889
864 890 #-------------------------------------------------------------------------
865 891 # Things related to injections into the sys module
866 892 #-------------------------------------------------------------------------
867 893
868 894 def save_sys_module_state(self):
869 895 """Save the state of hooks in the sys module.
870 896
871 897 This has to be called after self.user_module is created.
872 898 """
873 899 self._orig_sys_module_state = {'stdin': sys.stdin,
874 900 'stdout': sys.stdout,
875 901 'stderr': sys.stderr,
876 902 'excepthook': sys.excepthook}
877 903 self._orig_sys_modules_main_name = self.user_module.__name__
878 904 self._orig_sys_modules_main_mod = sys.modules.get(self.user_module.__name__)
879 905
880 906 def restore_sys_module_state(self):
881 907 """Restore the state of the sys module."""
882 908 try:
883 909 for k, v in self._orig_sys_module_state.items():
884 910 setattr(sys, k, v)
885 911 except AttributeError:
886 912 pass
887 913 # Reset what what done in self.init_sys_modules
888 914 if self._orig_sys_modules_main_mod is not None:
889 915 sys.modules[self._orig_sys_modules_main_name] = self._orig_sys_modules_main_mod
890 916
891 917 #-------------------------------------------------------------------------
892 918 # Things related to the banner
893 919 #-------------------------------------------------------------------------
894 920
895 921 @property
896 922 def banner(self):
897 923 banner = self.banner1
898 924 if self.profile and self.profile != 'default':
899 925 banner += '\nIPython profile: %s\n' % self.profile
900 926 if self.banner2:
901 927 banner += '\n' + self.banner2
902 928 return banner
903 929
904 930 def show_banner(self, banner=None):
905 931 if banner is None:
906 932 banner = self.banner
907 933 sys.stdout.write(banner)
908 934
909 935 #-------------------------------------------------------------------------
910 936 # Things related to hooks
911 937 #-------------------------------------------------------------------------
912 938
913 939 def init_hooks(self):
914 940 # hooks holds pointers used for user-side customizations
915 941 self.hooks = Struct()
916 942
917 943 self.strdispatchers = {}
918 944
919 945 # Set all default hooks, defined in the IPython.hooks module.
920 946 hooks = IPython.core.hooks
921 947 for hook_name in hooks.__all__:
922 948 # default hooks have priority 100, i.e. low; user hooks should have
923 949 # 0-100 priority
924 950 self.set_hook(hook_name, getattr(hooks, hook_name), 100)
925 951
926 952 if self.display_page:
927 953 self.set_hook('show_in_pager', page.as_hook(page.display_page), 90)
928 954
929 955 def set_hook(self, name, hook, priority=50, str_key=None, re_key=None):
930 956 """set_hook(name,hook) -> sets an internal IPython hook.
931 957
932 958 IPython exposes some of its internal API as user-modifiable hooks. By
933 959 adding your function to one of these hooks, you can modify IPython's
934 960 behavior to call at runtime your own routines."""
935 961
936 962 # At some point in the future, this should validate the hook before it
937 963 # accepts it. Probably at least check that the hook takes the number
938 964 # of args it's supposed to.
939 965
940 966 f = types.MethodType(hook,self)
941 967
942 968 # check if the hook is for strdispatcher first
943 969 if str_key is not None:
944 970 sdp = self.strdispatchers.get(name, StrDispatch())
945 971 sdp.add_s(str_key, f, priority )
946 972 self.strdispatchers[name] = sdp
947 973 return
948 974 if re_key is not None:
949 975 sdp = self.strdispatchers.get(name, StrDispatch())
950 976 sdp.add_re(re.compile(re_key), f, priority )
951 977 self.strdispatchers[name] = sdp
952 978 return
953 979
954 980 dp = getattr(self.hooks, name, None)
955 981 if name not in IPython.core.hooks.__all__:
956 982 print("Warning! Hook '%s' is not one of %s" % \
957 983 (name, IPython.core.hooks.__all__ ))
958 984
959 985 if name in IPython.core.hooks.deprecated:
960 986 alternative = IPython.core.hooks.deprecated[name]
961 987 raise ValueError(
962 988 "Hook {} has been deprecated since IPython 5.0. Use {} instead.".format(
963 989 name, alternative
964 990 )
965 991 )
966 992
967 993 if not dp:
968 994 dp = IPython.core.hooks.CommandChainDispatcher()
969 995
970 996 try:
971 997 dp.add(f,priority)
972 998 except AttributeError:
973 999 # it was not commandchain, plain old func - replace
974 1000 dp = f
975 1001
976 1002 setattr(self.hooks,name, dp)
977 1003
978 1004 #-------------------------------------------------------------------------
979 1005 # Things related to events
980 1006 #-------------------------------------------------------------------------
981 1007
982 1008 def init_events(self):
983 1009 self.events = EventManager(self, available_events)
984 1010
985 1011 self.events.register("pre_execute", self._clear_warning_registry)
986 1012
987 1013 def register_post_execute(self, func):
988 1014 """DEPRECATED: Use ip.events.register('post_run_cell', func)
989 1015
990 1016 Register a function for calling after code execution.
991 1017 """
992 1018 raise ValueError(
993 1019 "ip.register_post_execute is deprecated since IPython 1.0, use "
994 1020 "ip.events.register('post_run_cell', func) instead."
995 1021 )
996 1022
997 1023 def _clear_warning_registry(self):
998 1024 # clear the warning registry, so that different code blocks with
999 1025 # overlapping line number ranges don't cause spurious suppression of
1000 1026 # warnings (see gh-6611 for details)
1001 1027 if "__warningregistry__" in self.user_global_ns:
1002 1028 del self.user_global_ns["__warningregistry__"]
1003 1029
1004 1030 #-------------------------------------------------------------------------
1005 1031 # Things related to the "main" module
1006 1032 #-------------------------------------------------------------------------
1007 1033
1008 1034 def new_main_mod(self, filename, modname):
1009 1035 """Return a new 'main' module object for user code execution.
1010 1036
1011 1037 ``filename`` should be the path of the script which will be run in the
1012 1038 module. Requests with the same filename will get the same module, with
1013 1039 its namespace cleared.
1014 1040
1015 1041 ``modname`` should be the module name - normally either '__main__' or
1016 1042 the basename of the file without the extension.
1017 1043
1018 1044 When scripts are executed via %run, we must keep a reference to their
1019 1045 __main__ module around so that Python doesn't
1020 1046 clear it, rendering references to module globals useless.
1021 1047
1022 1048 This method keeps said reference in a private dict, keyed by the
1023 1049 absolute path of the script. This way, for multiple executions of the
1024 1050 same script we only keep one copy of the namespace (the last one),
1025 1051 thus preventing memory leaks from old references while allowing the
1026 1052 objects from the last execution to be accessible.
1027 1053 """
1028 1054 filename = os.path.abspath(filename)
1029 1055 try:
1030 1056 main_mod = self._main_mod_cache[filename]
1031 1057 except KeyError:
1032 1058 main_mod = self._main_mod_cache[filename] = types.ModuleType(
1033 1059 modname,
1034 1060 doc="Module created for script run in IPython")
1035 1061 else:
1036 1062 main_mod.__dict__.clear()
1037 1063 main_mod.__name__ = modname
1038 1064
1039 1065 main_mod.__file__ = filename
1040 1066 # It seems pydoc (and perhaps others) needs any module instance to
1041 1067 # implement a __nonzero__ method
1042 1068 main_mod.__nonzero__ = lambda : True
1043 1069
1044 1070 return main_mod
1045 1071
1046 1072 def clear_main_mod_cache(self):
1047 1073 """Clear the cache of main modules.
1048 1074
1049 1075 Mainly for use by utilities like %reset.
1050 1076
1051 1077 Examples
1052 1078 --------
1053 1079 In [15]: import IPython
1054 1080
1055 1081 In [16]: m = _ip.new_main_mod(IPython.__file__, 'IPython')
1056 1082
1057 1083 In [17]: len(_ip._main_mod_cache) > 0
1058 1084 Out[17]: True
1059 1085
1060 1086 In [18]: _ip.clear_main_mod_cache()
1061 1087
1062 1088 In [19]: len(_ip._main_mod_cache) == 0
1063 1089 Out[19]: True
1064 1090 """
1065 1091 self._main_mod_cache.clear()
1066 1092
1067 1093 #-------------------------------------------------------------------------
1068 1094 # Things related to debugging
1069 1095 #-------------------------------------------------------------------------
1070 1096
1071 1097 def init_pdb(self):
1072 1098 # Set calling of pdb on exceptions
1073 1099 # self.call_pdb is a property
1074 1100 self.call_pdb = self.pdb
1075 1101
1076 1102 def _get_call_pdb(self):
1077 1103 return self._call_pdb
1078 1104
1079 1105 def _set_call_pdb(self,val):
1080 1106
1081 1107 if val not in (0,1,False,True):
1082 1108 raise ValueError('new call_pdb value must be boolean')
1083 1109
1084 1110 # store value in instance
1085 1111 self._call_pdb = val
1086 1112
1087 1113 # notify the actual exception handlers
1088 1114 self.InteractiveTB.call_pdb = val
1089 1115
1090 1116 call_pdb = property(_get_call_pdb,_set_call_pdb,None,
1091 1117 'Control auto-activation of pdb at exceptions')
1092 1118
1093 1119 def debugger(self,force=False):
1094 1120 """Call the pdb debugger.
1095 1121
1096 1122 Keywords:
1097 1123
1098 1124 - force(False): by default, this routine checks the instance call_pdb
1099 1125 flag and does not actually invoke the debugger if the flag is false.
1100 1126 The 'force' option forces the debugger to activate even if the flag
1101 1127 is false.
1102 1128 """
1103 1129
1104 1130 if not (force or self.call_pdb):
1105 1131 return
1106 1132
1107 1133 if not hasattr(sys,'last_traceback'):
1108 1134 error('No traceback has been produced, nothing to debug.')
1109 1135 return
1110 1136
1111 1137 self.InteractiveTB.debugger(force=True)
1112 1138
1113 1139 #-------------------------------------------------------------------------
1114 1140 # Things related to IPython's various namespaces
1115 1141 #-------------------------------------------------------------------------
1116 1142 default_user_namespaces = True
1117 1143
1118 1144 def init_create_namespaces(self, user_module=None, user_ns=None):
1119 1145 # Create the namespace where the user will operate. user_ns is
1120 1146 # normally the only one used, and it is passed to the exec calls as
1121 1147 # the locals argument. But we do carry a user_global_ns namespace
1122 1148 # given as the exec 'globals' argument, This is useful in embedding
1123 1149 # situations where the ipython shell opens in a context where the
1124 1150 # distinction between locals and globals is meaningful. For
1125 1151 # non-embedded contexts, it is just the same object as the user_ns dict.
1126 1152
1127 1153 # FIXME. For some strange reason, __builtins__ is showing up at user
1128 1154 # level as a dict instead of a module. This is a manual fix, but I
1129 1155 # should really track down where the problem is coming from. Alex
1130 1156 # Schmolck reported this problem first.
1131 1157
1132 1158 # A useful post by Alex Martelli on this topic:
1133 1159 # Re: inconsistent value from __builtins__
1134 1160 # Von: Alex Martelli <aleaxit@yahoo.com>
1135 1161 # Datum: Freitag 01 Oktober 2004 04:45:34 nachmittags/abends
1136 1162 # Gruppen: comp.lang.python
1137 1163
1138 1164 # Michael Hohn <hohn@hooknose.lbl.gov> wrote:
1139 1165 # > >>> print type(builtin_check.get_global_binding('__builtins__'))
1140 1166 # > <type 'dict'>
1141 1167 # > >>> print type(__builtins__)
1142 1168 # > <type 'module'>
1143 1169 # > Is this difference in return value intentional?
1144 1170
1145 1171 # Well, it's documented that '__builtins__' can be either a dictionary
1146 1172 # or a module, and it's been that way for a long time. Whether it's
1147 1173 # intentional (or sensible), I don't know. In any case, the idea is
1148 1174 # that if you need to access the built-in namespace directly, you
1149 1175 # should start with "import __builtin__" (note, no 's') which will
1150 1176 # definitely give you a module. Yeah, it's somewhat confusing:-(.
1151 1177
1152 1178 # These routines return a properly built module and dict as needed by
1153 1179 # the rest of the code, and can also be used by extension writers to
1154 1180 # generate properly initialized namespaces.
1155 1181 if (user_ns is not None) or (user_module is not None):
1156 1182 self.default_user_namespaces = False
1157 1183 self.user_module, self.user_ns = self.prepare_user_module(user_module, user_ns)
1158 1184
1159 1185 # A record of hidden variables we have added to the user namespace, so
1160 1186 # we can list later only variables defined in actual interactive use.
1161 1187 self.user_ns_hidden = {}
1162 1188
1163 1189 # Now that FakeModule produces a real module, we've run into a nasty
1164 1190 # problem: after script execution (via %run), the module where the user
1165 1191 # code ran is deleted. Now that this object is a true module (needed
1166 1192 # so doctest and other tools work correctly), the Python module
1167 1193 # teardown mechanism runs over it, and sets to None every variable
1168 1194 # present in that module. Top-level references to objects from the
1169 1195 # script survive, because the user_ns is updated with them. However,
1170 1196 # calling functions defined in the script that use other things from
1171 1197 # the script will fail, because the function's closure had references
1172 1198 # to the original objects, which are now all None. So we must protect
1173 1199 # these modules from deletion by keeping a cache.
1174 1200 #
1175 1201 # To avoid keeping stale modules around (we only need the one from the
1176 1202 # last run), we use a dict keyed with the full path to the script, so
1177 1203 # only the last version of the module is held in the cache. Note,
1178 1204 # however, that we must cache the module *namespace contents* (their
1179 1205 # __dict__). Because if we try to cache the actual modules, old ones
1180 1206 # (uncached) could be destroyed while still holding references (such as
1181 1207 # those held by GUI objects that tend to be long-lived)>
1182 1208 #
1183 1209 # The %reset command will flush this cache. See the cache_main_mod()
1184 1210 # and clear_main_mod_cache() methods for details on use.
1185 1211
1186 1212 # This is the cache used for 'main' namespaces
1187 1213 self._main_mod_cache = {}
1188 1214
1189 1215 # A table holding all the namespaces IPython deals with, so that
1190 1216 # introspection facilities can search easily.
1191 1217 self.ns_table = {'user_global':self.user_module.__dict__,
1192 1218 'user_local':self.user_ns,
1193 1219 'builtin':builtin_mod.__dict__
1194 1220 }
1195 1221
1196 1222 @property
1197 1223 def user_global_ns(self):
1198 1224 return self.user_module.__dict__
1199 1225
1200 1226 def prepare_user_module(self, user_module=None, user_ns=None):
1201 1227 """Prepare the module and namespace in which user code will be run.
1202 1228
1203 1229 When IPython is started normally, both parameters are None: a new module
1204 1230 is created automatically, and its __dict__ used as the namespace.
1205 1231
1206 1232 If only user_module is provided, its __dict__ is used as the namespace.
1207 1233 If only user_ns is provided, a dummy module is created, and user_ns
1208 1234 becomes the global namespace. If both are provided (as they may be
1209 1235 when embedding), user_ns is the local namespace, and user_module
1210 1236 provides the global namespace.
1211 1237
1212 1238 Parameters
1213 1239 ----------
1214 1240 user_module : module, optional
1215 1241 The current user module in which IPython is being run. If None,
1216 1242 a clean module will be created.
1217 1243 user_ns : dict, optional
1218 1244 A namespace in which to run interactive commands.
1219 1245
1220 1246 Returns
1221 1247 -------
1222 1248 A tuple of user_module and user_ns, each properly initialised.
1223 1249 """
1224 1250 if user_module is None and user_ns is not None:
1225 1251 user_ns.setdefault("__name__", "__main__")
1226 1252 user_module = DummyMod()
1227 1253 user_module.__dict__ = user_ns
1228 1254
1229 1255 if user_module is None:
1230 1256 user_module = types.ModuleType("__main__",
1231 1257 doc="Automatically created module for IPython interactive environment")
1232 1258
1233 1259 # We must ensure that __builtin__ (without the final 's') is always
1234 1260 # available and pointing to the __builtin__ *module*. For more details:
1235 1261 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1236 1262 user_module.__dict__.setdefault('__builtin__', builtin_mod)
1237 1263 user_module.__dict__.setdefault('__builtins__', builtin_mod)
1238 1264
1239 1265 if user_ns is None:
1240 1266 user_ns = user_module.__dict__
1241 1267
1242 1268 return user_module, user_ns
1243 1269
1244 1270 def init_sys_modules(self):
1245 1271 # We need to insert into sys.modules something that looks like a
1246 1272 # module but which accesses the IPython namespace, for shelve and
1247 1273 # pickle to work interactively. Normally they rely on getting
1248 1274 # everything out of __main__, but for embedding purposes each IPython
1249 1275 # instance has its own private namespace, so we can't go shoving
1250 1276 # everything into __main__.
1251 1277
1252 1278 # note, however, that we should only do this for non-embedded
1253 1279 # ipythons, which really mimic the __main__.__dict__ with their own
1254 1280 # namespace. Embedded instances, on the other hand, should not do
1255 1281 # this because they need to manage the user local/global namespaces
1256 1282 # only, but they live within a 'normal' __main__ (meaning, they
1257 1283 # shouldn't overtake the execution environment of the script they're
1258 1284 # embedded in).
1259 1285
1260 1286 # This is overridden in the InteractiveShellEmbed subclass to a no-op.
1261 1287 main_name = self.user_module.__name__
1262 1288 sys.modules[main_name] = self.user_module
1263 1289
1264 1290 def init_user_ns(self):
1265 1291 """Initialize all user-visible namespaces to their minimum defaults.
1266 1292
1267 1293 Certain history lists are also initialized here, as they effectively
1268 1294 act as user namespaces.
1269 1295
1270 1296 Notes
1271 1297 -----
1272 1298 All data structures here are only filled in, they are NOT reset by this
1273 1299 method. If they were not empty before, data will simply be added to
1274 1300 them.
1275 1301 """
1276 1302 # This function works in two parts: first we put a few things in
1277 1303 # user_ns, and we sync that contents into user_ns_hidden so that these
1278 1304 # initial variables aren't shown by %who. After the sync, we add the
1279 1305 # rest of what we *do* want the user to see with %who even on a new
1280 1306 # session (probably nothing, so they really only see their own stuff)
1281 1307
1282 1308 # The user dict must *always* have a __builtin__ reference to the
1283 1309 # Python standard __builtin__ namespace, which must be imported.
1284 1310 # This is so that certain operations in prompt evaluation can be
1285 1311 # reliably executed with builtins. Note that we can NOT use
1286 1312 # __builtins__ (note the 's'), because that can either be a dict or a
1287 1313 # module, and can even mutate at runtime, depending on the context
1288 1314 # (Python makes no guarantees on it). In contrast, __builtin__ is
1289 1315 # always a module object, though it must be explicitly imported.
1290 1316
1291 1317 # For more details:
1292 1318 # http://mail.python.org/pipermail/python-dev/2001-April/014068.html
1293 1319 ns = {}
1294 1320
1295 1321 # make global variables for user access to the histories
1296 1322 ns['_ih'] = self.history_manager.input_hist_parsed
1297 1323 ns['_oh'] = self.history_manager.output_hist
1298 1324 ns['_dh'] = self.history_manager.dir_hist
1299 1325
1300 1326 # user aliases to input and output histories. These shouldn't show up
1301 1327 # in %who, as they can have very large reprs.
1302 1328 ns['In'] = self.history_manager.input_hist_parsed
1303 1329 ns['Out'] = self.history_manager.output_hist
1304 1330
1305 1331 # Store myself as the public api!!!
1306 1332 ns['get_ipython'] = self.get_ipython
1307 1333
1308 1334 ns['exit'] = self.exiter
1309 1335 ns['quit'] = self.exiter
1336 ns["open"] = _modified_open
1310 1337
1311 1338 # Sync what we've added so far to user_ns_hidden so these aren't seen
1312 1339 # by %who
1313 1340 self.user_ns_hidden.update(ns)
1314 1341
1315 1342 # Anything put into ns now would show up in %who. Think twice before
1316 1343 # putting anything here, as we really want %who to show the user their
1317 1344 # stuff, not our variables.
1318 1345
1319 1346 # Finally, update the real user's namespace
1320 1347 self.user_ns.update(ns)
1321 1348
1322 1349 @property
1323 1350 def all_ns_refs(self):
1324 1351 """Get a list of references to all the namespace dictionaries in which
1325 1352 IPython might store a user-created object.
1326 1353
1327 1354 Note that this does not include the displayhook, which also caches
1328 1355 objects from the output."""
1329 1356 return [self.user_ns, self.user_global_ns, self.user_ns_hidden] + \
1330 1357 [m.__dict__ for m in self._main_mod_cache.values()]
1331 1358
1332 1359 def reset(self, new_session=True, aggressive=False):
1333 1360 """Clear all internal namespaces, and attempt to release references to
1334 1361 user objects.
1335 1362
1336 1363 If new_session is True, a new history session will be opened.
1337 1364 """
1338 1365 # Clear histories
1339 1366 self.history_manager.reset(new_session)
1340 1367 # Reset counter used to index all histories
1341 1368 if new_session:
1342 1369 self.execution_count = 1
1343 1370
1344 1371 # Reset last execution result
1345 1372 self.last_execution_succeeded = True
1346 1373 self.last_execution_result = None
1347 1374
1348 1375 # Flush cached output items
1349 1376 if self.displayhook.do_full_cache:
1350 1377 self.displayhook.flush()
1351 1378
1352 1379 # The main execution namespaces must be cleared very carefully,
1353 1380 # skipping the deletion of the builtin-related keys, because doing so
1354 1381 # would cause errors in many object's __del__ methods.
1355 1382 if self.user_ns is not self.user_global_ns:
1356 1383 self.user_ns.clear()
1357 1384 ns = self.user_global_ns
1358 1385 drop_keys = set(ns.keys())
1359 1386 drop_keys.discard('__builtin__')
1360 1387 drop_keys.discard('__builtins__')
1361 1388 drop_keys.discard('__name__')
1362 1389 for k in drop_keys:
1363 1390 del ns[k]
1364 1391
1365 1392 self.user_ns_hidden.clear()
1366 1393
1367 1394 # Restore the user namespaces to minimal usability
1368 1395 self.init_user_ns()
1369 1396 if aggressive and not hasattr(self, "_sys_modules_keys"):
1370 1397 print("Cannot restore sys.module, no snapshot")
1371 1398 elif aggressive:
1372 1399 print("culling sys module...")
1373 1400 current_keys = set(sys.modules.keys())
1374 1401 for k in current_keys - self._sys_modules_keys:
1375 1402 if k.startswith("multiprocessing"):
1376 1403 continue
1377 1404 del sys.modules[k]
1378 1405
1379 1406 # Restore the default and user aliases
1380 1407 self.alias_manager.clear_aliases()
1381 1408 self.alias_manager.init_aliases()
1382 1409
1383 1410 # Now define aliases that only make sense on the terminal, because they
1384 1411 # need direct access to the console in a way that we can't emulate in
1385 1412 # GUI or web frontend
1386 1413 if os.name == 'posix':
1387 1414 for cmd in ('clear', 'more', 'less', 'man'):
1388 1415 if cmd not in self.magics_manager.magics['line']:
1389 1416 self.alias_manager.soft_define_alias(cmd, cmd)
1390 1417
1391 1418 # Flush the private list of module references kept for script
1392 1419 # execution protection
1393 1420 self.clear_main_mod_cache()
1394 1421
1395 1422 def del_var(self, varname, by_name=False):
1396 1423 """Delete a variable from the various namespaces, so that, as
1397 1424 far as possible, we're not keeping any hidden references to it.
1398 1425
1399 1426 Parameters
1400 1427 ----------
1401 1428 varname : str
1402 1429 The name of the variable to delete.
1403 1430 by_name : bool
1404 1431 If True, delete variables with the given name in each
1405 1432 namespace. If False (default), find the variable in the user
1406 1433 namespace, and delete references to it.
1407 1434 """
1408 1435 if varname in ('__builtin__', '__builtins__'):
1409 1436 raise ValueError("Refusing to delete %s" % varname)
1410 1437
1411 1438 ns_refs = self.all_ns_refs
1412 1439
1413 1440 if by_name: # Delete by name
1414 1441 for ns in ns_refs:
1415 1442 try:
1416 1443 del ns[varname]
1417 1444 except KeyError:
1418 1445 pass
1419 1446 else: # Delete by object
1420 1447 try:
1421 1448 obj = self.user_ns[varname]
1422 1449 except KeyError as e:
1423 1450 raise NameError("name '%s' is not defined" % varname) from e
1424 1451 # Also check in output history
1425 1452 ns_refs.append(self.history_manager.output_hist)
1426 1453 for ns in ns_refs:
1427 1454 to_delete = [n for n, o in ns.items() if o is obj]
1428 1455 for name in to_delete:
1429 1456 del ns[name]
1430 1457
1431 1458 # Ensure it is removed from the last execution result
1432 1459 if self.last_execution_result.result is obj:
1433 1460 self.last_execution_result = None
1434 1461
1435 1462 # displayhook keeps extra references, but not in a dictionary
1436 1463 for name in ('_', '__', '___'):
1437 1464 if getattr(self.displayhook, name) is obj:
1438 1465 setattr(self.displayhook, name, None)
1439 1466
1440 1467 def reset_selective(self, regex=None):
1441 1468 """Clear selective variables from internal namespaces based on a
1442 1469 specified regular expression.
1443 1470
1444 1471 Parameters
1445 1472 ----------
1446 1473 regex : string or compiled pattern, optional
1447 1474 A regular expression pattern that will be used in searching
1448 1475 variable names in the users namespaces.
1449 1476 """
1450 1477 if regex is not None:
1451 1478 try:
1452 1479 m = re.compile(regex)
1453 1480 except TypeError as e:
1454 1481 raise TypeError('regex must be a string or compiled pattern') from e
1455 1482 # Search for keys in each namespace that match the given regex
1456 1483 # If a match is found, delete the key/value pair.
1457 1484 for ns in self.all_ns_refs:
1458 1485 for var in ns:
1459 1486 if m.search(var):
1460 1487 del ns[var]
1461 1488
1462 1489 def push(self, variables, interactive=True):
1463 1490 """Inject a group of variables into the IPython user namespace.
1464 1491
1465 1492 Parameters
1466 1493 ----------
1467 1494 variables : dict, str or list/tuple of str
1468 1495 The variables to inject into the user's namespace. If a dict, a
1469 1496 simple update is done. If a str, the string is assumed to have
1470 1497 variable names separated by spaces. A list/tuple of str can also
1471 1498 be used to give the variable names. If just the variable names are
1472 1499 give (list/tuple/str) then the variable values looked up in the
1473 1500 callers frame.
1474 1501 interactive : bool
1475 1502 If True (default), the variables will be listed with the ``who``
1476 1503 magic.
1477 1504 """
1478 1505 vdict = None
1479 1506
1480 1507 # We need a dict of name/value pairs to do namespace updates.
1481 1508 if isinstance(variables, dict):
1482 1509 vdict = variables
1483 1510 elif isinstance(variables, (str, list, tuple)):
1484 1511 if isinstance(variables, str):
1485 1512 vlist = variables.split()
1486 1513 else:
1487 1514 vlist = variables
1488 1515 vdict = {}
1489 1516 cf = sys._getframe(1)
1490 1517 for name in vlist:
1491 1518 try:
1492 1519 vdict[name] = eval(name, cf.f_globals, cf.f_locals)
1493 1520 except:
1494 1521 print('Could not get variable %s from %s' %
1495 1522 (name,cf.f_code.co_name))
1496 1523 else:
1497 1524 raise ValueError('variables must be a dict/str/list/tuple')
1498 1525
1499 1526 # Propagate variables to user namespace
1500 1527 self.user_ns.update(vdict)
1501 1528
1502 1529 # And configure interactive visibility
1503 1530 user_ns_hidden = self.user_ns_hidden
1504 1531 if interactive:
1505 1532 for name in vdict:
1506 1533 user_ns_hidden.pop(name, None)
1507 1534 else:
1508 1535 user_ns_hidden.update(vdict)
1509 1536
1510 1537 def drop_by_id(self, variables):
1511 1538 """Remove a dict of variables from the user namespace, if they are the
1512 1539 same as the values in the dictionary.
1513 1540
1514 1541 This is intended for use by extensions: variables that they've added can
1515 1542 be taken back out if they are unloaded, without removing any that the
1516 1543 user has overwritten.
1517 1544
1518 1545 Parameters
1519 1546 ----------
1520 1547 variables : dict
1521 1548 A dictionary mapping object names (as strings) to the objects.
1522 1549 """
1523 1550 for name, obj in variables.items():
1524 1551 if name in self.user_ns and self.user_ns[name] is obj:
1525 1552 del self.user_ns[name]
1526 1553 self.user_ns_hidden.pop(name, None)
1527 1554
1528 1555 #-------------------------------------------------------------------------
1529 1556 # Things related to object introspection
1530 1557 #-------------------------------------------------------------------------
1531 1558
1532 1559 def _ofind(self, oname, namespaces=None):
1533 1560 """Find an object in the available namespaces.
1534 1561
1535 1562 self._ofind(oname) -> dict with keys: found,obj,ospace,ismagic
1536 1563
1537 1564 Has special code to detect magic functions.
1538 1565 """
1539 1566 oname = oname.strip()
1540 if not oname.startswith(ESC_MAGIC) and \
1541 not oname.startswith(ESC_MAGIC2) and \
1542 not all(a.isidentifier() for a in oname.split(".")):
1543 return {'found': False}
1567 raw_parts = oname.split(".")
1568 parts = []
1569 parts_ok = True
1570 for p in raw_parts:
1571 if p.endswith("]"):
1572 var, *indices = p.split("[")
1573 if not var.isidentifier():
1574 parts_ok = False
1575 break
1576 parts.append(var)
1577 for ind in indices:
1578 if ind[-1] != "]" and not is_integer_string(ind[:-1]):
1579 parts_ok = False
1580 break
1581 parts.append(ind[:-1])
1582 continue
1583
1584 if not p.isidentifier():
1585 parts_ok = False
1586 parts.append(p)
1587
1588 if (
1589 not oname.startswith(ESC_MAGIC)
1590 and not oname.startswith(ESC_MAGIC2)
1591 and not parts_ok
1592 ):
1593 return {"found": False}
1544 1594
1545 1595 if namespaces is None:
1546 1596 # Namespaces to search in:
1547 1597 # Put them in a list. The order is important so that we
1548 1598 # find things in the same order that Python finds them.
1549 1599 namespaces = [ ('Interactive', self.user_ns),
1550 1600 ('Interactive (global)', self.user_global_ns),
1551 1601 ('Python builtin', builtin_mod.__dict__),
1552 1602 ]
1553 1603
1554 1604 ismagic = False
1555 1605 isalias = False
1556 1606 found = False
1557 1607 ospace = None
1558 1608 parent = None
1559 1609 obj = None
1560 1610
1561 1611
1562 1612 # Look for the given name by splitting it in parts. If the head is
1563 1613 # found, then we look for all the remaining parts as members, and only
1564 1614 # declare success if we can find them all.
1565 oname_parts = oname.split('.')
1615 oname_parts = parts
1566 1616 oname_head, oname_rest = oname_parts[0],oname_parts[1:]
1567 1617 for nsname,ns in namespaces:
1568 1618 try:
1569 1619 obj = ns[oname_head]
1570 1620 except KeyError:
1571 1621 continue
1572 1622 else:
1573 1623 for idx, part in enumerate(oname_rest):
1574 1624 try:
1575 1625 parent = obj
1576 1626 # The last part is looked up in a special way to avoid
1577 1627 # descriptor invocation as it may raise or have side
1578 1628 # effects.
1579 1629 if idx == len(oname_rest) - 1:
1580 1630 obj = self._getattr_property(obj, part)
1581 1631 else:
1582 obj = getattr(obj, part)
1632 if is_integer_string(part):
1633 obj = obj[int(part)]
1634 else:
1635 obj = getattr(obj, part)
1583 1636 except:
1584 1637 # Blanket except b/c some badly implemented objects
1585 1638 # allow __getattr__ to raise exceptions other than
1586 1639 # AttributeError, which then crashes IPython.
1587 1640 break
1588 1641 else:
1589 1642 # If we finish the for loop (no break), we got all members
1590 1643 found = True
1591 1644 ospace = nsname
1592 1645 break # namespace loop
1593 1646
1594 1647 # Try to see if it's magic
1595 1648 if not found:
1596 1649 obj = None
1597 1650 if oname.startswith(ESC_MAGIC2):
1598 1651 oname = oname.lstrip(ESC_MAGIC2)
1599 1652 obj = self.find_cell_magic(oname)
1600 1653 elif oname.startswith(ESC_MAGIC):
1601 1654 oname = oname.lstrip(ESC_MAGIC)
1602 1655 obj = self.find_line_magic(oname)
1603 1656 else:
1604 1657 # search without prefix, so run? will find %run?
1605 1658 obj = self.find_line_magic(oname)
1606 1659 if obj is None:
1607 1660 obj = self.find_cell_magic(oname)
1608 1661 if obj is not None:
1609 1662 found = True
1610 1663 ospace = 'IPython internal'
1611 1664 ismagic = True
1612 1665 isalias = isinstance(obj, Alias)
1613 1666
1614 1667 # Last try: special-case some literals like '', [], {}, etc:
1615 1668 if not found and oname_head in ["''",'""','[]','{}','()']:
1616 1669 obj = eval(oname_head)
1617 1670 found = True
1618 1671 ospace = 'Interactive'
1619 1672
1620 1673 return {
1621 1674 'obj':obj,
1622 1675 'found':found,
1623 1676 'parent':parent,
1624 1677 'ismagic':ismagic,
1625 1678 'isalias':isalias,
1626 1679 'namespace':ospace
1627 1680 }
1628 1681
1629 1682 @staticmethod
1630 1683 def _getattr_property(obj, attrname):
1631 1684 """Property-aware getattr to use in object finding.
1632 1685
1633 1686 If attrname represents a property, return it unevaluated (in case it has
1634 1687 side effects or raises an error.
1635 1688
1636 1689 """
1637 1690 if not isinstance(obj, type):
1638 1691 try:
1639 1692 # `getattr(type(obj), attrname)` is not guaranteed to return
1640 1693 # `obj`, but does so for property:
1641 1694 #
1642 1695 # property.__get__(self, None, cls) -> self
1643 1696 #
1644 1697 # The universal alternative is to traverse the mro manually
1645 1698 # searching for attrname in class dicts.
1646 attr = getattr(type(obj), attrname)
1699 if is_integer_string(attrname):
1700 return obj[int(attrname)]
1701 else:
1702 attr = getattr(type(obj), attrname)
1647 1703 except AttributeError:
1648 1704 pass
1649 1705 else:
1650 1706 # This relies on the fact that data descriptors (with both
1651 1707 # __get__ & __set__ magic methods) take precedence over
1652 1708 # instance-level attributes:
1653 1709 #
1654 1710 # class A(object):
1655 1711 # @property
1656 1712 # def foobar(self): return 123
1657 1713 # a = A()
1658 1714 # a.__dict__['foobar'] = 345
1659 1715 # a.foobar # == 123
1660 1716 #
1661 1717 # So, a property may be returned right away.
1662 1718 if isinstance(attr, property):
1663 1719 return attr
1664 1720
1665 1721 # Nothing helped, fall back.
1666 1722 return getattr(obj, attrname)
1667 1723
1668 1724 def _object_find(self, oname, namespaces=None):
1669 1725 """Find an object and return a struct with info about it."""
1670 1726 return Struct(self._ofind(oname, namespaces))
1671 1727
1672 1728 def _inspect(self, meth, oname, namespaces=None, **kw):
1673 1729 """Generic interface to the inspector system.
1674 1730
1675 1731 This function is meant to be called by pdef, pdoc & friends.
1676 1732 """
1677 1733 info = self._object_find(oname, namespaces)
1678 1734 docformat = (
1679 1735 sphinxify(self.object_inspect(oname)) if self.sphinxify_docstring else None
1680 1736 )
1681 1737 if info.found:
1682 1738 pmethod = getattr(self.inspector, meth)
1683 1739 # TODO: only apply format_screen to the plain/text repr of the mime
1684 1740 # bundle.
1685 1741 formatter = format_screen if info.ismagic else docformat
1686 1742 if meth == 'pdoc':
1687 1743 pmethod(info.obj, oname, formatter)
1688 1744 elif meth == 'pinfo':
1689 1745 pmethod(
1690 1746 info.obj,
1691 1747 oname,
1692 1748 formatter,
1693 1749 info,
1694 1750 enable_html_pager=self.enable_html_pager,
1695 1751 **kw,
1696 1752 )
1697 1753 else:
1698 1754 pmethod(info.obj, oname)
1699 1755 else:
1700 1756 print('Object `%s` not found.' % oname)
1701 1757 return 'not found' # so callers can take other action
1702 1758
1703 1759 def object_inspect(self, oname, detail_level=0):
1704 1760 """Get object info about oname"""
1705 1761 with self.builtin_trap:
1706 1762 info = self._object_find(oname)
1707 1763 if info.found:
1708 1764 return self.inspector.info(info.obj, oname, info=info,
1709 1765 detail_level=detail_level
1710 1766 )
1711 1767 else:
1712 1768 return oinspect.object_info(name=oname, found=False)
1713 1769
1714 1770 def object_inspect_text(self, oname, detail_level=0):
1715 1771 """Get object info as formatted text"""
1716 1772 return self.object_inspect_mime(oname, detail_level)['text/plain']
1717 1773
1718 1774 def object_inspect_mime(self, oname, detail_level=0, omit_sections=()):
1719 1775 """Get object info as a mimebundle of formatted representations.
1720 1776
1721 1777 A mimebundle is a dictionary, keyed by mime-type.
1722 1778 It must always have the key `'text/plain'`.
1723 1779 """
1724 1780 with self.builtin_trap:
1725 1781 info = self._object_find(oname)
1726 1782 if info.found:
1727 1783 docformat = (
1728 1784 sphinxify(self.object_inspect(oname))
1729 1785 if self.sphinxify_docstring
1730 1786 else None
1731 1787 )
1732 1788 return self.inspector._get_info(
1733 1789 info.obj,
1734 1790 oname,
1735 1791 info=info,
1736 1792 detail_level=detail_level,
1737 1793 formatter=docformat,
1738 1794 omit_sections=omit_sections,
1739 1795 )
1740 1796 else:
1741 1797 raise KeyError(oname)
1742 1798
1743 1799 #-------------------------------------------------------------------------
1744 1800 # Things related to history management
1745 1801 #-------------------------------------------------------------------------
1746 1802
1747 1803 def init_history(self):
1748 1804 """Sets up the command history, and starts regular autosaves."""
1749 1805 self.history_manager = HistoryManager(shell=self, parent=self)
1750 1806 self.configurables.append(self.history_manager)
1751 1807
1752 1808 #-------------------------------------------------------------------------
1753 1809 # Things related to exception handling and tracebacks (not debugging)
1754 1810 #-------------------------------------------------------------------------
1755 1811
1756 1812 debugger_cls = InterruptiblePdb
1757 1813
1758 1814 def init_traceback_handlers(self, custom_exceptions):
1759 1815 # Syntax error handler.
1760 1816 self.SyntaxTB = ultratb.SyntaxTB(color_scheme='NoColor', parent=self)
1761 1817
1762 1818 # The interactive one is initialized with an offset, meaning we always
1763 1819 # want to remove the topmost item in the traceback, which is our own
1764 1820 # internal code. Valid modes: ['Plain','Context','Verbose','Minimal']
1765 1821 self.InteractiveTB = ultratb.AutoFormattedTB(mode = 'Plain',
1766 1822 color_scheme='NoColor',
1767 1823 tb_offset = 1,
1768 check_cache=check_linecache_ipython,
1769 1824 debugger_cls=self.debugger_cls, parent=self)
1770 1825
1771 1826 # The instance will store a pointer to the system-wide exception hook,
1772 1827 # so that runtime code (such as magics) can access it. This is because
1773 1828 # during the read-eval loop, it may get temporarily overwritten.
1774 1829 self.sys_excepthook = sys.excepthook
1775 1830
1776 1831 # and add any custom exception handlers the user may have specified
1777 1832 self.set_custom_exc(*custom_exceptions)
1778 1833
1779 1834 # Set the exception mode
1780 1835 self.InteractiveTB.set_mode(mode=self.xmode)
1781 1836
1782 1837 def set_custom_exc(self, exc_tuple, handler):
1783 1838 """set_custom_exc(exc_tuple, handler)
1784 1839
1785 1840 Set a custom exception handler, which will be called if any of the
1786 1841 exceptions in exc_tuple occur in the mainloop (specifically, in the
1787 1842 run_code() method).
1788 1843
1789 1844 Parameters
1790 1845 ----------
1791 1846 exc_tuple : tuple of exception classes
1792 1847 A *tuple* of exception classes, for which to call the defined
1793 1848 handler. It is very important that you use a tuple, and NOT A
1794 1849 LIST here, because of the way Python's except statement works. If
1795 1850 you only want to trap a single exception, use a singleton tuple::
1796 1851
1797 1852 exc_tuple == (MyCustomException,)
1798 1853
1799 1854 handler : callable
1800 1855 handler must have the following signature::
1801 1856
1802 1857 def my_handler(self, etype, value, tb, tb_offset=None):
1803 1858 ...
1804 1859 return structured_traceback
1805 1860
1806 1861 Your handler must return a structured traceback (a list of strings),
1807 1862 or None.
1808 1863
1809 1864 This will be made into an instance method (via types.MethodType)
1810 1865 of IPython itself, and it will be called if any of the exceptions
1811 1866 listed in the exc_tuple are caught. If the handler is None, an
1812 1867 internal basic one is used, which just prints basic info.
1813 1868
1814 1869 To protect IPython from crashes, if your handler ever raises an
1815 1870 exception or returns an invalid result, it will be immediately
1816 1871 disabled.
1817 1872
1818 1873 Notes
1819 1874 -----
1820 1875 WARNING: by putting in your own exception handler into IPython's main
1821 1876 execution loop, you run a very good chance of nasty crashes. This
1822 1877 facility should only be used if you really know what you are doing.
1823 1878 """
1824 1879
1825 1880 if not isinstance(exc_tuple, tuple):
1826 1881 raise TypeError("The custom exceptions must be given as a tuple.")
1827 1882
1828 1883 def dummy_handler(self, etype, value, tb, tb_offset=None):
1829 1884 print('*** Simple custom exception handler ***')
1830 1885 print('Exception type :', etype)
1831 1886 print('Exception value:', value)
1832 1887 print('Traceback :', tb)
1833 1888
1834 1889 def validate_stb(stb):
1835 1890 """validate structured traceback return type
1836 1891
1837 1892 return type of CustomTB *should* be a list of strings, but allow
1838 1893 single strings or None, which are harmless.
1839 1894
1840 1895 This function will *always* return a list of strings,
1841 1896 and will raise a TypeError if stb is inappropriate.
1842 1897 """
1843 1898 msg = "CustomTB must return list of strings, not %r" % stb
1844 1899 if stb is None:
1845 1900 return []
1846 1901 elif isinstance(stb, str):
1847 1902 return [stb]
1848 1903 elif not isinstance(stb, list):
1849 1904 raise TypeError(msg)
1850 1905 # it's a list
1851 1906 for line in stb:
1852 1907 # check every element
1853 1908 if not isinstance(line, str):
1854 1909 raise TypeError(msg)
1855 1910 return stb
1856 1911
1857 1912 if handler is None:
1858 1913 wrapped = dummy_handler
1859 1914 else:
1860 1915 def wrapped(self,etype,value,tb,tb_offset=None):
1861 1916 """wrap CustomTB handler, to protect IPython from user code
1862 1917
1863 1918 This makes it harder (but not impossible) for custom exception
1864 1919 handlers to crash IPython.
1865 1920 """
1866 1921 try:
1867 1922 stb = handler(self,etype,value,tb,tb_offset=tb_offset)
1868 1923 return validate_stb(stb)
1869 1924 except:
1870 1925 # clear custom handler immediately
1871 1926 self.set_custom_exc((), None)
1872 1927 print("Custom TB Handler failed, unregistering", file=sys.stderr)
1873 1928 # show the exception in handler first
1874 1929 stb = self.InteractiveTB.structured_traceback(*sys.exc_info())
1875 1930 print(self.InteractiveTB.stb2text(stb))
1876 1931 print("The original exception:")
1877 1932 stb = self.InteractiveTB.structured_traceback(
1878 1933 (etype,value,tb), tb_offset=tb_offset
1879 1934 )
1880 1935 return stb
1881 1936
1882 1937 self.CustomTB = types.MethodType(wrapped,self)
1883 1938 self.custom_exceptions = exc_tuple
1884 1939
1885 1940 def excepthook(self, etype, value, tb):
1886 1941 """One more defense for GUI apps that call sys.excepthook.
1887 1942
1888 1943 GUI frameworks like wxPython trap exceptions and call
1889 1944 sys.excepthook themselves. I guess this is a feature that
1890 1945 enables them to keep running after exceptions that would
1891 1946 otherwise kill their mainloop. This is a bother for IPython
1892 1947 which expects to catch all of the program exceptions with a try:
1893 1948 except: statement.
1894 1949
1895 1950 Normally, IPython sets sys.excepthook to a CrashHandler instance, so if
1896 1951 any app directly invokes sys.excepthook, it will look to the user like
1897 1952 IPython crashed. In order to work around this, we can disable the
1898 1953 CrashHandler and replace it with this excepthook instead, which prints a
1899 1954 regular traceback using our InteractiveTB. In this fashion, apps which
1900 1955 call sys.excepthook will generate a regular-looking exception from
1901 1956 IPython, and the CrashHandler will only be triggered by real IPython
1902 1957 crashes.
1903 1958
1904 1959 This hook should be used sparingly, only in places which are not likely
1905 1960 to be true IPython errors.
1906 1961 """
1907 1962 self.showtraceback((etype, value, tb), tb_offset=0)
1908 1963
1909 1964 def _get_exc_info(self, exc_tuple=None):
1910 1965 """get exc_info from a given tuple, sys.exc_info() or sys.last_type etc.
1911 1966
1912 1967 Ensures sys.last_type,value,traceback hold the exc_info we found,
1913 1968 from whichever source.
1914 1969
1915 1970 raises ValueError if none of these contain any information
1916 1971 """
1917 1972 if exc_tuple is None:
1918 1973 etype, value, tb = sys.exc_info()
1919 1974 else:
1920 1975 etype, value, tb = exc_tuple
1921 1976
1922 1977 if etype is None:
1923 1978 if hasattr(sys, 'last_type'):
1924 1979 etype, value, tb = sys.last_type, sys.last_value, \
1925 1980 sys.last_traceback
1926 1981
1927 1982 if etype is None:
1928 1983 raise ValueError("No exception to find")
1929 1984
1930 1985 # Now store the exception info in sys.last_type etc.
1931 1986 # WARNING: these variables are somewhat deprecated and not
1932 1987 # necessarily safe to use in a threaded environment, but tools
1933 1988 # like pdb depend on their existence, so let's set them. If we
1934 1989 # find problems in the field, we'll need to revisit their use.
1935 1990 sys.last_type = etype
1936 1991 sys.last_value = value
1937 1992 sys.last_traceback = tb
1938 1993
1939 1994 return etype, value, tb
1940 1995
1941 1996 def show_usage_error(self, exc):
1942 1997 """Show a short message for UsageErrors
1943 1998
1944 1999 These are special exceptions that shouldn't show a traceback.
1945 2000 """
1946 2001 print("UsageError: %s" % exc, file=sys.stderr)
1947 2002
1948 2003 def get_exception_only(self, exc_tuple=None):
1949 2004 """
1950 2005 Return as a string (ending with a newline) the exception that
1951 2006 just occurred, without any traceback.
1952 2007 """
1953 2008 etype, value, tb = self._get_exc_info(exc_tuple)
1954 2009 msg = traceback.format_exception_only(etype, value)
1955 2010 return ''.join(msg)
1956 2011
1957 2012 def showtraceback(self, exc_tuple=None, filename=None, tb_offset=None,
1958 2013 exception_only=False, running_compiled_code=False):
1959 2014 """Display the exception that just occurred.
1960 2015
1961 2016 If nothing is known about the exception, this is the method which
1962 2017 should be used throughout the code for presenting user tracebacks,
1963 2018 rather than directly invoking the InteractiveTB object.
1964 2019
1965 2020 A specific showsyntaxerror() also exists, but this method can take
1966 2021 care of calling it if needed, so unless you are explicitly catching a
1967 2022 SyntaxError exception, don't try to analyze the stack manually and
1968 2023 simply call this method."""
1969 2024
1970 2025 try:
1971 2026 try:
1972 2027 etype, value, tb = self._get_exc_info(exc_tuple)
1973 2028 except ValueError:
1974 2029 print('No traceback available to show.', file=sys.stderr)
1975 2030 return
1976 2031
1977 2032 if issubclass(etype, SyntaxError):
1978 2033 # Though this won't be called by syntax errors in the input
1979 2034 # line, there may be SyntaxError cases with imported code.
1980 2035 self.showsyntaxerror(filename, running_compiled_code)
1981 2036 elif etype is UsageError:
1982 2037 self.show_usage_error(value)
1983 2038 else:
1984 2039 if exception_only:
1985 2040 stb = ['An exception has occurred, use %tb to see '
1986 2041 'the full traceback.\n']
1987 2042 stb.extend(self.InteractiveTB.get_exception_only(etype,
1988 2043 value))
1989 2044 else:
1990 2045 try:
1991 2046 # Exception classes can customise their traceback - we
1992 2047 # use this in IPython.parallel for exceptions occurring
1993 2048 # in the engines. This should return a list of strings.
1994 2049 if hasattr(value, "_render_traceback_"):
1995 2050 stb = value._render_traceback_()
1996 2051 else:
1997 2052 stb = self.InteractiveTB.structured_traceback(
1998 2053 etype, value, tb, tb_offset=tb_offset
1999 2054 )
2000 2055
2001 2056 except Exception:
2002 2057 print(
2003 2058 "Unexpected exception formatting exception. Falling back to standard exception"
2004 2059 )
2005 2060 traceback.print_exc()
2006 2061 return None
2007 2062
2008 2063 self._showtraceback(etype, value, stb)
2009 2064 if self.call_pdb:
2010 2065 # drop into debugger
2011 2066 self.debugger(force=True)
2012 2067 return
2013 2068
2014 2069 # Actually show the traceback
2015 2070 self._showtraceback(etype, value, stb)
2016 2071
2017 2072 except KeyboardInterrupt:
2018 2073 print('\n' + self.get_exception_only(), file=sys.stderr)
2019 2074
2020 2075 def _showtraceback(self, etype, evalue, stb: str):
2021 2076 """Actually show a traceback.
2022 2077
2023 2078 Subclasses may override this method to put the traceback on a different
2024 2079 place, like a side channel.
2025 2080 """
2026 2081 val = self.InteractiveTB.stb2text(stb)
2027 2082 try:
2028 2083 print(val)
2029 2084 except UnicodeEncodeError:
2030 2085 print(val.encode("utf-8", "backslashreplace").decode())
2031 2086
2032 2087 def showsyntaxerror(self, filename=None, running_compiled_code=False):
2033 2088 """Display the syntax error that just occurred.
2034 2089
2035 2090 This doesn't display a stack trace because there isn't one.
2036 2091
2037 2092 If a filename is given, it is stuffed in the exception instead
2038 2093 of what was there before (because Python's parser always uses
2039 2094 "<string>" when reading from a string).
2040 2095
2041 2096 If the syntax error occurred when running a compiled code (i.e. running_compile_code=True),
2042 2097 longer stack trace will be displayed.
2043 2098 """
2044 2099 etype, value, last_traceback = self._get_exc_info()
2045 2100
2046 2101 if filename and issubclass(etype, SyntaxError):
2047 2102 try:
2048 2103 value.filename = filename
2049 2104 except:
2050 2105 # Not the format we expect; leave it alone
2051 2106 pass
2052 2107
2053 2108 # If the error occurred when executing compiled code, we should provide full stacktrace.
2054 2109 elist = traceback.extract_tb(last_traceback) if running_compiled_code else []
2055 2110 stb = self.SyntaxTB.structured_traceback(etype, value, elist)
2056 2111 self._showtraceback(etype, value, stb)
2057 2112
2058 2113 # This is overridden in TerminalInteractiveShell to show a message about
2059 2114 # the %paste magic.
2060 2115 def showindentationerror(self):
2061 2116 """Called by _run_cell when there's an IndentationError in code entered
2062 2117 at the prompt.
2063 2118
2064 2119 This is overridden in TerminalInteractiveShell to show a message about
2065 2120 the %paste magic."""
2066 2121 self.showsyntaxerror()
2067 2122
2068 2123 @skip_doctest
2069 2124 def set_next_input(self, s, replace=False):
2070 2125 """ Sets the 'default' input string for the next command line.
2071 2126
2072 2127 Example::
2073 2128
2074 2129 In [1]: _ip.set_next_input("Hello Word")
2075 2130 In [2]: Hello Word_ # cursor is here
2076 2131 """
2077 2132 self.rl_next_input = s
2078 2133
2079 2134 def _indent_current_str(self):
2080 2135 """return the current level of indentation as a string"""
2081 2136 return self.input_splitter.get_indent_spaces() * ' '
2082 2137
2083 2138 #-------------------------------------------------------------------------
2084 2139 # Things related to text completion
2085 2140 #-------------------------------------------------------------------------
2086 2141
2087 2142 def init_completer(self):
2088 2143 """Initialize the completion machinery.
2089 2144
2090 2145 This creates completion machinery that can be used by client code,
2091 2146 either interactively in-process (typically triggered by the readline
2092 2147 library), programmatically (such as in test suites) or out-of-process
2093 2148 (typically over the network by remote frontends).
2094 2149 """
2095 2150 from IPython.core.completer import IPCompleter
2096 2151 from IPython.core.completerlib import (
2097 2152 cd_completer,
2098 2153 magic_run_completer,
2099 2154 module_completer,
2100 2155 reset_completer,
2101 2156 )
2102 2157
2103 2158 self.Completer = IPCompleter(shell=self,
2104 2159 namespace=self.user_ns,
2105 2160 global_namespace=self.user_global_ns,
2106 2161 parent=self,
2107 2162 )
2108 2163 self.configurables.append(self.Completer)
2109 2164
2110 2165 # Add custom completers to the basic ones built into IPCompleter
2111 2166 sdisp = self.strdispatchers.get('complete_command', StrDispatch())
2112 2167 self.strdispatchers['complete_command'] = sdisp
2113 2168 self.Completer.custom_completers = sdisp
2114 2169
2115 2170 self.set_hook('complete_command', module_completer, str_key = 'import')
2116 2171 self.set_hook('complete_command', module_completer, str_key = 'from')
2117 2172 self.set_hook('complete_command', module_completer, str_key = '%aimport')
2118 2173 self.set_hook('complete_command', magic_run_completer, str_key = '%run')
2119 2174 self.set_hook('complete_command', cd_completer, str_key = '%cd')
2120 2175 self.set_hook('complete_command', reset_completer, str_key = '%reset')
2121 2176
2122 2177 @skip_doctest
2123 2178 def complete(self, text, line=None, cursor_pos=None):
2124 2179 """Return the completed text and a list of completions.
2125 2180
2126 2181 Parameters
2127 2182 ----------
2128 2183 text : string
2129 2184 A string of text to be completed on. It can be given as empty and
2130 2185 instead a line/position pair are given. In this case, the
2131 2186 completer itself will split the line like readline does.
2132 2187 line : string, optional
2133 2188 The complete line that text is part of.
2134 2189 cursor_pos : int, optional
2135 2190 The position of the cursor on the input line.
2136 2191
2137 2192 Returns
2138 2193 -------
2139 2194 text : string
2140 2195 The actual text that was completed.
2141 2196 matches : list
2142 2197 A sorted list with all possible completions.
2143 2198
2144 2199 Notes
2145 2200 -----
2146 2201 The optional arguments allow the completion to take more context into
2147 2202 account, and are part of the low-level completion API.
2148 2203
2149 2204 This is a wrapper around the completion mechanism, similar to what
2150 2205 readline does at the command line when the TAB key is hit. By
2151 2206 exposing it as a method, it can be used by other non-readline
2152 2207 environments (such as GUIs) for text completion.
2153 2208
2154 2209 Examples
2155 2210 --------
2156 2211 In [1]: x = 'hello'
2157 2212
2158 2213 In [2]: _ip.complete('x.l')
2159 2214 Out[2]: ('x.l', ['x.ljust', 'x.lower', 'x.lstrip'])
2160 2215 """
2161 2216
2162 2217 # Inject names into __builtin__ so we can complete on the added names.
2163 2218 with self.builtin_trap:
2164 2219 return self.Completer.complete(text, line, cursor_pos)
2165 2220
2166 2221 def set_custom_completer(self, completer, pos=0) -> None:
2167 2222 """Adds a new custom completer function.
2168 2223
2169 2224 The position argument (defaults to 0) is the index in the completers
2170 2225 list where you want the completer to be inserted.
2171 2226
2172 2227 `completer` should have the following signature::
2173 2228
2174 2229 def completion(self: Completer, text: string) -> List[str]:
2175 2230 raise NotImplementedError
2176 2231
2177 2232 It will be bound to the current Completer instance and pass some text
2178 2233 and return a list with current completions to suggest to the user.
2179 2234 """
2180 2235
2181 2236 newcomp = types.MethodType(completer, self.Completer)
2182 2237 self.Completer.custom_matchers.insert(pos,newcomp)
2183 2238
2184 2239 def set_completer_frame(self, frame=None):
2185 2240 """Set the frame of the completer."""
2186 2241 if frame:
2187 2242 self.Completer.namespace = frame.f_locals
2188 2243 self.Completer.global_namespace = frame.f_globals
2189 2244 else:
2190 2245 self.Completer.namespace = self.user_ns
2191 2246 self.Completer.global_namespace = self.user_global_ns
2192 2247
2193 2248 #-------------------------------------------------------------------------
2194 2249 # Things related to magics
2195 2250 #-------------------------------------------------------------------------
2196 2251
2197 2252 def init_magics(self):
2198 2253 from IPython.core import magics as m
2199 2254 self.magics_manager = magic.MagicsManager(shell=self,
2200 2255 parent=self,
2201 2256 user_magics=m.UserMagics(self))
2202 2257 self.configurables.append(self.magics_manager)
2203 2258
2204 2259 # Expose as public API from the magics manager
2205 2260 self.register_magics = self.magics_manager.register
2206 2261
2207 2262 self.register_magics(m.AutoMagics, m.BasicMagics, m.CodeMagics,
2208 2263 m.ConfigMagics, m.DisplayMagics, m.ExecutionMagics,
2209 2264 m.ExtensionMagics, m.HistoryMagics, m.LoggingMagics,
2210 2265 m.NamespaceMagics, m.OSMagics, m.PackagingMagics,
2211 2266 m.PylabMagics, m.ScriptMagics,
2212 2267 )
2213 2268 self.register_magics(m.AsyncMagics)
2214 2269
2215 2270 # Register Magic Aliases
2216 2271 mman = self.magics_manager
2217 2272 # FIXME: magic aliases should be defined by the Magics classes
2218 2273 # or in MagicsManager, not here
2219 2274 mman.register_alias('ed', 'edit')
2220 2275 mman.register_alias('hist', 'history')
2221 2276 mman.register_alias('rep', 'recall')
2222 2277 mman.register_alias('SVG', 'svg', 'cell')
2223 2278 mman.register_alias('HTML', 'html', 'cell')
2224 2279 mman.register_alias('file', 'writefile', 'cell')
2225 2280
2226 2281 # FIXME: Move the color initialization to the DisplayHook, which
2227 2282 # should be split into a prompt manager and displayhook. We probably
2228 2283 # even need a centralize colors management object.
2229 2284 self.run_line_magic('colors', self.colors)
2230 2285
2231 2286 # Defined here so that it's included in the documentation
2232 2287 @functools.wraps(magic.MagicsManager.register_function)
2233 2288 def register_magic_function(self, func, magic_kind='line', magic_name=None):
2234 2289 self.magics_manager.register_function(
2235 2290 func, magic_kind=magic_kind, magic_name=magic_name
2236 2291 )
2237 2292
2238 2293 def _find_with_lazy_load(self, /, type_, magic_name: str):
2239 2294 """
2240 2295 Try to find a magic potentially lazy-loading it.
2241 2296
2242 2297 Parameters
2243 2298 ----------
2244 2299
2245 2300 type_: "line"|"cell"
2246 2301 the type of magics we are trying to find/lazy load.
2247 2302 magic_name: str
2248 2303 The name of the magic we are trying to find/lazy load
2249 2304
2250 2305
2251 2306 Note that this may have any side effects
2252 2307 """
2253 2308 finder = {"line": self.find_line_magic, "cell": self.find_cell_magic}[type_]
2254 2309 fn = finder(magic_name)
2255 2310 if fn is not None:
2256 2311 return fn
2257 2312 lazy = self.magics_manager.lazy_magics.get(magic_name)
2258 2313 if lazy is None:
2259 2314 return None
2260 2315
2261 2316 self.run_line_magic("load_ext", lazy)
2262 2317 res = finder(magic_name)
2263 2318 return res
2264 2319
2265 2320 def run_line_magic(self, magic_name: str, line, _stack_depth=1):
2266 2321 """Execute the given line magic.
2267 2322
2268 2323 Parameters
2269 2324 ----------
2270 2325 magic_name : str
2271 2326 Name of the desired magic function, without '%' prefix.
2272 2327 line : str
2273 2328 The rest of the input line as a single string.
2274 2329 _stack_depth : int
2275 2330 If run_line_magic() is called from magic() then _stack_depth=2.
2276 2331 This is added to ensure backward compatibility for use of 'get_ipython().magic()'
2277 2332 """
2278 2333 fn = self._find_with_lazy_load("line", magic_name)
2279 2334 if fn is None:
2280 2335 lazy = self.magics_manager.lazy_magics.get(magic_name)
2281 2336 if lazy:
2282 2337 self.run_line_magic("load_ext", lazy)
2283 2338 fn = self.find_line_magic(magic_name)
2284 2339 if fn is None:
2285 2340 cm = self.find_cell_magic(magic_name)
2286 2341 etpl = "Line magic function `%%%s` not found%s."
2287 2342 extra = '' if cm is None else (' (But cell magic `%%%%%s` exists, '
2288 2343 'did you mean that instead?)' % magic_name )
2289 2344 raise UsageError(etpl % (magic_name, extra))
2290 2345 else:
2291 2346 # Note: this is the distance in the stack to the user's frame.
2292 2347 # This will need to be updated if the internal calling logic gets
2293 2348 # refactored, or else we'll be expanding the wrong variables.
2294 2349
2295 2350 # Determine stack_depth depending on where run_line_magic() has been called
2296 2351 stack_depth = _stack_depth
2297 2352 if getattr(fn, magic.MAGIC_NO_VAR_EXPAND_ATTR, False):
2298 2353 # magic has opted out of var_expand
2299 2354 magic_arg_s = line
2300 2355 else:
2301 2356 magic_arg_s = self.var_expand(line, stack_depth)
2302 2357 # Put magic args in a list so we can call with f(*a) syntax
2303 2358 args = [magic_arg_s]
2304 2359 kwargs = {}
2305 2360 # Grab local namespace if we need it:
2306 2361 if getattr(fn, "needs_local_scope", False):
2307 2362 kwargs['local_ns'] = self.get_local_scope(stack_depth)
2308 2363 with self.builtin_trap:
2309 2364 result = fn(*args, **kwargs)
2310 2365 return result
2311 2366
2312 2367 def get_local_scope(self, stack_depth):
2313 2368 """Get local scope at given stack depth.
2314 2369
2315 2370 Parameters
2316 2371 ----------
2317 2372 stack_depth : int
2318 2373 Depth relative to calling frame
2319 2374 """
2320 2375 return sys._getframe(stack_depth + 1).f_locals
2321 2376
2322 2377 def run_cell_magic(self, magic_name, line, cell):
2323 2378 """Execute the given cell magic.
2324 2379
2325 2380 Parameters
2326 2381 ----------
2327 2382 magic_name : str
2328 2383 Name of the desired magic function, without '%' prefix.
2329 2384 line : str
2330 2385 The rest of the first input line as a single string.
2331 2386 cell : str
2332 2387 The body of the cell as a (possibly multiline) string.
2333 2388 """
2334 2389 fn = self._find_with_lazy_load("cell", magic_name)
2335 2390 if fn is None:
2336 2391 lm = self.find_line_magic(magic_name)
2337 2392 etpl = "Cell magic `%%{0}` not found{1}."
2338 2393 extra = '' if lm is None else (' (But line magic `%{0}` exists, '
2339 2394 'did you mean that instead?)'.format(magic_name))
2340 2395 raise UsageError(etpl.format(magic_name, extra))
2341 2396 elif cell == '':
2342 2397 message = '%%{0} is a cell magic, but the cell body is empty.'.format(magic_name)
2343 2398 if self.find_line_magic(magic_name) is not None:
2344 2399 message += ' Did you mean the line magic %{0} (single %)?'.format(magic_name)
2345 2400 raise UsageError(message)
2346 2401 else:
2347 2402 # Note: this is the distance in the stack to the user's frame.
2348 2403 # This will need to be updated if the internal calling logic gets
2349 2404 # refactored, or else we'll be expanding the wrong variables.
2350 2405 stack_depth = 2
2351 2406 if getattr(fn, magic.MAGIC_NO_VAR_EXPAND_ATTR, False):
2352 2407 # magic has opted out of var_expand
2353 2408 magic_arg_s = line
2354 2409 else:
2355 2410 magic_arg_s = self.var_expand(line, stack_depth)
2356 2411 kwargs = {}
2357 2412 if getattr(fn, "needs_local_scope", False):
2358 2413 kwargs['local_ns'] = self.user_ns
2359 2414
2360 2415 with self.builtin_trap:
2361 2416 args = (magic_arg_s, cell)
2362 2417 result = fn(*args, **kwargs)
2363 2418 return result
2364 2419
2365 2420 def find_line_magic(self, magic_name):
2366 2421 """Find and return a line magic by name.
2367 2422
2368 2423 Returns None if the magic isn't found."""
2369 2424 return self.magics_manager.magics['line'].get(magic_name)
2370 2425
2371 2426 def find_cell_magic(self, magic_name):
2372 2427 """Find and return a cell magic by name.
2373 2428
2374 2429 Returns None if the magic isn't found."""
2375 2430 return self.magics_manager.magics['cell'].get(magic_name)
2376 2431
2377 2432 def find_magic(self, magic_name, magic_kind='line'):
2378 2433 """Find and return a magic of the given type by name.
2379 2434
2380 2435 Returns None if the magic isn't found."""
2381 2436 return self.magics_manager.magics[magic_kind].get(magic_name)
2382 2437
2383 2438 def magic(self, arg_s):
2384 2439 """
2385 2440 DEPRECATED
2386 2441
2387 2442 Deprecated since IPython 0.13 (warning added in
2388 2443 8.1), use run_line_magic(magic_name, parameter_s).
2389 2444
2390 2445 Call a magic function by name.
2391 2446
2392 2447 Input: a string containing the name of the magic function to call and
2393 2448 any additional arguments to be passed to the magic.
2394 2449
2395 2450 magic('name -opt foo bar') is equivalent to typing at the ipython
2396 2451 prompt:
2397 2452
2398 2453 In[1]: %name -opt foo bar
2399 2454
2400 2455 To call a magic without arguments, simply use magic('name').
2401 2456
2402 2457 This provides a proper Python function to call IPython's magics in any
2403 2458 valid Python code you can type at the interpreter, including loops and
2404 2459 compound statements.
2405 2460 """
2406 2461 warnings.warn(
2407 2462 "`magic(...)` is deprecated since IPython 0.13 (warning added in "
2408 2463 "8.1), use run_line_magic(magic_name, parameter_s).",
2409 2464 DeprecationWarning,
2410 2465 stacklevel=2,
2411 2466 )
2412 2467 # TODO: should we issue a loud deprecation warning here?
2413 2468 magic_name, _, magic_arg_s = arg_s.partition(' ')
2414 2469 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
2415 2470 return self.run_line_magic(magic_name, magic_arg_s, _stack_depth=2)
2416 2471
2417 2472 #-------------------------------------------------------------------------
2418 2473 # Things related to macros
2419 2474 #-------------------------------------------------------------------------
2420 2475
2421 2476 def define_macro(self, name, themacro):
2422 2477 """Define a new macro
2423 2478
2424 2479 Parameters
2425 2480 ----------
2426 2481 name : str
2427 2482 The name of the macro.
2428 2483 themacro : str or Macro
2429 2484 The action to do upon invoking the macro. If a string, a new
2430 2485 Macro object is created by passing the string to it.
2431 2486 """
2432 2487
2433 2488 from IPython.core import macro
2434 2489
2435 2490 if isinstance(themacro, str):
2436 2491 themacro = macro.Macro(themacro)
2437 2492 if not isinstance(themacro, macro.Macro):
2438 2493 raise ValueError('A macro must be a string or a Macro instance.')
2439 2494 self.user_ns[name] = themacro
2440 2495
2441 2496 #-------------------------------------------------------------------------
2442 2497 # Things related to the running of system commands
2443 2498 #-------------------------------------------------------------------------
2444 2499
2445 2500 def system_piped(self, cmd):
2446 2501 """Call the given cmd in a subprocess, piping stdout/err
2447 2502
2448 2503 Parameters
2449 2504 ----------
2450 2505 cmd : str
2451 2506 Command to execute (can not end in '&', as background processes are
2452 2507 not supported. Should not be a command that expects input
2453 2508 other than simple text.
2454 2509 """
2455 2510 if cmd.rstrip().endswith('&'):
2456 2511 # this is *far* from a rigorous test
2457 2512 # We do not support backgrounding processes because we either use
2458 2513 # pexpect or pipes to read from. Users can always just call
2459 2514 # os.system() or use ip.system=ip.system_raw
2460 2515 # if they really want a background process.
2461 2516 raise OSError("Background processes not supported.")
2462 2517
2463 2518 # we explicitly do NOT return the subprocess status code, because
2464 2519 # a non-None value would trigger :func:`sys.displayhook` calls.
2465 2520 # Instead, we store the exit_code in user_ns.
2466 2521 self.user_ns['_exit_code'] = system(self.var_expand(cmd, depth=1))
2467 2522
2468 2523 def system_raw(self, cmd):
2469 2524 """Call the given cmd in a subprocess using os.system on Windows or
2470 2525 subprocess.call using the system shell on other platforms.
2471 2526
2472 2527 Parameters
2473 2528 ----------
2474 2529 cmd : str
2475 2530 Command to execute.
2476 2531 """
2477 2532 cmd = self.var_expand(cmd, depth=1)
2478 2533 # warn if there is an IPython magic alternative.
2479 2534 main_cmd = cmd.split()[0]
2480 2535 has_magic_alternatives = ("pip", "conda", "cd")
2481 2536
2482 2537 if main_cmd in has_magic_alternatives:
2483 2538 warnings.warn(
2484 2539 (
2485 2540 "You executed the system command !{0} which may not work "
2486 2541 "as expected. Try the IPython magic %{0} instead."
2487 2542 ).format(main_cmd)
2488 2543 )
2489 2544
2490 2545 # protect os.system from UNC paths on Windows, which it can't handle:
2491 2546 if sys.platform == 'win32':
2492 2547 from IPython.utils._process_win32 import AvoidUNCPath
2493 2548 with AvoidUNCPath() as path:
2494 2549 if path is not None:
2495 2550 cmd = '"pushd %s &&"%s' % (path, cmd)
2496 2551 try:
2497 2552 ec = os.system(cmd)
2498 2553 except KeyboardInterrupt:
2499 2554 print('\n' + self.get_exception_only(), file=sys.stderr)
2500 2555 ec = -2
2501 2556 else:
2502 2557 # For posix the result of the subprocess.call() below is an exit
2503 2558 # code, which by convention is zero for success, positive for
2504 2559 # program failure. Exit codes above 128 are reserved for signals,
2505 2560 # and the formula for converting a signal to an exit code is usually
2506 2561 # signal_number+128. To more easily differentiate between exit
2507 2562 # codes and signals, ipython uses negative numbers. For instance
2508 2563 # since control-c is signal 2 but exit code 130, ipython's
2509 2564 # _exit_code variable will read -2. Note that some shells like
2510 2565 # csh and fish don't follow sh/bash conventions for exit codes.
2511 2566 executable = os.environ.get('SHELL', None)
2512 2567 try:
2513 2568 # Use env shell instead of default /bin/sh
2514 2569 ec = subprocess.call(cmd, shell=True, executable=executable)
2515 2570 except KeyboardInterrupt:
2516 2571 # intercept control-C; a long traceback is not useful here
2517 2572 print('\n' + self.get_exception_only(), file=sys.stderr)
2518 2573 ec = 130
2519 2574 if ec > 128:
2520 2575 ec = -(ec - 128)
2521 2576
2522 2577 # We explicitly do NOT return the subprocess status code, because
2523 2578 # a non-None value would trigger :func:`sys.displayhook` calls.
2524 2579 # Instead, we store the exit_code in user_ns. Note the semantics
2525 2580 # of _exit_code: for control-c, _exit_code == -signal.SIGNIT,
2526 2581 # but raising SystemExit(_exit_code) will give status 254!
2527 2582 self.user_ns['_exit_code'] = ec
2528 2583
2529 2584 # use piped system by default, because it is better behaved
2530 2585 system = system_piped
2531 2586
2532 2587 def getoutput(self, cmd, split=True, depth=0):
2533 2588 """Get output (possibly including stderr) from a subprocess.
2534 2589
2535 2590 Parameters
2536 2591 ----------
2537 2592 cmd : str
2538 2593 Command to execute (can not end in '&', as background processes are
2539 2594 not supported.
2540 2595 split : bool, optional
2541 2596 If True, split the output into an IPython SList. Otherwise, an
2542 2597 IPython LSString is returned. These are objects similar to normal
2543 2598 lists and strings, with a few convenience attributes for easier
2544 2599 manipulation of line-based output. You can use '?' on them for
2545 2600 details.
2546 2601 depth : int, optional
2547 2602 How many frames above the caller are the local variables which should
2548 2603 be expanded in the command string? The default (0) assumes that the
2549 2604 expansion variables are in the stack frame calling this function.
2550 2605 """
2551 2606 if cmd.rstrip().endswith('&'):
2552 2607 # this is *far* from a rigorous test
2553 2608 raise OSError("Background processes not supported.")
2554 2609 out = getoutput(self.var_expand(cmd, depth=depth+1))
2555 2610 if split:
2556 2611 out = SList(out.splitlines())
2557 2612 else:
2558 2613 out = LSString(out)
2559 2614 return out
2560 2615
2561 2616 #-------------------------------------------------------------------------
2562 2617 # Things related to aliases
2563 2618 #-------------------------------------------------------------------------
2564 2619
2565 2620 def init_alias(self):
2566 2621 self.alias_manager = AliasManager(shell=self, parent=self)
2567 2622 self.configurables.append(self.alias_manager)
2568 2623
2569 2624 #-------------------------------------------------------------------------
2570 2625 # Things related to extensions
2571 2626 #-------------------------------------------------------------------------
2572 2627
2573 2628 def init_extension_manager(self):
2574 2629 self.extension_manager = ExtensionManager(shell=self, parent=self)
2575 2630 self.configurables.append(self.extension_manager)
2576 2631
2577 2632 #-------------------------------------------------------------------------
2578 2633 # Things related to payloads
2579 2634 #-------------------------------------------------------------------------
2580 2635
2581 2636 def init_payload(self):
2582 2637 self.payload_manager = PayloadManager(parent=self)
2583 2638 self.configurables.append(self.payload_manager)
2584 2639
2585 2640 #-------------------------------------------------------------------------
2586 2641 # Things related to the prefilter
2587 2642 #-------------------------------------------------------------------------
2588 2643
2589 2644 def init_prefilter(self):
2590 2645 self.prefilter_manager = PrefilterManager(shell=self, parent=self)
2591 2646 self.configurables.append(self.prefilter_manager)
2592 2647 # Ultimately this will be refactored in the new interpreter code, but
2593 2648 # for now, we should expose the main prefilter method (there's legacy
2594 2649 # code out there that may rely on this).
2595 2650 self.prefilter = self.prefilter_manager.prefilter_lines
2596 2651
2597 2652 def auto_rewrite_input(self, cmd):
2598 2653 """Print to the screen the rewritten form of the user's command.
2599 2654
2600 2655 This shows visual feedback by rewriting input lines that cause
2601 2656 automatic calling to kick in, like::
2602 2657
2603 2658 /f x
2604 2659
2605 2660 into::
2606 2661
2607 2662 ------> f(x)
2608 2663
2609 2664 after the user's input prompt. This helps the user understand that the
2610 2665 input line was transformed automatically by IPython.
2611 2666 """
2612 2667 if not self.show_rewritten_input:
2613 2668 return
2614 2669
2615 2670 # This is overridden in TerminalInteractiveShell to use fancy prompts
2616 2671 print("------> " + cmd)
2617 2672
2618 2673 #-------------------------------------------------------------------------
2619 2674 # Things related to extracting values/expressions from kernel and user_ns
2620 2675 #-------------------------------------------------------------------------
2621 2676
2622 2677 def _user_obj_error(self):
2623 2678 """return simple exception dict
2624 2679
2625 2680 for use in user_expressions
2626 2681 """
2627 2682
2628 2683 etype, evalue, tb = self._get_exc_info()
2629 2684 stb = self.InteractiveTB.get_exception_only(etype, evalue)
2630 2685
2631 2686 exc_info = {
2632 2687 "status": "error",
2633 2688 "traceback": stb,
2634 2689 "ename": etype.__name__,
2635 2690 "evalue": py3compat.safe_unicode(evalue),
2636 2691 }
2637 2692
2638 2693 return exc_info
2639 2694
2640 2695 def _format_user_obj(self, obj):
2641 2696 """format a user object to display dict
2642 2697
2643 2698 for use in user_expressions
2644 2699 """
2645 2700
2646 2701 data, md = self.display_formatter.format(obj)
2647 2702 value = {
2648 2703 'status' : 'ok',
2649 2704 'data' : data,
2650 2705 'metadata' : md,
2651 2706 }
2652 2707 return value
2653 2708
2654 2709 def user_expressions(self, expressions):
2655 2710 """Evaluate a dict of expressions in the user's namespace.
2656 2711
2657 2712 Parameters
2658 2713 ----------
2659 2714 expressions : dict
2660 2715 A dict with string keys and string values. The expression values
2661 2716 should be valid Python expressions, each of which will be evaluated
2662 2717 in the user namespace.
2663 2718
2664 2719 Returns
2665 2720 -------
2666 2721 A dict, keyed like the input expressions dict, with the rich mime-typed
2667 2722 display_data of each value.
2668 2723 """
2669 2724 out = {}
2670 2725 user_ns = self.user_ns
2671 2726 global_ns = self.user_global_ns
2672 2727
2673 2728 for key, expr in expressions.items():
2674 2729 try:
2675 2730 value = self._format_user_obj(eval(expr, global_ns, user_ns))
2676 2731 except:
2677 2732 value = self._user_obj_error()
2678 2733 out[key] = value
2679 2734 return out
2680 2735
2681 2736 #-------------------------------------------------------------------------
2682 2737 # Things related to the running of code
2683 2738 #-------------------------------------------------------------------------
2684 2739
2685 2740 def ex(self, cmd):
2686 2741 """Execute a normal python statement in user namespace."""
2687 2742 with self.builtin_trap:
2688 2743 exec(cmd, self.user_global_ns, self.user_ns)
2689 2744
2690 2745 def ev(self, expr):
2691 2746 """Evaluate python expression expr in user namespace.
2692 2747
2693 2748 Returns the result of evaluation
2694 2749 """
2695 2750 with self.builtin_trap:
2696 2751 return eval(expr, self.user_global_ns, self.user_ns)
2697 2752
2698 2753 def safe_execfile(self, fname, *where, exit_ignore=False, raise_exceptions=False, shell_futures=False):
2699 2754 """A safe version of the builtin execfile().
2700 2755
2701 2756 This version will never throw an exception, but instead print
2702 2757 helpful error messages to the screen. This only works on pure
2703 2758 Python files with the .py extension.
2704 2759
2705 2760 Parameters
2706 2761 ----------
2707 2762 fname : string
2708 2763 The name of the file to be executed.
2709 2764 *where : tuple
2710 2765 One or two namespaces, passed to execfile() as (globals,locals).
2711 2766 If only one is given, it is passed as both.
2712 2767 exit_ignore : bool (False)
2713 2768 If True, then silence SystemExit for non-zero status (it is always
2714 2769 silenced for zero status, as it is so common).
2715 2770 raise_exceptions : bool (False)
2716 2771 If True raise exceptions everywhere. Meant for testing.
2717 2772 shell_futures : bool (False)
2718 2773 If True, the code will share future statements with the interactive
2719 2774 shell. It will both be affected by previous __future__ imports, and
2720 2775 any __future__ imports in the code will affect the shell. If False,
2721 2776 __future__ imports are not shared in either direction.
2722 2777
2723 2778 """
2724 2779 fname = Path(fname).expanduser().resolve()
2725 2780
2726 2781 # Make sure we can open the file
2727 2782 try:
2728 2783 with fname.open("rb"):
2729 2784 pass
2730 2785 except:
2731 2786 warn('Could not open file <%s> for safe execution.' % fname)
2732 2787 return
2733 2788
2734 2789 # Find things also in current directory. This is needed to mimic the
2735 2790 # behavior of running a script from the system command line, where
2736 2791 # Python inserts the script's directory into sys.path
2737 2792 dname = str(fname.parent)
2738 2793
2739 2794 with prepended_to_syspath(dname), self.builtin_trap:
2740 2795 try:
2741 2796 glob, loc = (where + (None, ))[:2]
2742 2797 py3compat.execfile(
2743 2798 fname, glob, loc,
2744 2799 self.compile if shell_futures else None)
2745 2800 except SystemExit as status:
2746 2801 # If the call was made with 0 or None exit status (sys.exit(0)
2747 2802 # or sys.exit() ), don't bother showing a traceback, as both of
2748 2803 # these are considered normal by the OS:
2749 2804 # > python -c'import sys;sys.exit(0)'; echo $?
2750 2805 # 0
2751 2806 # > python -c'import sys;sys.exit()'; echo $?
2752 2807 # 0
2753 2808 # For other exit status, we show the exception unless
2754 2809 # explicitly silenced, but only in short form.
2755 2810 if status.code:
2756 2811 if raise_exceptions:
2757 2812 raise
2758 2813 if not exit_ignore:
2759 2814 self.showtraceback(exception_only=True)
2760 2815 except:
2761 2816 if raise_exceptions:
2762 2817 raise
2763 2818 # tb offset is 2 because we wrap execfile
2764 2819 self.showtraceback(tb_offset=2)
2765 2820
2766 2821 def safe_execfile_ipy(self, fname, shell_futures=False, raise_exceptions=False):
2767 2822 """Like safe_execfile, but for .ipy or .ipynb files with IPython syntax.
2768 2823
2769 2824 Parameters
2770 2825 ----------
2771 2826 fname : str
2772 2827 The name of the file to execute. The filename must have a
2773 2828 .ipy or .ipynb extension.
2774 2829 shell_futures : bool (False)
2775 2830 If True, the code will share future statements with the interactive
2776 2831 shell. It will both be affected by previous __future__ imports, and
2777 2832 any __future__ imports in the code will affect the shell. If False,
2778 2833 __future__ imports are not shared in either direction.
2779 2834 raise_exceptions : bool (False)
2780 2835 If True raise exceptions everywhere. Meant for testing.
2781 2836 """
2782 2837 fname = Path(fname).expanduser().resolve()
2783 2838
2784 2839 # Make sure we can open the file
2785 2840 try:
2786 2841 with fname.open("rb"):
2787 2842 pass
2788 2843 except:
2789 2844 warn('Could not open file <%s> for safe execution.' % fname)
2790 2845 return
2791 2846
2792 2847 # Find things also in current directory. This is needed to mimic the
2793 2848 # behavior of running a script from the system command line, where
2794 2849 # Python inserts the script's directory into sys.path
2795 2850 dname = str(fname.parent)
2796 2851
2797 2852 def get_cells():
2798 2853 """generator for sequence of code blocks to run"""
2799 2854 if fname.suffix == ".ipynb":
2800 2855 from nbformat import read
2801 2856 nb = read(fname, as_version=4)
2802 2857 if not nb.cells:
2803 2858 return
2804 2859 for cell in nb.cells:
2805 2860 if cell.cell_type == 'code':
2806 2861 yield cell.source
2807 2862 else:
2808 2863 yield fname.read_text(encoding="utf-8")
2809 2864
2810 2865 with prepended_to_syspath(dname):
2811 2866 try:
2812 2867 for cell in get_cells():
2813 2868 result = self.run_cell(cell, silent=True, shell_futures=shell_futures)
2814 2869 if raise_exceptions:
2815 2870 result.raise_error()
2816 2871 elif not result.success:
2817 2872 break
2818 2873 except:
2819 2874 if raise_exceptions:
2820 2875 raise
2821 2876 self.showtraceback()
2822 2877 warn('Unknown failure executing file: <%s>' % fname)
2823 2878
2824 2879 def safe_run_module(self, mod_name, where):
2825 2880 """A safe version of runpy.run_module().
2826 2881
2827 2882 This version will never throw an exception, but instead print
2828 2883 helpful error messages to the screen.
2829 2884
2830 2885 `SystemExit` exceptions with status code 0 or None are ignored.
2831 2886
2832 2887 Parameters
2833 2888 ----------
2834 2889 mod_name : string
2835 2890 The name of the module to be executed.
2836 2891 where : dict
2837 2892 The globals namespace.
2838 2893 """
2839 2894 try:
2840 2895 try:
2841 2896 where.update(
2842 2897 runpy.run_module(str(mod_name), run_name="__main__",
2843 2898 alter_sys=True)
2844 2899 )
2845 2900 except SystemExit as status:
2846 2901 if status.code:
2847 2902 raise
2848 2903 except:
2849 2904 self.showtraceback()
2850 2905 warn('Unknown failure executing module: <%s>' % mod_name)
2851 2906
2852 2907 def run_cell(
2853 2908 self,
2854 2909 raw_cell,
2855 2910 store_history=False,
2856 2911 silent=False,
2857 2912 shell_futures=True,
2858 2913 cell_id=None,
2859 2914 ):
2860 2915 """Run a complete IPython cell.
2861 2916
2862 2917 Parameters
2863 2918 ----------
2864 2919 raw_cell : str
2865 2920 The code (including IPython code such as %magic functions) to run.
2866 2921 store_history : bool
2867 2922 If True, the raw and translated cell will be stored in IPython's
2868 2923 history. For user code calling back into IPython's machinery, this
2869 2924 should be set to False.
2870 2925 silent : bool
2871 2926 If True, avoid side-effects, such as implicit displayhooks and
2872 2927 and logging. silent=True forces store_history=False.
2873 2928 shell_futures : bool
2874 2929 If True, the code will share future statements with the interactive
2875 2930 shell. It will both be affected by previous __future__ imports, and
2876 2931 any __future__ imports in the code will affect the shell. If False,
2877 2932 __future__ imports are not shared in either direction.
2878 2933
2879 2934 Returns
2880 2935 -------
2881 2936 result : :class:`ExecutionResult`
2882 2937 """
2883 2938 result = None
2884 2939 try:
2885 2940 result = self._run_cell(
2886 2941 raw_cell, store_history, silent, shell_futures, cell_id
2887 2942 )
2888 2943 finally:
2889 2944 self.events.trigger('post_execute')
2890 2945 if not silent:
2891 2946 self.events.trigger('post_run_cell', result)
2892 2947 return result
2893 2948
2894 2949 def _run_cell(
2895 2950 self,
2896 2951 raw_cell: str,
2897 2952 store_history: bool,
2898 2953 silent: bool,
2899 2954 shell_futures: bool,
2900 2955 cell_id: str,
2901 2956 ) -> ExecutionResult:
2902 2957 """Internal method to run a complete IPython cell."""
2903 2958
2904 2959 # we need to avoid calling self.transform_cell multiple time on the same thing
2905 2960 # so we need to store some results:
2906 2961 preprocessing_exc_tuple = None
2907 2962 try:
2908 2963 transformed_cell = self.transform_cell(raw_cell)
2909 2964 except Exception:
2910 2965 transformed_cell = raw_cell
2911 2966 preprocessing_exc_tuple = sys.exc_info()
2912 2967
2913 2968 assert transformed_cell is not None
2914 2969 coro = self.run_cell_async(
2915 2970 raw_cell,
2916 2971 store_history=store_history,
2917 2972 silent=silent,
2918 2973 shell_futures=shell_futures,
2919 2974 transformed_cell=transformed_cell,
2920 2975 preprocessing_exc_tuple=preprocessing_exc_tuple,
2921 2976 cell_id=cell_id,
2922 2977 )
2923 2978
2924 2979 # run_cell_async is async, but may not actually need an eventloop.
2925 2980 # when this is the case, we want to run it using the pseudo_sync_runner
2926 2981 # so that code can invoke eventloops (for example via the %run , and
2927 2982 # `%paste` magic.
2928 2983 if self.trio_runner:
2929 2984 runner = self.trio_runner
2930 2985 elif self.should_run_async(
2931 2986 raw_cell,
2932 2987 transformed_cell=transformed_cell,
2933 2988 preprocessing_exc_tuple=preprocessing_exc_tuple,
2934 2989 ):
2935 2990 runner = self.loop_runner
2936 2991 else:
2937 2992 runner = _pseudo_sync_runner
2938 2993
2939 2994 try:
2940 2995 return runner(coro)
2941 2996 except BaseException as e:
2942 2997 info = ExecutionInfo(
2943 2998 raw_cell, store_history, silent, shell_futures, cell_id
2944 2999 )
2945 3000 result = ExecutionResult(info)
2946 3001 result.error_in_exec = e
2947 3002 self.showtraceback(running_compiled_code=True)
2948 3003 return result
2949 3004
2950 3005 def should_run_async(
2951 3006 self, raw_cell: str, *, transformed_cell=None, preprocessing_exc_tuple=None
2952 3007 ) -> bool:
2953 3008 """Return whether a cell should be run asynchronously via a coroutine runner
2954 3009
2955 3010 Parameters
2956 3011 ----------
2957 3012 raw_cell : str
2958 3013 The code to be executed
2959 3014
2960 3015 Returns
2961 3016 -------
2962 3017 result: bool
2963 3018 Whether the code needs to be run with a coroutine runner or not
2964 3019 .. versionadded:: 7.0
2965 3020 """
2966 3021 if not self.autoawait:
2967 3022 return False
2968 3023 if preprocessing_exc_tuple is not None:
2969 3024 return False
2970 3025 assert preprocessing_exc_tuple is None
2971 3026 if transformed_cell is None:
2972 3027 warnings.warn(
2973 3028 "`should_run_async` will not call `transform_cell`"
2974 3029 " automatically in the future. Please pass the result to"
2975 3030 " `transformed_cell` argument and any exception that happen"
2976 3031 " during the"
2977 3032 "transform in `preprocessing_exc_tuple` in"
2978 3033 " IPython 7.17 and above.",
2979 3034 DeprecationWarning,
2980 3035 stacklevel=2,
2981 3036 )
2982 3037 try:
2983 3038 cell = self.transform_cell(raw_cell)
2984 3039 except Exception:
2985 3040 # any exception during transform will be raised
2986 3041 # prior to execution
2987 3042 return False
2988 3043 else:
2989 3044 cell = transformed_cell
2990 3045 return _should_be_async(cell)
2991 3046
2992 3047 async def run_cell_async(
2993 3048 self,
2994 3049 raw_cell: str,
2995 3050 store_history=False,
2996 3051 silent=False,
2997 3052 shell_futures=True,
2998 3053 *,
2999 3054 transformed_cell: Optional[str] = None,
3000 3055 preprocessing_exc_tuple: Optional[Any] = None,
3001 3056 cell_id=None,
3002 3057 ) -> ExecutionResult:
3003 3058 """Run a complete IPython cell asynchronously.
3004 3059
3005 3060 Parameters
3006 3061 ----------
3007 3062 raw_cell : str
3008 3063 The code (including IPython code such as %magic functions) to run.
3009 3064 store_history : bool
3010 3065 If True, the raw and translated cell will be stored in IPython's
3011 3066 history. For user code calling back into IPython's machinery, this
3012 3067 should be set to False.
3013 3068 silent : bool
3014 3069 If True, avoid side-effects, such as implicit displayhooks and
3015 3070 and logging. silent=True forces store_history=False.
3016 3071 shell_futures : bool
3017 3072 If True, the code will share future statements with the interactive
3018 3073 shell. It will both be affected by previous __future__ imports, and
3019 3074 any __future__ imports in the code will affect the shell. If False,
3020 3075 __future__ imports are not shared in either direction.
3021 3076 transformed_cell: str
3022 3077 cell that was passed through transformers
3023 3078 preprocessing_exc_tuple:
3024 3079 trace if the transformation failed.
3025 3080
3026 3081 Returns
3027 3082 -------
3028 3083 result : :class:`ExecutionResult`
3029 3084
3030 3085 .. versionadded:: 7.0
3031 3086 """
3032 3087 info = ExecutionInfo(raw_cell, store_history, silent, shell_futures, cell_id)
3033 3088 result = ExecutionResult(info)
3034 3089
3035 3090 if (not raw_cell) or raw_cell.isspace():
3036 3091 self.last_execution_succeeded = True
3037 3092 self.last_execution_result = result
3038 3093 return result
3039 3094
3040 3095 if silent:
3041 3096 store_history = False
3042 3097
3043 3098 if store_history:
3044 3099 result.execution_count = self.execution_count
3045 3100
3046 3101 def error_before_exec(value):
3047 3102 if store_history:
3048 3103 self.execution_count += 1
3049 3104 result.error_before_exec = value
3050 3105 self.last_execution_succeeded = False
3051 3106 self.last_execution_result = result
3052 3107 return result
3053 3108
3054 3109 self.events.trigger('pre_execute')
3055 3110 if not silent:
3056 3111 self.events.trigger('pre_run_cell', info)
3057 3112
3058 3113 if transformed_cell is None:
3059 3114 warnings.warn(
3060 3115 "`run_cell_async` will not call `transform_cell`"
3061 3116 " automatically in the future. Please pass the result to"
3062 3117 " `transformed_cell` argument and any exception that happen"
3063 3118 " during the"
3064 3119 "transform in `preprocessing_exc_tuple` in"
3065 3120 " IPython 7.17 and above.",
3066 3121 DeprecationWarning,
3067 3122 stacklevel=2,
3068 3123 )
3069 3124 # If any of our input transformation (input_transformer_manager or
3070 3125 # prefilter_manager) raises an exception, we store it in this variable
3071 3126 # so that we can display the error after logging the input and storing
3072 3127 # it in the history.
3073 3128 try:
3074 3129 cell = self.transform_cell(raw_cell)
3075 3130 except Exception:
3076 3131 preprocessing_exc_tuple = sys.exc_info()
3077 3132 cell = raw_cell # cell has to exist so it can be stored/logged
3078 3133 else:
3079 3134 preprocessing_exc_tuple = None
3080 3135 else:
3081 3136 if preprocessing_exc_tuple is None:
3082 3137 cell = transformed_cell
3083 3138 else:
3084 3139 cell = raw_cell
3085 3140
3086 3141 # Store raw and processed history
3087 3142 if store_history and raw_cell.strip(" %") != "paste":
3088 3143 self.history_manager.store_inputs(self.execution_count, cell, raw_cell)
3089 3144 if not silent:
3090 3145 self.logger.log(cell, raw_cell)
3091 3146
3092 3147 # Display the exception if input processing failed.
3093 3148 if preprocessing_exc_tuple is not None:
3094 3149 self.showtraceback(preprocessing_exc_tuple)
3095 3150 if store_history:
3096 3151 self.execution_count += 1
3097 3152 return error_before_exec(preprocessing_exc_tuple[1])
3098 3153
3099 3154 # Our own compiler remembers the __future__ environment. If we want to
3100 3155 # run code with a separate __future__ environment, use the default
3101 3156 # compiler
3102 3157 compiler = self.compile if shell_futures else self.compiler_class()
3103 3158
3104 3159 _run_async = False
3105 3160
3106 3161 with self.builtin_trap:
3107 3162 cell_name = compiler.cache(cell, self.execution_count, raw_code=raw_cell)
3108 3163
3109 3164 with self.display_trap:
3110 3165 # Compile to bytecode
3111 3166 try:
3112 3167 code_ast = compiler.ast_parse(cell, filename=cell_name)
3113 3168 except self.custom_exceptions as e:
3114 3169 etype, value, tb = sys.exc_info()
3115 3170 self.CustomTB(etype, value, tb)
3116 3171 return error_before_exec(e)
3117 3172 except IndentationError as e:
3118 3173 self.showindentationerror()
3119 3174 return error_before_exec(e)
3120 3175 except (OverflowError, SyntaxError, ValueError, TypeError,
3121 3176 MemoryError) as e:
3122 3177 self.showsyntaxerror()
3123 3178 return error_before_exec(e)
3124 3179
3125 3180 # Apply AST transformations
3126 3181 try:
3127 3182 code_ast = self.transform_ast(code_ast)
3128 3183 except InputRejected as e:
3129 3184 self.showtraceback()
3130 3185 return error_before_exec(e)
3131 3186
3132 3187 # Give the displayhook a reference to our ExecutionResult so it
3133 3188 # can fill in the output value.
3134 3189 self.displayhook.exec_result = result
3135 3190
3136 3191 # Execute the user code
3137 3192 interactivity = "none" if silent else self.ast_node_interactivity
3138 3193
3139 3194 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
3140 3195 interactivity=interactivity, compiler=compiler, result=result)
3141 3196
3142 3197 self.last_execution_succeeded = not has_raised
3143 3198 self.last_execution_result = result
3144 3199
3145 3200 # Reset this so later displayed values do not modify the
3146 3201 # ExecutionResult
3147 3202 self.displayhook.exec_result = None
3148 3203
3149 3204 if store_history:
3150 3205 # Write output to the database. Does nothing unless
3151 3206 # history output logging is enabled.
3152 3207 self.history_manager.store_output(self.execution_count)
3153 3208 # Each cell is a *single* input, regardless of how many lines it has
3154 3209 self.execution_count += 1
3155 3210
3156 3211 return result
3157 3212
3158 3213 def transform_cell(self, raw_cell):
3159 3214 """Transform an input cell before parsing it.
3160 3215
3161 3216 Static transformations, implemented in IPython.core.inputtransformer2,
3162 3217 deal with things like ``%magic`` and ``!system`` commands.
3163 3218 These run on all input.
3164 3219 Dynamic transformations, for things like unescaped magics and the exit
3165 3220 autocall, depend on the state of the interpreter.
3166 3221 These only apply to single line inputs.
3167 3222
3168 3223 These string-based transformations are followed by AST transformations;
3169 3224 see :meth:`transform_ast`.
3170 3225 """
3171 3226 # Static input transformations
3172 3227 cell = self.input_transformer_manager.transform_cell(raw_cell)
3173 3228
3174 3229 if len(cell.splitlines()) == 1:
3175 3230 # Dynamic transformations - only applied for single line commands
3176 3231 with self.builtin_trap:
3177 3232 # use prefilter_lines to handle trailing newlines
3178 3233 # restore trailing newline for ast.parse
3179 3234 cell = self.prefilter_manager.prefilter_lines(cell) + '\n'
3180 3235
3181 3236 lines = cell.splitlines(keepends=True)
3182 3237 for transform in self.input_transformers_post:
3183 3238 lines = transform(lines)
3184 3239 cell = ''.join(lines)
3185 3240
3186 3241 return cell
3187 3242
3188 3243 def transform_ast(self, node):
3189 3244 """Apply the AST transformations from self.ast_transformers
3190 3245
3191 3246 Parameters
3192 3247 ----------
3193 3248 node : ast.Node
3194 3249 The root node to be transformed. Typically called with the ast.Module
3195 3250 produced by parsing user input.
3196 3251
3197 3252 Returns
3198 3253 -------
3199 3254 An ast.Node corresponding to the node it was called with. Note that it
3200 3255 may also modify the passed object, so don't rely on references to the
3201 3256 original AST.
3202 3257 """
3203 3258 for transformer in self.ast_transformers:
3204 3259 try:
3205 3260 node = transformer.visit(node)
3206 3261 except InputRejected:
3207 3262 # User-supplied AST transformers can reject an input by raising
3208 3263 # an InputRejected. Short-circuit in this case so that we
3209 3264 # don't unregister the transform.
3210 3265 raise
3211 3266 except Exception:
3212 3267 warn("AST transformer %r threw an error. It will be unregistered." % transformer)
3213 3268 self.ast_transformers.remove(transformer)
3214 3269
3215 3270 if self.ast_transformers:
3216 3271 ast.fix_missing_locations(node)
3217 3272 return node
3218 3273
3219 3274 async def run_ast_nodes(
3220 3275 self,
3221 3276 nodelist: ListType[stmt],
3222 3277 cell_name: str,
3223 3278 interactivity="last_expr",
3224 3279 compiler=compile,
3225 3280 result=None,
3226 3281 ):
3227 3282 """Run a sequence of AST nodes. The execution mode depends on the
3228 3283 interactivity parameter.
3229 3284
3230 3285 Parameters
3231 3286 ----------
3232 3287 nodelist : list
3233 3288 A sequence of AST nodes to run.
3234 3289 cell_name : str
3235 3290 Will be passed to the compiler as the filename of the cell. Typically
3236 3291 the value returned by ip.compile.cache(cell).
3237 3292 interactivity : str
3238 3293 'all', 'last', 'last_expr' , 'last_expr_or_assign' or 'none',
3239 3294 specifying which nodes should be run interactively (displaying output
3240 3295 from expressions). 'last_expr' will run the last node interactively
3241 3296 only if it is an expression (i.e. expressions in loops or other blocks
3242 3297 are not displayed) 'last_expr_or_assign' will run the last expression
3243 3298 or the last assignment. Other values for this parameter will raise a
3244 3299 ValueError.
3245 3300
3246 3301 compiler : callable
3247 3302 A function with the same interface as the built-in compile(), to turn
3248 3303 the AST nodes into code objects. Default is the built-in compile().
3249 3304 result : ExecutionResult, optional
3250 3305 An object to store exceptions that occur during execution.
3251 3306
3252 3307 Returns
3253 3308 -------
3254 3309 True if an exception occurred while running code, False if it finished
3255 3310 running.
3256 3311 """
3257 3312 if not nodelist:
3258 3313 return
3259 3314
3260 3315
3261 3316 if interactivity == 'last_expr_or_assign':
3262 3317 if isinstance(nodelist[-1], _assign_nodes):
3263 3318 asg = nodelist[-1]
3264 3319 if isinstance(asg, ast.Assign) and len(asg.targets) == 1:
3265 3320 target = asg.targets[0]
3266 3321 elif isinstance(asg, _single_targets_nodes):
3267 3322 target = asg.target
3268 3323 else:
3269 3324 target = None
3270 3325 if isinstance(target, ast.Name):
3271 3326 nnode = ast.Expr(ast.Name(target.id, ast.Load()))
3272 3327 ast.fix_missing_locations(nnode)
3273 3328 nodelist.append(nnode)
3274 3329 interactivity = 'last_expr'
3275 3330
3276 3331 _async = False
3277 3332 if interactivity == 'last_expr':
3278 3333 if isinstance(nodelist[-1], ast.Expr):
3279 3334 interactivity = "last"
3280 3335 else:
3281 3336 interactivity = "none"
3282 3337
3283 3338 if interactivity == 'none':
3284 3339 to_run_exec, to_run_interactive = nodelist, []
3285 3340 elif interactivity == 'last':
3286 3341 to_run_exec, to_run_interactive = nodelist[:-1], nodelist[-1:]
3287 3342 elif interactivity == 'all':
3288 3343 to_run_exec, to_run_interactive = [], nodelist
3289 3344 else:
3290 3345 raise ValueError("Interactivity was %r" % interactivity)
3291 3346
3292 3347 try:
3293 3348
3294 3349 def compare(code):
3295 3350 is_async = inspect.CO_COROUTINE & code.co_flags == inspect.CO_COROUTINE
3296 3351 return is_async
3297 3352
3298 3353 # refactor that to just change the mod constructor.
3299 3354 to_run = []
3300 3355 for node in to_run_exec:
3301 3356 to_run.append((node, "exec"))
3302 3357
3303 3358 for node in to_run_interactive:
3304 3359 to_run.append((node, "single"))
3305 3360
3306 3361 for node, mode in to_run:
3307 3362 if mode == "exec":
3308 3363 mod = Module([node], [])
3309 3364 elif mode == "single":
3310 3365 mod = ast.Interactive([node])
3311 3366 with compiler.extra_flags(
3312 3367 getattr(ast, "PyCF_ALLOW_TOP_LEVEL_AWAIT", 0x0)
3313 3368 if self.autoawait
3314 3369 else 0x0
3315 3370 ):
3316 3371 code = compiler(mod, cell_name, mode)
3317 3372 asy = compare(code)
3318 3373 if await self.run_code(code, result, async_=asy):
3319 3374 return True
3320 3375
3321 3376 # Flush softspace
3322 3377 if softspace(sys.stdout, 0):
3323 3378 print()
3324 3379
3325 3380 except:
3326 3381 # It's possible to have exceptions raised here, typically by
3327 3382 # compilation of odd code (such as a naked 'return' outside a
3328 3383 # function) that did parse but isn't valid. Typically the exception
3329 3384 # is a SyntaxError, but it's safest just to catch anything and show
3330 3385 # the user a traceback.
3331 3386
3332 3387 # We do only one try/except outside the loop to minimize the impact
3333 3388 # on runtime, and also because if any node in the node list is
3334 3389 # broken, we should stop execution completely.
3335 3390 if result:
3336 3391 result.error_before_exec = sys.exc_info()[1]
3337 3392 self.showtraceback()
3338 3393 return True
3339 3394
3340 3395 return False
3341 3396
3342 3397 async def run_code(self, code_obj, result=None, *, async_=False):
3343 3398 """Execute a code object.
3344 3399
3345 3400 When an exception occurs, self.showtraceback() is called to display a
3346 3401 traceback.
3347 3402
3348 3403 Parameters
3349 3404 ----------
3350 3405 code_obj : code object
3351 3406 A compiled code object, to be executed
3352 3407 result : ExecutionResult, optional
3353 3408 An object to store exceptions that occur during execution.
3354 3409 async_ : Bool (Experimental)
3355 3410 Attempt to run top-level asynchronous code in a default loop.
3356 3411
3357 3412 Returns
3358 3413 -------
3359 3414 False : successful execution.
3360 3415 True : an error occurred.
3361 3416 """
3362 3417 # special value to say that anything above is IPython and should be
3363 3418 # hidden.
3364 3419 __tracebackhide__ = "__ipython_bottom__"
3365 3420 # Set our own excepthook in case the user code tries to call it
3366 3421 # directly, so that the IPython crash handler doesn't get triggered
3367 3422 old_excepthook, sys.excepthook = sys.excepthook, self.excepthook
3368 3423
3369 3424 # we save the original sys.excepthook in the instance, in case config
3370 3425 # code (such as magics) needs access to it.
3371 3426 self.sys_excepthook = old_excepthook
3372 3427 outflag = True # happens in more places, so it's easier as default
3373 3428 try:
3374 3429 try:
3375 3430 if async_:
3376 3431 await eval(code_obj, self.user_global_ns, self.user_ns)
3377 3432 else:
3378 3433 exec(code_obj, self.user_global_ns, self.user_ns)
3379 3434 finally:
3380 3435 # Reset our crash handler in place
3381 3436 sys.excepthook = old_excepthook
3382 3437 except SystemExit as e:
3383 3438 if result is not None:
3384 3439 result.error_in_exec = e
3385 3440 self.showtraceback(exception_only=True)
3386 3441 warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)
3387 3442 except bdb.BdbQuit:
3388 3443 etype, value, tb = sys.exc_info()
3389 3444 if result is not None:
3390 3445 result.error_in_exec = value
3391 3446 # the BdbQuit stops here
3392 3447 except self.custom_exceptions:
3393 3448 etype, value, tb = sys.exc_info()
3394 3449 if result is not None:
3395 3450 result.error_in_exec = value
3396 3451 self.CustomTB(etype, value, tb)
3397 3452 except:
3398 3453 if result is not None:
3399 3454 result.error_in_exec = sys.exc_info()[1]
3400 3455 self.showtraceback(running_compiled_code=True)
3401 3456 else:
3402 3457 outflag = False
3403 3458 return outflag
3404 3459
3405 3460 # For backwards compatibility
3406 3461 runcode = run_code
3407 3462
3408 3463 def check_complete(self, code: str) -> Tuple[str, str]:
3409 3464 """Return whether a block of code is ready to execute, or should be continued
3410 3465
3411 3466 Parameters
3412 3467 ----------
3413 3468 code : string
3414 3469 Python input code, which can be multiline.
3415 3470
3416 3471 Returns
3417 3472 -------
3418 3473 status : str
3419 3474 One of 'complete', 'incomplete', or 'invalid' if source is not a
3420 3475 prefix of valid code.
3421 3476 indent : str
3422 3477 When status is 'incomplete', this is some whitespace to insert on
3423 3478 the next line of the prompt.
3424 3479 """
3425 3480 status, nspaces = self.input_transformer_manager.check_complete(code)
3426 3481 return status, ' ' * (nspaces or 0)
3427 3482
3428 3483 #-------------------------------------------------------------------------
3429 3484 # Things related to GUI support and pylab
3430 3485 #-------------------------------------------------------------------------
3431 3486
3432 3487 active_eventloop = None
3433 3488
3434 3489 def enable_gui(self, gui=None):
3435 3490 raise NotImplementedError('Implement enable_gui in a subclass')
3436 3491
3437 3492 def enable_matplotlib(self, gui=None):
3438 3493 """Enable interactive matplotlib and inline figure support.
3439 3494
3440 3495 This takes the following steps:
3441 3496
3442 3497 1. select the appropriate eventloop and matplotlib backend
3443 3498 2. set up matplotlib for interactive use with that backend
3444 3499 3. configure formatters for inline figure display
3445 3500 4. enable the selected gui eventloop
3446 3501
3447 3502 Parameters
3448 3503 ----------
3449 3504 gui : optional, string
3450 3505 If given, dictates the choice of matplotlib GUI backend to use
3451 3506 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
3452 3507 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
3453 3508 matplotlib (as dictated by the matplotlib build-time options plus the
3454 3509 user's matplotlibrc configuration file). Note that not all backends
3455 3510 make sense in all contexts, for example a terminal ipython can't
3456 3511 display figures inline.
3457 3512 """
3458 3513 from matplotlib_inline.backend_inline import configure_inline_support
3459 3514
3460 3515 from IPython.core import pylabtools as pt
3461 3516 gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
3462 3517
3463 3518 if gui != 'inline':
3464 3519 # If we have our first gui selection, store it
3465 3520 if self.pylab_gui_select is None:
3466 3521 self.pylab_gui_select = gui
3467 3522 # Otherwise if they are different
3468 3523 elif gui != self.pylab_gui_select:
3469 3524 print('Warning: Cannot change to a different GUI toolkit: %s.'
3470 3525 ' Using %s instead.' % (gui, self.pylab_gui_select))
3471 3526 gui, backend = pt.find_gui_and_backend(self.pylab_gui_select)
3472 3527
3473 3528 pt.activate_matplotlib(backend)
3474 3529 configure_inline_support(self, backend)
3475 3530
3476 3531 # Now we must activate the gui pylab wants to use, and fix %run to take
3477 3532 # plot updates into account
3478 3533 self.enable_gui(gui)
3479 3534 self.magics_manager.registry['ExecutionMagics'].default_runner = \
3480 3535 pt.mpl_runner(self.safe_execfile)
3481 3536
3482 3537 return gui, backend
3483 3538
3484 3539 def enable_pylab(self, gui=None, import_all=True, welcome_message=False):
3485 3540 """Activate pylab support at runtime.
3486 3541
3487 3542 This turns on support for matplotlib, preloads into the interactive
3488 3543 namespace all of numpy and pylab, and configures IPython to correctly
3489 3544 interact with the GUI event loop. The GUI backend to be used can be
3490 3545 optionally selected with the optional ``gui`` argument.
3491 3546
3492 3547 This method only adds preloading the namespace to InteractiveShell.enable_matplotlib.
3493 3548
3494 3549 Parameters
3495 3550 ----------
3496 3551 gui : optional, string
3497 3552 If given, dictates the choice of matplotlib GUI backend to use
3498 3553 (should be one of IPython's supported backends, 'qt', 'osx', 'tk',
3499 3554 'gtk', 'wx' or 'inline'), otherwise we use the default chosen by
3500 3555 matplotlib (as dictated by the matplotlib build-time options plus the
3501 3556 user's matplotlibrc configuration file). Note that not all backends
3502 3557 make sense in all contexts, for example a terminal ipython can't
3503 3558 display figures inline.
3504 3559 import_all : optional, bool, default: True
3505 3560 Whether to do `from numpy import *` and `from pylab import *`
3506 3561 in addition to module imports.
3507 3562 welcome_message : deprecated
3508 3563 This argument is ignored, no welcome message will be displayed.
3509 3564 """
3510 3565 from IPython.core.pylabtools import import_pylab
3511 3566
3512 3567 gui, backend = self.enable_matplotlib(gui)
3513 3568
3514 3569 # We want to prevent the loading of pylab to pollute the user's
3515 3570 # namespace as shown by the %who* magics, so we execute the activation
3516 3571 # code in an empty namespace, and we update *both* user_ns and
3517 3572 # user_ns_hidden with this information.
3518 3573 ns = {}
3519 3574 import_pylab(ns, import_all)
3520 3575 # warn about clobbered names
3521 3576 ignored = {"__builtins__"}
3522 3577 both = set(ns).intersection(self.user_ns).difference(ignored)
3523 3578 clobbered = [ name for name in both if self.user_ns[name] is not ns[name] ]
3524 3579 self.user_ns.update(ns)
3525 3580 self.user_ns_hidden.update(ns)
3526 3581 return gui, backend, clobbered
3527 3582
3528 3583 #-------------------------------------------------------------------------
3529 3584 # Utilities
3530 3585 #-------------------------------------------------------------------------
3531 3586
3532 3587 def var_expand(self, cmd, depth=0, formatter=DollarFormatter()):
3533 3588 """Expand python variables in a string.
3534 3589
3535 3590 The depth argument indicates how many frames above the caller should
3536 3591 be walked to look for the local namespace where to expand variables.
3537 3592
3538 3593 The global namespace for expansion is always the user's interactive
3539 3594 namespace.
3540 3595 """
3541 3596 ns = self.user_ns.copy()
3542 3597 try:
3543 3598 frame = sys._getframe(depth+1)
3544 3599 except ValueError:
3545 3600 # This is thrown if there aren't that many frames on the stack,
3546 3601 # e.g. if a script called run_line_magic() directly.
3547 3602 pass
3548 3603 else:
3549 3604 ns.update(frame.f_locals)
3550 3605
3551 3606 try:
3552 3607 # We have to use .vformat() here, because 'self' is a valid and common
3553 3608 # name, and expanding **ns for .format() would make it collide with
3554 3609 # the 'self' argument of the method.
3555 3610 cmd = formatter.vformat(cmd, args=[], kwargs=ns)
3556 3611 except Exception:
3557 3612 # if formatter couldn't format, just let it go untransformed
3558 3613 pass
3559 3614 return cmd
3560 3615
3561 3616 def mktempfile(self, data=None, prefix='ipython_edit_'):
3562 3617 """Make a new tempfile and return its filename.
3563 3618
3564 3619 This makes a call to tempfile.mkstemp (created in a tempfile.mkdtemp),
3565 3620 but it registers the created filename internally so ipython cleans it up
3566 3621 at exit time.
3567 3622
3568 3623 Optional inputs:
3569 3624
3570 3625 - data(None): if data is given, it gets written out to the temp file
3571 3626 immediately, and the file is closed again."""
3572 3627
3573 3628 dir_path = Path(tempfile.mkdtemp(prefix=prefix))
3574 3629 self.tempdirs.append(dir_path)
3575 3630
3576 3631 handle, filename = tempfile.mkstemp(".py", prefix, dir=str(dir_path))
3577 3632 os.close(handle) # On Windows, there can only be one open handle on a file
3578 3633
3579 3634 file_path = Path(filename)
3580 3635 self.tempfiles.append(file_path)
3581 3636
3582 3637 if data:
3583 3638 file_path.write_text(data, encoding="utf-8")
3584 3639 return filename
3585 3640
3586 3641 def ask_yes_no(self, prompt, default=None, interrupt=None):
3587 3642 if self.quiet:
3588 3643 return True
3589 3644 return ask_yes_no(prompt,default,interrupt)
3590 3645
3591 3646 def show_usage(self):
3592 3647 """Show a usage message"""
3593 3648 page.page(IPython.core.usage.interactive_usage)
3594 3649
3595 3650 def extract_input_lines(self, range_str, raw=False):
3596 3651 """Return as a string a set of input history slices.
3597 3652
3598 3653 Parameters
3599 3654 ----------
3600 3655 range_str : str
3601 3656 The set of slices is given as a string, like "~5/6-~4/2 4:8 9",
3602 3657 since this function is for use by magic functions which get their
3603 3658 arguments as strings. The number before the / is the session
3604 3659 number: ~n goes n back from the current session.
3605 3660
3606 3661 If empty string is given, returns history of current session
3607 3662 without the last input.
3608 3663
3609 3664 raw : bool, optional
3610 3665 By default, the processed input is used. If this is true, the raw
3611 3666 input history is used instead.
3612 3667
3613 3668 Notes
3614 3669 -----
3615 3670 Slices can be described with two notations:
3616 3671
3617 3672 * ``N:M`` -> standard python form, means including items N...(M-1).
3618 3673 * ``N-M`` -> include items N..M (closed endpoint).
3619 3674 """
3620 3675 lines = self.history_manager.get_range_by_str(range_str, raw=raw)
3621 3676 text = "\n".join(x for _, _, x in lines)
3622 3677
3623 3678 # Skip the last line, as it's probably the magic that called this
3624 3679 if not range_str:
3625 3680 if "\n" not in text:
3626 3681 text = ""
3627 3682 else:
3628 3683 text = text[: text.rfind("\n")]
3629 3684
3630 3685 return text
3631 3686
3632 3687 def find_user_code(self, target, raw=True, py_only=False, skip_encoding_cookie=True, search_ns=False):
3633 3688 """Get a code string from history, file, url, or a string or macro.
3634 3689
3635 3690 This is mainly used by magic functions.
3636 3691
3637 3692 Parameters
3638 3693 ----------
3639 3694 target : str
3640 3695 A string specifying code to retrieve. This will be tried respectively
3641 3696 as: ranges of input history (see %history for syntax), url,
3642 3697 corresponding .py file, filename, or an expression evaluating to a
3643 3698 string or Macro in the user namespace.
3644 3699
3645 3700 If empty string is given, returns complete history of current
3646 3701 session, without the last line.
3647 3702
3648 3703 raw : bool
3649 3704 If true (default), retrieve raw history. Has no effect on the other
3650 3705 retrieval mechanisms.
3651 3706
3652 3707 py_only : bool (default False)
3653 3708 Only try to fetch python code, do not try alternative methods to decode file
3654 3709 if unicode fails.
3655 3710
3656 3711 Returns
3657 3712 -------
3658 3713 A string of code.
3659 3714 ValueError is raised if nothing is found, and TypeError if it evaluates
3660 3715 to an object of another type. In each case, .args[0] is a printable
3661 3716 message.
3662 3717 """
3663 3718 code = self.extract_input_lines(target, raw=raw) # Grab history
3664 3719 if code:
3665 3720 return code
3666 3721 try:
3667 3722 if target.startswith(('http://', 'https://')):
3668 3723 return openpy.read_py_url(target, skip_encoding_cookie=skip_encoding_cookie)
3669 3724 except UnicodeDecodeError as e:
3670 3725 if not py_only :
3671 3726 # Deferred import
3672 3727 from urllib.request import urlopen
3673 3728 response = urlopen(target)
3674 3729 return response.read().decode('latin1')
3675 3730 raise ValueError(("'%s' seem to be unreadable.") % target) from e
3676 3731
3677 3732 potential_target = [target]
3678 3733 try :
3679 3734 potential_target.insert(0,get_py_filename(target))
3680 3735 except IOError:
3681 3736 pass
3682 3737
3683 3738 for tgt in potential_target :
3684 3739 if os.path.isfile(tgt): # Read file
3685 3740 try :
3686 3741 return openpy.read_py_file(tgt, skip_encoding_cookie=skip_encoding_cookie)
3687 3742 except UnicodeDecodeError as e:
3688 3743 if not py_only :
3689 3744 with io_open(tgt,'r', encoding='latin1') as f :
3690 3745 return f.read()
3691 3746 raise ValueError(("'%s' seem to be unreadable.") % target) from e
3692 3747 elif os.path.isdir(os.path.expanduser(tgt)):
3693 3748 raise ValueError("'%s' is a directory, not a regular file." % target)
3694 3749
3695 3750 if search_ns:
3696 3751 # Inspect namespace to load object source
3697 3752 object_info = self.object_inspect(target, detail_level=1)
3698 3753 if object_info['found'] and object_info['source']:
3699 3754 return object_info['source']
3700 3755
3701 3756 try: # User namespace
3702 3757 codeobj = eval(target, self.user_ns)
3703 3758 except Exception as e:
3704 3759 raise ValueError(("'%s' was not found in history, as a file, url, "
3705 3760 "nor in the user namespace.") % target) from e
3706 3761
3707 3762 if isinstance(codeobj, str):
3708 3763 return codeobj
3709 3764 elif isinstance(codeobj, Macro):
3710 3765 return codeobj.value
3711 3766
3712 3767 raise TypeError("%s is neither a string nor a macro." % target,
3713 3768 codeobj)
3714 3769
3715 3770 def _atexit_once(self):
3716 3771 """
3717 3772 At exist operation that need to be called at most once.
3718 3773 Second call to this function per instance will do nothing.
3719 3774 """
3720 3775
3721 3776 if not getattr(self, "_atexit_once_called", False):
3722 3777 self._atexit_once_called = True
3723 3778 # Clear all user namespaces to release all references cleanly.
3724 3779 self.reset(new_session=False)
3725 3780 # Close the history session (this stores the end time and line count)
3726 3781 # this must be *before* the tempfile cleanup, in case of temporary
3727 3782 # history db
3728 3783 self.history_manager.end_session()
3729 3784 self.history_manager = None
3730 3785
3731 3786 #-------------------------------------------------------------------------
3732 3787 # Things related to IPython exiting
3733 3788 #-------------------------------------------------------------------------
3734 3789 def atexit_operations(self):
3735 3790 """This will be executed at the time of exit.
3736 3791
3737 3792 Cleanup operations and saving of persistent data that is done
3738 3793 unconditionally by IPython should be performed here.
3739 3794
3740 3795 For things that may depend on startup flags or platform specifics (such
3741 3796 as having readline or not), register a separate atexit function in the
3742 3797 code that has the appropriate information, rather than trying to
3743 3798 clutter
3744 3799 """
3745 3800 self._atexit_once()
3746 3801
3747 3802 # Cleanup all tempfiles and folders left around
3748 3803 for tfile in self.tempfiles:
3749 3804 try:
3750 3805 tfile.unlink()
3751 3806 self.tempfiles.remove(tfile)
3752 3807 except FileNotFoundError:
3753 3808 pass
3754 3809 del self.tempfiles
3755 3810 for tdir in self.tempdirs:
3756 3811 try:
3757 3812 tdir.rmdir()
3758 3813 self.tempdirs.remove(tdir)
3759 3814 except FileNotFoundError:
3760 3815 pass
3761 3816 del self.tempdirs
3762 3817
3763 3818 # Restore user's cursor
3764 3819 if hasattr(self, "editing_mode") and self.editing_mode == "vi":
3765 3820 sys.stdout.write("\x1b[0 q")
3766 3821 sys.stdout.flush()
3767 3822
3768 3823 def cleanup(self):
3769 3824 self.restore_sys_module_state()
3770 3825
3771 3826
3772 3827 # Overridden in terminal subclass to change prompts
3773 3828 def switch_doctest_mode(self, mode):
3774 3829 pass
3775 3830
3776 3831
3777 3832 class InteractiveShellABC(metaclass=abc.ABCMeta):
3778 3833 """An abstract base class for InteractiveShell."""
3779 3834
3780 3835 InteractiveShellABC.register(InteractiveShell)
@@ -1,658 +1,661 b''
1 1 """Implementation of basic magic functions."""
2 2
3 3
4 4 from logging import error
5 5 import io
6 6 import os
7 7 from pprint import pformat
8 8 import sys
9 9 from warnings import warn
10 10
11 11 from traitlets.utils.importstring import import_item
12 12 from IPython.core import magic_arguments, page
13 13 from IPython.core.error import UsageError
14 14 from IPython.core.magic import Magics, magics_class, line_magic, magic_escapes
15 15 from IPython.utils.text import format_screen, dedent, indent
16 16 from IPython.testing.skipdoctest import skip_doctest
17 17 from IPython.utils.ipstruct import Struct
18 18
19 19
20 20 class MagicsDisplay(object):
21 21 def __init__(self, magics_manager, ignore=None):
22 22 self.ignore = ignore if ignore else []
23 23 self.magics_manager = magics_manager
24 24
25 25 def _lsmagic(self):
26 26 """The main implementation of the %lsmagic"""
27 27 mesc = magic_escapes['line']
28 28 cesc = magic_escapes['cell']
29 29 mman = self.magics_manager
30 30 magics = mman.lsmagic()
31 31 out = ['Available line magics:',
32 32 mesc + (' '+mesc).join(sorted([m for m,v in magics['line'].items() if (v not in self.ignore)])),
33 33 '',
34 34 'Available cell magics:',
35 35 cesc + (' '+cesc).join(sorted([m for m,v in magics['cell'].items() if (v not in self.ignore)])),
36 36 '',
37 37 mman.auto_status()]
38 38 return '\n'.join(out)
39 39
40 40 def _repr_pretty_(self, p, cycle):
41 41 p.text(self._lsmagic())
42 42
43 43 def __str__(self):
44 44 return self._lsmagic()
45 45
46 46 def _jsonable(self):
47 47 """turn magics dict into jsonable dict of the same structure
48 48
49 49 replaces object instances with their class names as strings
50 50 """
51 51 magic_dict = {}
52 52 mman = self.magics_manager
53 53 magics = mman.lsmagic()
54 54 for key, subdict in magics.items():
55 55 d = {}
56 56 magic_dict[key] = d
57 57 for name, obj in subdict.items():
58 58 try:
59 59 classname = obj.__self__.__class__.__name__
60 60 except AttributeError:
61 61 classname = 'Other'
62 62
63 63 d[name] = classname
64 64 return magic_dict
65 65
66 66 def _repr_json_(self):
67 67 return self._jsonable()
68 68
69 69
70 70 @magics_class
71 71 class BasicMagics(Magics):
72 72 """Magics that provide central IPython functionality.
73 73
74 74 These are various magics that don't fit into specific categories but that
75 75 are all part of the base 'IPython experience'."""
76 76
77 77 @skip_doctest
78 78 @magic_arguments.magic_arguments()
79 79 @magic_arguments.argument(
80 80 '-l', '--line', action='store_true',
81 81 help="""Create a line magic alias."""
82 82 )
83 83 @magic_arguments.argument(
84 84 '-c', '--cell', action='store_true',
85 85 help="""Create a cell magic alias."""
86 86 )
87 87 @magic_arguments.argument(
88 88 'name',
89 89 help="""Name of the magic to be created."""
90 90 )
91 91 @magic_arguments.argument(
92 92 'target',
93 93 help="""Name of the existing line or cell magic."""
94 94 )
95 95 @magic_arguments.argument(
96 96 '-p', '--params', default=None,
97 97 help="""Parameters passed to the magic function."""
98 98 )
99 99 @line_magic
100 100 def alias_magic(self, line=''):
101 101 """Create an alias for an existing line or cell magic.
102 102
103 103 Examples
104 104 --------
105 105 ::
106 106
107 107 In [1]: %alias_magic t timeit
108 108 Created `%t` as an alias for `%timeit`.
109 109 Created `%%t` as an alias for `%%timeit`.
110 110
111 111 In [2]: %t -n1 pass
112 112 1 loops, best of 3: 954 ns per loop
113 113
114 114 In [3]: %%t -n1
115 115 ...: pass
116 116 ...:
117 117 1 loops, best of 3: 954 ns per loop
118 118
119 119 In [4]: %alias_magic --cell whereami pwd
120 120 UsageError: Cell magic function `%%pwd` not found.
121 121 In [5]: %alias_magic --line whereami pwd
122 122 Created `%whereami` as an alias for `%pwd`.
123 123
124 124 In [6]: %whereami
125 125 Out[6]: u'/home/testuser'
126 126
127 127 In [7]: %alias_magic h history "-p -l 30" --line
128 128 Created `%h` as an alias for `%history -l 30`.
129 129 """
130 130
131 131 args = magic_arguments.parse_argstring(self.alias_magic, line)
132 132 shell = self.shell
133 133 mman = self.shell.magics_manager
134 134 escs = ''.join(magic_escapes.values())
135 135
136 136 target = args.target.lstrip(escs)
137 137 name = args.name.lstrip(escs)
138 138
139 139 params = args.params
140 140 if (params and
141 141 ((params.startswith('"') and params.endswith('"'))
142 142 or (params.startswith("'") and params.endswith("'")))):
143 143 params = params[1:-1]
144 144
145 145 # Find the requested magics.
146 146 m_line = shell.find_magic(target, 'line')
147 147 m_cell = shell.find_magic(target, 'cell')
148 148 if args.line and m_line is None:
149 149 raise UsageError('Line magic function `%s%s` not found.' %
150 150 (magic_escapes['line'], target))
151 151 if args.cell and m_cell is None:
152 152 raise UsageError('Cell magic function `%s%s` not found.' %
153 153 (magic_escapes['cell'], target))
154 154
155 155 # If --line and --cell are not specified, default to the ones
156 156 # that are available.
157 157 if not args.line and not args.cell:
158 158 if not m_line and not m_cell:
159 159 raise UsageError(
160 160 'No line or cell magic with name `%s` found.' % target
161 161 )
162 162 args.line = bool(m_line)
163 163 args.cell = bool(m_cell)
164 164
165 165 params_str = "" if params is None else " " + params
166 166
167 167 if args.line:
168 168 mman.register_alias(name, target, 'line', params)
169 169 print('Created `%s%s` as an alias for `%s%s%s`.' % (
170 170 magic_escapes['line'], name,
171 171 magic_escapes['line'], target, params_str))
172 172
173 173 if args.cell:
174 174 mman.register_alias(name, target, 'cell', params)
175 175 print('Created `%s%s` as an alias for `%s%s%s`.' % (
176 176 magic_escapes['cell'], name,
177 177 magic_escapes['cell'], target, params_str))
178 178
179 179 @line_magic
180 180 def lsmagic(self, parameter_s=''):
181 181 """List currently available magic functions."""
182 182 return MagicsDisplay(self.shell.magics_manager, ignore=[])
183 183
184 184 def _magic_docs(self, brief=False, rest=False):
185 185 """Return docstrings from magic functions."""
186 186 mman = self.shell.magics_manager
187 187 docs = mman.lsmagic_docs(brief, missing='No documentation')
188 188
189 189 if rest:
190 190 format_string = '**%s%s**::\n\n%s\n\n'
191 191 else:
192 192 format_string = '%s%s:\n%s\n'
193 193
194 194 return ''.join(
195 195 [format_string % (magic_escapes['line'], fname,
196 196 indent(dedent(fndoc)))
197 197 for fname, fndoc in sorted(docs['line'].items())]
198 198 +
199 199 [format_string % (magic_escapes['cell'], fname,
200 200 indent(dedent(fndoc)))
201 201 for fname, fndoc in sorted(docs['cell'].items())]
202 202 )
203 203
204 204 @line_magic
205 205 def magic(self, parameter_s=''):
206 206 """Print information about the magic function system.
207 207
208 208 Supported formats: -latex, -brief, -rest
209 209 """
210 210
211 211 mode = ''
212 212 try:
213 213 mode = parameter_s.split()[0][1:]
214 214 except IndexError:
215 215 pass
216 216
217 217 brief = (mode == 'brief')
218 218 rest = (mode == 'rest')
219 219 magic_docs = self._magic_docs(brief, rest)
220 220
221 221 if mode == 'latex':
222 222 print(self.format_latex(magic_docs))
223 223 return
224 224 else:
225 225 magic_docs = format_screen(magic_docs)
226 226
227 227 out = ["""
228 228 IPython's 'magic' functions
229 229 ===========================
230 230
231 231 The magic function system provides a series of functions which allow you to
232 232 control the behavior of IPython itself, plus a lot of system-type
233 233 features. There are two kinds of magics, line-oriented and cell-oriented.
234 234
235 235 Line magics are prefixed with the % character and work much like OS
236 236 command-line calls: they get as an argument the rest of the line, where
237 237 arguments are passed without parentheses or quotes. For example, this will
238 238 time the given statement::
239 239
240 240 %timeit range(1000)
241 241
242 242 Cell magics are prefixed with a double %%, and they are functions that get as
243 243 an argument not only the rest of the line, but also the lines below it in a
244 244 separate argument. These magics are called with two arguments: the rest of the
245 245 call line and the body of the cell, consisting of the lines below the first.
246 246 For example::
247 247
248 248 %%timeit x = numpy.random.randn((100, 100))
249 249 numpy.linalg.svd(x)
250 250
251 251 will time the execution of the numpy svd routine, running the assignment of x
252 252 as part of the setup phase, which is not timed.
253 253
254 254 In a line-oriented client (the terminal or Qt console IPython), starting a new
255 255 input with %% will automatically enter cell mode, and IPython will continue
256 256 reading input until a blank line is given. In the notebook, simply type the
257 257 whole cell as one entity, but keep in mind that the %% escape can only be at
258 258 the very start of the cell.
259 259
260 260 NOTE: If you have 'automagic' enabled (via the command line option or with the
261 261 %automagic function), you don't need to type in the % explicitly for line
262 262 magics; cell magics always require an explicit '%%' escape. By default,
263 263 IPython ships with automagic on, so you should only rarely need the % escape.
264 264
265 265 Example: typing '%cd mydir' (without the quotes) changes your working directory
266 266 to 'mydir', if it exists.
267 267
268 268 For a list of the available magic functions, use %lsmagic. For a description
269 269 of any of them, type %magic_name?, e.g. '%cd?'.
270 270
271 271 Currently the magic system has the following functions:""",
272 272 magic_docs,
273 273 "Summary of magic functions (from %slsmagic):" % magic_escapes['line'],
274 274 str(self.lsmagic()),
275 275 ]
276 276 page.page('\n'.join(out))
277 277
278 278
279 279 @line_magic
280 280 def page(self, parameter_s=''):
281 281 """Pretty print the object and display it through a pager.
282 282
283 283 %page [options] OBJECT
284 284
285 285 If no object is given, use _ (last output).
286 286
287 287 Options:
288 288
289 289 -r: page str(object), don't pretty-print it."""
290 290
291 291 # After a function contributed by Olivier Aubert, slightly modified.
292 292
293 293 # Process options/args
294 294 opts, args = self.parse_options(parameter_s, 'r')
295 295 raw = 'r' in opts
296 296
297 297 oname = args and args or '_'
298 298 info = self.shell._ofind(oname)
299 299 if info['found']:
300 txt = (raw and str or pformat)( info['obj'] )
300 if raw:
301 txt = str(info["obj"])
302 else:
303 txt = pformat(info["obj"])
301 304 page.page(txt)
302 305 else:
303 306 print('Object `%s` not found' % oname)
304 307
305 308 @line_magic
306 309 def pprint(self, parameter_s=''):
307 310 """Toggle pretty printing on/off."""
308 311 ptformatter = self.shell.display_formatter.formatters['text/plain']
309 312 ptformatter.pprint = bool(1 - ptformatter.pprint)
310 313 print('Pretty printing has been turned',
311 314 ['OFF','ON'][ptformatter.pprint])
312 315
313 316 @line_magic
314 317 def colors(self, parameter_s=''):
315 318 """Switch color scheme for prompts, info system and exception handlers.
316 319
317 320 Currently implemented schemes: NoColor, Linux, LightBG.
318 321
319 322 Color scheme names are not case-sensitive.
320 323
321 324 Examples
322 325 --------
323 326 To get a plain black and white terminal::
324 327
325 328 %colors nocolor
326 329 """
327 330 def color_switch_err(name):
328 331 warn('Error changing %s color schemes.\n%s' %
329 332 (name, sys.exc_info()[1]), stacklevel=2)
330 333
331 334
332 335 new_scheme = parameter_s.strip()
333 336 if not new_scheme:
334 337 raise UsageError(
335 338 "%colors: you must specify a color scheme. See '%colors?'")
336 339 # local shortcut
337 340 shell = self.shell
338 341
339 342 # Set shell colour scheme
340 343 try:
341 344 shell.colors = new_scheme
342 345 shell.refresh_style()
343 346 except:
344 347 color_switch_err('shell')
345 348
346 349 # Set exception colors
347 350 try:
348 351 shell.InteractiveTB.set_colors(scheme = new_scheme)
349 352 shell.SyntaxTB.set_colors(scheme = new_scheme)
350 353 except:
351 354 color_switch_err('exception')
352 355
353 356 # Set info (for 'object?') colors
354 357 if shell.color_info:
355 358 try:
356 359 shell.inspector.set_active_scheme(new_scheme)
357 360 except:
358 361 color_switch_err('object inspector')
359 362 else:
360 363 shell.inspector.set_active_scheme('NoColor')
361 364
362 365 @line_magic
363 366 def xmode(self, parameter_s=''):
364 367 """Switch modes for the exception handlers.
365 368
366 369 Valid modes: Plain, Context, Verbose, and Minimal.
367 370
368 371 If called without arguments, acts as a toggle.
369 372
370 373 When in verbose mode the value `--show` (and `--hide`)
371 374 will respectively show (or hide) frames with ``__tracebackhide__ =
372 375 True`` value set.
373 376 """
374 377
375 378 def xmode_switch_err(name):
376 379 warn('Error changing %s exception modes.\n%s' %
377 380 (name,sys.exc_info()[1]))
378 381
379 382 shell = self.shell
380 383 if parameter_s.strip() == "--show":
381 384 shell.InteractiveTB.skip_hidden = False
382 385 return
383 386 if parameter_s.strip() == "--hide":
384 387 shell.InteractiveTB.skip_hidden = True
385 388 return
386 389
387 390 new_mode = parameter_s.strip().capitalize()
388 391 try:
389 392 shell.InteractiveTB.set_mode(mode=new_mode)
390 393 print('Exception reporting mode:',shell.InteractiveTB.mode)
391 394 except:
392 395 xmode_switch_err('user')
393 396
394 397 @line_magic
395 398 def quickref(self, arg):
396 399 """ Show a quick reference sheet """
397 400 from IPython.core.usage import quick_reference
398 401 qr = quick_reference + self._magic_docs(brief=True)
399 402 page.page(qr)
400 403
401 404 @line_magic
402 405 def doctest_mode(self, parameter_s=''):
403 406 """Toggle doctest mode on and off.
404 407
405 408 This mode is intended to make IPython behave as much as possible like a
406 409 plain Python shell, from the perspective of how its prompts, exceptions
407 410 and output look. This makes it easy to copy and paste parts of a
408 411 session into doctests. It does so by:
409 412
410 413 - Changing the prompts to the classic ``>>>`` ones.
411 414 - Changing the exception reporting mode to 'Plain'.
412 415 - Disabling pretty-printing of output.
413 416
414 417 Note that IPython also supports the pasting of code snippets that have
415 418 leading '>>>' and '...' prompts in them. This means that you can paste
416 419 doctests from files or docstrings (even if they have leading
417 420 whitespace), and the code will execute correctly. You can then use
418 421 '%history -t' to see the translated history; this will give you the
419 422 input after removal of all the leading prompts and whitespace, which
420 423 can be pasted back into an editor.
421 424
422 425 With these features, you can switch into this mode easily whenever you
423 426 need to do testing and changes to doctests, without having to leave
424 427 your existing IPython session.
425 428 """
426 429
427 430 # Shorthands
428 431 shell = self.shell
429 432 meta = shell.meta
430 433 disp_formatter = self.shell.display_formatter
431 434 ptformatter = disp_formatter.formatters['text/plain']
432 435 # dstore is a data store kept in the instance metadata bag to track any
433 436 # changes we make, so we can undo them later.
434 437 dstore = meta.setdefault('doctest_mode',Struct())
435 438 save_dstore = dstore.setdefault
436 439
437 440 # save a few values we'll need to recover later
438 441 mode = save_dstore('mode',False)
439 442 save_dstore('rc_pprint',ptformatter.pprint)
440 443 save_dstore('xmode',shell.InteractiveTB.mode)
441 444 save_dstore('rc_separate_out',shell.separate_out)
442 445 save_dstore('rc_separate_out2',shell.separate_out2)
443 446 save_dstore('rc_separate_in',shell.separate_in)
444 447 save_dstore('rc_active_types',disp_formatter.active_types)
445 448
446 449 if not mode:
447 450 # turn on
448 451
449 452 # Prompt separators like plain python
450 453 shell.separate_in = ''
451 454 shell.separate_out = ''
452 455 shell.separate_out2 = ''
453 456
454 457
455 458 ptformatter.pprint = False
456 459 disp_formatter.active_types = ['text/plain']
457 460
458 461 shell.magic('xmode Plain')
459 462 else:
460 463 # turn off
461 464 shell.separate_in = dstore.rc_separate_in
462 465
463 466 shell.separate_out = dstore.rc_separate_out
464 467 shell.separate_out2 = dstore.rc_separate_out2
465 468
466 469 ptformatter.pprint = dstore.rc_pprint
467 470 disp_formatter.active_types = dstore.rc_active_types
468 471
469 472 shell.magic('xmode ' + dstore.xmode)
470 473
471 474 # mode here is the state before we switch; switch_doctest_mode takes
472 475 # the mode we're switching to.
473 476 shell.switch_doctest_mode(not mode)
474 477
475 478 # Store new mode and inform
476 479 dstore.mode = bool(not mode)
477 480 mode_label = ['OFF','ON'][dstore.mode]
478 481 print('Doctest mode is:', mode_label)
479 482
480 483 @line_magic
481 484 def gui(self, parameter_s=''):
482 485 """Enable or disable IPython GUI event loop integration.
483 486
484 487 %gui [GUINAME]
485 488
486 489 This magic replaces IPython's threaded shells that were activated
487 490 using the (pylab/wthread/etc.) command line flags. GUI toolkits
488 491 can now be enabled at runtime and keyboard
489 492 interrupts should work without any problems. The following toolkits
490 493 are supported: wxPython, PyQt4, PyGTK, Tk and Cocoa (OSX)::
491 494
492 495 %gui wx # enable wxPython event loop integration
493 496 %gui qt4|qt # enable PyQt4 event loop integration
494 497 %gui qt5 # enable PyQt5 event loop integration
495 498 %gui gtk # enable PyGTK event loop integration
496 499 %gui gtk3 # enable Gtk3 event loop integration
497 500 %gui gtk4 # enable Gtk4 event loop integration
498 501 %gui tk # enable Tk event loop integration
499 502 %gui osx # enable Cocoa event loop integration
500 503 # (requires %matplotlib 1.1)
501 504 %gui # disable all event loop integration
502 505
503 506 WARNING: after any of these has been called you can simply create
504 507 an application object, but DO NOT start the event loop yourself, as
505 508 we have already handled that.
506 509 """
507 510 opts, arg = self.parse_options(parameter_s, '')
508 511 if arg=='': arg = None
509 512 try:
510 513 return self.shell.enable_gui(arg)
511 514 except Exception as e:
512 515 # print simple error message, rather than traceback if we can't
513 516 # hook up the GUI
514 517 error(str(e))
515 518
516 519 @skip_doctest
517 520 @line_magic
518 521 def precision(self, s=''):
519 522 """Set floating point precision for pretty printing.
520 523
521 524 Can set either integer precision or a format string.
522 525
523 526 If numpy has been imported and precision is an int,
524 527 numpy display precision will also be set, via ``numpy.set_printoptions``.
525 528
526 529 If no argument is given, defaults will be restored.
527 530
528 531 Examples
529 532 --------
530 533 ::
531 534
532 535 In [1]: from math import pi
533 536
534 537 In [2]: %precision 3
535 538 Out[2]: u'%.3f'
536 539
537 540 In [3]: pi
538 541 Out[3]: 3.142
539 542
540 543 In [4]: %precision %i
541 544 Out[4]: u'%i'
542 545
543 546 In [5]: pi
544 547 Out[5]: 3
545 548
546 549 In [6]: %precision %e
547 550 Out[6]: u'%e'
548 551
549 552 In [7]: pi**10
550 553 Out[7]: 9.364805e+04
551 554
552 555 In [8]: %precision
553 556 Out[8]: u'%r'
554 557
555 558 In [9]: pi**10
556 559 Out[9]: 93648.047476082982
557 560 """
558 561 ptformatter = self.shell.display_formatter.formatters['text/plain']
559 562 ptformatter.float_precision = s
560 563 return ptformatter.float_format
561 564
562 565 @magic_arguments.magic_arguments()
563 566 @magic_arguments.argument(
564 567 'filename', type=str,
565 568 help='Notebook name or filename'
566 569 )
567 570 @line_magic
568 571 def notebook(self, s):
569 572 """Export and convert IPython notebooks.
570 573
571 574 This function can export the current IPython history to a notebook file.
572 575 For example, to export the history to "foo.ipynb" do "%notebook foo.ipynb".
573 576 """
574 577 args = magic_arguments.parse_argstring(self.notebook, s)
575 578 outfname = os.path.expanduser(args.filename)
576 579
577 580 from nbformat import write, v4
578 581
579 582 cells = []
580 583 hist = list(self.shell.history_manager.get_range())
581 584 if(len(hist)<=1):
582 585 raise ValueError('History is empty, cannot export')
583 586 for session, execution_count, source in hist[:-1]:
584 587 cells.append(v4.new_code_cell(
585 588 execution_count=execution_count,
586 589 source=source
587 590 ))
588 591 nb = v4.new_notebook(cells=cells)
589 592 with io.open(outfname, "w", encoding="utf-8") as f:
590 593 write(nb, f, version=4)
591 594
592 595 @magics_class
593 596 class AsyncMagics(BasicMagics):
594 597
595 598 @line_magic
596 599 def autoawait(self, parameter_s):
597 600 """
598 601 Allow to change the status of the autoawait option.
599 602
600 603 This allow you to set a specific asynchronous code runner.
601 604
602 605 If no value is passed, print the currently used asynchronous integration
603 606 and whether it is activated.
604 607
605 608 It can take a number of value evaluated in the following order:
606 609
607 610 - False/false/off deactivate autoawait integration
608 611 - True/true/on activate autoawait integration using configured default
609 612 loop
610 613 - asyncio/curio/trio activate autoawait integration and use integration
611 614 with said library.
612 615
613 616 - `sync` turn on the pseudo-sync integration (mostly used for
614 617 `IPython.embed()` which does not run IPython with a real eventloop and
615 618 deactivate running asynchronous code. Turning on Asynchronous code with
616 619 the pseudo sync loop is undefined behavior and may lead IPython to crash.
617 620
618 621 If the passed parameter does not match any of the above and is a python
619 622 identifier, get said object from user namespace and set it as the
620 623 runner, and activate autoawait.
621 624
622 625 If the object is a fully qualified object name, attempt to import it and
623 626 set it as the runner, and activate autoawait.
624 627
625 628 The exact behavior of autoawait is experimental and subject to change
626 629 across version of IPython and Python.
627 630 """
628 631
629 632 param = parameter_s.strip()
630 633 d = {True: "on", False: "off"}
631 634
632 635 if not param:
633 636 print("IPython autoawait is `{}`, and set to use `{}`".format(
634 637 d[self.shell.autoawait],
635 638 self.shell.loop_runner
636 639 ))
637 640 return None
638 641
639 642 if param.lower() in ('false', 'off'):
640 643 self.shell.autoawait = False
641 644 return None
642 645 if param.lower() in ('true', 'on'):
643 646 self.shell.autoawait = True
644 647 return None
645 648
646 649 if param in self.shell.loop_runner_map:
647 650 self.shell.loop_runner, self.shell.autoawait = self.shell.loop_runner_map[param]
648 651 return None
649 652
650 653 if param in self.shell.user_ns :
651 654 self.shell.loop_runner = self.shell.user_ns[param]
652 655 self.shell.autoawait = True
653 656 return None
654 657
655 658 runner = import_item(param)
656 659
657 660 self.shell.loop_runner = runner
658 661 self.shell.autoawait = True
@@ -1,187 +1,210 b''
1 1 """Implementation of configuration-related magic functions.
2 2 """
3 3 #-----------------------------------------------------------------------------
4 4 # Copyright (c) 2012 The IPython Development Team.
5 5 #
6 6 # Distributed under the terms of the Modified BSD License.
7 7 #
8 8 # The full license is in the file COPYING.txt, distributed with this software.
9 9 #-----------------------------------------------------------------------------
10 10
11 11 #-----------------------------------------------------------------------------
12 12 # Imports
13 13 #-----------------------------------------------------------------------------
14 14
15 15 # Stdlib
16 16 import re
17 17
18 18 # Our own packages
19 19 from IPython.core.error import UsageError
20 20 from IPython.core.magic import Magics, magics_class, line_magic
21 21 from logging import error
22 22
23 23 #-----------------------------------------------------------------------------
24 24 # Magic implementation classes
25 25 #-----------------------------------------------------------------------------
26 26
27 27 reg = re.compile(r'^\w+\.\w+$')
28 28 @magics_class
29 29 class ConfigMagics(Magics):
30 30
31 31 def __init__(self, shell):
32 32 super(ConfigMagics, self).__init__(shell)
33 33 self.configurables = []
34 34
35 35 @line_magic
36 36 def config(self, s):
37 37 """configure IPython
38 38
39 39 %config Class[.trait=value]
40 40
41 41 This magic exposes most of the IPython config system. Any
42 42 Configurable class should be able to be configured with the simple
43 43 line::
44 44
45 45 %config Class.trait=value
46 46
47 47 Where `value` will be resolved in the user's namespace, if it is an
48 48 expression or variable name.
49 49
50 50 Examples
51 51 --------
52 52
53 53 To see what classes are available for config, pass no arguments::
54 54
55 55 In [1]: %config
56 56 Available objects for config:
57 57 AliasManager
58 58 DisplayFormatter
59 59 HistoryManager
60 60 IPCompleter
61 61 LoggingMagics
62 62 MagicsManager
63 63 OSMagics
64 64 PrefilterManager
65 65 ScriptMagics
66 66 TerminalInteractiveShell
67 67
68 68 To view what is configurable on a given class, just pass the class
69 69 name::
70 70
71 71 In [2]: %config IPCompleter
72 72 IPCompleter(Completer) options
73 73 ----------------------------
74 74 IPCompleter.backslash_combining_completions=<Bool>
75 75 Enable unicode completions, e.g. \\alpha<tab> . Includes completion of latex
76 76 commands, unicode names, and expanding unicode characters back to latex
77 77 commands.
78 78 Current: True
79 79 IPCompleter.debug=<Bool>
80 80 Enable debug for the Completer. Mostly print extra information for
81 81 experimental jedi integration.
82 82 Current: False
83 IPCompleter.disable_matchers=<list-item-1>...
84 List of matchers to disable.
85 Current: []
83 86 IPCompleter.greedy=<Bool>
84 87 Activate greedy completion
85 88 PENDING DEPRECATION. this is now mostly taken care of with Jedi.
86 89 This will enable completion on elements of lists, results of function calls, etc.,
87 90 but can be unsafe because the code is actually evaluated on TAB.
88 91 Current: False
89 92 IPCompleter.jedi_compute_type_timeout=<Int>
90 93 Experimental: restrict time (in milliseconds) during which Jedi can compute types.
91 94 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
92 95 performance by preventing jedi to build its cache.
93 96 Current: 400
94 97 IPCompleter.limit_to__all__=<Bool>
95 98 DEPRECATED as of version 5.0.
96 99 Instruct the completer to use __all__ for the completion
97 100 Specifically, when completing on ``object.<tab>``.
98 101 When True: only those names in obj.__all__ will be included.
99 102 When False [default]: the __all__ attribute is ignored
100 103 Current: False
101 104 IPCompleter.merge_completions=<Bool>
102 105 Whether to merge completion results into a single list
103 106 If False, only the completion results from the first non-empty
104 107 completer will be returned.
108 As of version 8.6.0, setting the value to ``False`` is an alias for:
109 ``IPCompleter.suppress_competing_matchers = True.``.
105 110 Current: True
106 111 IPCompleter.omit__names=<Enum>
107 112 Instruct the completer to omit private method names
108 113 Specifically, when completing on ``object.<tab>``.
109 114 When 2 [default]: all names that start with '_' will be excluded.
110 115 When 1: all 'magic' names (``__foo__``) will be excluded.
111 116 When 0: nothing will be excluded.
112 117 Choices: any of [0, 1, 2]
113 118 Current: 2
114 119 IPCompleter.profile_completions=<Bool>
115 120 If True, emit profiling data for completion subsystem using cProfile.
116 121 Current: False
117 122 IPCompleter.profiler_output_dir=<Unicode>
118 123 Template for path at which to output profile data for completions.
119 124 Current: '.completion_profiles'
125 IPCompleter.suppress_competing_matchers=<Union>
126 Whether to suppress completions from other *Matchers*.
127 When set to ``None`` (default) the matchers will attempt to auto-detect
128 whether suppression of other matchers is desirable. For example, at the
129 beginning of a line followed by `%` we expect a magic completion to be the
130 only applicable option, and after ``my_dict['`` we usually expect a
131 completion with an existing dictionary key.
132 If you want to disable this heuristic and see completions from all matchers,
133 set ``IPCompleter.suppress_competing_matchers = False``. To disable the
134 heuristic for specific matchers provide a dictionary mapping:
135 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher':
136 False}``.
137 Set ``IPCompleter.suppress_competing_matchers = True`` to limit completions
138 to the set of matchers with the highest priority; this is equivalent to
139 ``IPCompleter.merge_completions`` and can be beneficial for performance, but
140 will sometimes omit relevant candidates from matchers further down the
141 priority list.
142 Current: None
120 143 IPCompleter.use_jedi=<Bool>
121 144 Experimental: Use Jedi to generate autocompletions. Default to True if jedi
122 145 is installed.
123 146 Current: True
124 147
125 148 but the real use is in setting values::
126 149
127 150 In [3]: %config IPCompleter.greedy = True
128 151
129 152 and these values are read from the user_ns if they are variables::
130 153
131 154 In [4]: feeling_greedy=False
132 155
133 156 In [5]: %config IPCompleter.greedy = feeling_greedy
134 157
135 158 """
136 159 from traitlets.config.loader import Config
137 160 # some IPython objects are Configurable, but do not yet have
138 161 # any configurable traits. Exclude them from the effects of
139 162 # this magic, as their presence is just noise:
140 163 configurables = sorted(set([ c for c in self.shell.configurables
141 164 if c.__class__.class_traits(config=True)
142 165 ]), key=lambda x: x.__class__.__name__)
143 166 classnames = [ c.__class__.__name__ for c in configurables ]
144 167
145 168 line = s.strip()
146 169 if not line:
147 170 # print available configurable names
148 171 print("Available objects for config:")
149 172 for name in classnames:
150 173 print(" ", name)
151 174 return
152 175 elif line in classnames:
153 176 # `%config TerminalInteractiveShell` will print trait info for
154 177 # TerminalInteractiveShell
155 178 c = configurables[classnames.index(line)]
156 179 cls = c.__class__
157 180 help = cls.class_get_help(c)
158 181 # strip leading '--' from cl-args:
159 182 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
160 183 print(help)
161 184 return
162 185 elif reg.match(line):
163 186 cls, attr = line.split('.')
164 187 return getattr(configurables[classnames.index(cls)],attr)
165 188 elif '=' not in line:
166 189 msg = "Invalid config statement: %r, "\
167 190 "should be `Class.trait = value`."
168 191
169 192 ll = line.lower()
170 193 for classname in classnames:
171 194 if ll == classname.lower():
172 195 msg = msg + '\nDid you mean %s (note the case)?' % classname
173 196 break
174 197
175 198 raise UsageError( msg % line)
176 199
177 200 # otherwise, assume we are setting configurables.
178 201 # leave quotes on args when splitting, because we want
179 202 # unquoted args to eval in user_ns
180 203 cfg = Config()
181 204 exec("cfg."+line, self.shell.user_ns, locals())
182 205
183 206 for configurable in configurables:
184 207 try:
185 208 configurable.update_config(cfg)
186 209 except Exception as e:
187 210 error(e)
@@ -1,711 +1,711 b''
1 1 """Implementation of namespace-related magic functions.
2 2 """
3 3 #-----------------------------------------------------------------------------
4 4 # Copyright (c) 2012 The IPython Development Team.
5 5 #
6 6 # Distributed under the terms of the Modified BSD License.
7 7 #
8 8 # The full license is in the file COPYING.txt, distributed with this software.
9 9 #-----------------------------------------------------------------------------
10 10
11 11 #-----------------------------------------------------------------------------
12 12 # Imports
13 13 #-----------------------------------------------------------------------------
14 14
15 15 # Stdlib
16 16 import gc
17 17 import re
18 18 import sys
19 19
20 20 # Our own packages
21 21 from IPython.core import page
22 22 from IPython.core.error import StdinNotImplementedError, UsageError
23 23 from IPython.core.magic import Magics, magics_class, line_magic
24 24 from IPython.testing.skipdoctest import skip_doctest
25 25 from IPython.utils.encoding import DEFAULT_ENCODING
26 26 from IPython.utils.openpy import read_py_file
27 27 from IPython.utils.path import get_py_filename
28 28
29 29 #-----------------------------------------------------------------------------
30 30 # Magic implementation classes
31 31 #-----------------------------------------------------------------------------
32 32
33 33 @magics_class
34 34 class NamespaceMagics(Magics):
35 35 """Magics to manage various aspects of the user's namespace.
36 36
37 37 These include listing variables, introspecting into them, etc.
38 38 """
39 39
40 40 @line_magic
41 41 def pinfo(self, parameter_s='', namespaces=None):
42 42 """Provide detailed information about an object.
43 43
44 44 '%pinfo object' is just a synonym for object? or ?object."""
45 45
46 46 #print 'pinfo par: <%s>' % parameter_s # dbg
47 47 # detail_level: 0 -> obj? , 1 -> obj??
48 48 detail_level = 0
49 49 # We need to detect if we got called as 'pinfo pinfo foo', which can
50 50 # happen if the user types 'pinfo foo?' at the cmd line.
51 51 pinfo,qmark1,oname,qmark2 = \
52 52 re.match(r'(pinfo )?(\?*)(.*?)(\??$)',parameter_s).groups()
53 53 if pinfo or qmark1 or qmark2:
54 54 detail_level = 1
55 55 if "*" in oname:
56 56 self.psearch(oname)
57 57 else:
58 58 self.shell._inspect('pinfo', oname, detail_level=detail_level,
59 59 namespaces=namespaces)
60 60
61 61 @line_magic
62 62 def pinfo2(self, parameter_s='', namespaces=None):
63 63 """Provide extra detailed information about an object.
64 64
65 65 '%pinfo2 object' is just a synonym for object?? or ??object."""
66 66 self.shell._inspect('pinfo', parameter_s, detail_level=1,
67 67 namespaces=namespaces)
68 68
69 69 @skip_doctest
70 70 @line_magic
71 71 def pdef(self, parameter_s='', namespaces=None):
72 72 """Print the call signature for any callable object.
73 73
74 74 If the object is a class, print the constructor information.
75 75
76 76 Examples
77 77 --------
78 78 ::
79 79
80 80 In [3]: %pdef urllib.urlopen
81 81 urllib.urlopen(url, data=None, proxies=None)
82 82 """
83 83 self.shell._inspect('pdef',parameter_s, namespaces)
84 84
85 85 @line_magic
86 86 def pdoc(self, parameter_s='', namespaces=None):
87 87 """Print the docstring for an object.
88 88
89 89 If the given object is a class, it will print both the class and the
90 90 constructor docstrings."""
91 91 self.shell._inspect('pdoc',parameter_s, namespaces)
92 92
93 93 @line_magic
94 94 def psource(self, parameter_s='', namespaces=None):
95 95 """Print (or run through pager) the source code for an object."""
96 96 if not parameter_s:
97 97 raise UsageError('Missing object name.')
98 98 self.shell._inspect('psource',parameter_s, namespaces)
99 99
100 100 @line_magic
101 101 def pfile(self, parameter_s='', namespaces=None):
102 102 """Print (or run through pager) the file where an object is defined.
103 103
104 104 The file opens at the line where the object definition begins. IPython
105 105 will honor the environment variable PAGER if set, and otherwise will
106 106 do its best to print the file in a convenient form.
107 107
108 108 If the given argument is not an object currently defined, IPython will
109 109 try to interpret it as a filename (automatically adding a .py extension
110 110 if needed). You can thus use %pfile as a syntax highlighting code
111 111 viewer."""
112 112
113 113 # first interpret argument as an object name
114 114 out = self.shell._inspect('pfile',parameter_s, namespaces)
115 115 # if not, try the input as a filename
116 116 if out == 'not found':
117 117 try:
118 118 filename = get_py_filename(parameter_s)
119 119 except IOError as msg:
120 120 print(msg)
121 121 return
122 122 page.page(self.shell.pycolorize(read_py_file(filename, skip_encoding_cookie=False)))
123 123
124 124 @line_magic
125 125 def psearch(self, parameter_s=''):
126 126 """Search for object in namespaces by wildcard.
127 127
128 128 %psearch [options] PATTERN [OBJECT TYPE]
129 129
130 130 Note: ? can be used as a synonym for %psearch, at the beginning or at
131 131 the end: both a*? and ?a* are equivalent to '%psearch a*'. Still, the
132 132 rest of the command line must be unchanged (options come first), so
133 133 for example the following forms are equivalent
134 134
135 135 %psearch -i a* function
136 136 -i a* function?
137 137 ?-i a* function
138 138
139 139 Arguments:
140 140
141 141 PATTERN
142 142
143 143 where PATTERN is a string containing * as a wildcard similar to its
144 144 use in a shell. The pattern is matched in all namespaces on the
145 145 search path. By default objects starting with a single _ are not
146 146 matched, many IPython generated objects have a single
147 147 underscore. The default is case insensitive matching. Matching is
148 148 also done on the attributes of objects and not only on the objects
149 149 in a module.
150 150
151 151 [OBJECT TYPE]
152 152
153 153 Is the name of a python type from the types module. The name is
154 154 given in lowercase without the ending type, ex. StringType is
155 155 written string. By adding a type here only objects matching the
156 156 given type are matched. Using all here makes the pattern match all
157 157 types (this is the default).
158 158
159 159 Options:
160 160
161 161 -a: makes the pattern match even objects whose names start with a
162 162 single underscore. These names are normally omitted from the
163 163 search.
164 164
165 165 -i/-c: make the pattern case insensitive/sensitive. If neither of
166 166 these options are given, the default is read from your configuration
167 167 file, with the option ``InteractiveShell.wildcards_case_sensitive``.
168 168 If this option is not specified in your configuration file, IPython's
169 169 internal default is to do a case sensitive search.
170 170
171 171 -e/-s NAMESPACE: exclude/search a given namespace. The pattern you
172 172 specify can be searched in any of the following namespaces:
173 173 'builtin', 'user', 'user_global','internal', 'alias', where
174 174 'builtin' and 'user' are the search defaults. Note that you should
175 175 not use quotes when specifying namespaces.
176 176
177 177 -l: List all available object types for object matching. This function
178 178 can be used without arguments.
179 179
180 180 'Builtin' contains the python module builtin, 'user' contains all
181 181 user data, 'alias' only contain the shell aliases and no python
182 182 objects, 'internal' contains objects used by IPython. The
183 183 'user_global' namespace is only used by embedded IPython instances,
184 184 and it contains module-level globals. You can add namespaces to the
185 185 search with -s or exclude them with -e (these options can be given
186 186 more than once).
187 187
188 188 Examples
189 189 --------
190 190 ::
191 191
192 192 %psearch a* -> objects beginning with an a
193 193 %psearch -e builtin a* -> objects NOT in the builtin space starting in a
194 194 %psearch a* function -> all functions beginning with an a
195 195 %psearch re.e* -> objects beginning with an e in module re
196 196 %psearch r*.e* -> objects that start with e in modules starting in r
197 197 %psearch r*.* string -> all strings in modules beginning with r
198 198
199 199 Case sensitive search::
200 200
201 201 %psearch -c a* list all object beginning with lower case a
202 202
203 203 Show objects beginning with a single _::
204 204
205 205 %psearch -a _* list objects beginning with a single underscore
206 206
207 207 List available objects::
208 208
209 209 %psearch -l list all available object types
210 210 """
211 211 # default namespaces to be searched
212 212 def_search = ['user_local', 'user_global', 'builtin']
213 213
214 214 # Process options/args
215 215 opts,args = self.parse_options(parameter_s,'cias:e:l',list_all=True)
216 216 opt = opts.get
217 217 shell = self.shell
218 218 psearch = shell.inspector.psearch
219 219
220 220 # select list object types
221 221 list_types = False
222 222 if 'l' in opts:
223 223 list_types = True
224 224
225 225 # select case options
226 226 if 'i' in opts:
227 227 ignore_case = True
228 228 elif 'c' in opts:
229 229 ignore_case = False
230 230 else:
231 231 ignore_case = not shell.wildcards_case_sensitive
232 232
233 233 # Build list of namespaces to search from user options
234 234 def_search.extend(opt('s',[]))
235 235 ns_exclude = ns_exclude=opt('e',[])
236 236 ns_search = [nm for nm in def_search if nm not in ns_exclude]
237 237
238 238 # Call the actual search
239 239 try:
240 240 psearch(args,shell.ns_table,ns_search,
241 241 show_all=opt('a'),ignore_case=ignore_case, list_types=list_types)
242 242 except:
243 243 shell.showtraceback()
244 244
245 245 @skip_doctest
246 246 @line_magic
247 247 def who_ls(self, parameter_s=''):
248 248 """Return a sorted list of all interactive variables.
249 249
250 250 If arguments are given, only variables of types matching these
251 251 arguments are returned.
252 252
253 253 Examples
254 254 --------
255 255 Define two variables and list them with who_ls::
256 256
257 257 In [1]: alpha = 123
258 258
259 259 In [2]: beta = 'test'
260 260
261 261 In [3]: %who_ls
262 262 Out[3]: ['alpha', 'beta']
263 263
264 264 In [4]: %who_ls int
265 265 Out[4]: ['alpha']
266 266
267 267 In [5]: %who_ls str
268 268 Out[5]: ['beta']
269 269 """
270 270
271 271 user_ns = self.shell.user_ns
272 272 user_ns_hidden = self.shell.user_ns_hidden
273 273 nonmatching = object() # This can never be in user_ns
274 274 out = [ i for i in user_ns
275 275 if not i.startswith('_') \
276 276 and (user_ns[i] is not user_ns_hidden.get(i, nonmatching)) ]
277 277
278 278 typelist = parameter_s.split()
279 279 if typelist:
280 280 typeset = set(typelist)
281 281 out = [i for i in out if type(user_ns[i]).__name__ in typeset]
282 282
283 283 out.sort()
284 284 return out
285 285
286 286 @skip_doctest
287 287 @line_magic
288 288 def who(self, parameter_s=''):
289 289 """Print all interactive variables, with some minimal formatting.
290 290
291 291 If any arguments are given, only variables whose type matches one of
292 292 these are printed. For example::
293 293
294 294 %who function str
295 295
296 296 will only list functions and strings, excluding all other types of
297 297 variables. To find the proper type names, simply use type(var) at a
298 298 command line to see how python prints type names. For example:
299 299
300 300 ::
301 301
302 302 In [1]: type('hello')\\
303 303 Out[1]: <type 'str'>
304 304
305 305 indicates that the type name for strings is 'str'.
306 306
307 307 ``%who`` always excludes executed names loaded through your configuration
308 308 file and things which are internal to IPython.
309 309
310 310 This is deliberate, as typically you may load many modules and the
311 311 purpose of %who is to show you only what you've manually defined.
312 312
313 313 Examples
314 314 --------
315 315
316 316 Define two variables and list them with who::
317 317
318 318 In [1]: alpha = 123
319 319
320 320 In [2]: beta = 'test'
321 321
322 322 In [3]: %who
323 323 alpha beta
324 324
325 325 In [4]: %who int
326 326 alpha
327 327
328 328 In [5]: %who str
329 329 beta
330 330 """
331 331
332 332 varlist = self.who_ls(parameter_s)
333 333 if not varlist:
334 334 if parameter_s:
335 335 print('No variables match your requested type.')
336 336 else:
337 337 print('Interactive namespace is empty.')
338 338 return
339 339
340 340 # if we have variables, move on...
341 341 count = 0
342 342 for i in varlist:
343 343 print(i+'\t', end=' ')
344 344 count += 1
345 345 if count > 8:
346 346 count = 0
347 347 print()
348 348 print()
349 349
350 350 @skip_doctest
351 351 @line_magic
352 352 def whos(self, parameter_s=''):
353 353 """Like %who, but gives some extra information about each variable.
354 354
355 355 The same type filtering of %who can be applied here.
356 356
357 357 For all variables, the type is printed. Additionally it prints:
358 358
359 359 - For {},[],(): their length.
360 360
361 361 - For numpy arrays, a summary with shape, number of
362 362 elements, typecode and size in memory.
363 363
364 364 - Everything else: a string representation, snipping their middle if
365 365 too long.
366 366
367 367 Examples
368 368 --------
369 369 Define two variables and list them with whos::
370 370
371 371 In [1]: alpha = 123
372 372
373 373 In [2]: beta = 'test'
374 374
375 375 In [3]: %whos
376 376 Variable Type Data/Info
377 377 --------------------------------
378 378 alpha int 123
379 379 beta str test
380 380 """
381 381
382 382 varnames = self.who_ls(parameter_s)
383 383 if not varnames:
384 384 if parameter_s:
385 385 print('No variables match your requested type.')
386 386 else:
387 387 print('Interactive namespace is empty.')
388 388 return
389 389
390 390 # if we have variables, move on...
391 391
392 392 # for these types, show len() instead of data:
393 393 seq_types = ['dict', 'list', 'tuple']
394 394
395 395 # for numpy arrays, display summary info
396 396 ndarray_type = None
397 397 if 'numpy' in sys.modules:
398 398 try:
399 399 from numpy import ndarray
400 400 except ImportError:
401 401 pass
402 402 else:
403 403 ndarray_type = ndarray.__name__
404 404
405 405 # Find all variable names and types so we can figure out column sizes
406 406
407 407 # some types are well known and can be shorter
408 408 abbrevs = {'IPython.core.macro.Macro' : 'Macro'}
409 409 def type_name(v):
410 410 tn = type(v).__name__
411 411 return abbrevs.get(tn,tn)
412 412
413 413 varlist = [self.shell.user_ns[n] for n in varnames]
414 414
415 415 typelist = []
416 416 for vv in varlist:
417 417 tt = type_name(vv)
418 418
419 419 if tt=='instance':
420 420 typelist.append( abbrevs.get(str(vv.__class__),
421 421 str(vv.__class__)))
422 422 else:
423 423 typelist.append(tt)
424 424
425 425 # column labels and # of spaces as separator
426 426 varlabel = 'Variable'
427 427 typelabel = 'Type'
428 428 datalabel = 'Data/Info'
429 429 colsep = 3
430 430 # variable format strings
431 431 vformat = "{0:<{varwidth}}{1:<{typewidth}}"
432 432 aformat = "%s: %s elems, type `%s`, %s bytes"
433 433 # find the size of the columns to format the output nicely
434 434 varwidth = max(max(map(len,varnames)), len(varlabel)) + colsep
435 435 typewidth = max(max(map(len,typelist)), len(typelabel)) + colsep
436 436 # table header
437 437 print(varlabel.ljust(varwidth) + typelabel.ljust(typewidth) + \
438 438 ' '+datalabel+'\n' + '-'*(varwidth+typewidth+len(datalabel)+1))
439 439 # and the table itself
440 440 kb = 1024
441 441 Mb = 1048576 # kb**2
442 442 for vname,var,vtype in zip(varnames,varlist,typelist):
443 443 print(vformat.format(vname, vtype, varwidth=varwidth, typewidth=typewidth), end=' ')
444 444 if vtype in seq_types:
445 445 print("n="+str(len(var)))
446 446 elif vtype == ndarray_type:
447 447 vshape = str(var.shape).replace(',','').replace(' ','x')[1:-1]
448 448 if vtype==ndarray_type:
449 449 # numpy
450 450 vsize = var.size
451 451 vbytes = vsize*var.itemsize
452 452 vdtype = var.dtype
453 453
454 454 if vbytes < 100000:
455 455 print(aformat % (vshape, vsize, vdtype, vbytes))
456 456 else:
457 457 print(aformat % (vshape, vsize, vdtype, vbytes), end=' ')
458 458 if vbytes < Mb:
459 459 print('(%s kb)' % (vbytes/kb,))
460 460 else:
461 461 print('(%s Mb)' % (vbytes/Mb,))
462 462 else:
463 463 try:
464 464 vstr = str(var)
465 465 except UnicodeEncodeError:
466 466 vstr = var.encode(DEFAULT_ENCODING,
467 467 'backslashreplace')
468 468 except:
469 469 vstr = "<object with id %d (str() failed)>" % id(var)
470 470 vstr = vstr.replace('\n', '\\n')
471 471 if len(vstr) < 50:
472 472 print(vstr)
473 473 else:
474 474 print(vstr[:25] + "<...>" + vstr[-25:])
475 475
476 476 @line_magic
477 477 def reset(self, parameter_s=''):
478 478 """Resets the namespace by removing all names defined by the user, if
479 479 called without arguments, or by removing some types of objects, such
480 480 as everything currently in IPython's In[] and Out[] containers (see
481 481 the parameters for details).
482 482
483 483 Parameters
484 484 ----------
485 485 -f
486 486 force reset without asking for confirmation.
487 487 -s
488 488 'Soft' reset: Only clears your namespace, leaving history intact.
489 489 References to objects may be kept. By default (without this option),
490 490 we do a 'hard' reset, giving you a new session and removing all
491 491 references to objects from the current session.
492 492 --aggressive
493 493 Try to aggressively remove modules from sys.modules ; this
494 494 may allow you to reimport Python modules that have been updated and
495 pick up changes, but can have unattended consequences.
495 pick up changes, but can have unintended consequences.
496 496
497 497 in
498 498 reset input history
499 499 out
500 500 reset output history
501 501 dhist
502 502 reset directory history
503 503 array
504 504 reset only variables that are NumPy arrays
505 505
506 506 See Also
507 507 --------
508 508 reset_selective : invoked as ``%reset_selective``
509 509
510 510 Examples
511 511 --------
512 512 ::
513 513
514 514 In [6]: a = 1
515 515
516 516 In [7]: a
517 517 Out[7]: 1
518 518
519 519 In [8]: 'a' in get_ipython().user_ns
520 520 Out[8]: True
521 521
522 522 In [9]: %reset -f
523 523
524 524 In [1]: 'a' in get_ipython().user_ns
525 525 Out[1]: False
526 526
527 527 In [2]: %reset -f in
528 528 Flushing input history
529 529
530 530 In [3]: %reset -f dhist in
531 531 Flushing directory history
532 532 Flushing input history
533 533
534 534 Notes
535 535 -----
536 536 Calling this magic from clients that do not implement standard input,
537 537 such as the ipython notebook interface, will reset the namespace
538 538 without confirmation.
539 539 """
540 540 opts, args = self.parse_options(parameter_s, "sf", "aggressive", mode="list")
541 541 if "f" in opts:
542 542 ans = True
543 543 else:
544 544 try:
545 545 ans = self.shell.ask_yes_no(
546 546 "Once deleted, variables cannot be recovered. Proceed (y/[n])?",
547 547 default='n')
548 548 except StdinNotImplementedError:
549 549 ans = True
550 550 if not ans:
551 551 print('Nothing done.')
552 552 return
553 553
554 554 if 's' in opts: # Soft reset
555 555 user_ns = self.shell.user_ns
556 556 for i in self.who_ls():
557 557 del(user_ns[i])
558 558 elif len(args) == 0: # Hard reset
559 559 self.shell.reset(new_session=False, aggressive=("aggressive" in opts))
560 560
561 561 # reset in/out/dhist/array: previously extensinions/clearcmd.py
562 562 ip = self.shell
563 563 user_ns = self.shell.user_ns # local lookup, heavily used
564 564
565 565 for target in args:
566 566 target = target.lower() # make matches case insensitive
567 567 if target == 'out':
568 568 print("Flushing output cache (%d entries)" % len(user_ns['_oh']))
569 569 self.shell.displayhook.flush()
570 570
571 571 elif target == 'in':
572 572 print("Flushing input history")
573 573 pc = self.shell.displayhook.prompt_count + 1
574 574 for n in range(1, pc):
575 575 key = '_i'+repr(n)
576 576 user_ns.pop(key,None)
577 577 user_ns.update(dict(_i=u'',_ii=u'',_iii=u''))
578 578 hm = ip.history_manager
579 579 # don't delete these, as %save and %macro depending on the
580 580 # length of these lists to be preserved
581 581 hm.input_hist_parsed[:] = [''] * pc
582 582 hm.input_hist_raw[:] = [''] * pc
583 583 # hm has internal machinery for _i,_ii,_iii, clear it out
584 584 hm._i = hm._ii = hm._iii = hm._i00 = u''
585 585
586 586 elif target == 'array':
587 587 # Support cleaning up numpy arrays
588 588 try:
589 589 from numpy import ndarray
590 590 # This must be done with items and not iteritems because
591 591 # we're going to modify the dict in-place.
592 592 for x,val in list(user_ns.items()):
593 593 if isinstance(val,ndarray):
594 594 del user_ns[x]
595 595 except ImportError:
596 596 print("reset array only works if Numpy is available.")
597 597
598 598 elif target == 'dhist':
599 599 print("Flushing directory history")
600 600 del user_ns['_dh'][:]
601 601
602 602 else:
603 603 print("Don't know how to reset ", end=' ')
604 604 print(target + ", please run `%reset?` for details")
605 605
606 606 gc.collect()
607 607
608 608 @line_magic
609 609 def reset_selective(self, parameter_s=''):
610 610 """Resets the namespace by removing names defined by the user.
611 611
612 612 Input/Output history are left around in case you need them.
613 613
614 614 %reset_selective [-f] regex
615 615
616 616 No action is taken if regex is not included
617 617
618 618 Options
619 619 -f : force reset without asking for confirmation.
620 620
621 621 See Also
622 622 --------
623 623 reset : invoked as ``%reset``
624 624
625 625 Examples
626 626 --------
627 627 We first fully reset the namespace so your output looks identical to
628 628 this example for pedagogical reasons; in practice you do not need a
629 629 full reset::
630 630
631 631 In [1]: %reset -f
632 632
633 633 Now, with a clean namespace we can make a few variables and use
634 634 ``%reset_selective`` to only delete names that match our regexp::
635 635
636 636 In [2]: a=1; b=2; c=3; b1m=4; b2m=5; b3m=6; b4m=7; b2s=8
637 637
638 638 In [3]: who_ls
639 639 Out[3]: ['a', 'b', 'b1m', 'b2m', 'b2s', 'b3m', 'b4m', 'c']
640 640
641 641 In [4]: %reset_selective -f b[2-3]m
642 642
643 643 In [5]: who_ls
644 644 Out[5]: ['a', 'b', 'b1m', 'b2s', 'b4m', 'c']
645 645
646 646 In [6]: %reset_selective -f d
647 647
648 648 In [7]: who_ls
649 649 Out[7]: ['a', 'b', 'b1m', 'b2s', 'b4m', 'c']
650 650
651 651 In [8]: %reset_selective -f c
652 652
653 653 In [9]: who_ls
654 654 Out[9]: ['a', 'b', 'b1m', 'b2s', 'b4m']
655 655
656 656 In [10]: %reset_selective -f b
657 657
658 658 In [11]: who_ls
659 659 Out[11]: ['a']
660 660
661 661 Notes
662 662 -----
663 663 Calling this magic from clients that do not implement standard input,
664 664 such as the ipython notebook interface, will reset the namespace
665 665 without confirmation.
666 666 """
667 667
668 668 opts, regex = self.parse_options(parameter_s,'f')
669 669
670 670 if 'f' in opts:
671 671 ans = True
672 672 else:
673 673 try:
674 674 ans = self.shell.ask_yes_no(
675 675 "Once deleted, variables cannot be recovered. Proceed (y/[n])? ",
676 676 default='n')
677 677 except StdinNotImplementedError:
678 678 ans = True
679 679 if not ans:
680 680 print('Nothing done.')
681 681 return
682 682 user_ns = self.shell.user_ns
683 683 if not regex:
684 684 print('No regex pattern specified. Nothing done.')
685 685 return
686 686 else:
687 687 try:
688 688 m = re.compile(regex)
689 689 except TypeError as e:
690 690 raise TypeError('regex must be a string or compiled pattern') from e
691 691 for i in self.who_ls():
692 692 if m.search(i):
693 693 del(user_ns[i])
694 694
695 695 @line_magic
696 696 def xdel(self, parameter_s=''):
697 697 """Delete a variable, trying to clear it from anywhere that
698 698 IPython's machinery has references to it. By default, this uses
699 699 the identity of the named object in the user namespace to remove
700 700 references held under other names. The object is also removed
701 701 from the output history.
702 702
703 703 Options
704 704 -n : Delete the specified name from all namespaces, without
705 705 checking their identity.
706 706 """
707 707 opts, varname = self.parse_options(parameter_s,'n')
708 708 try:
709 709 self.shell.del_var(varname, ('n' in opts))
710 710 except (NameError, ValueError) as e:
711 711 print(type(e).__name__ +": "+ str(e))
@@ -1,424 +1,425 b''
1 1 # -*- coding: utf-8 -*-
2 2 """Pylab (matplotlib) support utilities."""
3 3
4 4 # Copyright (c) IPython Development Team.
5 5 # Distributed under the terms of the Modified BSD License.
6 6
7 7 from io import BytesIO
8 8 from binascii import b2a_base64
9 9 from functools import partial
10 10 import warnings
11 11
12 12 from IPython.core.display import _pngxy
13 13 from IPython.utils.decorators import flag_calls
14 14
15 15 # If user specifies a GUI, that dictates the backend, otherwise we read the
16 16 # user's mpl default from the mpl rc structure
17 17 backends = {
18 18 "tk": "TkAgg",
19 19 "gtk": "GTKAgg",
20 20 "gtk3": "GTK3Agg",
21 21 "gtk4": "GTK4Agg",
22 22 "wx": "WXAgg",
23 23 "qt4": "Qt4Agg",
24 24 "qt5": "Qt5Agg",
25 25 "qt6": "QtAgg",
26 26 "qt": "Qt5Agg",
27 27 "osx": "MacOSX",
28 28 "nbagg": "nbAgg",
29 "webagg": "WebAgg",
29 30 "notebook": "nbAgg",
30 31 "agg": "agg",
31 32 "svg": "svg",
32 33 "pdf": "pdf",
33 34 "ps": "ps",
34 35 "inline": "module://matplotlib_inline.backend_inline",
35 36 "ipympl": "module://ipympl.backend_nbagg",
36 37 "widget": "module://ipympl.backend_nbagg",
37 38 }
38 39
39 40 # We also need a reverse backends2guis mapping that will properly choose which
40 41 # GUI support to activate based on the desired matplotlib backend. For the
41 42 # most part it's just a reverse of the above dict, but we also need to add a
42 43 # few others that map to the same GUI manually:
43 44 backend2gui = dict(zip(backends.values(), backends.keys()))
44 45 # In the reverse mapping, there are a few extra valid matplotlib backends that
45 46 # map to the same GUI support
46 47 backend2gui["GTK"] = backend2gui["GTKCairo"] = "gtk"
47 48 backend2gui["GTK3Cairo"] = "gtk3"
48 49 backend2gui["GTK4Cairo"] = "gtk4"
49 50 backend2gui["WX"] = "wx"
50 51 backend2gui["CocoaAgg"] = "osx"
51 52 # There needs to be a hysteresis here as the new QtAgg Matplotlib backend
52 53 # supports either Qt5 or Qt6 and the IPython qt event loop support Qt4, Qt5,
53 54 # and Qt6.
54 55 backend2gui["QtAgg"] = "qt"
55 56 backend2gui["Qt4Agg"] = "qt"
56 57 backend2gui["Qt5Agg"] = "qt"
57 58
58 59 # And some backends that don't need GUI integration
59 60 del backend2gui["nbAgg"]
60 61 del backend2gui["agg"]
61 62 del backend2gui["svg"]
62 63 del backend2gui["pdf"]
63 64 del backend2gui["ps"]
64 65 del backend2gui["module://matplotlib_inline.backend_inline"]
65 66 del backend2gui["module://ipympl.backend_nbagg"]
66 67
67 68 #-----------------------------------------------------------------------------
68 69 # Matplotlib utilities
69 70 #-----------------------------------------------------------------------------
70 71
71 72
72 73 def getfigs(*fig_nums):
73 74 """Get a list of matplotlib figures by figure numbers.
74 75
75 76 If no arguments are given, all available figures are returned. If the
76 77 argument list contains references to invalid figures, a warning is printed
77 78 but the function continues pasting further figures.
78 79
79 80 Parameters
80 81 ----------
81 82 figs : tuple
82 83 A tuple of ints giving the figure numbers of the figures to return.
83 84 """
84 85 from matplotlib._pylab_helpers import Gcf
85 86 if not fig_nums:
86 87 fig_managers = Gcf.get_all_fig_managers()
87 88 return [fm.canvas.figure for fm in fig_managers]
88 89 else:
89 90 figs = []
90 91 for num in fig_nums:
91 92 f = Gcf.figs.get(num)
92 93 if f is None:
93 94 print('Warning: figure %s not available.' % num)
94 95 else:
95 96 figs.append(f.canvas.figure)
96 97 return figs
97 98
98 99
99 100 def figsize(sizex, sizey):
100 101 """Set the default figure size to be [sizex, sizey].
101 102
102 103 This is just an easy to remember, convenience wrapper that sets::
103 104
104 105 matplotlib.rcParams['figure.figsize'] = [sizex, sizey]
105 106 """
106 107 import matplotlib
107 108 matplotlib.rcParams['figure.figsize'] = [sizex, sizey]
108 109
109 110
110 111 def print_figure(fig, fmt="png", bbox_inches="tight", base64=False, **kwargs):
111 112 """Print a figure to an image, and return the resulting file data
112 113
113 114 Returned data will be bytes unless ``fmt='svg'``,
114 115 in which case it will be unicode.
115 116
116 117 Any keyword args are passed to fig.canvas.print_figure,
117 118 such as ``quality`` or ``bbox_inches``.
118 119
119 120 If `base64` is True, return base64-encoded str instead of raw bytes
120 121 for binary-encoded image formats
121 122
122 123 .. versionadded:: 7.29
123 124 base64 argument
124 125 """
125 126 # When there's an empty figure, we shouldn't return anything, otherwise we
126 127 # get big blank areas in the qt console.
127 128 if not fig.axes and not fig.lines:
128 129 return
129 130
130 131 dpi = fig.dpi
131 132 if fmt == 'retina':
132 133 dpi = dpi * 2
133 134 fmt = 'png'
134 135
135 136 # build keyword args
136 137 kw = {
137 138 "format":fmt,
138 139 "facecolor":fig.get_facecolor(),
139 140 "edgecolor":fig.get_edgecolor(),
140 141 "dpi":dpi,
141 142 "bbox_inches":bbox_inches,
142 143 }
143 144 # **kwargs get higher priority
144 145 kw.update(kwargs)
145 146
146 147 bytes_io = BytesIO()
147 148 if fig.canvas is None:
148 149 from matplotlib.backend_bases import FigureCanvasBase
149 150 FigureCanvasBase(fig)
150 151
151 152 fig.canvas.print_figure(bytes_io, **kw)
152 153 data = bytes_io.getvalue()
153 154 if fmt == 'svg':
154 155 data = data.decode('utf-8')
155 156 elif base64:
156 157 data = b2a_base64(data).decode("ascii")
157 158 return data
158 159
159 160 def retina_figure(fig, base64=False, **kwargs):
160 161 """format a figure as a pixel-doubled (retina) PNG
161 162
162 163 If `base64` is True, return base64-encoded str instead of raw bytes
163 164 for binary-encoded image formats
164 165
165 166 .. versionadded:: 7.29
166 167 base64 argument
167 168 """
168 169 pngdata = print_figure(fig, fmt="retina", base64=False, **kwargs)
169 170 # Make sure that retina_figure acts just like print_figure and returns
170 171 # None when the figure is empty.
171 172 if pngdata is None:
172 173 return
173 174 w, h = _pngxy(pngdata)
174 175 metadata = {"width": w//2, "height":h//2}
175 176 if base64:
176 177 pngdata = b2a_base64(pngdata).decode("ascii")
177 178 return pngdata, metadata
178 179
179 180
180 181 # We need a little factory function here to create the closure where
181 182 # safe_execfile can live.
182 183 def mpl_runner(safe_execfile):
183 184 """Factory to return a matplotlib-enabled runner for %run.
184 185
185 186 Parameters
186 187 ----------
187 188 safe_execfile : function
188 189 This must be a function with the same interface as the
189 190 :meth:`safe_execfile` method of IPython.
190 191
191 192 Returns
192 193 -------
193 194 A function suitable for use as the ``runner`` argument of the %run magic
194 195 function.
195 196 """
196 197
197 198 def mpl_execfile(fname,*where,**kw):
198 199 """matplotlib-aware wrapper around safe_execfile.
199 200
200 201 Its interface is identical to that of the :func:`execfile` builtin.
201 202
202 203 This is ultimately a call to execfile(), but wrapped in safeties to
203 204 properly handle interactive rendering."""
204 205
205 206 import matplotlib
206 207 import matplotlib.pyplot as plt
207 208
208 209 #print '*** Matplotlib runner ***' # dbg
209 210 # turn off rendering until end of script
210 211 is_interactive = matplotlib.rcParams['interactive']
211 212 matplotlib.interactive(False)
212 213 safe_execfile(fname,*where,**kw)
213 214 matplotlib.interactive(is_interactive)
214 215 # make rendering call now, if the user tried to do it
215 216 if plt.draw_if_interactive.called:
216 217 plt.draw()
217 218 plt.draw_if_interactive.called = False
218 219
219 220 # re-draw everything that is stale
220 221 try:
221 222 da = plt.draw_all
222 223 except AttributeError:
223 224 pass
224 225 else:
225 226 da()
226 227
227 228 return mpl_execfile
228 229
229 230
230 231 def _reshow_nbagg_figure(fig):
231 232 """reshow an nbagg figure"""
232 233 try:
233 234 reshow = fig.canvas.manager.reshow
234 235 except AttributeError as e:
235 236 raise NotImplementedError() from e
236 237 else:
237 238 reshow()
238 239
239 240
240 241 def select_figure_formats(shell, formats, **kwargs):
241 242 """Select figure formats for the inline backend.
242 243
243 244 Parameters
244 245 ----------
245 246 shell : InteractiveShell
246 247 The main IPython instance.
247 248 formats : str or set
248 249 One or a set of figure formats to enable: 'png', 'retina', 'jpeg', 'svg', 'pdf'.
249 250 **kwargs : any
250 251 Extra keyword arguments to be passed to fig.canvas.print_figure.
251 252 """
252 253 import matplotlib
253 254 from matplotlib.figure import Figure
254 255
255 256 svg_formatter = shell.display_formatter.formatters['image/svg+xml']
256 257 png_formatter = shell.display_formatter.formatters['image/png']
257 258 jpg_formatter = shell.display_formatter.formatters['image/jpeg']
258 259 pdf_formatter = shell.display_formatter.formatters['application/pdf']
259 260
260 261 if isinstance(formats, str):
261 262 formats = {formats}
262 263 # cast in case of list / tuple
263 264 formats = set(formats)
264 265
265 266 [ f.pop(Figure, None) for f in shell.display_formatter.formatters.values() ]
266 267 mplbackend = matplotlib.get_backend().lower()
267 268 if mplbackend == 'nbagg' or mplbackend == 'module://ipympl.backend_nbagg':
268 269 formatter = shell.display_formatter.ipython_display_formatter
269 270 formatter.for_type(Figure, _reshow_nbagg_figure)
270 271
271 272 supported = {'png', 'png2x', 'retina', 'jpg', 'jpeg', 'svg', 'pdf'}
272 273 bad = formats.difference(supported)
273 274 if bad:
274 275 bs = "%s" % ','.join([repr(f) for f in bad])
275 276 gs = "%s" % ','.join([repr(f) for f in supported])
276 277 raise ValueError("supported formats are: %s not %s" % (gs, bs))
277 278
278 279 if "png" in formats:
279 280 png_formatter.for_type(
280 281 Figure, partial(print_figure, fmt="png", base64=True, **kwargs)
281 282 )
282 283 if "retina" in formats or "png2x" in formats:
283 284 png_formatter.for_type(Figure, partial(retina_figure, base64=True, **kwargs))
284 285 if "jpg" in formats or "jpeg" in formats:
285 286 jpg_formatter.for_type(
286 287 Figure, partial(print_figure, fmt="jpg", base64=True, **kwargs)
287 288 )
288 289 if "svg" in formats:
289 290 svg_formatter.for_type(Figure, partial(print_figure, fmt="svg", **kwargs))
290 291 if "pdf" in formats:
291 292 pdf_formatter.for_type(
292 293 Figure, partial(print_figure, fmt="pdf", base64=True, **kwargs)
293 294 )
294 295
295 296 #-----------------------------------------------------------------------------
296 297 # Code for initializing matplotlib and importing pylab
297 298 #-----------------------------------------------------------------------------
298 299
299 300
300 301 def find_gui_and_backend(gui=None, gui_select=None):
301 302 """Given a gui string return the gui and mpl backend.
302 303
303 304 Parameters
304 305 ----------
305 306 gui : str
306 307 Can be one of ('tk','gtk','wx','qt','qt4','inline','agg').
307 308 gui_select : str
308 309 Can be one of ('tk','gtk','wx','qt','qt4','inline').
309 310 This is any gui already selected by the shell.
310 311
311 312 Returns
312 313 -------
313 314 A tuple of (gui, backend) where backend is one of ('TkAgg','GTKAgg',
314 315 'WXAgg','Qt4Agg','module://matplotlib_inline.backend_inline','agg').
315 316 """
316 317
317 318 import matplotlib
318 319
319 320 if gui and gui != 'auto':
320 321 # select backend based on requested gui
321 322 backend = backends[gui]
322 323 if gui == 'agg':
323 324 gui = None
324 325 else:
325 326 # We need to read the backend from the original data structure, *not*
326 327 # from mpl.rcParams, since a prior invocation of %matplotlib may have
327 328 # overwritten that.
328 329 # WARNING: this assumes matplotlib 1.1 or newer!!
329 330 backend = matplotlib.rcParamsOrig['backend']
330 331 # In this case, we need to find what the appropriate gui selection call
331 332 # should be for IPython, so we can activate inputhook accordingly
332 333 gui = backend2gui.get(backend, None)
333 334
334 335 # If we have already had a gui active, we need it and inline are the
335 336 # ones allowed.
336 337 if gui_select and gui != gui_select:
337 338 gui = gui_select
338 339 backend = backends[gui]
339 340
340 341 return gui, backend
341 342
342 343
343 344 def activate_matplotlib(backend):
344 345 """Activate the given backend and set interactive to True."""
345 346
346 347 import matplotlib
347 348 matplotlib.interactive(True)
348 349
349 350 # Matplotlib had a bug where even switch_backend could not force
350 351 # the rcParam to update. This needs to be set *before* the module
351 352 # magic of switch_backend().
352 353 matplotlib.rcParams['backend'] = backend
353 354
354 355 # Due to circular imports, pyplot may be only partially initialised
355 356 # when this function runs.
356 357 # So avoid needing matplotlib attribute-lookup to access pyplot.
357 358 from matplotlib import pyplot as plt
358 359
359 360 plt.switch_backend(backend)
360 361
361 362 plt.show._needmain = False
362 363 # We need to detect at runtime whether show() is called by the user.
363 364 # For this, we wrap it into a decorator which adds a 'called' flag.
364 365 plt.draw_if_interactive = flag_calls(plt.draw_if_interactive)
365 366
366 367
367 368 def import_pylab(user_ns, import_all=True):
368 369 """Populate the namespace with pylab-related values.
369 370
370 371 Imports matplotlib, pylab, numpy, and everything from pylab and numpy.
371 372
372 373 Also imports a few names from IPython (figsize, display, getfigs)
373 374
374 375 """
375 376
376 377 # Import numpy as np/pyplot as plt are conventions we're trying to
377 378 # somewhat standardize on. Making them available to users by default
378 379 # will greatly help this.
379 380 s = ("import numpy\n"
380 381 "import matplotlib\n"
381 382 "from matplotlib import pylab, mlab, pyplot\n"
382 383 "np = numpy\n"
383 384 "plt = pyplot\n"
384 385 )
385 386 exec(s, user_ns)
386 387
387 388 if import_all:
388 389 s = ("from matplotlib.pylab import *\n"
389 390 "from numpy import *\n")
390 391 exec(s, user_ns)
391 392
392 393 # IPython symbols to add
393 394 user_ns['figsize'] = figsize
394 395 from IPython.display import display
395 396 # Add display and getfigs to the user's namespace
396 397 user_ns['display'] = display
397 398 user_ns['getfigs'] = getfigs
398 399
399 400
400 401 def configure_inline_support(shell, backend):
401 402 """
402 403 .. deprecated:: 7.23
403 404
404 405 use `matplotlib_inline.backend_inline.configure_inline_support()`
405 406
406 407 Configure an IPython shell object for matplotlib use.
407 408
408 409 Parameters
409 410 ----------
410 411 shell : InteractiveShell instance
411 412 backend : matplotlib backend
412 413 """
413 414 warnings.warn(
414 415 "`configure_inline_support` is deprecated since IPython 7.23, directly "
415 416 "use `matplotlib_inline.backend_inline.configure_inline_support()`",
416 417 DeprecationWarning,
417 418 stacklevel=2,
418 419 )
419 420
420 421 from matplotlib_inline.backend_inline import (
421 422 configure_inline_support as configure_inline_support_orig,
422 423 )
423 424
424 425 configure_inline_support_orig(shell, backend)
@@ -1,54 +1,54 b''
1 1 # -*- coding: utf-8 -*-
2 2 """Release data for the IPython project."""
3 3
4 4 #-----------------------------------------------------------------------------
5 5 # Copyright (c) 2008, IPython Development Team.
6 6 # Copyright (c) 2001, Fernando Perez <fernando.perez@colorado.edu>
7 7 # Copyright (c) 2001, Janko Hauser <jhauser@zscout.de>
8 8 # Copyright (c) 2001, Nathaniel Gray <n8gray@caltech.edu>
9 9 #
10 10 # Distributed under the terms of the Modified BSD License.
11 11 #
12 12 # The full license is in the file COPYING.txt, distributed with this software.
13 13 #-----------------------------------------------------------------------------
14 14
15 15 # IPython version information. An empty _version_extra corresponds to a full
16 16 # release. 'dev' as a _version_extra string means this is a development
17 17 # version
18 18 _version_major = 8
19 _version_minor = 6
19 _version_minor = 7
20 20 _version_patch = 0
21 21 _version_extra = ".dev"
22 22 # _version_extra = "rc1"
23 23 # _version_extra = "" # Uncomment this for full releases
24 24
25 25 # Construct full version string from these.
26 26 _ver = [_version_major, _version_minor, _version_patch]
27 27
28 28 __version__ = '.'.join(map(str, _ver))
29 29 if _version_extra:
30 30 __version__ = __version__ + _version_extra
31 31
32 32 version = __version__ # backwards compatibility name
33 33 version_info = (_version_major, _version_minor, _version_patch, _version_extra)
34 34
35 35 # Change this when incrementing the kernel protocol version
36 36 kernel_protocol_version_info = (5, 0)
37 37 kernel_protocol_version = "%i.%i" % kernel_protocol_version_info
38 38
39 license = 'BSD'
39 license = "BSD-3-Clause"
40 40
41 41 authors = {'Fernando' : ('Fernando Perez','fperez.net@gmail.com'),
42 42 'Janko' : ('Janko Hauser','jhauser@zscout.de'),
43 43 'Nathan' : ('Nathaniel Gray','n8gray@caltech.edu'),
44 44 'Ville' : ('Ville Vainio','vivainio@gmail.com'),
45 45 'Brian' : ('Brian E Granger', 'ellisonbg@gmail.com'),
46 46 'Min' : ('Min Ragan-Kelley', 'benjaminrk@gmail.com'),
47 47 'Thomas' : ('Thomas A. Kluyver', 'takowl@gmail.com'),
48 48 'Jorgen' : ('Jorgen Stenarson', 'jorgen.stenarson@bostream.nu'),
49 49 'Matthias' : ('Matthias Bussonnier', 'bussonniermatthias@gmail.com'),
50 50 }
51 51
52 52 author = 'The IPython Development Team'
53 53
54 54 author_email = 'ipython-dev@python.org'
@@ -1,4 +1,4 b''
1 1 # coding: iso-8859-5
2 2 # (Unlikely to be the default encoding for most testers.)
3 3 # ������������������� <- Cyrillic characters
4 u = '����'
4 u = "����"
@@ -1,1275 +1,1446 b''
1 1 # encoding: utf-8
2 2 """Tests for the IPython tab-completion machinery."""
3 3
4 4 # Copyright (c) IPython Development Team.
5 5 # Distributed under the terms of the Modified BSD License.
6 6
7 7 import os
8 8 import pytest
9 9 import sys
10 10 import textwrap
11 11 import unittest
12 12
13 13 from contextlib import contextmanager
14 14
15 15 from traitlets.config.loader import Config
16 16 from IPython import get_ipython
17 17 from IPython.core import completer
18 18 from IPython.utils.tempdir import TemporaryDirectory, TemporaryWorkingDirectory
19 19 from IPython.utils.generics import complete_object
20 20 from IPython.testing import decorators as dec
21 21
22 22 from IPython.core.completer import (
23 23 Completion,
24 24 provisionalcompleter,
25 25 match_dict_keys,
26 26 _deduplicate_completions,
27 completion_matcher,
28 SimpleCompletion,
29 CompletionContext,
27 30 )
28 31
29 32 # -----------------------------------------------------------------------------
30 33 # Test functions
31 34 # -----------------------------------------------------------------------------
32 35
33 36 def recompute_unicode_ranges():
34 37 """
35 38 utility to recompute the largest unicode range without any characters
36 39
37 40 use to recompute the gap in the global _UNICODE_RANGES of completer.py
38 41 """
39 42 import itertools
40 43 import unicodedata
41 44 valid = []
42 45 for c in range(0,0x10FFFF + 1):
43 46 try:
44 47 unicodedata.name(chr(c))
45 48 except ValueError:
46 49 continue
47 50 valid.append(c)
48 51
49 52 def ranges(i):
50 53 for a, b in itertools.groupby(enumerate(i), lambda pair: pair[1] - pair[0]):
51 54 b = list(b)
52 55 yield b[0][1], b[-1][1]
53 56
54 57 rg = list(ranges(valid))
55 58 lens = []
56 59 gap_lens = []
57 60 pstart, pstop = 0,0
58 61 for start, stop in rg:
59 62 lens.append(stop-start)
60 63 gap_lens.append((start - pstop, hex(pstop), hex(start), f'{round((start - pstop)/0xe01f0*100)}%'))
61 64 pstart, pstop = start, stop
62 65
63 66 return sorted(gap_lens)[-1]
64 67
65 68
66 69
67 70 def test_unicode_range():
68 71 """
69 72 Test that the ranges we test for unicode names give the same number of
70 73 results than testing the full length.
71 74 """
72 75 from IPython.core.completer import _unicode_name_compute, _UNICODE_RANGES
73 76
74 77 expected_list = _unicode_name_compute([(0, 0x110000)])
75 78 test = _unicode_name_compute(_UNICODE_RANGES)
76 79 len_exp = len(expected_list)
77 80 len_test = len(test)
78 81
79 82 # do not inline the len() or on error pytest will try to print the 130 000 +
80 83 # elements.
81 84 message = None
82 85 if len_exp != len_test or len_exp > 131808:
83 86 size, start, stop, prct = recompute_unicode_ranges()
84 87 message = f"""_UNICODE_RANGES likely wrong and need updating. This is
85 88 likely due to a new release of Python. We've find that the biggest gap
86 89 in unicode characters has reduces in size to be {size} characters
87 90 ({prct}), from {start}, to {stop}. In completer.py likely update to
88 91
89 92 _UNICODE_RANGES = [(32, {start}), ({stop}, 0xe01f0)]
90 93
91 94 And update the assertion below to use
92 95
93 96 len_exp <= {len_exp}
94 97 """
95 98 assert len_exp == len_test, message
96 99
97 100 # fail if new unicode symbols have been added.
98 101 assert len_exp <= 138552, message
99 102
100 103
101 104 @contextmanager
102 105 def greedy_completion():
103 106 ip = get_ipython()
104 107 greedy_original = ip.Completer.greedy
105 108 try:
106 109 ip.Completer.greedy = True
107 110 yield
108 111 finally:
109 112 ip.Completer.greedy = greedy_original
110 113
111 114
115 @contextmanager
116 def custom_matchers(matchers):
117 ip = get_ipython()
118 try:
119 ip.Completer.custom_matchers.extend(matchers)
120 yield
121 finally:
122 ip.Completer.custom_matchers.clear()
123
124
112 125 def test_protect_filename():
113 126 if sys.platform == "win32":
114 127 pairs = [
115 128 ("abc", "abc"),
116 129 (" abc", '" abc"'),
117 130 ("a bc", '"a bc"'),
118 131 ("a bc", '"a bc"'),
119 132 (" bc", '" bc"'),
120 133 ]
121 134 else:
122 135 pairs = [
123 136 ("abc", "abc"),
124 137 (" abc", r"\ abc"),
125 138 ("a bc", r"a\ bc"),
126 139 ("a bc", r"a\ \ bc"),
127 140 (" bc", r"\ \ bc"),
128 141 # On posix, we also protect parens and other special characters.
129 142 ("a(bc", r"a\(bc"),
130 143 ("a)bc", r"a\)bc"),
131 144 ("a( )bc", r"a\(\ \)bc"),
132 145 ("a[1]bc", r"a\[1\]bc"),
133 146 ("a{1}bc", r"a\{1\}bc"),
134 147 ("a#bc", r"a\#bc"),
135 148 ("a?bc", r"a\?bc"),
136 149 ("a=bc", r"a\=bc"),
137 150 ("a\\bc", r"a\\bc"),
138 151 ("a|bc", r"a\|bc"),
139 152 ("a;bc", r"a\;bc"),
140 153 ("a:bc", r"a\:bc"),
141 154 ("a'bc", r"a\'bc"),
142 155 ("a*bc", r"a\*bc"),
143 156 ('a"bc', r"a\"bc"),
144 157 ("a^bc", r"a\^bc"),
145 158 ("a&bc", r"a\&bc"),
146 159 ]
147 160 # run the actual tests
148 161 for s1, s2 in pairs:
149 162 s1p = completer.protect_filename(s1)
150 163 assert s1p == s2
151 164
152 165
153 166 def check_line_split(splitter, test_specs):
154 167 for part1, part2, split in test_specs:
155 168 cursor_pos = len(part1)
156 169 line = part1 + part2
157 170 out = splitter.split_line(line, cursor_pos)
158 171 assert out == split
159 172
160 173
161 174 def test_line_split():
162 175 """Basic line splitter test with default specs."""
163 176 sp = completer.CompletionSplitter()
164 177 # The format of the test specs is: part1, part2, expected answer. Parts 1
165 178 # and 2 are joined into the 'line' sent to the splitter, as if the cursor
166 179 # was at the end of part1. So an empty part2 represents someone hitting
167 180 # tab at the end of the line, the most common case.
168 181 t = [
169 182 ("run some/scrip", "", "some/scrip"),
170 183 ("run scripts/er", "ror.py foo", "scripts/er"),
171 184 ("echo $HOM", "", "HOM"),
172 185 ("print sys.pa", "", "sys.pa"),
173 186 ("print(sys.pa", "", "sys.pa"),
174 187 ("execfile('scripts/er", "", "scripts/er"),
175 188 ("a[x.", "", "x."),
176 189 ("a[x.", "y", "x."),
177 190 ('cd "some_file/', "", "some_file/"),
178 191 ]
179 192 check_line_split(sp, t)
180 193 # Ensure splitting works OK with unicode by re-running the tests with
181 194 # all inputs turned into unicode
182 195 check_line_split(sp, [map(str, p) for p in t])
183 196
184 197
185 198 class NamedInstanceClass:
186 199 instances = {}
187 200
188 201 def __init__(self, name):
189 202 self.instances[name] = self
190 203
191 204 @classmethod
192 205 def _ipython_key_completions_(cls):
193 206 return cls.instances.keys()
194 207
195 208
196 209 class KeyCompletable:
197 210 def __init__(self, things=()):
198 211 self.things = things
199 212
200 213 def _ipython_key_completions_(self):
201 214 return list(self.things)
202 215
203 216
204 217 class TestCompleter(unittest.TestCase):
205 218 def setUp(self):
206 219 """
207 220 We want to silence all PendingDeprecationWarning when testing the completer
208 221 """
209 222 self._assertwarns = self.assertWarns(PendingDeprecationWarning)
210 223 self._assertwarns.__enter__()
211 224
212 225 def tearDown(self):
213 226 try:
214 227 self._assertwarns.__exit__(None, None, None)
215 228 except AssertionError:
216 229 pass
217 230
218 231 def test_custom_completion_error(self):
219 232 """Test that errors from custom attribute completers are silenced."""
220 233 ip = get_ipython()
221 234
222 235 class A:
223 236 pass
224 237
225 238 ip.user_ns["x"] = A()
226 239
227 240 @complete_object.register(A)
228 241 def complete_A(a, existing_completions):
229 242 raise TypeError("this should be silenced")
230 243
231 244 ip.complete("x.")
232 245
233 246 def test_custom_completion_ordering(self):
234 247 """Test that errors from custom attribute completers are silenced."""
235 248 ip = get_ipython()
236 249
237 250 _, matches = ip.complete('in')
238 251 assert matches.index('input') < matches.index('int')
239 252
240 253 def complete_example(a):
241 254 return ['example2', 'example1']
242 255
243 256 ip.Completer.custom_completers.add_re('ex*', complete_example)
244 257 _, matches = ip.complete('ex')
245 258 assert matches.index('example2') < matches.index('example1')
246 259
247 260 def test_unicode_completions(self):
248 261 ip = get_ipython()
249 262 # Some strings that trigger different types of completion. Check them both
250 263 # in str and unicode forms
251 264 s = ["ru", "%ru", "cd /", "floa", "float(x)/"]
252 265 for t in s + list(map(str, s)):
253 266 # We don't need to check exact completion values (they may change
254 267 # depending on the state of the namespace, but at least no exceptions
255 268 # should be thrown and the return value should be a pair of text, list
256 269 # values.
257 270 text, matches = ip.complete(t)
258 271 self.assertIsInstance(text, str)
259 272 self.assertIsInstance(matches, list)
260 273
261 274 def test_latex_completions(self):
262 275 from IPython.core.latex_symbols import latex_symbols
263 276 import random
264 277
265 278 ip = get_ipython()
266 279 # Test some random unicode symbols
267 280 keys = random.sample(sorted(latex_symbols), 10)
268 281 for k in keys:
269 282 text, matches = ip.complete(k)
270 283 self.assertEqual(text, k)
271 284 self.assertEqual(matches, [latex_symbols[k]])
272 285 # Test a more complex line
273 286 text, matches = ip.complete("print(\\alpha")
274 287 self.assertEqual(text, "\\alpha")
275 288 self.assertEqual(matches[0], latex_symbols["\\alpha"])
276 289 # Test multiple matching latex symbols
277 290 text, matches = ip.complete("\\al")
278 291 self.assertIn("\\alpha", matches)
279 292 self.assertIn("\\aleph", matches)
280 293
281 294 def test_latex_no_results(self):
282 295 """
283 296 forward latex should really return nothing in either field if nothing is found.
284 297 """
285 298 ip = get_ipython()
286 299 text, matches = ip.Completer.latex_matches("\\really_i_should_match_nothing")
287 300 self.assertEqual(text, "")
288 301 self.assertEqual(matches, ())
289 302
290 303 def test_back_latex_completion(self):
291 304 ip = get_ipython()
292 305
293 306 # do not return more than 1 matches for \beta, only the latex one.
294 307 name, matches = ip.complete("\\β")
295 308 self.assertEqual(matches, ["\\beta"])
296 309
297 310 def test_back_unicode_completion(self):
298 311 ip = get_ipython()
299 312
300 313 name, matches = ip.complete("\\Ⅴ")
301 self.assertEqual(matches, ("\\ROMAN NUMERAL FIVE",))
314 self.assertEqual(matches, ["\\ROMAN NUMERAL FIVE"])
302 315
303 316 def test_forward_unicode_completion(self):
304 317 ip = get_ipython()
305 318
306 319 name, matches = ip.complete("\\ROMAN NUMERAL FIVE")
307 320 self.assertEqual(matches, ["Ⅴ"]) # This is not a V
308 321 self.assertEqual(matches, ["\u2164"]) # same as above but explicit.
309 322
310 323 def test_delim_setting(self):
311 324 sp = completer.CompletionSplitter()
312 325 sp.delims = " "
313 326 self.assertEqual(sp.delims, " ")
314 327 self.assertEqual(sp._delim_expr, r"[\ ]")
315 328
316 329 def test_spaces(self):
317 330 """Test with only spaces as split chars."""
318 331 sp = completer.CompletionSplitter()
319 332 sp.delims = " "
320 333 t = [("foo", "", "foo"), ("run foo", "", "foo"), ("run foo", "bar", "foo")]
321 334 check_line_split(sp, t)
322 335
323 336 def test_has_open_quotes1(self):
324 337 for s in ["'", "'''", "'hi' '"]:
325 338 self.assertEqual(completer.has_open_quotes(s), "'")
326 339
327 340 def test_has_open_quotes2(self):
328 341 for s in ['"', '"""', '"hi" "']:
329 342 self.assertEqual(completer.has_open_quotes(s), '"')
330 343
331 344 def test_has_open_quotes3(self):
332 345 for s in ["''", "''' '''", "'hi' 'ipython'"]:
333 346 self.assertFalse(completer.has_open_quotes(s))
334 347
335 348 def test_has_open_quotes4(self):
336 349 for s in ['""', '""" """', '"hi" "ipython"']:
337 350 self.assertFalse(completer.has_open_quotes(s))
338 351
339 352 @pytest.mark.xfail(
340 353 sys.platform == "win32", reason="abspath completions fail on Windows"
341 354 )
342 355 def test_abspath_file_completions(self):
343 356 ip = get_ipython()
344 357 with TemporaryDirectory() as tmpdir:
345 358 prefix = os.path.join(tmpdir, "foo")
346 359 suffixes = ["1", "2"]
347 360 names = [prefix + s for s in suffixes]
348 361 for n in names:
349 362 open(n, "w", encoding="utf-8").close()
350 363
351 364 # Check simple completion
352 365 c = ip.complete(prefix)[1]
353 366 self.assertEqual(c, names)
354 367
355 368 # Now check with a function call
356 369 cmd = 'a = f("%s' % prefix
357 370 c = ip.complete(prefix, cmd)[1]
358 371 comp = [prefix + s for s in suffixes]
359 372 self.assertEqual(c, comp)
360 373
361 374 def test_local_file_completions(self):
362 375 ip = get_ipython()
363 376 with TemporaryWorkingDirectory():
364 377 prefix = "./foo"
365 378 suffixes = ["1", "2"]
366 379 names = [prefix + s for s in suffixes]
367 380 for n in names:
368 381 open(n, "w", encoding="utf-8").close()
369 382
370 383 # Check simple completion
371 384 c = ip.complete(prefix)[1]
372 385 self.assertEqual(c, names)
373 386
374 387 # Now check with a function call
375 388 cmd = 'a = f("%s' % prefix
376 389 c = ip.complete(prefix, cmd)[1]
377 390 comp = {prefix + s for s in suffixes}
378 391 self.assertTrue(comp.issubset(set(c)))
379 392
380 393 def test_quoted_file_completions(self):
381 394 ip = get_ipython()
395
396 def _(text):
397 return ip.Completer._complete(
398 cursor_line=0, cursor_pos=len(text), full_text=text
399 )["IPCompleter.file_matcher"]["completions"]
400
382 401 with TemporaryWorkingDirectory():
383 402 name = "foo'bar"
384 403 open(name, "w", encoding="utf-8").close()
385 404
386 405 # Don't escape Windows
387 406 escaped = name if sys.platform == "win32" else "foo\\'bar"
388 407
389 408 # Single quote matches embedded single quote
390 text = "open('foo"
391 c = ip.Completer._complete(
392 cursor_line=0, cursor_pos=len(text), full_text=text
393 )[1]
394 self.assertEqual(c, [escaped])
409 c = _("open('foo")[0]
410 self.assertEqual(c.text, escaped)
395 411
396 412 # Double quote requires no escape
397 text = 'open("foo'
398 c = ip.Completer._complete(
399 cursor_line=0, cursor_pos=len(text), full_text=text
400 )[1]
401 self.assertEqual(c, [name])
413 c = _('open("foo')[0]
414 self.assertEqual(c.text, name)
402 415
403 416 # No quote requires an escape
404 text = "%ls foo"
405 c = ip.Completer._complete(
406 cursor_line=0, cursor_pos=len(text), full_text=text
407 )[1]
408 self.assertEqual(c, [escaped])
417 c = _("%ls foo")[0]
418 self.assertEqual(c.text, escaped)
409 419
410 420 def test_all_completions_dups(self):
411 421 """
412 422 Make sure the output of `IPCompleter.all_completions` does not have
413 423 duplicated prefixes.
414 424 """
415 425 ip = get_ipython()
416 426 c = ip.Completer
417 427 ip.ex("class TestClass():\n\ta=1\n\ta1=2")
418 428 for jedi_status in [True, False]:
419 429 with provisionalcompleter():
420 430 ip.Completer.use_jedi = jedi_status
421 431 matches = c.all_completions("TestCl")
422 432 assert matches == ["TestClass"], (jedi_status, matches)
423 433 matches = c.all_completions("TestClass.")
424 434 assert len(matches) > 2, (jedi_status, matches)
425 435 matches = c.all_completions("TestClass.a")
426 436 assert matches == ['TestClass.a', 'TestClass.a1'], jedi_status
427 437
428 438 def test_jedi(self):
429 439 """
430 440 A couple of issue we had with Jedi
431 441 """
432 442 ip = get_ipython()
433 443
434 444 def _test_complete(reason, s, comp, start=None, end=None):
435 445 l = len(s)
436 446 start = start if start is not None else l
437 447 end = end if end is not None else l
438 448 with provisionalcompleter():
439 449 ip.Completer.use_jedi = True
440 450 completions = set(ip.Completer.completions(s, l))
441 451 ip.Completer.use_jedi = False
442 452 assert Completion(start, end, comp) in completions, reason
443 453
444 454 def _test_not_complete(reason, s, comp):
445 455 l = len(s)
446 456 with provisionalcompleter():
447 457 ip.Completer.use_jedi = True
448 458 completions = set(ip.Completer.completions(s, l))
449 459 ip.Completer.use_jedi = False
450 460 assert Completion(l, l, comp) not in completions, reason
451 461
452 462 import jedi
453 463
454 464 jedi_version = tuple(int(i) for i in jedi.__version__.split(".")[:3])
455 465 if jedi_version > (0, 10):
456 466 _test_complete("jedi >0.9 should complete and not crash", "a=1;a.", "real")
457 467 _test_complete("can infer first argument", 'a=(1,"foo");a[0].', "real")
458 468 _test_complete("can infer second argument", 'a=(1,"foo");a[1].', "capitalize")
459 469 _test_complete("cover duplicate completions", "im", "import", 0, 2)
460 470
461 471 _test_not_complete("does not mix types", 'a=(1,"foo");a[0].', "capitalize")
462 472
463 473 def test_completion_have_signature(self):
464 474 """
465 475 Lets make sure jedi is capable of pulling out the signature of the function we are completing.
466 476 """
467 477 ip = get_ipython()
468 478 with provisionalcompleter():
469 479 ip.Completer.use_jedi = True
470 480 completions = ip.Completer.completions("ope", 3)
471 481 c = next(completions) # should be `open`
472 482 ip.Completer.use_jedi = False
473 483 assert "file" in c.signature, "Signature of function was not found by completer"
474 484 assert (
475 485 "encoding" in c.signature
476 486 ), "Signature of function was not found by completer"
477 487
488 def test_completions_have_type(self):
489 """
490 Lets make sure matchers provide completion type.
491 """
492 ip = get_ipython()
493 with provisionalcompleter():
494 ip.Completer.use_jedi = False
495 completions = ip.Completer.completions("%tim", 3)
496 c = next(completions) # should be `%time` or similar
497 assert c.type == "magic", "Type of magic was not assigned by completer"
498
478 499 @pytest.mark.xfail(reason="Known failure on jedi<=0.18.0")
479 500 def test_deduplicate_completions(self):
480 501 """
481 502 Test that completions are correctly deduplicated (even if ranges are not the same)
482 503 """
483 504 ip = get_ipython()
484 505 ip.ex(
485 506 textwrap.dedent(
486 507 """
487 508 class Z:
488 509 zoo = 1
489 510 """
490 511 )
491 512 )
492 513 with provisionalcompleter():
493 514 ip.Completer.use_jedi = True
494 515 l = list(
495 516 _deduplicate_completions("Z.z", ip.Completer.completions("Z.z", 3))
496 517 )
497 518 ip.Completer.use_jedi = False
498 519
499 520 assert len(l) == 1, "Completions (Z.z<tab>) correctly deduplicate: %s " % l
500 521 assert l[0].text == "zoo" # and not `it.accumulate`
501 522
502 523 def test_greedy_completions(self):
503 524 """
504 525 Test the capability of the Greedy completer.
505 526
506 527 Most of the test here does not really show off the greedy completer, for proof
507 528 each of the text below now pass with Jedi. The greedy completer is capable of more.
508 529
509 530 See the :any:`test_dict_key_completion_contexts`
510 531
511 532 """
512 533 ip = get_ipython()
513 534 ip.ex("a=list(range(5))")
514 535 _, c = ip.complete(".", line="a[0].")
515 536 self.assertFalse(".real" in c, "Shouldn't have completed on a[0]: %s" % c)
516 537
517 538 def _(line, cursor_pos, expect, message, completion):
518 539 with greedy_completion(), provisionalcompleter():
519 540 ip.Completer.use_jedi = False
520 541 _, c = ip.complete(".", line=line, cursor_pos=cursor_pos)
521 542 self.assertIn(expect, c, message % c)
522 543
523 544 ip.Completer.use_jedi = True
524 545 with provisionalcompleter():
525 546 completions = ip.Completer.completions(line, cursor_pos)
526 547 self.assertIn(completion, completions)
527 548
528 549 with provisionalcompleter():
529 550 _(
530 551 "a[0].",
531 552 5,
532 553 "a[0].real",
533 554 "Should have completed on a[0].: %s",
534 555 Completion(5, 5, "real"),
535 556 )
536 557 _(
537 558 "a[0].r",
538 559 6,
539 560 "a[0].real",
540 561 "Should have completed on a[0].r: %s",
541 562 Completion(5, 6, "real"),
542 563 )
543 564
544 565 _(
545 566 "a[0].from_",
546 567 10,
547 568 "a[0].from_bytes",
548 569 "Should have completed on a[0].from_: %s",
549 570 Completion(5, 10, "from_bytes"),
550 571 )
551 572
552 573 def test_omit__names(self):
553 574 # also happens to test IPCompleter as a configurable
554 575 ip = get_ipython()
555 576 ip._hidden_attr = 1
556 577 ip._x = {}
557 578 c = ip.Completer
558 579 ip.ex("ip=get_ipython()")
559 580 cfg = Config()
560 581 cfg.IPCompleter.omit__names = 0
561 582 c.update_config(cfg)
562 583 with provisionalcompleter():
563 584 c.use_jedi = False
564 585 s, matches = c.complete("ip.")
565 586 self.assertIn("ip.__str__", matches)
566 587 self.assertIn("ip._hidden_attr", matches)
567 588
568 589 # c.use_jedi = True
569 590 # completions = set(c.completions('ip.', 3))
570 591 # self.assertIn(Completion(3, 3, '__str__'), completions)
571 592 # self.assertIn(Completion(3,3, "_hidden_attr"), completions)
572 593
573 594 cfg = Config()
574 595 cfg.IPCompleter.omit__names = 1
575 596 c.update_config(cfg)
576 597 with provisionalcompleter():
577 598 c.use_jedi = False
578 599 s, matches = c.complete("ip.")
579 600 self.assertNotIn("ip.__str__", matches)
580 601 # self.assertIn('ip._hidden_attr', matches)
581 602
582 603 # c.use_jedi = True
583 604 # completions = set(c.completions('ip.', 3))
584 605 # self.assertNotIn(Completion(3,3,'__str__'), completions)
585 606 # self.assertIn(Completion(3,3, "_hidden_attr"), completions)
586 607
587 608 cfg = Config()
588 609 cfg.IPCompleter.omit__names = 2
589 610 c.update_config(cfg)
590 611 with provisionalcompleter():
591 612 c.use_jedi = False
592 613 s, matches = c.complete("ip.")
593 614 self.assertNotIn("ip.__str__", matches)
594 615 self.assertNotIn("ip._hidden_attr", matches)
595 616
596 617 # c.use_jedi = True
597 618 # completions = set(c.completions('ip.', 3))
598 619 # self.assertNotIn(Completion(3,3,'__str__'), completions)
599 620 # self.assertNotIn(Completion(3,3, "_hidden_attr"), completions)
600 621
601 622 with provisionalcompleter():
602 623 c.use_jedi = False
603 624 s, matches = c.complete("ip._x.")
604 625 self.assertIn("ip._x.keys", matches)
605 626
606 627 # c.use_jedi = True
607 628 # completions = set(c.completions('ip._x.', 6))
608 629 # self.assertIn(Completion(6,6, "keys"), completions)
609 630
610 631 del ip._hidden_attr
611 632 del ip._x
612 633
613 634 def test_limit_to__all__False_ok(self):
614 635 """
615 636 Limit to all is deprecated, once we remove it this test can go away.
616 637 """
617 638 ip = get_ipython()
618 639 c = ip.Completer
619 640 c.use_jedi = False
620 641 ip.ex("class D: x=24")
621 642 ip.ex("d=D()")
622 643 cfg = Config()
623 644 cfg.IPCompleter.limit_to__all__ = False
624 645 c.update_config(cfg)
625 646 s, matches = c.complete("d.")
626 647 self.assertIn("d.x", matches)
627 648
628 649 def test_get__all__entries_ok(self):
629 650 class A:
630 651 __all__ = ["x", 1]
631 652
632 653 words = completer.get__all__entries(A())
633 654 self.assertEqual(words, ["x"])
634 655
635 656 def test_get__all__entries_no__all__ok(self):
636 657 class A:
637 658 pass
638 659
639 660 words = completer.get__all__entries(A())
640 661 self.assertEqual(words, [])
641 662
642 663 def test_func_kw_completions(self):
643 664 ip = get_ipython()
644 665 c = ip.Completer
645 666 c.use_jedi = False
646 667 ip.ex("def myfunc(a=1,b=2): return a+b")
647 668 s, matches = c.complete(None, "myfunc(1,b")
648 669 self.assertIn("b=", matches)
649 670 # Simulate completing with cursor right after b (pos==10):
650 671 s, matches = c.complete(None, "myfunc(1,b)", 10)
651 672 self.assertIn("b=", matches)
652 673 s, matches = c.complete(None, 'myfunc(a="escaped\\")string",b')
653 674 self.assertIn("b=", matches)
654 675 # builtin function
655 676 s, matches = c.complete(None, "min(k, k")
656 677 self.assertIn("key=", matches)
657 678
658 679 def test_default_arguments_from_docstring(self):
659 680 ip = get_ipython()
660 681 c = ip.Completer
661 682 kwd = c._default_arguments_from_docstring("min(iterable[, key=func]) -> value")
662 683 self.assertEqual(kwd, ["key"])
663 684 # with cython type etc
664 685 kwd = c._default_arguments_from_docstring(
665 686 "Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)\n"
666 687 )
667 688 self.assertEqual(kwd, ["ncall", "resume", "nsplit"])
668 689 # white spaces
669 690 kwd = c._default_arguments_from_docstring(
670 691 "\n Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)\n"
671 692 )
672 693 self.assertEqual(kwd, ["ncall", "resume", "nsplit"])
673 694
674 695 def test_line_magics(self):
675 696 ip = get_ipython()
676 697 c = ip.Completer
677 698 s, matches = c.complete(None, "lsmag")
678 699 self.assertIn("%lsmagic", matches)
679 700 s, matches = c.complete(None, "%lsmag")
680 701 self.assertIn("%lsmagic", matches)
681 702
682 703 def test_cell_magics(self):
683 704 from IPython.core.magic import register_cell_magic
684 705
685 706 @register_cell_magic
686 707 def _foo_cellm(line, cell):
687 708 pass
688 709
689 710 ip = get_ipython()
690 711 c = ip.Completer
691 712
692 713 s, matches = c.complete(None, "_foo_ce")
693 714 self.assertIn("%%_foo_cellm", matches)
694 715 s, matches = c.complete(None, "%%_foo_ce")
695 716 self.assertIn("%%_foo_cellm", matches)
696 717
697 718 def test_line_cell_magics(self):
698 719 from IPython.core.magic import register_line_cell_magic
699 720
700 721 @register_line_cell_magic
701 722 def _bar_cellm(line, cell):
702 723 pass
703 724
704 725 ip = get_ipython()
705 726 c = ip.Completer
706 727
707 728 # The policy here is trickier, see comments in completion code. The
708 729 # returned values depend on whether the user passes %% or not explicitly,
709 730 # and this will show a difference if the same name is both a line and cell
710 731 # magic.
711 732 s, matches = c.complete(None, "_bar_ce")
712 733 self.assertIn("%_bar_cellm", matches)
713 734 self.assertIn("%%_bar_cellm", matches)
714 735 s, matches = c.complete(None, "%_bar_ce")
715 736 self.assertIn("%_bar_cellm", matches)
716 737 self.assertIn("%%_bar_cellm", matches)
717 738 s, matches = c.complete(None, "%%_bar_ce")
718 739 self.assertNotIn("%_bar_cellm", matches)
719 740 self.assertIn("%%_bar_cellm", matches)
720 741
721 742 def test_magic_completion_order(self):
722 743 ip = get_ipython()
723 744 c = ip.Completer
724 745
725 746 # Test ordering of line and cell magics.
726 747 text, matches = c.complete("timeit")
727 748 self.assertEqual(matches, ["%timeit", "%%timeit"])
728 749
729 750 def test_magic_completion_shadowing(self):
730 751 ip = get_ipython()
731 752 c = ip.Completer
732 753 c.use_jedi = False
733 754
734 755 # Before importing matplotlib, %matplotlib magic should be the only option.
735 756 text, matches = c.complete("mat")
736 757 self.assertEqual(matches, ["%matplotlib"])
737 758
738 759 # The newly introduced name should shadow the magic.
739 760 ip.run_cell("matplotlib = 1")
740 761 text, matches = c.complete("mat")
741 762 self.assertEqual(matches, ["matplotlib"])
742 763
743 764 # After removing matplotlib from namespace, the magic should again be
744 765 # the only option.
745 766 del ip.user_ns["matplotlib"]
746 767 text, matches = c.complete("mat")
747 768 self.assertEqual(matches, ["%matplotlib"])
748 769
749 770 def test_magic_completion_shadowing_explicit(self):
750 771 """
751 772 If the user try to complete a shadowed magic, and explicit % start should
752 773 still return the completions.
753 774 """
754 775 ip = get_ipython()
755 776 c = ip.Completer
756 777
757 778 # Before importing matplotlib, %matplotlib magic should be the only option.
758 779 text, matches = c.complete("%mat")
759 780 self.assertEqual(matches, ["%matplotlib"])
760 781
761 782 ip.run_cell("matplotlib = 1")
762 783
763 784 # After removing matplotlib from namespace, the magic should still be
764 785 # the only option.
765 786 text, matches = c.complete("%mat")
766 787 self.assertEqual(matches, ["%matplotlib"])
767 788
768 789 def test_magic_config(self):
769 790 ip = get_ipython()
770 791 c = ip.Completer
771 792
772 793 s, matches = c.complete(None, "conf")
773 794 self.assertIn("%config", matches)
774 795 s, matches = c.complete(None, "conf")
775 796 self.assertNotIn("AliasManager", matches)
776 797 s, matches = c.complete(None, "config ")
777 798 self.assertIn("AliasManager", matches)
778 799 s, matches = c.complete(None, "%config ")
779 800 self.assertIn("AliasManager", matches)
780 801 s, matches = c.complete(None, "config Ali")
781 802 self.assertListEqual(["AliasManager"], matches)
782 803 s, matches = c.complete(None, "%config Ali")
783 804 self.assertListEqual(["AliasManager"], matches)
784 805 s, matches = c.complete(None, "config AliasManager")
785 806 self.assertListEqual(["AliasManager"], matches)
786 807 s, matches = c.complete(None, "%config AliasManager")
787 808 self.assertListEqual(["AliasManager"], matches)
788 809 s, matches = c.complete(None, "config AliasManager.")
789 810 self.assertIn("AliasManager.default_aliases", matches)
790 811 s, matches = c.complete(None, "%config AliasManager.")
791 812 self.assertIn("AliasManager.default_aliases", matches)
792 813 s, matches = c.complete(None, "config AliasManager.de")
793 814 self.assertListEqual(["AliasManager.default_aliases"], matches)
794 815 s, matches = c.complete(None, "config AliasManager.de")
795 816 self.assertListEqual(["AliasManager.default_aliases"], matches)
796 817
797 818 def test_magic_color(self):
798 819 ip = get_ipython()
799 820 c = ip.Completer
800 821
801 822 s, matches = c.complete(None, "colo")
802 823 self.assertIn("%colors", matches)
803 824 s, matches = c.complete(None, "colo")
804 825 self.assertNotIn("NoColor", matches)
805 826 s, matches = c.complete(None, "%colors") # No trailing space
806 827 self.assertNotIn("NoColor", matches)
807 828 s, matches = c.complete(None, "colors ")
808 829 self.assertIn("NoColor", matches)
809 830 s, matches = c.complete(None, "%colors ")
810 831 self.assertIn("NoColor", matches)
811 832 s, matches = c.complete(None, "colors NoCo")
812 833 self.assertListEqual(["NoColor"], matches)
813 834 s, matches = c.complete(None, "%colors NoCo")
814 835 self.assertListEqual(["NoColor"], matches)
815 836
816 837 def test_match_dict_keys(self):
817 838 """
818 839 Test that match_dict_keys works on a couple of use case does return what
819 840 expected, and does not crash
820 841 """
821 842 delims = " \t\n`!@#$^&*()=+[{]}\\|;:'\",<>?"
822 843
823 844 keys = ["foo", b"far"]
824 845 assert match_dict_keys(keys, "b'", delims=delims) == ("'", 2, ["far"])
825 846 assert match_dict_keys(keys, "b'f", delims=delims) == ("'", 2, ["far"])
826 847 assert match_dict_keys(keys, 'b"', delims=delims) == ('"', 2, ["far"])
827 848 assert match_dict_keys(keys, 'b"f', delims=delims) == ('"', 2, ["far"])
828 849
829 850 assert match_dict_keys(keys, "'", delims=delims) == ("'", 1, ["foo"])
830 851 assert match_dict_keys(keys, "'f", delims=delims) == ("'", 1, ["foo"])
831 852 assert match_dict_keys(keys, '"', delims=delims) == ('"', 1, ["foo"])
832 853 assert match_dict_keys(keys, '"f', delims=delims) == ('"', 1, ["foo"])
833 854
834 855 match_dict_keys
835 856
836 857 def test_match_dict_keys_tuple(self):
837 858 """
838 859 Test that match_dict_keys called with extra prefix works on a couple of use case,
839 860 does return what expected, and does not crash.
840 861 """
841 862 delims = " \t\n`!@#$^&*()=+[{]}\\|;:'\",<>?"
842 863
843 864 keys = [("foo", "bar"), ("foo", "oof"), ("foo", b"bar"), ('other', 'test')]
844 865
845 866 # Completion on first key == "foo"
846 867 assert match_dict_keys(keys, "'", delims=delims, extra_prefix=("foo",)) == ("'", 1, ["bar", "oof"])
847 868 assert match_dict_keys(keys, "\"", delims=delims, extra_prefix=("foo",)) == ("\"", 1, ["bar", "oof"])
848 869 assert match_dict_keys(keys, "'o", delims=delims, extra_prefix=("foo",)) == ("'", 1, ["oof"])
849 870 assert match_dict_keys(keys, "\"o", delims=delims, extra_prefix=("foo",)) == ("\"", 1, ["oof"])
850 871 assert match_dict_keys(keys, "b'", delims=delims, extra_prefix=("foo",)) == ("'", 2, ["bar"])
851 872 assert match_dict_keys(keys, "b\"", delims=delims, extra_prefix=("foo",)) == ("\"", 2, ["bar"])
852 873 assert match_dict_keys(keys, "b'b", delims=delims, extra_prefix=("foo",)) == ("'", 2, ["bar"])
853 874 assert match_dict_keys(keys, "b\"b", delims=delims, extra_prefix=("foo",)) == ("\"", 2, ["bar"])
854 875
855 876 # No Completion
856 877 assert match_dict_keys(keys, "'", delims=delims, extra_prefix=("no_foo",)) == ("'", 1, [])
857 878 assert match_dict_keys(keys, "'", delims=delims, extra_prefix=("fo",)) == ("'", 1, [])
858 879
859 880 keys = [('foo1', 'foo2', 'foo3', 'foo4'), ('foo1', 'foo2', 'bar', 'foo4')]
860 881 assert match_dict_keys(keys, "'foo", delims=delims, extra_prefix=('foo1',)) == ("'", 1, ["foo2", "foo2"])
861 882 assert match_dict_keys(keys, "'foo", delims=delims, extra_prefix=('foo1', 'foo2')) == ("'", 1, ["foo3"])
862 883 assert match_dict_keys(keys, "'foo", delims=delims, extra_prefix=('foo1', 'foo2', 'foo3')) == ("'", 1, ["foo4"])
863 884 assert match_dict_keys(keys, "'foo", delims=delims, extra_prefix=('foo1', 'foo2', 'foo3', 'foo4')) == ("'", 1, [])
864 885
865 886 def test_dict_key_completion_string(self):
866 887 """Test dictionary key completion for string keys"""
867 888 ip = get_ipython()
868 889 complete = ip.Completer.complete
869 890
870 891 ip.user_ns["d"] = {"abc": None}
871 892
872 893 # check completion at different stages
873 894 _, matches = complete(line_buffer="d[")
874 895 self.assertIn("'abc'", matches)
875 896 self.assertNotIn("'abc']", matches)
876 897
877 898 _, matches = complete(line_buffer="d['")
878 899 self.assertIn("abc", matches)
879 900 self.assertNotIn("abc']", matches)
880 901
881 902 _, matches = complete(line_buffer="d['a")
882 903 self.assertIn("abc", matches)
883 904 self.assertNotIn("abc']", matches)
884 905
885 906 # check use of different quoting
886 907 _, matches = complete(line_buffer='d["')
887 908 self.assertIn("abc", matches)
888 909 self.assertNotIn('abc"]', matches)
889 910
890 911 _, matches = complete(line_buffer='d["a')
891 912 self.assertIn("abc", matches)
892 913 self.assertNotIn('abc"]', matches)
893 914
894 915 # check sensitivity to following context
895 916 _, matches = complete(line_buffer="d[]", cursor_pos=2)
896 917 self.assertIn("'abc'", matches)
897 918
898 919 _, matches = complete(line_buffer="d['']", cursor_pos=3)
899 920 self.assertIn("abc", matches)
900 921 self.assertNotIn("abc'", matches)
901 922 self.assertNotIn("abc']", matches)
902 923
903 924 # check multiple solutions are correctly returned and that noise is not
904 925 ip.user_ns["d"] = {
905 926 "abc": None,
906 927 "abd": None,
907 928 "bad": None,
908 929 object(): None,
909 930 5: None,
910 931 ("abe", None): None,
911 932 (None, "abf"): None
912 933 }
913 934
914 935 _, matches = complete(line_buffer="d['a")
915 936 self.assertIn("abc", matches)
916 937 self.assertIn("abd", matches)
917 938 self.assertNotIn("bad", matches)
918 939 self.assertNotIn("abe", matches)
919 940 self.assertNotIn("abf", matches)
920 941 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
921 942
922 943 # check escaping and whitespace
923 944 ip.user_ns["d"] = {"a\nb": None, "a'b": None, 'a"b': None, "a word": None}
924 945 _, matches = complete(line_buffer="d['a")
925 946 self.assertIn("a\\nb", matches)
926 947 self.assertIn("a\\'b", matches)
927 948 self.assertIn('a"b', matches)
928 949 self.assertIn("a word", matches)
929 950 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
930 951
931 952 # - can complete on non-initial word of the string
932 953 _, matches = complete(line_buffer="d['a w")
933 954 self.assertIn("word", matches)
934 955
935 956 # - understands quote escaping
936 957 _, matches = complete(line_buffer="d['a\\'")
937 958 self.assertIn("b", matches)
938 959
939 960 # - default quoting should work like repr
940 961 _, matches = complete(line_buffer="d[")
941 962 self.assertIn('"a\'b"', matches)
942 963
943 964 # - when opening quote with ", possible to match with unescaped apostrophe
944 965 _, matches = complete(line_buffer="d[\"a'")
945 966 self.assertIn("b", matches)
946 967
947 968 # need to not split at delims that readline won't split at
948 969 if "-" not in ip.Completer.splitter.delims:
949 970 ip.user_ns["d"] = {"before-after": None}
950 971 _, matches = complete(line_buffer="d['before-af")
951 972 self.assertIn("before-after", matches)
952 973
953 974 # check completion on tuple-of-string keys at different stage - on first key
954 975 ip.user_ns["d"] = {('foo', 'bar'): None}
955 976 _, matches = complete(line_buffer="d[")
956 977 self.assertIn("'foo'", matches)
957 978 self.assertNotIn("'foo']", matches)
958 979 self.assertNotIn("'bar'", matches)
959 980 self.assertNotIn("foo", matches)
960 981 self.assertNotIn("bar", matches)
961 982
962 983 # - match the prefix
963 984 _, matches = complete(line_buffer="d['f")
964 985 self.assertIn("foo", matches)
965 986 self.assertNotIn("foo']", matches)
966 987 self.assertNotIn('foo"]', matches)
967 988 _, matches = complete(line_buffer="d['foo")
968 989 self.assertIn("foo", matches)
969 990
970 991 # - can complete on second key
971 992 _, matches = complete(line_buffer="d['foo', ")
972 993 self.assertIn("'bar'", matches)
973 994 _, matches = complete(line_buffer="d['foo', 'b")
974 995 self.assertIn("bar", matches)
975 996 self.assertNotIn("foo", matches)
976 997
977 998 # - does not propose missing keys
978 999 _, matches = complete(line_buffer="d['foo', 'f")
979 1000 self.assertNotIn("bar", matches)
980 1001 self.assertNotIn("foo", matches)
981 1002
982 1003 # check sensitivity to following context
983 1004 _, matches = complete(line_buffer="d['foo',]", cursor_pos=8)
984 1005 self.assertIn("'bar'", matches)
985 1006 self.assertNotIn("bar", matches)
986 1007 self.assertNotIn("'foo'", matches)
987 1008 self.assertNotIn("foo", matches)
988 1009
989 1010 _, matches = complete(line_buffer="d['']", cursor_pos=3)
990 1011 self.assertIn("foo", matches)
991 1012 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
992 1013
993 1014 _, matches = complete(line_buffer='d[""]', cursor_pos=3)
994 1015 self.assertIn("foo", matches)
995 1016 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
996 1017
997 1018 _, matches = complete(line_buffer='d["foo","]', cursor_pos=9)
998 1019 self.assertIn("bar", matches)
999 1020 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
1000 1021
1001 1022 _, matches = complete(line_buffer='d["foo",]', cursor_pos=8)
1002 1023 self.assertIn("'bar'", matches)
1003 1024 self.assertNotIn("bar", matches)
1004 1025
1005 1026 # Can complete with longer tuple keys
1006 1027 ip.user_ns["d"] = {('foo', 'bar', 'foobar'): None}
1007 1028
1008 1029 # - can complete second key
1009 1030 _, matches = complete(line_buffer="d['foo', 'b")
1010 1031 self.assertIn("bar", matches)
1011 1032 self.assertNotIn("foo", matches)
1012 1033 self.assertNotIn("foobar", matches)
1013 1034
1014 1035 # - can complete third key
1015 1036 _, matches = complete(line_buffer="d['foo', 'bar', 'fo")
1016 1037 self.assertIn("foobar", matches)
1017 1038 self.assertNotIn("foo", matches)
1018 1039 self.assertNotIn("bar", matches)
1019 1040
1020 1041 def test_dict_key_completion_contexts(self):
1021 1042 """Test expression contexts in which dict key completion occurs"""
1022 1043 ip = get_ipython()
1023 1044 complete = ip.Completer.complete
1024 1045 d = {"abc": None}
1025 1046 ip.user_ns["d"] = d
1026 1047
1027 1048 class C:
1028 1049 data = d
1029 1050
1030 1051 ip.user_ns["C"] = C
1031 1052 ip.user_ns["get"] = lambda: d
1032 1053
1033 1054 def assert_no_completion(**kwargs):
1034 1055 _, matches = complete(**kwargs)
1035 1056 self.assertNotIn("abc", matches)
1036 1057 self.assertNotIn("abc'", matches)
1037 1058 self.assertNotIn("abc']", matches)
1038 1059 self.assertNotIn("'abc'", matches)
1039 1060 self.assertNotIn("'abc']", matches)
1040 1061
1041 1062 def assert_completion(**kwargs):
1042 1063 _, matches = complete(**kwargs)
1043 1064 self.assertIn("'abc'", matches)
1044 1065 self.assertNotIn("'abc']", matches)
1045 1066
1046 1067 # no completion after string closed, even if reopened
1047 1068 assert_no_completion(line_buffer="d['a'")
1048 1069 assert_no_completion(line_buffer='d["a"')
1049 1070 assert_no_completion(line_buffer="d['a' + ")
1050 1071 assert_no_completion(line_buffer="d['a' + '")
1051 1072
1052 1073 # completion in non-trivial expressions
1053 1074 assert_completion(line_buffer="+ d[")
1054 1075 assert_completion(line_buffer="(d[")
1055 1076 assert_completion(line_buffer="C.data[")
1056 1077
1057 1078 # greedy flag
1058 1079 def assert_completion(**kwargs):
1059 1080 _, matches = complete(**kwargs)
1060 1081 self.assertIn("get()['abc']", matches)
1061 1082
1062 1083 assert_no_completion(line_buffer="get()[")
1063 1084 with greedy_completion():
1064 1085 assert_completion(line_buffer="get()[")
1065 1086 assert_completion(line_buffer="get()['")
1066 1087 assert_completion(line_buffer="get()['a")
1067 1088 assert_completion(line_buffer="get()['ab")
1068 1089 assert_completion(line_buffer="get()['abc")
1069 1090
1070 1091 def test_dict_key_completion_bytes(self):
1071 1092 """Test handling of bytes in dict key completion"""
1072 1093 ip = get_ipython()
1073 1094 complete = ip.Completer.complete
1074 1095
1075 1096 ip.user_ns["d"] = {"abc": None, b"abd": None}
1076 1097
1077 1098 _, matches = complete(line_buffer="d[")
1078 1099 self.assertIn("'abc'", matches)
1079 1100 self.assertIn("b'abd'", matches)
1080 1101
1081 1102 if False: # not currently implemented
1082 1103 _, matches = complete(line_buffer="d[b")
1083 1104 self.assertIn("b'abd'", matches)
1084 1105 self.assertNotIn("b'abc'", matches)
1085 1106
1086 1107 _, matches = complete(line_buffer="d[b'")
1087 1108 self.assertIn("abd", matches)
1088 1109 self.assertNotIn("abc", matches)
1089 1110
1090 1111 _, matches = complete(line_buffer="d[B'")
1091 1112 self.assertIn("abd", matches)
1092 1113 self.assertNotIn("abc", matches)
1093 1114
1094 1115 _, matches = complete(line_buffer="d['")
1095 1116 self.assertIn("abc", matches)
1096 1117 self.assertNotIn("abd", matches)
1097 1118
1098 1119 def test_dict_key_completion_unicode_py3(self):
1099 1120 """Test handling of unicode in dict key completion"""
1100 1121 ip = get_ipython()
1101 1122 complete = ip.Completer.complete
1102 1123
1103 1124 ip.user_ns["d"] = {"a\u05d0": None}
1104 1125
1105 1126 # query using escape
1106 1127 if sys.platform != "win32":
1107 1128 # Known failure on Windows
1108 1129 _, matches = complete(line_buffer="d['a\\u05d0")
1109 1130 self.assertIn("u05d0", matches) # tokenized after \\
1110 1131
1111 1132 # query using character
1112 1133 _, matches = complete(line_buffer="d['a\u05d0")
1113 1134 self.assertIn("a\u05d0", matches)
1114 1135
1115 1136 with greedy_completion():
1116 1137 # query using escape
1117 1138 _, matches = complete(line_buffer="d['a\\u05d0")
1118 1139 self.assertIn("d['a\\u05d0']", matches) # tokenized after \\
1119 1140
1120 1141 # query using character
1121 1142 _, matches = complete(line_buffer="d['a\u05d0")
1122 1143 self.assertIn("d['a\u05d0']", matches)
1123 1144
1124 1145 @dec.skip_without("numpy")
1125 1146 def test_struct_array_key_completion(self):
1126 1147 """Test dict key completion applies to numpy struct arrays"""
1127 1148 import numpy
1128 1149
1129 1150 ip = get_ipython()
1130 1151 complete = ip.Completer.complete
1131 1152 ip.user_ns["d"] = numpy.array([], dtype=[("hello", "f"), ("world", "f")])
1132 1153 _, matches = complete(line_buffer="d['")
1133 1154 self.assertIn("hello", matches)
1134 1155 self.assertIn("world", matches)
1135 1156 # complete on the numpy struct itself
1136 1157 dt = numpy.dtype(
1137 1158 [("my_head", [("my_dt", ">u4"), ("my_df", ">u4")]), ("my_data", ">f4", 5)]
1138 1159 )
1139 1160 x = numpy.zeros(2, dtype=dt)
1140 1161 ip.user_ns["d"] = x[1]
1141 1162 _, matches = complete(line_buffer="d['")
1142 1163 self.assertIn("my_head", matches)
1143 1164 self.assertIn("my_data", matches)
1144 1165 # complete on a nested level
1145 1166 with greedy_completion():
1146 1167 ip.user_ns["d"] = numpy.zeros(2, dtype=dt)
1147 1168 _, matches = complete(line_buffer="d[1]['my_head']['")
1148 1169 self.assertTrue(any(["my_dt" in m for m in matches]))
1149 1170 self.assertTrue(any(["my_df" in m for m in matches]))
1150 1171
1151 1172 @dec.skip_without("pandas")
1152 1173 def test_dataframe_key_completion(self):
1153 1174 """Test dict key completion applies to pandas DataFrames"""
1154 1175 import pandas
1155 1176
1156 1177 ip = get_ipython()
1157 1178 complete = ip.Completer.complete
1158 1179 ip.user_ns["d"] = pandas.DataFrame({"hello": [1], "world": [2]})
1159 1180 _, matches = complete(line_buffer="d['")
1160 1181 self.assertIn("hello", matches)
1161 1182 self.assertIn("world", matches)
1162 1183
1163 1184 def test_dict_key_completion_invalids(self):
1164 1185 """Smoke test cases dict key completion can't handle"""
1165 1186 ip = get_ipython()
1166 1187 complete = ip.Completer.complete
1167 1188
1168 1189 ip.user_ns["no_getitem"] = None
1169 1190 ip.user_ns["no_keys"] = []
1170 1191 ip.user_ns["cant_call_keys"] = dict
1171 1192 ip.user_ns["empty"] = {}
1172 1193 ip.user_ns["d"] = {"abc": 5}
1173 1194
1174 1195 _, matches = complete(line_buffer="no_getitem['")
1175 1196 _, matches = complete(line_buffer="no_keys['")
1176 1197 _, matches = complete(line_buffer="cant_call_keys['")
1177 1198 _, matches = complete(line_buffer="empty['")
1178 1199 _, matches = complete(line_buffer="name_error['")
1179 1200 _, matches = complete(line_buffer="d['\\") # incomplete escape
1180 1201
1181 1202 def test_object_key_completion(self):
1182 1203 ip = get_ipython()
1183 1204 ip.user_ns["key_completable"] = KeyCompletable(["qwerty", "qwick"])
1184 1205
1185 1206 _, matches = ip.Completer.complete(line_buffer="key_completable['qw")
1186 1207 self.assertIn("qwerty", matches)
1187 1208 self.assertIn("qwick", matches)
1188 1209
1189 1210 def test_class_key_completion(self):
1190 1211 ip = get_ipython()
1191 1212 NamedInstanceClass("qwerty")
1192 1213 NamedInstanceClass("qwick")
1193 1214 ip.user_ns["named_instance_class"] = NamedInstanceClass
1194 1215
1195 1216 _, matches = ip.Completer.complete(line_buffer="named_instance_class['qw")
1196 1217 self.assertIn("qwerty", matches)
1197 1218 self.assertIn("qwick", matches)
1198 1219
1199 1220 def test_tryimport(self):
1200 1221 """
1201 1222 Test that try-import don't crash on trailing dot, and import modules before
1202 1223 """
1203 1224 from IPython.core.completerlib import try_import
1204 1225
1205 1226 assert try_import("IPython.")
1206 1227
1207 1228 def test_aimport_module_completer(self):
1208 1229 ip = get_ipython()
1209 1230 _, matches = ip.complete("i", "%aimport i")
1210 1231 self.assertIn("io", matches)
1211 1232 self.assertNotIn("int", matches)
1212 1233
1213 1234 def test_nested_import_module_completer(self):
1214 1235 ip = get_ipython()
1215 1236 _, matches = ip.complete(None, "import IPython.co", 17)
1216 1237 self.assertIn("IPython.core", matches)
1217 1238 self.assertNotIn("import IPython.core", matches)
1218 1239 self.assertNotIn("IPython.display", matches)
1219 1240
1220 1241 def test_import_module_completer(self):
1221 1242 ip = get_ipython()
1222 1243 _, matches = ip.complete("i", "import i")
1223 1244 self.assertIn("io", matches)
1224 1245 self.assertNotIn("int", matches)
1225 1246
1226 1247 def test_from_module_completer(self):
1227 1248 ip = get_ipython()
1228 1249 _, matches = ip.complete("B", "from io import B", 16)
1229 1250 self.assertIn("BytesIO", matches)
1230 1251 self.assertNotIn("BaseException", matches)
1231 1252
1232 1253 def test_snake_case_completion(self):
1233 1254 ip = get_ipython()
1234 1255 ip.Completer.use_jedi = False
1235 1256 ip.user_ns["some_three"] = 3
1236 1257 ip.user_ns["some_four"] = 4
1237 1258 _, matches = ip.complete("s_", "print(s_f")
1238 1259 self.assertIn("some_three", matches)
1239 1260 self.assertIn("some_four", matches)
1240 1261
1241 1262 def test_mix_terms(self):
1242 1263 ip = get_ipython()
1243 1264 from textwrap import dedent
1244 1265
1245 1266 ip.Completer.use_jedi = False
1246 1267 ip.ex(
1247 1268 dedent(
1248 1269 """
1249 1270 class Test:
1250 1271 def meth(self, meth_arg1):
1251 1272 print("meth")
1252 1273
1253 1274 def meth_1(self, meth1_arg1, meth1_arg2):
1254 1275 print("meth1")
1255 1276
1256 1277 def meth_2(self, meth2_arg1, meth2_arg2):
1257 1278 print("meth2")
1258 1279 test = Test()
1259 1280 """
1260 1281 )
1261 1282 )
1262 1283 _, matches = ip.complete(None, "test.meth(")
1263 1284 self.assertIn("meth_arg1=", matches)
1264 1285 self.assertNotIn("meth2_arg1=", matches)
1265 1286
1266 1287 def test_percent_symbol_restrict_to_magic_completions(self):
1267 1288 ip = get_ipython()
1268 1289 completer = ip.Completer
1269 1290 text = "%a"
1270 1291
1271 1292 with provisionalcompleter():
1272 1293 completer.use_jedi = True
1273 1294 completions = completer.completions(text, len(text))
1274 1295 for c in completions:
1275 1296 self.assertEqual(c.text[0], "%")
1297
1298 def test_fwd_unicode_restricts(self):
1299 ip = get_ipython()
1300 completer = ip.Completer
1301 text = "\\ROMAN NUMERAL FIVE"
1302
1303 with provisionalcompleter():
1304 completer.use_jedi = True
1305 completions = [
1306 completion.text for completion in completer.completions(text, len(text))
1307 ]
1308 self.assertEqual(completions, ["\u2164"])
1309
1310 def test_dict_key_restrict_to_dicts(self):
1311 """Test that dict key suppresses non-dict completion items"""
1312 ip = get_ipython()
1313 c = ip.Completer
1314 d = {"abc": None}
1315 ip.user_ns["d"] = d
1316
1317 text = 'd["a'
1318
1319 def _():
1320 with provisionalcompleter():
1321 c.use_jedi = True
1322 return [
1323 completion.text for completion in c.completions(text, len(text))
1324 ]
1325
1326 completions = _()
1327 self.assertEqual(completions, ["abc"])
1328
1329 # check that it can be disabled in granular manner:
1330 cfg = Config()
1331 cfg.IPCompleter.suppress_competing_matchers = {
1332 "IPCompleter.dict_key_matcher": False
1333 }
1334 c.update_config(cfg)
1335
1336 completions = _()
1337 self.assertIn("abc", completions)
1338 self.assertGreater(len(completions), 1)
1339
1340 def test_matcher_suppression(self):
1341 @completion_matcher(identifier="a_matcher")
1342 def a_matcher(text):
1343 return ["completion_a"]
1344
1345 @completion_matcher(identifier="b_matcher", api_version=2)
1346 def b_matcher(context: CompletionContext):
1347 text = context.token
1348 result = {"completions": [SimpleCompletion("completion_b")]}
1349
1350 if text == "suppress c":
1351 result["suppress"] = {"c_matcher"}
1352
1353 if text.startswith("suppress all"):
1354 result["suppress"] = True
1355 if text == "suppress all but c":
1356 result["do_not_suppress"] = {"c_matcher"}
1357 if text == "suppress all but a":
1358 result["do_not_suppress"] = {"a_matcher"}
1359
1360 return result
1361
1362 @completion_matcher(identifier="c_matcher")
1363 def c_matcher(text):
1364 return ["completion_c"]
1365
1366 with custom_matchers([a_matcher, b_matcher, c_matcher]):
1367 ip = get_ipython()
1368 c = ip.Completer
1369
1370 def _(text, expected):
1371 c.use_jedi = False
1372 s, matches = c.complete(text)
1373 self.assertEqual(expected, matches)
1374
1375 _("do not suppress", ["completion_a", "completion_b", "completion_c"])
1376 _("suppress all", ["completion_b"])
1377 _("suppress all but a", ["completion_a", "completion_b"])
1378 _("suppress all but c", ["completion_b", "completion_c"])
1379
1380 def configure(suppression_config):
1381 cfg = Config()
1382 cfg.IPCompleter.suppress_competing_matchers = suppression_config
1383 c.update_config(cfg)
1384
1385 # test that configuration takes priority over the run-time decisions
1386
1387 configure(False)
1388 _("suppress all", ["completion_a", "completion_b", "completion_c"])
1389
1390 configure({"b_matcher": False})
1391 _("suppress all", ["completion_a", "completion_b", "completion_c"])
1392
1393 configure({"a_matcher": False})
1394 _("suppress all", ["completion_b"])
1395
1396 configure({"b_matcher": True})
1397 _("do not suppress", ["completion_b"])
1398
1399 def test_matcher_disabling(self):
1400 @completion_matcher(identifier="a_matcher")
1401 def a_matcher(text):
1402 return ["completion_a"]
1403
1404 @completion_matcher(identifier="b_matcher")
1405 def b_matcher(text):
1406 return ["completion_b"]
1407
1408 def _(expected):
1409 s, matches = c.complete("completion_")
1410 self.assertEqual(expected, matches)
1411
1412 with custom_matchers([a_matcher, b_matcher]):
1413 ip = get_ipython()
1414 c = ip.Completer
1415
1416 _(["completion_a", "completion_b"])
1417
1418 cfg = Config()
1419 cfg.IPCompleter.disable_matchers = ["b_matcher"]
1420 c.update_config(cfg)
1421
1422 _(["completion_a"])
1423
1424 cfg.IPCompleter.disable_matchers = []
1425 c.update_config(cfg)
1426
1427 def test_matcher_priority(self):
1428 @completion_matcher(identifier="a_matcher", priority=0, api_version=2)
1429 def a_matcher(text):
1430 return {"completions": [SimpleCompletion("completion_a")], "suppress": True}
1431
1432 @completion_matcher(identifier="b_matcher", priority=2, api_version=2)
1433 def b_matcher(text):
1434 return {"completions": [SimpleCompletion("completion_b")], "suppress": True}
1435
1436 def _(expected):
1437 s, matches = c.complete("completion_")
1438 self.assertEqual(expected, matches)
1439
1440 with custom_matchers([a_matcher, b_matcher]):
1441 ip = get_ipython()
1442 c = ip.Completer
1443
1444 _(["completion_b"])
1445 a_matcher.matcher_priority = 3
1446 _(["completion_a"])
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
General Comments 0
You need to be logged in to leave comments. Login now