##// END OF EJS Templates
Minor documentation escape sequence updatre
Matthias Bussonnier -
Show More
@@ -1,3346 +1,3346 b''
1 """Completion for IPython.
1 """Completion for IPython.
2
2
3 This module started as fork of the rlcompleter module in the Python standard
3 This module started as fork of the rlcompleter module in the Python standard
4 library. The original enhancements made to rlcompleter have been sent
4 library. The original enhancements made to rlcompleter have been sent
5 upstream and were accepted as of Python 2.3,
5 upstream and were accepted as of Python 2.3,
6
6
7 This module now support a wide variety of completion mechanism both available
7 This module now support a wide variety of completion mechanism both available
8 for normal classic Python code, as well as completer for IPython specific
8 for normal classic Python code, as well as completer for IPython specific
9 Syntax like magics.
9 Syntax like magics.
10
10
11 Latex and Unicode completion
11 Latex and Unicode completion
12 ============================
12 ============================
13
13
14 IPython and compatible frontends not only can complete your code, but can help
14 IPython and compatible frontends not only can complete your code, but can help
15 you to input a wide range of characters. In particular we allow you to insert
15 you to input a wide range of characters. In particular we allow you to insert
16 a unicode character using the tab completion mechanism.
16 a unicode character using the tab completion mechanism.
17
17
18 Forward latex/unicode completion
18 Forward latex/unicode completion
19 --------------------------------
19 --------------------------------
20
20
21 Forward completion allows you to easily type a unicode character using its latex
21 Forward completion allows you to easily type a unicode character using its latex
22 name, or unicode long description. To do so type a backslash follow by the
22 name, or unicode long description. To do so type a backslash follow by the
23 relevant name and press tab:
23 relevant name and press tab:
24
24
25
25
26 Using latex completion:
26 Using latex completion:
27
27
28 .. code::
28 .. code::
29
29
30 \\alpha<tab>
30 \\alpha<tab>
31 Ξ±
31 Ξ±
32
32
33 or using unicode completion:
33 or using unicode completion:
34
34
35
35
36 .. code::
36 .. code::
37
37
38 \\GREEK SMALL LETTER ALPHA<tab>
38 \\GREEK SMALL LETTER ALPHA<tab>
39 Ξ±
39 Ξ±
40
40
41
41
42 Only valid Python identifiers will complete. Combining characters (like arrow or
42 Only valid Python identifiers will complete. Combining characters (like arrow or
43 dots) are also available, unlike latex they need to be put after the their
43 dots) are also available, unlike latex they need to be put after the their
44 counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
44 counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
45
45
46 Some browsers are known to display combining characters incorrectly.
46 Some browsers are known to display combining characters incorrectly.
47
47
48 Backward latex completion
48 Backward latex completion
49 -------------------------
49 -------------------------
50
50
51 It is sometime challenging to know how to type a character, if you are using
51 It is sometime challenging to know how to type a character, if you are using
52 IPython, or any compatible frontend you can prepend backslash to the character
52 IPython, or any compatible frontend you can prepend backslash to the character
53 and press :kbd:`Tab` to expand it to its latex form.
53 and press :kbd:`Tab` to expand it to its latex form.
54
54
55 .. code::
55 .. code::
56
56
57 \\Ξ±<tab>
57 \\Ξ±<tab>
58 \\alpha
58 \\alpha
59
59
60
60
61 Both forward and backward completions can be deactivated by setting the
61 Both forward and backward completions can be deactivated by setting the
62 :std:configtrait:`Completer.backslash_combining_completions` option to
62 :std:configtrait:`Completer.backslash_combining_completions` option to
63 ``False``.
63 ``False``.
64
64
65
65
66 Experimental
66 Experimental
67 ============
67 ============
68
68
69 Starting with IPython 6.0, this module can make use of the Jedi library to
69 Starting with IPython 6.0, this module can make use of the Jedi library to
70 generate completions both using static analysis of the code, and dynamically
70 generate completions both using static analysis of the code, and dynamically
71 inspecting multiple namespaces. Jedi is an autocompletion and static analysis
71 inspecting multiple namespaces. Jedi is an autocompletion and static analysis
72 for Python. The APIs attached to this new mechanism is unstable and will
72 for Python. The APIs attached to this new mechanism is unstable and will
73 raise unless use in an :any:`provisionalcompleter` context manager.
73 raise unless use in an :any:`provisionalcompleter` context manager.
74
74
75 You will find that the following are experimental:
75 You will find that the following are experimental:
76
76
77 - :any:`provisionalcompleter`
77 - :any:`provisionalcompleter`
78 - :any:`IPCompleter.completions`
78 - :any:`IPCompleter.completions`
79 - :any:`Completion`
79 - :any:`Completion`
80 - :any:`rectify_completions`
80 - :any:`rectify_completions`
81
81
82 .. note::
82 .. note::
83
83
84 better name for :any:`rectify_completions` ?
84 better name for :any:`rectify_completions` ?
85
85
86 We welcome any feedback on these new API, and we also encourage you to try this
86 We welcome any feedback on these new API, and we also encourage you to try this
87 module in debug mode (start IPython with ``--Completer.debug=True``) in order
87 module in debug mode (start IPython with ``--Completer.debug=True``) in order
88 to have extra logging information if :any:`jedi` is crashing, or if current
88 to have extra logging information if :any:`jedi` is crashing, or if current
89 IPython completer pending deprecations are returning results not yet handled
89 IPython completer pending deprecations are returning results not yet handled
90 by :any:`jedi`
90 by :any:`jedi`
91
91
92 Using Jedi for tab completion allow snippets like the following to work without
92 Using Jedi for tab completion allow snippets like the following to work without
93 having to execute any code:
93 having to execute any code:
94
94
95 >>> myvar = ['hello', 42]
95 >>> myvar = ['hello', 42]
96 ... myvar[1].bi<tab>
96 ... myvar[1].bi<tab>
97
97
98 Tab completion will be able to infer that ``myvar[1]`` is a real number without
98 Tab completion will be able to infer that ``myvar[1]`` is a real number without
99 executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
99 executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
100 option.
100 option.
101
101
102 Be sure to update :any:`jedi` to the latest stable version or to try the
102 Be sure to update :any:`jedi` to the latest stable version or to try the
103 current development version to get better completions.
103 current development version to get better completions.
104
104
105 Matchers
105 Matchers
106 ========
106 ========
107
107
108 All completions routines are implemented using unified *Matchers* API.
108 All completions routines are implemented using unified *Matchers* API.
109 The matchers API is provisional and subject to change without notice.
109 The matchers API is provisional and subject to change without notice.
110
110
111 The built-in matchers include:
111 The built-in matchers include:
112
112
113 - :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
113 - :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
114 - :any:`IPCompleter.magic_matcher`: completions for magics,
114 - :any:`IPCompleter.magic_matcher`: completions for magics,
115 - :any:`IPCompleter.unicode_name_matcher`,
115 - :any:`IPCompleter.unicode_name_matcher`,
116 :any:`IPCompleter.fwd_unicode_matcher`
116 :any:`IPCompleter.fwd_unicode_matcher`
117 and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
117 and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
118 - :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
118 - :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
119 - :any:`IPCompleter.file_matcher`: paths to files and directories,
119 - :any:`IPCompleter.file_matcher`: paths to files and directories,
120 - :any:`IPCompleter.python_func_kw_matcher` - function keywords,
120 - :any:`IPCompleter.python_func_kw_matcher` - function keywords,
121 - :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
121 - :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
122 - ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
122 - ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
123 - :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
123 - :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
124 implementation in :any:`InteractiveShell` which uses IPython hooks system
124 implementation in :any:`InteractiveShell` which uses IPython hooks system
125 (`complete_command`) with string dispatch (including regular expressions).
125 (`complete_command`) with string dispatch (including regular expressions).
126 Differently to other matchers, ``custom_completer_matcher`` will not suppress
126 Differently to other matchers, ``custom_completer_matcher`` will not suppress
127 Jedi results to match behaviour in earlier IPython versions.
127 Jedi results to match behaviour in earlier IPython versions.
128
128
129 Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
129 Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
130
130
131 Matcher API
131 Matcher API
132 -----------
132 -----------
133
133
134 Simplifying some details, the ``Matcher`` interface can described as
134 Simplifying some details, the ``Matcher`` interface can described as
135
135
136 .. code-block::
136 .. code-block::
137
137
138 MatcherAPIv1 = Callable[[str], list[str]]
138 MatcherAPIv1 = Callable[[str], list[str]]
139 MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
139 MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
140
140
141 Matcher = MatcherAPIv1 | MatcherAPIv2
141 Matcher = MatcherAPIv1 | MatcherAPIv2
142
142
143 The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
143 The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
144 and remains supported as a simplest way for generating completions. This is also
144 and remains supported as a simplest way for generating completions. This is also
145 currently the only API supported by the IPython hooks system `complete_command`.
145 currently the only API supported by the IPython hooks system `complete_command`.
146
146
147 To distinguish between matcher versions ``matcher_api_version`` attribute is used.
147 To distinguish between matcher versions ``matcher_api_version`` attribute is used.
148 More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
148 More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
149 and requires a literal ``2`` for v2 Matchers.
149 and requires a literal ``2`` for v2 Matchers.
150
150
151 Once the API stabilises future versions may relax the requirement for specifying
151 Once the API stabilises future versions may relax the requirement for specifying
152 ``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
152 ``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
153 please do not rely on the presence of ``matcher_api_version`` for any purposes.
153 please do not rely on the presence of ``matcher_api_version`` for any purposes.
154
154
155 Suppression of competing matchers
155 Suppression of competing matchers
156 ---------------------------------
156 ---------------------------------
157
157
158 By default results from all matchers are combined, in the order determined by
158 By default results from all matchers are combined, in the order determined by
159 their priority. Matchers can request to suppress results from subsequent
159 their priority. Matchers can request to suppress results from subsequent
160 matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
160 matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
161
161
162 When multiple matchers simultaneously request surpression, the results from of
162 When multiple matchers simultaneously request surpression, the results from of
163 the matcher with higher priority will be returned.
163 the matcher with higher priority will be returned.
164
164
165 Sometimes it is desirable to suppress most but not all other matchers;
165 Sometimes it is desirable to suppress most but not all other matchers;
166 this can be achieved by adding a set of identifiers of matchers which
166 this can be achieved by adding a set of identifiers of matchers which
167 should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
167 should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
168
168
169 The suppression behaviour can is user-configurable via
169 The suppression behaviour can is user-configurable via
170 :std:configtrait:`IPCompleter.suppress_competing_matchers`.
170 :std:configtrait:`IPCompleter.suppress_competing_matchers`.
171 """
171 """
172
172
173
173
174 # Copyright (c) IPython Development Team.
174 # Copyright (c) IPython Development Team.
175 # Distributed under the terms of the Modified BSD License.
175 # Distributed under the terms of the Modified BSD License.
176 #
176 #
177 # Some of this code originated from rlcompleter in the Python standard library
177 # Some of this code originated from rlcompleter in the Python standard library
178 # Copyright (C) 2001 Python Software Foundation, www.python.org
178 # Copyright (C) 2001 Python Software Foundation, www.python.org
179
179
180 from __future__ import annotations
180 from __future__ import annotations
181 import builtins as builtin_mod
181 import builtins as builtin_mod
182 import enum
182 import enum
183 import glob
183 import glob
184 import inspect
184 import inspect
185 import itertools
185 import itertools
186 import keyword
186 import keyword
187 import os
187 import os
188 import re
188 import re
189 import string
189 import string
190 import sys
190 import sys
191 import tokenize
191 import tokenize
192 import time
192 import time
193 import unicodedata
193 import unicodedata
194 import uuid
194 import uuid
195 import warnings
195 import warnings
196 from ast import literal_eval
196 from ast import literal_eval
197 from collections import defaultdict
197 from collections import defaultdict
198 from contextlib import contextmanager
198 from contextlib import contextmanager
199 from dataclasses import dataclass
199 from dataclasses import dataclass
200 from functools import cached_property, partial
200 from functools import cached_property, partial
201 from types import SimpleNamespace
201 from types import SimpleNamespace
202 from typing import (
202 from typing import (
203 Iterable,
203 Iterable,
204 Iterator,
204 Iterator,
205 List,
205 List,
206 Tuple,
206 Tuple,
207 Union,
207 Union,
208 Any,
208 Any,
209 Sequence,
209 Sequence,
210 Dict,
210 Dict,
211 Optional,
211 Optional,
212 TYPE_CHECKING,
212 TYPE_CHECKING,
213 Set,
213 Set,
214 Sized,
214 Sized,
215 TypeVar,
215 TypeVar,
216 Literal,
216 Literal,
217 )
217 )
218
218
219 from IPython.core.guarded_eval import guarded_eval, EvaluationContext
219 from IPython.core.guarded_eval import guarded_eval, EvaluationContext
220 from IPython.core.error import TryNext
220 from IPython.core.error import TryNext
221 from IPython.core.inputtransformer2 import ESC_MAGIC
221 from IPython.core.inputtransformer2 import ESC_MAGIC
222 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
222 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
223 from IPython.core.oinspect import InspectColors
223 from IPython.core.oinspect import InspectColors
224 from IPython.testing.skipdoctest import skip_doctest
224 from IPython.testing.skipdoctest import skip_doctest
225 from IPython.utils import generics
225 from IPython.utils import generics
226 from IPython.utils.decorators import sphinx_options
226 from IPython.utils.decorators import sphinx_options
227 from IPython.utils.dir2 import dir2, get_real_method
227 from IPython.utils.dir2 import dir2, get_real_method
228 from IPython.utils.docs import GENERATING_DOCUMENTATION
228 from IPython.utils.docs import GENERATING_DOCUMENTATION
229 from IPython.utils.path import ensure_dir_exists
229 from IPython.utils.path import ensure_dir_exists
230 from IPython.utils.process import arg_split
230 from IPython.utils.process import arg_split
231 from traitlets import (
231 from traitlets import (
232 Bool,
232 Bool,
233 Enum,
233 Enum,
234 Int,
234 Int,
235 List as ListTrait,
235 List as ListTrait,
236 Unicode,
236 Unicode,
237 Dict as DictTrait,
237 Dict as DictTrait,
238 Union as UnionTrait,
238 Union as UnionTrait,
239 observe,
239 observe,
240 )
240 )
241 from traitlets.config.configurable import Configurable
241 from traitlets.config.configurable import Configurable
242
242
243 import __main__
243 import __main__
244
244
245 # skip module docstests
245 # skip module docstests
246 __skip_doctest__ = True
246 __skip_doctest__ = True
247
247
248
248
249 try:
249 try:
250 import jedi
250 import jedi
251 jedi.settings.case_insensitive_completion = False
251 jedi.settings.case_insensitive_completion = False
252 import jedi.api.helpers
252 import jedi.api.helpers
253 import jedi.api.classes
253 import jedi.api.classes
254 JEDI_INSTALLED = True
254 JEDI_INSTALLED = True
255 except ImportError:
255 except ImportError:
256 JEDI_INSTALLED = False
256 JEDI_INSTALLED = False
257
257
258
258
259 if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
259 if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
260 from typing import cast
260 from typing import cast
261 from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
261 from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
262 else:
262 else:
263 from typing import Generic
263 from typing import Generic
264
264
265 def cast(type_, obj):
265 def cast(type_, obj):
266 """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
266 """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
267 return obj
267 return obj
268
268
269 # do not require on runtime
269 # do not require on runtime
270 NotRequired = Tuple # requires Python >=3.11
270 NotRequired = Tuple # requires Python >=3.11
271 TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
271 TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
272 Protocol = object # requires Python >=3.8
272 Protocol = object # requires Python >=3.8
273 TypeAlias = Any # requires Python >=3.10
273 TypeAlias = Any # requires Python >=3.10
274 TypeGuard = Generic # requires Python >=3.10
274 TypeGuard = Generic # requires Python >=3.10
275 if GENERATING_DOCUMENTATION:
275 if GENERATING_DOCUMENTATION:
276 from typing import TypedDict
276 from typing import TypedDict
277
277
278 # -----------------------------------------------------------------------------
278 # -----------------------------------------------------------------------------
279 # Globals
279 # Globals
280 #-----------------------------------------------------------------------------
280 #-----------------------------------------------------------------------------
281
281
282 # ranges where we have most of the valid unicode names. We could be more finer
282 # ranges where we have most of the valid unicode names. We could be more finer
283 # grained but is it worth it for performance While unicode have character in the
283 # grained but is it worth it for performance While unicode have character in the
284 # range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
284 # range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
285 # write this). With below range we cover them all, with a density of ~67%
285 # write this). With below range we cover them all, with a density of ~67%
286 # biggest next gap we consider only adds up about 1% density and there are 600
286 # biggest next gap we consider only adds up about 1% density and there are 600
287 # gaps that would need hard coding.
287 # gaps that would need hard coding.
288 _UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
288 _UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
289
289
290 # Public API
290 # Public API
291 __all__ = ["Completer", "IPCompleter"]
291 __all__ = ["Completer", "IPCompleter"]
292
292
293 if sys.platform == 'win32':
293 if sys.platform == 'win32':
294 PROTECTABLES = ' '
294 PROTECTABLES = ' '
295 else:
295 else:
296 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
296 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
297
297
298 # Protect against returning an enormous number of completions which the frontend
298 # Protect against returning an enormous number of completions which the frontend
299 # may have trouble processing.
299 # may have trouble processing.
300 MATCHES_LIMIT = 500
300 MATCHES_LIMIT = 500
301
301
302 # Completion type reported when no type can be inferred.
302 # Completion type reported when no type can be inferred.
303 _UNKNOWN_TYPE = "<unknown>"
303 _UNKNOWN_TYPE = "<unknown>"
304
304
305 # sentinel value to signal lack of a match
305 # sentinel value to signal lack of a match
306 not_found = object()
306 not_found = object()
307
307
308 class ProvisionalCompleterWarning(FutureWarning):
308 class ProvisionalCompleterWarning(FutureWarning):
309 """
309 """
310 Exception raise by an experimental feature in this module.
310 Exception raise by an experimental feature in this module.
311
311
312 Wrap code in :any:`provisionalcompleter` context manager if you
312 Wrap code in :any:`provisionalcompleter` context manager if you
313 are certain you want to use an unstable feature.
313 are certain you want to use an unstable feature.
314 """
314 """
315 pass
315 pass
316
316
317 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
317 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
318
318
319
319
320 @skip_doctest
320 @skip_doctest
321 @contextmanager
321 @contextmanager
322 def provisionalcompleter(action='ignore'):
322 def provisionalcompleter(action='ignore'):
323 """
323 """
324 This context manager has to be used in any place where unstable completer
324 This context manager has to be used in any place where unstable completer
325 behavior and API may be called.
325 behavior and API may be called.
326
326
327 >>> with provisionalcompleter():
327 >>> with provisionalcompleter():
328 ... completer.do_experimental_things() # works
328 ... completer.do_experimental_things() # works
329
329
330 >>> completer.do_experimental_things() # raises.
330 >>> completer.do_experimental_things() # raises.
331
331
332 .. note::
332 .. note::
333
333
334 Unstable
334 Unstable
335
335
336 By using this context manager you agree that the API in use may change
336 By using this context manager you agree that the API in use may change
337 without warning, and that you won't complain if they do so.
337 without warning, and that you won't complain if they do so.
338
338
339 You also understand that, if the API is not to your liking, you should report
339 You also understand that, if the API is not to your liking, you should report
340 a bug to explain your use case upstream.
340 a bug to explain your use case upstream.
341
341
342 We'll be happy to get your feedback, feature requests, and improvements on
342 We'll be happy to get your feedback, feature requests, and improvements on
343 any of the unstable APIs!
343 any of the unstable APIs!
344 """
344 """
345 with warnings.catch_warnings():
345 with warnings.catch_warnings():
346 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
346 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
347 yield
347 yield
348
348
349
349
350 def has_open_quotes(s):
350 def has_open_quotes(s):
351 """Return whether a string has open quotes.
351 """Return whether a string has open quotes.
352
352
353 This simply counts whether the number of quote characters of either type in
353 This simply counts whether the number of quote characters of either type in
354 the string is odd.
354 the string is odd.
355
355
356 Returns
356 Returns
357 -------
357 -------
358 If there is an open quote, the quote character is returned. Else, return
358 If there is an open quote, the quote character is returned. Else, return
359 False.
359 False.
360 """
360 """
361 # We check " first, then ', so complex cases with nested quotes will get
361 # We check " first, then ', so complex cases with nested quotes will get
362 # the " to take precedence.
362 # the " to take precedence.
363 if s.count('"') % 2:
363 if s.count('"') % 2:
364 return '"'
364 return '"'
365 elif s.count("'") % 2:
365 elif s.count("'") % 2:
366 return "'"
366 return "'"
367 else:
367 else:
368 return False
368 return False
369
369
370
370
371 def protect_filename(s, protectables=PROTECTABLES):
371 def protect_filename(s, protectables=PROTECTABLES):
372 """Escape a string to protect certain characters."""
372 """Escape a string to protect certain characters."""
373 if set(s) & set(protectables):
373 if set(s) & set(protectables):
374 if sys.platform == "win32":
374 if sys.platform == "win32":
375 return '"' + s + '"'
375 return '"' + s + '"'
376 else:
376 else:
377 return "".join(("\\" + c if c in protectables else c) for c in s)
377 return "".join(("\\" + c if c in protectables else c) for c in s)
378 else:
378 else:
379 return s
379 return s
380
380
381
381
382 def expand_user(path:str) -> Tuple[str, bool, str]:
382 def expand_user(path:str) -> Tuple[str, bool, str]:
383 """Expand ``~``-style usernames in strings.
383 """Expand ``~``-style usernames in strings.
384
384
385 This is similar to :func:`os.path.expanduser`, but it computes and returns
385 This is similar to :func:`os.path.expanduser`, but it computes and returns
386 extra information that will be useful if the input was being used in
386 extra information that will be useful if the input was being used in
387 computing completions, and you wish to return the completions with the
387 computing completions, and you wish to return the completions with the
388 original '~' instead of its expanded value.
388 original '~' instead of its expanded value.
389
389
390 Parameters
390 Parameters
391 ----------
391 ----------
392 path : str
392 path : str
393 String to be expanded. If no ~ is present, the output is the same as the
393 String to be expanded. If no ~ is present, the output is the same as the
394 input.
394 input.
395
395
396 Returns
396 Returns
397 -------
397 -------
398 newpath : str
398 newpath : str
399 Result of ~ expansion in the input path.
399 Result of ~ expansion in the input path.
400 tilde_expand : bool
400 tilde_expand : bool
401 Whether any expansion was performed or not.
401 Whether any expansion was performed or not.
402 tilde_val : str
402 tilde_val : str
403 The value that ~ was replaced with.
403 The value that ~ was replaced with.
404 """
404 """
405 # Default values
405 # Default values
406 tilde_expand = False
406 tilde_expand = False
407 tilde_val = ''
407 tilde_val = ''
408 newpath = path
408 newpath = path
409
409
410 if path.startswith('~'):
410 if path.startswith('~'):
411 tilde_expand = True
411 tilde_expand = True
412 rest = len(path)-1
412 rest = len(path)-1
413 newpath = os.path.expanduser(path)
413 newpath = os.path.expanduser(path)
414 if rest:
414 if rest:
415 tilde_val = newpath[:-rest]
415 tilde_val = newpath[:-rest]
416 else:
416 else:
417 tilde_val = newpath
417 tilde_val = newpath
418
418
419 return newpath, tilde_expand, tilde_val
419 return newpath, tilde_expand, tilde_val
420
420
421
421
422 def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
422 def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
423 """Does the opposite of expand_user, with its outputs.
423 """Does the opposite of expand_user, with its outputs.
424 """
424 """
425 if tilde_expand:
425 if tilde_expand:
426 return path.replace(tilde_val, '~')
426 return path.replace(tilde_val, '~')
427 else:
427 else:
428 return path
428 return path
429
429
430
430
431 def completions_sorting_key(word):
431 def completions_sorting_key(word):
432 """key for sorting completions
432 """key for sorting completions
433
433
434 This does several things:
434 This does several things:
435
435
436 - Demote any completions starting with underscores to the end
436 - Demote any completions starting with underscores to the end
437 - Insert any %magic and %%cellmagic completions in the alphabetical order
437 - Insert any %magic and %%cellmagic completions in the alphabetical order
438 by their name
438 by their name
439 """
439 """
440 prio1, prio2 = 0, 0
440 prio1, prio2 = 0, 0
441
441
442 if word.startswith('__'):
442 if word.startswith('__'):
443 prio1 = 2
443 prio1 = 2
444 elif word.startswith('_'):
444 elif word.startswith('_'):
445 prio1 = 1
445 prio1 = 1
446
446
447 if word.endswith('='):
447 if word.endswith('='):
448 prio1 = -1
448 prio1 = -1
449
449
450 if word.startswith('%%'):
450 if word.startswith('%%'):
451 # If there's another % in there, this is something else, so leave it alone
451 # If there's another % in there, this is something else, so leave it alone
452 if not "%" in word[2:]:
452 if not "%" in word[2:]:
453 word = word[2:]
453 word = word[2:]
454 prio2 = 2
454 prio2 = 2
455 elif word.startswith('%'):
455 elif word.startswith('%'):
456 if not "%" in word[1:]:
456 if not "%" in word[1:]:
457 word = word[1:]
457 word = word[1:]
458 prio2 = 1
458 prio2 = 1
459
459
460 return prio1, word, prio2
460 return prio1, word, prio2
461
461
462
462
463 class _FakeJediCompletion:
463 class _FakeJediCompletion:
464 """
464 """
465 This is a workaround to communicate to the UI that Jedi has crashed and to
465 This is a workaround to communicate to the UI that Jedi has crashed and to
466 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
466 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
467
467
468 Added in IPython 6.0 so should likely be removed for 7.0
468 Added in IPython 6.0 so should likely be removed for 7.0
469
469
470 """
470 """
471
471
472 def __init__(self, name):
472 def __init__(self, name):
473
473
474 self.name = name
474 self.name = name
475 self.complete = name
475 self.complete = name
476 self.type = 'crashed'
476 self.type = 'crashed'
477 self.name_with_symbols = name
477 self.name_with_symbols = name
478 self.signature = ""
478 self.signature = ""
479 self._origin = "fake"
479 self._origin = "fake"
480 self.text = "crashed"
480 self.text = "crashed"
481
481
482 def __repr__(self):
482 def __repr__(self):
483 return '<Fake completion object jedi has crashed>'
483 return '<Fake completion object jedi has crashed>'
484
484
485
485
486 _JediCompletionLike = Union["jedi.api.Completion", _FakeJediCompletion]
486 _JediCompletionLike = Union["jedi.api.Completion", _FakeJediCompletion]
487
487
488
488
489 class Completion:
489 class Completion:
490 """
490 """
491 Completion object used and returned by IPython completers.
491 Completion object used and returned by IPython completers.
492
492
493 .. warning::
493 .. warning::
494
494
495 Unstable
495 Unstable
496
496
497 This function is unstable, API may change without warning.
497 This function is unstable, API may change without warning.
498 It will also raise unless use in proper context manager.
498 It will also raise unless use in proper context manager.
499
499
500 This act as a middle ground :any:`Completion` object between the
500 This act as a middle ground :any:`Completion` object between the
501 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
501 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
502 object. While Jedi need a lot of information about evaluator and how the
502 object. While Jedi need a lot of information about evaluator and how the
503 code should be ran/inspected, PromptToolkit (and other frontend) mostly
503 code should be ran/inspected, PromptToolkit (and other frontend) mostly
504 need user facing information.
504 need user facing information.
505
505
506 - Which range should be replaced replaced by what.
506 - Which range should be replaced replaced by what.
507 - Some metadata (like completion type), or meta information to displayed to
507 - Some metadata (like completion type), or meta information to displayed to
508 the use user.
508 the use user.
509
509
510 For debugging purpose we can also store the origin of the completion (``jedi``,
510 For debugging purpose we can also store the origin of the completion (``jedi``,
511 ``IPython.python_matches``, ``IPython.magics_matches``...).
511 ``IPython.python_matches``, ``IPython.magics_matches``...).
512 """
512 """
513
513
514 __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
514 __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
515
515
516 def __init__(
516 def __init__(
517 self,
517 self,
518 start: int,
518 start: int,
519 end: int,
519 end: int,
520 text: str,
520 text: str,
521 *,
521 *,
522 type: Optional[str] = None,
522 type: Optional[str] = None,
523 _origin="",
523 _origin="",
524 signature="",
524 signature="",
525 ) -> None:
525 ) -> None:
526 warnings.warn(
526 warnings.warn(
527 "``Completion`` is a provisional API (as of IPython 6.0). "
527 "``Completion`` is a provisional API (as of IPython 6.0). "
528 "It may change without warnings. "
528 "It may change without warnings. "
529 "Use in corresponding context manager.",
529 "Use in corresponding context manager.",
530 category=ProvisionalCompleterWarning,
530 category=ProvisionalCompleterWarning,
531 stacklevel=2,
531 stacklevel=2,
532 )
532 )
533
533
534 self.start = start
534 self.start = start
535 self.end = end
535 self.end = end
536 self.text = text
536 self.text = text
537 self.type = type
537 self.type = type
538 self.signature = signature
538 self.signature = signature
539 self._origin = _origin
539 self._origin = _origin
540
540
541 def __repr__(self):
541 def __repr__(self):
542 return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
542 return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
543 (self.start, self.end, self.text, self.type or '?', self.signature or '?')
543 (self.start, self.end, self.text, self.type or '?', self.signature or '?')
544
544
545 def __eq__(self, other) -> bool:
545 def __eq__(self, other) -> bool:
546 """
546 """
547 Equality and hash do not hash the type (as some completer may not be
547 Equality and hash do not hash the type (as some completer may not be
548 able to infer the type), but are use to (partially) de-duplicate
548 able to infer the type), but are use to (partially) de-duplicate
549 completion.
549 completion.
550
550
551 Completely de-duplicating completion is a bit tricker that just
551 Completely de-duplicating completion is a bit tricker that just
552 comparing as it depends on surrounding text, which Completions are not
552 comparing as it depends on surrounding text, which Completions are not
553 aware of.
553 aware of.
554 """
554 """
555 return self.start == other.start and \
555 return self.start == other.start and \
556 self.end == other.end and \
556 self.end == other.end and \
557 self.text == other.text
557 self.text == other.text
558
558
559 def __hash__(self):
559 def __hash__(self):
560 return hash((self.start, self.end, self.text))
560 return hash((self.start, self.end, self.text))
561
561
562
562
563 class SimpleCompletion:
563 class SimpleCompletion:
564 """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
564 """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
565
565
566 .. warning::
566 .. warning::
567
567
568 Provisional
568 Provisional
569
569
570 This class is used to describe the currently supported attributes of
570 This class is used to describe the currently supported attributes of
571 simple completion items, and any additional implementation details
571 simple completion items, and any additional implementation details
572 should not be relied on. Additional attributes may be included in
572 should not be relied on. Additional attributes may be included in
573 future versions, and meaning of text disambiguated from the current
573 future versions, and meaning of text disambiguated from the current
574 dual meaning of "text to insert" and "text to used as a label".
574 dual meaning of "text to insert" and "text to used as a label".
575 """
575 """
576
576
577 __slots__ = ["text", "type"]
577 __slots__ = ["text", "type"]
578
578
579 def __init__(self, text: str, *, type: Optional[str] = None):
579 def __init__(self, text: str, *, type: Optional[str] = None):
580 self.text = text
580 self.text = text
581 self.type = type
581 self.type = type
582
582
583 def __repr__(self):
583 def __repr__(self):
584 return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
584 return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
585
585
586
586
587 class _MatcherResultBase(TypedDict):
587 class _MatcherResultBase(TypedDict):
588 """Definition of dictionary to be returned by new-style Matcher (API v2)."""
588 """Definition of dictionary to be returned by new-style Matcher (API v2)."""
589
589
590 #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
590 #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
591 matched_fragment: NotRequired[str]
591 matched_fragment: NotRequired[str]
592
592
593 #: Whether to suppress results from all other matchers (True), some
593 #: Whether to suppress results from all other matchers (True), some
594 #: matchers (set of identifiers) or none (False); default is False.
594 #: matchers (set of identifiers) or none (False); default is False.
595 suppress: NotRequired[Union[bool, Set[str]]]
595 suppress: NotRequired[Union[bool, Set[str]]]
596
596
597 #: Identifiers of matchers which should NOT be suppressed when this matcher
597 #: Identifiers of matchers which should NOT be suppressed when this matcher
598 #: requests to suppress all other matchers; defaults to an empty set.
598 #: requests to suppress all other matchers; defaults to an empty set.
599 do_not_suppress: NotRequired[Set[str]]
599 do_not_suppress: NotRequired[Set[str]]
600
600
601 #: Are completions already ordered and should be left as-is? default is False.
601 #: Are completions already ordered and should be left as-is? default is False.
602 ordered: NotRequired[bool]
602 ordered: NotRequired[bool]
603
603
604
604
605 @sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
605 @sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
606 class SimpleMatcherResult(_MatcherResultBase, TypedDict):
606 class SimpleMatcherResult(_MatcherResultBase, TypedDict):
607 """Result of new-style completion matcher."""
607 """Result of new-style completion matcher."""
608
608
609 # note: TypedDict is added again to the inheritance chain
609 # note: TypedDict is added again to the inheritance chain
610 # in order to get __orig_bases__ for documentation
610 # in order to get __orig_bases__ for documentation
611
611
612 #: List of candidate completions
612 #: List of candidate completions
613 completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
613 completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
614
614
615
615
616 class _JediMatcherResult(_MatcherResultBase):
616 class _JediMatcherResult(_MatcherResultBase):
617 """Matching result returned by Jedi (will be processed differently)"""
617 """Matching result returned by Jedi (will be processed differently)"""
618
618
619 #: list of candidate completions
619 #: list of candidate completions
620 completions: Iterator[_JediCompletionLike]
620 completions: Iterator[_JediCompletionLike]
621
621
622
622
623 AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
623 AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
624 AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
624 AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
625
625
626
626
627 @dataclass
627 @dataclass
628 class CompletionContext:
628 class CompletionContext:
629 """Completion context provided as an argument to matchers in the Matcher API v2."""
629 """Completion context provided as an argument to matchers in the Matcher API v2."""
630
630
631 # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
631 # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
632 # which was not explicitly visible as an argument of the matcher, making any refactor
632 # which was not explicitly visible as an argument of the matcher, making any refactor
633 # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
633 # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
634 # from the completer, and make substituting them in sub-classes easier.
634 # from the completer, and make substituting them in sub-classes easier.
635
635
636 #: Relevant fragment of code directly preceding the cursor.
636 #: Relevant fragment of code directly preceding the cursor.
637 #: The extraction of token is implemented via splitter heuristic
637 #: The extraction of token is implemented via splitter heuristic
638 #: (following readline behaviour for legacy reasons), which is user configurable
638 #: (following readline behaviour for legacy reasons), which is user configurable
639 #: (by switching the greedy mode).
639 #: (by switching the greedy mode).
640 token: str
640 token: str
641
641
642 #: The full available content of the editor or buffer
642 #: The full available content of the editor or buffer
643 full_text: str
643 full_text: str
644
644
645 #: Cursor position in the line (the same for ``full_text`` and ``text``).
645 #: Cursor position in the line (the same for ``full_text`` and ``text``).
646 cursor_position: int
646 cursor_position: int
647
647
648 #: Cursor line in ``full_text``.
648 #: Cursor line in ``full_text``.
649 cursor_line: int
649 cursor_line: int
650
650
651 #: The maximum number of completions that will be used downstream.
651 #: The maximum number of completions that will be used downstream.
652 #: Matchers can use this information to abort early.
652 #: Matchers can use this information to abort early.
653 #: The built-in Jedi matcher is currently excepted from this limit.
653 #: The built-in Jedi matcher is currently excepted from this limit.
654 # If not given, return all possible completions.
654 # If not given, return all possible completions.
655 limit: Optional[int]
655 limit: Optional[int]
656
656
657 @cached_property
657 @cached_property
658 def text_until_cursor(self) -> str:
658 def text_until_cursor(self) -> str:
659 return self.line_with_cursor[: self.cursor_position]
659 return self.line_with_cursor[: self.cursor_position]
660
660
661 @cached_property
661 @cached_property
662 def line_with_cursor(self) -> str:
662 def line_with_cursor(self) -> str:
663 return self.full_text.split("\n")[self.cursor_line]
663 return self.full_text.split("\n")[self.cursor_line]
664
664
665
665
666 #: Matcher results for API v2.
666 #: Matcher results for API v2.
667 MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
667 MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
668
668
669
669
670 class _MatcherAPIv1Base(Protocol):
670 class _MatcherAPIv1Base(Protocol):
671 def __call__(self, text: str) -> List[str]:
671 def __call__(self, text: str) -> List[str]:
672 """Call signature."""
672 """Call signature."""
673 ...
673 ...
674
674
675 #: Used to construct the default matcher identifier
675 #: Used to construct the default matcher identifier
676 __qualname__: str
676 __qualname__: str
677
677
678
678
679 class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
679 class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
680 #: API version
680 #: API version
681 matcher_api_version: Optional[Literal[1]]
681 matcher_api_version: Optional[Literal[1]]
682
682
683 def __call__(self, text: str) -> List[str]:
683 def __call__(self, text: str) -> List[str]:
684 """Call signature."""
684 """Call signature."""
685 ...
685 ...
686
686
687
687
688 #: Protocol describing Matcher API v1.
688 #: Protocol describing Matcher API v1.
689 MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
689 MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
690
690
691
691
692 class MatcherAPIv2(Protocol):
692 class MatcherAPIv2(Protocol):
693 """Protocol describing Matcher API v2."""
693 """Protocol describing Matcher API v2."""
694
694
695 #: API version
695 #: API version
696 matcher_api_version: Literal[2] = 2
696 matcher_api_version: Literal[2] = 2
697
697
698 def __call__(self, context: CompletionContext) -> MatcherResult:
698 def __call__(self, context: CompletionContext) -> MatcherResult:
699 """Call signature."""
699 """Call signature."""
700 ...
700 ...
701
701
702 #: Used to construct the default matcher identifier
702 #: Used to construct the default matcher identifier
703 __qualname__: str
703 __qualname__: str
704
704
705
705
706 Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
706 Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
707
707
708
708
709 def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
709 def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
710 api_version = _get_matcher_api_version(matcher)
710 api_version = _get_matcher_api_version(matcher)
711 return api_version == 1
711 return api_version == 1
712
712
713
713
714 def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
714 def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
715 api_version = _get_matcher_api_version(matcher)
715 api_version = _get_matcher_api_version(matcher)
716 return api_version == 2
716 return api_version == 2
717
717
718
718
719 def _is_sizable(value: Any) -> TypeGuard[Sized]:
719 def _is_sizable(value: Any) -> TypeGuard[Sized]:
720 """Determines whether objects is sizable"""
720 """Determines whether objects is sizable"""
721 return hasattr(value, "__len__")
721 return hasattr(value, "__len__")
722
722
723
723
724 def _is_iterator(value: Any) -> TypeGuard[Iterator]:
724 def _is_iterator(value: Any) -> TypeGuard[Iterator]:
725 """Determines whether objects is sizable"""
725 """Determines whether objects is sizable"""
726 return hasattr(value, "__next__")
726 return hasattr(value, "__next__")
727
727
728
728
729 def has_any_completions(result: MatcherResult) -> bool:
729 def has_any_completions(result: MatcherResult) -> bool:
730 """Check if any result includes any completions."""
730 """Check if any result includes any completions."""
731 completions = result["completions"]
731 completions = result["completions"]
732 if _is_sizable(completions):
732 if _is_sizable(completions):
733 return len(completions) != 0
733 return len(completions) != 0
734 if _is_iterator(completions):
734 if _is_iterator(completions):
735 try:
735 try:
736 old_iterator = completions
736 old_iterator = completions
737 first = next(old_iterator)
737 first = next(old_iterator)
738 result["completions"] = cast(
738 result["completions"] = cast(
739 Iterator[SimpleCompletion],
739 Iterator[SimpleCompletion],
740 itertools.chain([first], old_iterator),
740 itertools.chain([first], old_iterator),
741 )
741 )
742 return True
742 return True
743 except StopIteration:
743 except StopIteration:
744 return False
744 return False
745 raise ValueError(
745 raise ValueError(
746 "Completions returned by matcher need to be an Iterator or a Sizable"
746 "Completions returned by matcher need to be an Iterator or a Sizable"
747 )
747 )
748
748
749
749
750 def completion_matcher(
750 def completion_matcher(
751 *,
751 *,
752 priority: Optional[float] = None,
752 priority: Optional[float] = None,
753 identifier: Optional[str] = None,
753 identifier: Optional[str] = None,
754 api_version: int = 1,
754 api_version: int = 1,
755 ):
755 ):
756 """Adds attributes describing the matcher.
756 """Adds attributes describing the matcher.
757
757
758 Parameters
758 Parameters
759 ----------
759 ----------
760 priority : Optional[float]
760 priority : Optional[float]
761 The priority of the matcher, determines the order of execution of matchers.
761 The priority of the matcher, determines the order of execution of matchers.
762 Higher priority means that the matcher will be executed first. Defaults to 0.
762 Higher priority means that the matcher will be executed first. Defaults to 0.
763 identifier : Optional[str]
763 identifier : Optional[str]
764 identifier of the matcher allowing users to modify the behaviour via traitlets,
764 identifier of the matcher allowing users to modify the behaviour via traitlets,
765 and also used to for debugging (will be passed as ``origin`` with the completions).
765 and also used to for debugging (will be passed as ``origin`` with the completions).
766
766
767 Defaults to matcher function's ``__qualname__`` (for example,
767 Defaults to matcher function's ``__qualname__`` (for example,
768 ``IPCompleter.file_matcher`` for the built-in matched defined
768 ``IPCompleter.file_matcher`` for the built-in matched defined
769 as a ``file_matcher`` method of the ``IPCompleter`` class).
769 as a ``file_matcher`` method of the ``IPCompleter`` class).
770 api_version: Optional[int]
770 api_version: Optional[int]
771 version of the Matcher API used by this matcher.
771 version of the Matcher API used by this matcher.
772 Currently supported values are 1 and 2.
772 Currently supported values are 1 and 2.
773 Defaults to 1.
773 Defaults to 1.
774 """
774 """
775
775
776 def wrapper(func: Matcher):
776 def wrapper(func: Matcher):
777 func.matcher_priority = priority or 0 # type: ignore
777 func.matcher_priority = priority or 0 # type: ignore
778 func.matcher_identifier = identifier or func.__qualname__ # type: ignore
778 func.matcher_identifier = identifier or func.__qualname__ # type: ignore
779 func.matcher_api_version = api_version # type: ignore
779 func.matcher_api_version = api_version # type: ignore
780 if TYPE_CHECKING:
780 if TYPE_CHECKING:
781 if api_version == 1:
781 if api_version == 1:
782 func = cast(MatcherAPIv1, func)
782 func = cast(MatcherAPIv1, func)
783 elif api_version == 2:
783 elif api_version == 2:
784 func = cast(MatcherAPIv2, func)
784 func = cast(MatcherAPIv2, func)
785 return func
785 return func
786
786
787 return wrapper
787 return wrapper
788
788
789
789
790 def _get_matcher_priority(matcher: Matcher):
790 def _get_matcher_priority(matcher: Matcher):
791 return getattr(matcher, "matcher_priority", 0)
791 return getattr(matcher, "matcher_priority", 0)
792
792
793
793
794 def _get_matcher_id(matcher: Matcher):
794 def _get_matcher_id(matcher: Matcher):
795 return getattr(matcher, "matcher_identifier", matcher.__qualname__)
795 return getattr(matcher, "matcher_identifier", matcher.__qualname__)
796
796
797
797
798 def _get_matcher_api_version(matcher):
798 def _get_matcher_api_version(matcher):
799 return getattr(matcher, "matcher_api_version", 1)
799 return getattr(matcher, "matcher_api_version", 1)
800
800
801
801
802 context_matcher = partial(completion_matcher, api_version=2)
802 context_matcher = partial(completion_matcher, api_version=2)
803
803
804
804
805 _IC = Iterable[Completion]
805 _IC = Iterable[Completion]
806
806
807
807
808 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
808 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
809 """
809 """
810 Deduplicate a set of completions.
810 Deduplicate a set of completions.
811
811
812 .. warning::
812 .. warning::
813
813
814 Unstable
814 Unstable
815
815
816 This function is unstable, API may change without warning.
816 This function is unstable, API may change without warning.
817
817
818 Parameters
818 Parameters
819 ----------
819 ----------
820 text : str
820 text : str
821 text that should be completed.
821 text that should be completed.
822 completions : Iterator[Completion]
822 completions : Iterator[Completion]
823 iterator over the completions to deduplicate
823 iterator over the completions to deduplicate
824
824
825 Yields
825 Yields
826 ------
826 ------
827 `Completions` objects
827 `Completions` objects
828 Completions coming from multiple sources, may be different but end up having
828 Completions coming from multiple sources, may be different but end up having
829 the same effect when applied to ``text``. If this is the case, this will
829 the same effect when applied to ``text``. If this is the case, this will
830 consider completions as equal and only emit the first encountered.
830 consider completions as equal and only emit the first encountered.
831 Not folded in `completions()` yet for debugging purpose, and to detect when
831 Not folded in `completions()` yet for debugging purpose, and to detect when
832 the IPython completer does return things that Jedi does not, but should be
832 the IPython completer does return things that Jedi does not, but should be
833 at some point.
833 at some point.
834 """
834 """
835 completions = list(completions)
835 completions = list(completions)
836 if not completions:
836 if not completions:
837 return
837 return
838
838
839 new_start = min(c.start for c in completions)
839 new_start = min(c.start for c in completions)
840 new_end = max(c.end for c in completions)
840 new_end = max(c.end for c in completions)
841
841
842 seen = set()
842 seen = set()
843 for c in completions:
843 for c in completions:
844 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
844 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
845 if new_text not in seen:
845 if new_text not in seen:
846 yield c
846 yield c
847 seen.add(new_text)
847 seen.add(new_text)
848
848
849
849
850 def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
850 def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
851 """
851 """
852 Rectify a set of completions to all have the same ``start`` and ``end``
852 Rectify a set of completions to all have the same ``start`` and ``end``
853
853
854 .. warning::
854 .. warning::
855
855
856 Unstable
856 Unstable
857
857
858 This function is unstable, API may change without warning.
858 This function is unstable, API may change without warning.
859 It will also raise unless use in proper context manager.
859 It will also raise unless use in proper context manager.
860
860
861 Parameters
861 Parameters
862 ----------
862 ----------
863 text : str
863 text : str
864 text that should be completed.
864 text that should be completed.
865 completions : Iterator[Completion]
865 completions : Iterator[Completion]
866 iterator over the completions to rectify
866 iterator over the completions to rectify
867 _debug : bool
867 _debug : bool
868 Log failed completion
868 Log failed completion
869
869
870 Notes
870 Notes
871 -----
871 -----
872 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
872 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
873 the Jupyter Protocol requires them to behave like so. This will readjust
873 the Jupyter Protocol requires them to behave like so. This will readjust
874 the completion to have the same ``start`` and ``end`` by padding both
874 the completion to have the same ``start`` and ``end`` by padding both
875 extremities with surrounding text.
875 extremities with surrounding text.
876
876
877 During stabilisation should support a ``_debug`` option to log which
877 During stabilisation should support a ``_debug`` option to log which
878 completion are return by the IPython completer and not found in Jedi in
878 completion are return by the IPython completer and not found in Jedi in
879 order to make upstream bug report.
879 order to make upstream bug report.
880 """
880 """
881 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
881 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
882 "It may change without warnings. "
882 "It may change without warnings. "
883 "Use in corresponding context manager.",
883 "Use in corresponding context manager.",
884 category=ProvisionalCompleterWarning, stacklevel=2)
884 category=ProvisionalCompleterWarning, stacklevel=2)
885
885
886 completions = list(completions)
886 completions = list(completions)
887 if not completions:
887 if not completions:
888 return
888 return
889 starts = (c.start for c in completions)
889 starts = (c.start for c in completions)
890 ends = (c.end for c in completions)
890 ends = (c.end for c in completions)
891
891
892 new_start = min(starts)
892 new_start = min(starts)
893 new_end = max(ends)
893 new_end = max(ends)
894
894
895 seen_jedi = set()
895 seen_jedi = set()
896 seen_python_matches = set()
896 seen_python_matches = set()
897 for c in completions:
897 for c in completions:
898 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
898 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
899 if c._origin == 'jedi':
899 if c._origin == 'jedi':
900 seen_jedi.add(new_text)
900 seen_jedi.add(new_text)
901 elif c._origin == 'IPCompleter.python_matches':
901 elif c._origin == 'IPCompleter.python_matches':
902 seen_python_matches.add(new_text)
902 seen_python_matches.add(new_text)
903 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
903 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
904 diff = seen_python_matches.difference(seen_jedi)
904 diff = seen_python_matches.difference(seen_jedi)
905 if diff and _debug:
905 if diff and _debug:
906 print('IPython.python matches have extras:', diff)
906 print('IPython.python matches have extras:', diff)
907
907
908
908
909 if sys.platform == 'win32':
909 if sys.platform == 'win32':
910 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
910 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
911 else:
911 else:
912 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
912 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
913
913
914 GREEDY_DELIMS = ' =\r\n'
914 GREEDY_DELIMS = ' =\r\n'
915
915
916
916
917 class CompletionSplitter(object):
917 class CompletionSplitter(object):
918 """An object to split an input line in a manner similar to readline.
918 """An object to split an input line in a manner similar to readline.
919
919
920 By having our own implementation, we can expose readline-like completion in
920 By having our own implementation, we can expose readline-like completion in
921 a uniform manner to all frontends. This object only needs to be given the
921 a uniform manner to all frontends. This object only needs to be given the
922 line of text to be split and the cursor position on said line, and it
922 line of text to be split and the cursor position on said line, and it
923 returns the 'word' to be completed on at the cursor after splitting the
923 returns the 'word' to be completed on at the cursor after splitting the
924 entire line.
924 entire line.
925
925
926 What characters are used as splitting delimiters can be controlled by
926 What characters are used as splitting delimiters can be controlled by
927 setting the ``delims`` attribute (this is a property that internally
927 setting the ``delims`` attribute (this is a property that internally
928 automatically builds the necessary regular expression)"""
928 automatically builds the necessary regular expression)"""
929
929
930 # Private interface
930 # Private interface
931
931
932 # A string of delimiter characters. The default value makes sense for
932 # A string of delimiter characters. The default value makes sense for
933 # IPython's most typical usage patterns.
933 # IPython's most typical usage patterns.
934 _delims = DELIMS
934 _delims = DELIMS
935
935
936 # The expression (a normal string) to be compiled into a regular expression
936 # The expression (a normal string) to be compiled into a regular expression
937 # for actual splitting. We store it as an attribute mostly for ease of
937 # for actual splitting. We store it as an attribute mostly for ease of
938 # debugging, since this type of code can be so tricky to debug.
938 # debugging, since this type of code can be so tricky to debug.
939 _delim_expr = None
939 _delim_expr = None
940
940
941 # The regular expression that does the actual splitting
941 # The regular expression that does the actual splitting
942 _delim_re = None
942 _delim_re = None
943
943
944 def __init__(self, delims=None):
944 def __init__(self, delims=None):
945 delims = CompletionSplitter._delims if delims is None else delims
945 delims = CompletionSplitter._delims if delims is None else delims
946 self.delims = delims
946 self.delims = delims
947
947
948 @property
948 @property
949 def delims(self):
949 def delims(self):
950 """Return the string of delimiter characters."""
950 """Return the string of delimiter characters."""
951 return self._delims
951 return self._delims
952
952
953 @delims.setter
953 @delims.setter
954 def delims(self, delims):
954 def delims(self, delims):
955 """Set the delimiters for line splitting."""
955 """Set the delimiters for line splitting."""
956 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
956 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
957 self._delim_re = re.compile(expr)
957 self._delim_re = re.compile(expr)
958 self._delims = delims
958 self._delims = delims
959 self._delim_expr = expr
959 self._delim_expr = expr
960
960
961 def split_line(self, line, cursor_pos=None):
961 def split_line(self, line, cursor_pos=None):
962 """Split a line of text with a cursor at the given position.
962 """Split a line of text with a cursor at the given position.
963 """
963 """
964 l = line if cursor_pos is None else line[:cursor_pos]
964 l = line if cursor_pos is None else line[:cursor_pos]
965 return self._delim_re.split(l)[-1]
965 return self._delim_re.split(l)[-1]
966
966
967
967
968
968
969 class Completer(Configurable):
969 class Completer(Configurable):
970
970
971 greedy = Bool(
971 greedy = Bool(
972 False,
972 False,
973 help="""Activate greedy completion.
973 help="""Activate greedy completion.
974
974
975 .. deprecated:: 8.8
975 .. deprecated:: 8.8
976 Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
976 Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
977
977
978 When enabled in IPython 8.8 or newer, changes configuration as follows:
978 When enabled in IPython 8.8 or newer, changes configuration as follows:
979
979
980 - ``Completer.evaluation = 'unsafe'``
980 - ``Completer.evaluation = 'unsafe'``
981 - ``Completer.auto_close_dict_keys = True``
981 - ``Completer.auto_close_dict_keys = True``
982 """,
982 """,
983 ).tag(config=True)
983 ).tag(config=True)
984
984
985 evaluation = Enum(
985 evaluation = Enum(
986 ("forbidden", "minimal", "limited", "unsafe", "dangerous"),
986 ("forbidden", "minimal", "limited", "unsafe", "dangerous"),
987 default_value="limited",
987 default_value="limited",
988 help="""Policy for code evaluation under completion.
988 help="""Policy for code evaluation under completion.
989
989
990 Successive options allow to enable more eager evaluation for better
990 Successive options allow to enable more eager evaluation for better
991 completion suggestions, including for nested dictionaries, nested lists,
991 completion suggestions, including for nested dictionaries, nested lists,
992 or even results of function calls.
992 or even results of function calls.
993 Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
993 Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
994 code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
994 code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
995
995
996 Allowed values are:
996 Allowed values are:
997
997
998 - ``forbidden``: no evaluation of code is permitted,
998 - ``forbidden``: no evaluation of code is permitted,
999 - ``minimal``: evaluation of literals and access to built-in namespace;
999 - ``minimal``: evaluation of literals and access to built-in namespace;
1000 no item/attribute evaluationm no access to locals/globals,
1000 no item/attribute evaluationm no access to locals/globals,
1001 no evaluation of any operations or comparisons.
1001 no evaluation of any operations or comparisons.
1002 - ``limited``: access to all namespaces, evaluation of hard-coded methods
1002 - ``limited``: access to all namespaces, evaluation of hard-coded methods
1003 (for example: :any:`dict.keys`, :any:`object.__getattr__`,
1003 (for example: :any:`dict.keys`, :any:`object.__getattr__`,
1004 :any:`object.__getitem__`) on allow-listed objects (for example:
1004 :any:`object.__getitem__`) on allow-listed objects (for example:
1005 :any:`dict`, :any:`list`, :any:`tuple`, ``pandas.Series``),
1005 :any:`dict`, :any:`list`, :any:`tuple`, ``pandas.Series``),
1006 - ``unsafe``: evaluation of all methods and function calls but not of
1006 - ``unsafe``: evaluation of all methods and function calls but not of
1007 syntax with side-effects like `del x`,
1007 syntax with side-effects like `del x`,
1008 - ``dangerous``: completely arbitrary evaluation.
1008 - ``dangerous``: completely arbitrary evaluation.
1009 """,
1009 """,
1010 ).tag(config=True)
1010 ).tag(config=True)
1011
1011
1012 use_jedi = Bool(default_value=JEDI_INSTALLED,
1012 use_jedi = Bool(default_value=JEDI_INSTALLED,
1013 help="Experimental: Use Jedi to generate autocompletions. "
1013 help="Experimental: Use Jedi to generate autocompletions. "
1014 "Default to True if jedi is installed.").tag(config=True)
1014 "Default to True if jedi is installed.").tag(config=True)
1015
1015
1016 jedi_compute_type_timeout = Int(default_value=400,
1016 jedi_compute_type_timeout = Int(default_value=400,
1017 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
1017 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
1018 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
1018 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
1019 performance by preventing jedi to build its cache.
1019 performance by preventing jedi to build its cache.
1020 """).tag(config=True)
1020 """).tag(config=True)
1021
1021
1022 debug = Bool(default_value=False,
1022 debug = Bool(default_value=False,
1023 help='Enable debug for the Completer. Mostly print extra '
1023 help='Enable debug for the Completer. Mostly print extra '
1024 'information for experimental jedi integration.')\
1024 'information for experimental jedi integration.')\
1025 .tag(config=True)
1025 .tag(config=True)
1026
1026
1027 backslash_combining_completions = Bool(True,
1027 backslash_combining_completions = Bool(True,
1028 help="Enable unicode completions, e.g. \\alpha<tab> . "
1028 help="Enable unicode completions, e.g. \\alpha<tab> . "
1029 "Includes completion of latex commands, unicode names, and expanding "
1029 "Includes completion of latex commands, unicode names, and expanding "
1030 "unicode characters back to latex commands.").tag(config=True)
1030 "unicode characters back to latex commands.").tag(config=True)
1031
1031
1032 auto_close_dict_keys = Bool(
1032 auto_close_dict_keys = Bool(
1033 False,
1033 False,
1034 help="""
1034 help="""
1035 Enable auto-closing dictionary keys.
1035 Enable auto-closing dictionary keys.
1036
1036
1037 When enabled string keys will be suffixed with a final quote
1037 When enabled string keys will be suffixed with a final quote
1038 (matching the opening quote), tuple keys will also receive a
1038 (matching the opening quote), tuple keys will also receive a
1039 separating comma if needed, and keys which are final will
1039 separating comma if needed, and keys which are final will
1040 receive a closing bracket (``]``).
1040 receive a closing bracket (``]``).
1041 """,
1041 """,
1042 ).tag(config=True)
1042 ).tag(config=True)
1043
1043
1044 def __init__(self, namespace=None, global_namespace=None, **kwargs):
1044 def __init__(self, namespace=None, global_namespace=None, **kwargs):
1045 """Create a new completer for the command line.
1045 """Create a new completer for the command line.
1046
1046
1047 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
1047 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
1048
1048
1049 If unspecified, the default namespace where completions are performed
1049 If unspecified, the default namespace where completions are performed
1050 is __main__ (technically, __main__.__dict__). Namespaces should be
1050 is __main__ (technically, __main__.__dict__). Namespaces should be
1051 given as dictionaries.
1051 given as dictionaries.
1052
1052
1053 An optional second namespace can be given. This allows the completer
1053 An optional second namespace can be given. This allows the completer
1054 to handle cases where both the local and global scopes need to be
1054 to handle cases where both the local and global scopes need to be
1055 distinguished.
1055 distinguished.
1056 """
1056 """
1057
1057
1058 # Don't bind to namespace quite yet, but flag whether the user wants a
1058 # Don't bind to namespace quite yet, but flag whether the user wants a
1059 # specific namespace or to use __main__.__dict__. This will allow us
1059 # specific namespace or to use __main__.__dict__. This will allow us
1060 # to bind to __main__.__dict__ at completion time, not now.
1060 # to bind to __main__.__dict__ at completion time, not now.
1061 if namespace is None:
1061 if namespace is None:
1062 self.use_main_ns = True
1062 self.use_main_ns = True
1063 else:
1063 else:
1064 self.use_main_ns = False
1064 self.use_main_ns = False
1065 self.namespace = namespace
1065 self.namespace = namespace
1066
1066
1067 # The global namespace, if given, can be bound directly
1067 # The global namespace, if given, can be bound directly
1068 if global_namespace is None:
1068 if global_namespace is None:
1069 self.global_namespace = {}
1069 self.global_namespace = {}
1070 else:
1070 else:
1071 self.global_namespace = global_namespace
1071 self.global_namespace = global_namespace
1072
1072
1073 self.custom_matchers = []
1073 self.custom_matchers = []
1074
1074
1075 super(Completer, self).__init__(**kwargs)
1075 super(Completer, self).__init__(**kwargs)
1076
1076
1077 def complete(self, text, state):
1077 def complete(self, text, state):
1078 """Return the next possible completion for 'text'.
1078 """Return the next possible completion for 'text'.
1079
1079
1080 This is called successively with state == 0, 1, 2, ... until it
1080 This is called successively with state == 0, 1, 2, ... until it
1081 returns None. The completion should begin with 'text'.
1081 returns None. The completion should begin with 'text'.
1082
1082
1083 """
1083 """
1084 if self.use_main_ns:
1084 if self.use_main_ns:
1085 self.namespace = __main__.__dict__
1085 self.namespace = __main__.__dict__
1086
1086
1087 if state == 0:
1087 if state == 0:
1088 if "." in text:
1088 if "." in text:
1089 self.matches = self.attr_matches(text)
1089 self.matches = self.attr_matches(text)
1090 else:
1090 else:
1091 self.matches = self.global_matches(text)
1091 self.matches = self.global_matches(text)
1092 try:
1092 try:
1093 return self.matches[state]
1093 return self.matches[state]
1094 except IndexError:
1094 except IndexError:
1095 return None
1095 return None
1096
1096
1097 def global_matches(self, text):
1097 def global_matches(self, text):
1098 """Compute matches when text is a simple name.
1098 """Compute matches when text is a simple name.
1099
1099
1100 Return a list of all keywords, built-in functions and names currently
1100 Return a list of all keywords, built-in functions and names currently
1101 defined in self.namespace or self.global_namespace that match.
1101 defined in self.namespace or self.global_namespace that match.
1102
1102
1103 """
1103 """
1104 matches = []
1104 matches = []
1105 match_append = matches.append
1105 match_append = matches.append
1106 n = len(text)
1106 n = len(text)
1107 for lst in [
1107 for lst in [
1108 keyword.kwlist,
1108 keyword.kwlist,
1109 builtin_mod.__dict__.keys(),
1109 builtin_mod.__dict__.keys(),
1110 list(self.namespace.keys()),
1110 list(self.namespace.keys()),
1111 list(self.global_namespace.keys()),
1111 list(self.global_namespace.keys()),
1112 ]:
1112 ]:
1113 for word in lst:
1113 for word in lst:
1114 if word[:n] == text and word != "__builtins__":
1114 if word[:n] == text and word != "__builtins__":
1115 match_append(word)
1115 match_append(word)
1116
1116
1117 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
1117 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
1118 for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
1118 for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
1119 shortened = {
1119 shortened = {
1120 "_".join([sub[0] for sub in word.split("_")]): word
1120 "_".join([sub[0] for sub in word.split("_")]): word
1121 for word in lst
1121 for word in lst
1122 if snake_case_re.match(word)
1122 if snake_case_re.match(word)
1123 }
1123 }
1124 for word in shortened.keys():
1124 for word in shortened.keys():
1125 if word[:n] == text and word != "__builtins__":
1125 if word[:n] == text and word != "__builtins__":
1126 match_append(shortened[word])
1126 match_append(shortened[word])
1127 return matches
1127 return matches
1128
1128
1129 def attr_matches(self, text):
1129 def attr_matches(self, text):
1130 """Compute matches when text contains a dot.
1130 """Compute matches when text contains a dot.
1131
1131
1132 Assuming the text is of the form NAME.NAME....[NAME], and is
1132 Assuming the text is of the form NAME.NAME....[NAME], and is
1133 evaluatable in self.namespace or self.global_namespace, it will be
1133 evaluatable in self.namespace or self.global_namespace, it will be
1134 evaluated and its attributes (as revealed by dir()) are used as
1134 evaluated and its attributes (as revealed by dir()) are used as
1135 possible completions. (For class instances, class members are
1135 possible completions. (For class instances, class members are
1136 also considered.)
1136 also considered.)
1137
1137
1138 WARNING: this can still invoke arbitrary C code, if an object
1138 WARNING: this can still invoke arbitrary C code, if an object
1139 with a __getattr__ hook is evaluated.
1139 with a __getattr__ hook is evaluated.
1140
1140
1141 """
1141 """
1142 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
1142 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
1143 if not m2:
1143 if not m2:
1144 return []
1144 return []
1145 expr, attr = m2.group(1, 2)
1145 expr, attr = m2.group(1, 2)
1146
1146
1147 obj = self._evaluate_expr(expr)
1147 obj = self._evaluate_expr(expr)
1148
1148
1149 if obj is not_found:
1149 if obj is not_found:
1150 return []
1150 return []
1151
1151
1152 if self.limit_to__all__ and hasattr(obj, '__all__'):
1152 if self.limit_to__all__ and hasattr(obj, '__all__'):
1153 words = get__all__entries(obj)
1153 words = get__all__entries(obj)
1154 else:
1154 else:
1155 words = dir2(obj)
1155 words = dir2(obj)
1156
1156
1157 try:
1157 try:
1158 words = generics.complete_object(obj, words)
1158 words = generics.complete_object(obj, words)
1159 except TryNext:
1159 except TryNext:
1160 pass
1160 pass
1161 except AssertionError:
1161 except AssertionError:
1162 raise
1162 raise
1163 except Exception:
1163 except Exception:
1164 # Silence errors from completion function
1164 # Silence errors from completion function
1165 pass
1165 pass
1166 # Build match list to return
1166 # Build match list to return
1167 n = len(attr)
1167 n = len(attr)
1168
1168
1169 # Note: ideally we would just return words here and the prefix
1169 # Note: ideally we would just return words here and the prefix
1170 # reconciliator would know that we intend to append to rather than
1170 # reconciliator would know that we intend to append to rather than
1171 # replace the input text; this requires refactoring to return range
1171 # replace the input text; this requires refactoring to return range
1172 # which ought to be replaced (as does jedi).
1172 # which ought to be replaced (as does jedi).
1173 tokens = _parse_tokens(expr)
1173 tokens = _parse_tokens(expr)
1174 rev_tokens = reversed(tokens)
1174 rev_tokens = reversed(tokens)
1175 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1175 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1176 name_turn = True
1176 name_turn = True
1177
1177
1178 parts = []
1178 parts = []
1179 for token in rev_tokens:
1179 for token in rev_tokens:
1180 if token.type in skip_over:
1180 if token.type in skip_over:
1181 continue
1181 continue
1182 if token.type == tokenize.NAME and name_turn:
1182 if token.type == tokenize.NAME and name_turn:
1183 parts.append(token.string)
1183 parts.append(token.string)
1184 name_turn = False
1184 name_turn = False
1185 elif token.type == tokenize.OP and token.string == "." and not name_turn:
1185 elif token.type == tokenize.OP and token.string == "." and not name_turn:
1186 parts.append(token.string)
1186 parts.append(token.string)
1187 name_turn = True
1187 name_turn = True
1188 else:
1188 else:
1189 # short-circuit if not empty nor name token
1189 # short-circuit if not empty nor name token
1190 break
1190 break
1191
1191
1192 prefix_after_space = "".join(reversed(parts))
1192 prefix_after_space = "".join(reversed(parts))
1193
1193
1194 return ["%s.%s" % (prefix_after_space, w) for w in words if w[:n] == attr]
1194 return ["%s.%s" % (prefix_after_space, w) for w in words if w[:n] == attr]
1195
1195
1196 def _evaluate_expr(self, expr):
1196 def _evaluate_expr(self, expr):
1197 obj = not_found
1197 obj = not_found
1198 done = False
1198 done = False
1199 while not done and expr:
1199 while not done and expr:
1200 try:
1200 try:
1201 obj = guarded_eval(
1201 obj = guarded_eval(
1202 expr,
1202 expr,
1203 EvaluationContext(
1203 EvaluationContext(
1204 globals=self.global_namespace,
1204 globals=self.global_namespace,
1205 locals=self.namespace,
1205 locals=self.namespace,
1206 evaluation=self.evaluation,
1206 evaluation=self.evaluation,
1207 ),
1207 ),
1208 )
1208 )
1209 done = True
1209 done = True
1210 except Exception as e:
1210 except Exception as e:
1211 if self.debug:
1211 if self.debug:
1212 print("Evaluation exception", e)
1212 print("Evaluation exception", e)
1213 # trim the expression to remove any invalid prefix
1213 # trim the expression to remove any invalid prefix
1214 # e.g. user starts `(d[`, so we get `expr = '(d'`,
1214 # e.g. user starts `(d[`, so we get `expr = '(d'`,
1215 # where parenthesis is not closed.
1215 # where parenthesis is not closed.
1216 # TODO: make this faster by reusing parts of the computation?
1216 # TODO: make this faster by reusing parts of the computation?
1217 expr = expr[1:]
1217 expr = expr[1:]
1218 return obj
1218 return obj
1219
1219
1220 def get__all__entries(obj):
1220 def get__all__entries(obj):
1221 """returns the strings in the __all__ attribute"""
1221 """returns the strings in the __all__ attribute"""
1222 try:
1222 try:
1223 words = getattr(obj, '__all__')
1223 words = getattr(obj, '__all__')
1224 except:
1224 except:
1225 return []
1225 return []
1226
1226
1227 return [w for w in words if isinstance(w, str)]
1227 return [w for w in words if isinstance(w, str)]
1228
1228
1229
1229
1230 class _DictKeyState(enum.Flag):
1230 class _DictKeyState(enum.Flag):
1231 """Represent state of the key match in context of other possible matches.
1231 """Represent state of the key match in context of other possible matches.
1232
1232
1233 - given `d1 = {'a': 1}` completion on `d1['<tab>` will yield `{'a': END_OF_ITEM}` as there is no tuple.
1233 - given `d1 = {'a': 1}` completion on `d1['<tab>` will yield `{'a': END_OF_ITEM}` as there is no tuple.
1234 - given `d2 = {('a', 'b'): 1}`: `d2['a', '<tab>` will yield `{'b': END_OF_TUPLE}` as there is no tuple members to add beyond `'b'`.
1234 - given `d2 = {('a', 'b'): 1}`: `d2['a', '<tab>` will yield `{'b': END_OF_TUPLE}` as there is no tuple members to add beyond `'b'`.
1235 - given `d3 = {('a', 'b'): 1}`: `d3['<tab>` will yield `{'a': IN_TUPLE}` as `'a'` can be added.
1235 - given `d3 = {('a', 'b'): 1}`: `d3['<tab>` will yield `{'a': IN_TUPLE}` as `'a'` can be added.
1236 - given `d4 = {'a': 1, ('a', 'b'): 2}`: `d4['<tab>` will yield `{'a': END_OF_ITEM & END_OF_TUPLE}`
1236 - given `d4 = {'a': 1, ('a', 'b'): 2}`: `d4['<tab>` will yield `{'a': END_OF_ITEM & END_OF_TUPLE}`
1237 """
1237 """
1238
1238
1239 BASELINE = 0
1239 BASELINE = 0
1240 END_OF_ITEM = enum.auto()
1240 END_OF_ITEM = enum.auto()
1241 END_OF_TUPLE = enum.auto()
1241 END_OF_TUPLE = enum.auto()
1242 IN_TUPLE = enum.auto()
1242 IN_TUPLE = enum.auto()
1243
1243
1244
1244
1245 def _parse_tokens(c):
1245 def _parse_tokens(c):
1246 """Parse tokens even if there is an error."""
1246 """Parse tokens even if there is an error."""
1247 tokens = []
1247 tokens = []
1248 token_generator = tokenize.generate_tokens(iter(c.splitlines()).__next__)
1248 token_generator = tokenize.generate_tokens(iter(c.splitlines()).__next__)
1249 while True:
1249 while True:
1250 try:
1250 try:
1251 tokens.append(next(token_generator))
1251 tokens.append(next(token_generator))
1252 except tokenize.TokenError:
1252 except tokenize.TokenError:
1253 return tokens
1253 return tokens
1254 except StopIteration:
1254 except StopIteration:
1255 return tokens
1255 return tokens
1256
1256
1257
1257
1258 def _match_number_in_dict_key_prefix(prefix: str) -> Union[str, None]:
1258 def _match_number_in_dict_key_prefix(prefix: str) -> Union[str, None]:
1259 """Match any valid Python numeric literal in a prefix of dictionary keys.
1259 """Match any valid Python numeric literal in a prefix of dictionary keys.
1260
1260
1261 References:
1261 References:
1262 - https://docs.python.org/3/reference/lexical_analysis.html#numeric-literals
1262 - https://docs.python.org/3/reference/lexical_analysis.html#numeric-literals
1263 - https://docs.python.org/3/library/tokenize.html
1263 - https://docs.python.org/3/library/tokenize.html
1264 """
1264 """
1265 if prefix[-1].isspace():
1265 if prefix[-1].isspace():
1266 # if user typed a space we do not have anything to complete
1266 # if user typed a space we do not have anything to complete
1267 # even if there was a valid number token before
1267 # even if there was a valid number token before
1268 return None
1268 return None
1269 tokens = _parse_tokens(prefix)
1269 tokens = _parse_tokens(prefix)
1270 rev_tokens = reversed(tokens)
1270 rev_tokens = reversed(tokens)
1271 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1271 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1272 number = None
1272 number = None
1273 for token in rev_tokens:
1273 for token in rev_tokens:
1274 if token.type in skip_over:
1274 if token.type in skip_over:
1275 continue
1275 continue
1276 if number is None:
1276 if number is None:
1277 if token.type == tokenize.NUMBER:
1277 if token.type == tokenize.NUMBER:
1278 number = token.string
1278 number = token.string
1279 continue
1279 continue
1280 else:
1280 else:
1281 # we did not match a number
1281 # we did not match a number
1282 return None
1282 return None
1283 if token.type == tokenize.OP:
1283 if token.type == tokenize.OP:
1284 if token.string == ",":
1284 if token.string == ",":
1285 break
1285 break
1286 if token.string in {"+", "-"}:
1286 if token.string in {"+", "-"}:
1287 number = token.string + number
1287 number = token.string + number
1288 else:
1288 else:
1289 return None
1289 return None
1290 return number
1290 return number
1291
1291
1292
1292
1293 _INT_FORMATS = {
1293 _INT_FORMATS = {
1294 "0b": bin,
1294 "0b": bin,
1295 "0o": oct,
1295 "0o": oct,
1296 "0x": hex,
1296 "0x": hex,
1297 }
1297 }
1298
1298
1299
1299
1300 def match_dict_keys(
1300 def match_dict_keys(
1301 keys: List[Union[str, bytes, Tuple[Union[str, bytes], ...]]],
1301 keys: List[Union[str, bytes, Tuple[Union[str, bytes], ...]]],
1302 prefix: str,
1302 prefix: str,
1303 delims: str,
1303 delims: str,
1304 extra_prefix: Optional[Tuple[Union[str, bytes], ...]] = None,
1304 extra_prefix: Optional[Tuple[Union[str, bytes], ...]] = None,
1305 ) -> Tuple[str, int, Dict[str, _DictKeyState]]:
1305 ) -> Tuple[str, int, Dict[str, _DictKeyState]]:
1306 """Used by dict_key_matches, matching the prefix to a list of keys
1306 """Used by dict_key_matches, matching the prefix to a list of keys
1307
1307
1308 Parameters
1308 Parameters
1309 ----------
1309 ----------
1310 keys
1310 keys
1311 list of keys in dictionary currently being completed.
1311 list of keys in dictionary currently being completed.
1312 prefix
1312 prefix
1313 Part of the text already typed by the user. E.g. `mydict[b'fo`
1313 Part of the text already typed by the user. E.g. `mydict[b'fo`
1314 delims
1314 delims
1315 String of delimiters to consider when finding the current key.
1315 String of delimiters to consider when finding the current key.
1316 extra_prefix : optional
1316 extra_prefix : optional
1317 Part of the text already typed in multi-key index cases. E.g. for
1317 Part of the text already typed in multi-key index cases. E.g. for
1318 `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
1318 `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
1319
1319
1320 Returns
1320 Returns
1321 -------
1321 -------
1322 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
1322 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
1323 ``quote`` being the quote that need to be used to close current string.
1323 ``quote`` being the quote that need to be used to close current string.
1324 ``token_start`` the position where the replacement should start occurring,
1324 ``token_start`` the position where the replacement should start occurring,
1325 ``matches`` a dictionary of replacement/completion keys on keys and values
1325 ``matches`` a dictionary of replacement/completion keys on keys and values
1326 indicating whether the state.
1326 indicating whether the state.
1327 """
1327 """
1328 prefix_tuple = extra_prefix if extra_prefix else ()
1328 prefix_tuple = extra_prefix if extra_prefix else ()
1329
1329
1330 prefix_tuple_size = sum(
1330 prefix_tuple_size = sum(
1331 [
1331 [
1332 # for pandas, do not count slices as taking space
1332 # for pandas, do not count slices as taking space
1333 not isinstance(k, slice)
1333 not isinstance(k, slice)
1334 for k in prefix_tuple
1334 for k in prefix_tuple
1335 ]
1335 ]
1336 )
1336 )
1337 text_serializable_types = (str, bytes, int, float, slice)
1337 text_serializable_types = (str, bytes, int, float, slice)
1338
1338
1339 def filter_prefix_tuple(key):
1339 def filter_prefix_tuple(key):
1340 # Reject too short keys
1340 # Reject too short keys
1341 if len(key) <= prefix_tuple_size:
1341 if len(key) <= prefix_tuple_size:
1342 return False
1342 return False
1343 # Reject keys which cannot be serialised to text
1343 # Reject keys which cannot be serialised to text
1344 for k in key:
1344 for k in key:
1345 if not isinstance(k, text_serializable_types):
1345 if not isinstance(k, text_serializable_types):
1346 return False
1346 return False
1347 # Reject keys that do not match the prefix
1347 # Reject keys that do not match the prefix
1348 for k, pt in zip(key, prefix_tuple):
1348 for k, pt in zip(key, prefix_tuple):
1349 if k != pt and not isinstance(pt, slice):
1349 if k != pt and not isinstance(pt, slice):
1350 return False
1350 return False
1351 # All checks passed!
1351 # All checks passed!
1352 return True
1352 return True
1353
1353
1354 filtered_key_is_final: Dict[
1354 filtered_key_is_final: Dict[
1355 Union[str, bytes, int, float], _DictKeyState
1355 Union[str, bytes, int, float], _DictKeyState
1356 ] = defaultdict(lambda: _DictKeyState.BASELINE)
1356 ] = defaultdict(lambda: _DictKeyState.BASELINE)
1357
1357
1358 for k in keys:
1358 for k in keys:
1359 # If at least one of the matches is not final, mark as undetermined.
1359 # If at least one of the matches is not final, mark as undetermined.
1360 # This can happen with `d = {111: 'b', (111, 222): 'a'}` where
1360 # This can happen with `d = {111: 'b', (111, 222): 'a'}` where
1361 # `111` appears final on first match but is not final on the second.
1361 # `111` appears final on first match but is not final on the second.
1362
1362
1363 if isinstance(k, tuple):
1363 if isinstance(k, tuple):
1364 if filter_prefix_tuple(k):
1364 if filter_prefix_tuple(k):
1365 key_fragment = k[prefix_tuple_size]
1365 key_fragment = k[prefix_tuple_size]
1366 filtered_key_is_final[key_fragment] |= (
1366 filtered_key_is_final[key_fragment] |= (
1367 _DictKeyState.END_OF_TUPLE
1367 _DictKeyState.END_OF_TUPLE
1368 if len(k) == prefix_tuple_size + 1
1368 if len(k) == prefix_tuple_size + 1
1369 else _DictKeyState.IN_TUPLE
1369 else _DictKeyState.IN_TUPLE
1370 )
1370 )
1371 elif prefix_tuple_size > 0:
1371 elif prefix_tuple_size > 0:
1372 # we are completing a tuple but this key is not a tuple,
1372 # we are completing a tuple but this key is not a tuple,
1373 # so we should ignore it
1373 # so we should ignore it
1374 pass
1374 pass
1375 else:
1375 else:
1376 if isinstance(k, text_serializable_types):
1376 if isinstance(k, text_serializable_types):
1377 filtered_key_is_final[k] |= _DictKeyState.END_OF_ITEM
1377 filtered_key_is_final[k] |= _DictKeyState.END_OF_ITEM
1378
1378
1379 filtered_keys = filtered_key_is_final.keys()
1379 filtered_keys = filtered_key_is_final.keys()
1380
1380
1381 if not prefix:
1381 if not prefix:
1382 return "", 0, {repr(k): v for k, v in filtered_key_is_final.items()}
1382 return "", 0, {repr(k): v for k, v in filtered_key_is_final.items()}
1383
1383
1384 quote_match = re.search("(?:\"|')", prefix)
1384 quote_match = re.search("(?:\"|')", prefix)
1385 is_user_prefix_numeric = False
1385 is_user_prefix_numeric = False
1386
1386
1387 if quote_match:
1387 if quote_match:
1388 quote = quote_match.group()
1388 quote = quote_match.group()
1389 valid_prefix = prefix + quote
1389 valid_prefix = prefix + quote
1390 try:
1390 try:
1391 prefix_str = literal_eval(valid_prefix)
1391 prefix_str = literal_eval(valid_prefix)
1392 except Exception:
1392 except Exception:
1393 return "", 0, {}
1393 return "", 0, {}
1394 else:
1394 else:
1395 # If it does not look like a string, let's assume
1395 # If it does not look like a string, let's assume
1396 # we are dealing with a number or variable.
1396 # we are dealing with a number or variable.
1397 number_match = _match_number_in_dict_key_prefix(prefix)
1397 number_match = _match_number_in_dict_key_prefix(prefix)
1398
1398
1399 # We do not want the key matcher to suggest variable names so we yield:
1399 # We do not want the key matcher to suggest variable names so we yield:
1400 if number_match is None:
1400 if number_match is None:
1401 # The alternative would be to assume that user forgort the quote
1401 # The alternative would be to assume that user forgort the quote
1402 # and if the substring matches, suggest adding it at the start.
1402 # and if the substring matches, suggest adding it at the start.
1403 return "", 0, {}
1403 return "", 0, {}
1404
1404
1405 prefix_str = number_match
1405 prefix_str = number_match
1406 is_user_prefix_numeric = True
1406 is_user_prefix_numeric = True
1407 quote = ""
1407 quote = ""
1408
1408
1409 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
1409 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
1410 token_match = re.search(pattern, prefix, re.UNICODE)
1410 token_match = re.search(pattern, prefix, re.UNICODE)
1411 assert token_match is not None # silence mypy
1411 assert token_match is not None # silence mypy
1412 token_start = token_match.start()
1412 token_start = token_match.start()
1413 token_prefix = token_match.group()
1413 token_prefix = token_match.group()
1414
1414
1415 matched: Dict[str, _DictKeyState] = {}
1415 matched: Dict[str, _DictKeyState] = {}
1416
1416
1417 str_key: Union[str, bytes]
1417 str_key: Union[str, bytes]
1418
1418
1419 for key in filtered_keys:
1419 for key in filtered_keys:
1420 if isinstance(key, (int, float)):
1420 if isinstance(key, (int, float)):
1421 # User typed a number but this key is not a number.
1421 # User typed a number but this key is not a number.
1422 if not is_user_prefix_numeric:
1422 if not is_user_prefix_numeric:
1423 continue
1423 continue
1424 str_key = str(key)
1424 str_key = str(key)
1425 if isinstance(key, int):
1425 if isinstance(key, int):
1426 int_base = prefix_str[:2].lower()
1426 int_base = prefix_str[:2].lower()
1427 # if user typed integer using binary/oct/hex notation:
1427 # if user typed integer using binary/oct/hex notation:
1428 if int_base in _INT_FORMATS:
1428 if int_base in _INT_FORMATS:
1429 int_format = _INT_FORMATS[int_base]
1429 int_format = _INT_FORMATS[int_base]
1430 str_key = int_format(key)
1430 str_key = int_format(key)
1431 else:
1431 else:
1432 # User typed a string but this key is a number.
1432 # User typed a string but this key is a number.
1433 if is_user_prefix_numeric:
1433 if is_user_prefix_numeric:
1434 continue
1434 continue
1435 str_key = key
1435 str_key = key
1436 try:
1436 try:
1437 if not str_key.startswith(prefix_str):
1437 if not str_key.startswith(prefix_str):
1438 continue
1438 continue
1439 except (AttributeError, TypeError, UnicodeError) as e:
1439 except (AttributeError, TypeError, UnicodeError) as e:
1440 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
1440 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
1441 continue
1441 continue
1442
1442
1443 # reformat remainder of key to begin with prefix
1443 # reformat remainder of key to begin with prefix
1444 rem = str_key[len(prefix_str) :]
1444 rem = str_key[len(prefix_str) :]
1445 # force repr wrapped in '
1445 # force repr wrapped in '
1446 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
1446 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
1447 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
1447 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
1448 if quote == '"':
1448 if quote == '"':
1449 # The entered prefix is quoted with ",
1449 # The entered prefix is quoted with ",
1450 # but the match is quoted with '.
1450 # but the match is quoted with '.
1451 # A contained " hence needs escaping for comparison:
1451 # A contained " hence needs escaping for comparison:
1452 rem_repr = rem_repr.replace('"', '\\"')
1452 rem_repr = rem_repr.replace('"', '\\"')
1453
1453
1454 # then reinsert prefix from start of token
1454 # then reinsert prefix from start of token
1455 match = "%s%s" % (token_prefix, rem_repr)
1455 match = "%s%s" % (token_prefix, rem_repr)
1456
1456
1457 matched[match] = filtered_key_is_final[key]
1457 matched[match] = filtered_key_is_final[key]
1458 return quote, token_start, matched
1458 return quote, token_start, matched
1459
1459
1460
1460
1461 def cursor_to_position(text:str, line:int, column:int)->int:
1461 def cursor_to_position(text:str, line:int, column:int)->int:
1462 """
1462 """
1463 Convert the (line,column) position of the cursor in text to an offset in a
1463 Convert the (line,column) position of the cursor in text to an offset in a
1464 string.
1464 string.
1465
1465
1466 Parameters
1466 Parameters
1467 ----------
1467 ----------
1468 text : str
1468 text : str
1469 The text in which to calculate the cursor offset
1469 The text in which to calculate the cursor offset
1470 line : int
1470 line : int
1471 Line of the cursor; 0-indexed
1471 Line of the cursor; 0-indexed
1472 column : int
1472 column : int
1473 Column of the cursor 0-indexed
1473 Column of the cursor 0-indexed
1474
1474
1475 Returns
1475 Returns
1476 -------
1476 -------
1477 Position of the cursor in ``text``, 0-indexed.
1477 Position of the cursor in ``text``, 0-indexed.
1478
1478
1479 See Also
1479 See Also
1480 --------
1480 --------
1481 position_to_cursor : reciprocal of this function
1481 position_to_cursor : reciprocal of this function
1482
1482
1483 """
1483 """
1484 lines = text.split('\n')
1484 lines = text.split('\n')
1485 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
1485 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
1486
1486
1487 return sum(len(l) + 1 for l in lines[:line]) + column
1487 return sum(len(l) + 1 for l in lines[:line]) + column
1488
1488
1489 def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
1489 def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
1490 """
1490 """
1491 Convert the position of the cursor in text (0 indexed) to a line
1491 Convert the position of the cursor in text (0 indexed) to a line
1492 number(0-indexed) and a column number (0-indexed) pair
1492 number(0-indexed) and a column number (0-indexed) pair
1493
1493
1494 Position should be a valid position in ``text``.
1494 Position should be a valid position in ``text``.
1495
1495
1496 Parameters
1496 Parameters
1497 ----------
1497 ----------
1498 text : str
1498 text : str
1499 The text in which to calculate the cursor offset
1499 The text in which to calculate the cursor offset
1500 offset : int
1500 offset : int
1501 Position of the cursor in ``text``, 0-indexed.
1501 Position of the cursor in ``text``, 0-indexed.
1502
1502
1503 Returns
1503 Returns
1504 -------
1504 -------
1505 (line, column) : (int, int)
1505 (line, column) : (int, int)
1506 Line of the cursor; 0-indexed, column of the cursor 0-indexed
1506 Line of the cursor; 0-indexed, column of the cursor 0-indexed
1507
1507
1508 See Also
1508 See Also
1509 --------
1509 --------
1510 cursor_to_position : reciprocal of this function
1510 cursor_to_position : reciprocal of this function
1511
1511
1512 """
1512 """
1513
1513
1514 assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
1514 assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
1515
1515
1516 before = text[:offset]
1516 before = text[:offset]
1517 blines = before.split('\n') # ! splitnes trim trailing \n
1517 blines = before.split('\n') # ! splitnes trim trailing \n
1518 line = before.count('\n')
1518 line = before.count('\n')
1519 col = len(blines[-1])
1519 col = len(blines[-1])
1520 return line, col
1520 return line, col
1521
1521
1522
1522
1523 def _safe_isinstance(obj, module, class_name, *attrs):
1523 def _safe_isinstance(obj, module, class_name, *attrs):
1524 """Checks if obj is an instance of module.class_name if loaded
1524 """Checks if obj is an instance of module.class_name if loaded
1525 """
1525 """
1526 if module in sys.modules:
1526 if module in sys.modules:
1527 m = sys.modules[module]
1527 m = sys.modules[module]
1528 for attr in [class_name, *attrs]:
1528 for attr in [class_name, *attrs]:
1529 m = getattr(m, attr)
1529 m = getattr(m, attr)
1530 return isinstance(obj, m)
1530 return isinstance(obj, m)
1531
1531
1532
1532
1533 @context_matcher()
1533 @context_matcher()
1534 def back_unicode_name_matcher(context: CompletionContext):
1534 def back_unicode_name_matcher(context: CompletionContext):
1535 """Match Unicode characters back to Unicode name
1535 """Match Unicode characters back to Unicode name
1536
1536
1537 Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1537 Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1538 """
1538 """
1539 fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1539 fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1540 return _convert_matcher_v1_result_to_v2(
1540 return _convert_matcher_v1_result_to_v2(
1541 matches, type="unicode", fragment=fragment, suppress_if_matches=True
1541 matches, type="unicode", fragment=fragment, suppress_if_matches=True
1542 )
1542 )
1543
1543
1544
1544
1545 def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1545 def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1546 """Match Unicode characters back to Unicode name
1546 """Match Unicode characters back to Unicode name
1547
1547
1548 This does ``β˜ƒ`` -> ``\\snowman``
1548 This does ``β˜ƒ`` -> ``\\snowman``
1549
1549
1550 Note that snowman is not a valid python3 combining character but will be expanded.
1550 Note that snowman is not a valid python3 combining character but will be expanded.
1551 Though it will not recombine back to the snowman character by the completion machinery.
1551 Though it will not recombine back to the snowman character by the completion machinery.
1552
1552
1553 This will not either back-complete standard sequences like \\n, \\b ...
1553 This will not either back-complete standard sequences like \\n, \\b ...
1554
1554
1555 .. deprecated:: 8.6
1555 .. deprecated:: 8.6
1556 You can use :meth:`back_unicode_name_matcher` instead.
1556 You can use :meth:`back_unicode_name_matcher` instead.
1557
1557
1558 Returns
1558 Returns
1559 =======
1559 =======
1560
1560
1561 Return a tuple with two elements:
1561 Return a tuple with two elements:
1562
1562
1563 - The Unicode character that was matched (preceded with a backslash), or
1563 - The Unicode character that was matched (preceded with a backslash), or
1564 empty string,
1564 empty string,
1565 - a sequence (of 1), name for the match Unicode character, preceded by
1565 - a sequence (of 1), name for the match Unicode character, preceded by
1566 backslash, or empty if no match.
1566 backslash, or empty if no match.
1567 """
1567 """
1568 if len(text)<2:
1568 if len(text)<2:
1569 return '', ()
1569 return '', ()
1570 maybe_slash = text[-2]
1570 maybe_slash = text[-2]
1571 if maybe_slash != '\\':
1571 if maybe_slash != '\\':
1572 return '', ()
1572 return '', ()
1573
1573
1574 char = text[-1]
1574 char = text[-1]
1575 # no expand on quote for completion in strings.
1575 # no expand on quote for completion in strings.
1576 # nor backcomplete standard ascii keys
1576 # nor backcomplete standard ascii keys
1577 if char in string.ascii_letters or char in ('"',"'"):
1577 if char in string.ascii_letters or char in ('"',"'"):
1578 return '', ()
1578 return '', ()
1579 try :
1579 try :
1580 unic = unicodedata.name(char)
1580 unic = unicodedata.name(char)
1581 return '\\'+char,('\\'+unic,)
1581 return '\\'+char,('\\'+unic,)
1582 except KeyError:
1582 except KeyError:
1583 pass
1583 pass
1584 return '', ()
1584 return '', ()
1585
1585
1586
1586
1587 @context_matcher()
1587 @context_matcher()
1588 def back_latex_name_matcher(context: CompletionContext):
1588 def back_latex_name_matcher(context: CompletionContext):
1589 """Match latex characters back to unicode name
1589 """Match latex characters back to unicode name
1590
1590
1591 Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1591 Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1592 """
1592 """
1593 fragment, matches = back_latex_name_matches(context.text_until_cursor)
1593 fragment, matches = back_latex_name_matches(context.text_until_cursor)
1594 return _convert_matcher_v1_result_to_v2(
1594 return _convert_matcher_v1_result_to_v2(
1595 matches, type="latex", fragment=fragment, suppress_if_matches=True
1595 matches, type="latex", fragment=fragment, suppress_if_matches=True
1596 )
1596 )
1597
1597
1598
1598
1599 def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1599 def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1600 """Match latex characters back to unicode name
1600 """Match latex characters back to unicode name
1601
1601
1602 This does ``\\β„΅`` -> ``\\aleph``
1602 This does ``\\β„΅`` -> ``\\aleph``
1603
1603
1604 .. deprecated:: 8.6
1604 .. deprecated:: 8.6
1605 You can use :meth:`back_latex_name_matcher` instead.
1605 You can use :meth:`back_latex_name_matcher` instead.
1606 """
1606 """
1607 if len(text)<2:
1607 if len(text)<2:
1608 return '', ()
1608 return '', ()
1609 maybe_slash = text[-2]
1609 maybe_slash = text[-2]
1610 if maybe_slash != '\\':
1610 if maybe_slash != '\\':
1611 return '', ()
1611 return '', ()
1612
1612
1613
1613
1614 char = text[-1]
1614 char = text[-1]
1615 # no expand on quote for completion in strings.
1615 # no expand on quote for completion in strings.
1616 # nor backcomplete standard ascii keys
1616 # nor backcomplete standard ascii keys
1617 if char in string.ascii_letters or char in ('"',"'"):
1617 if char in string.ascii_letters or char in ('"',"'"):
1618 return '', ()
1618 return '', ()
1619 try :
1619 try :
1620 latex = reverse_latex_symbol[char]
1620 latex = reverse_latex_symbol[char]
1621 # '\\' replace the \ as well
1621 # '\\' replace the \ as well
1622 return '\\'+char,[latex]
1622 return '\\'+char,[latex]
1623 except KeyError:
1623 except KeyError:
1624 pass
1624 pass
1625 return '', ()
1625 return '', ()
1626
1626
1627
1627
1628 def _formatparamchildren(parameter) -> str:
1628 def _formatparamchildren(parameter) -> str:
1629 """
1629 """
1630 Get parameter name and value from Jedi Private API
1630 Get parameter name and value from Jedi Private API
1631
1631
1632 Jedi does not expose a simple way to get `param=value` from its API.
1632 Jedi does not expose a simple way to get `param=value` from its API.
1633
1633
1634 Parameters
1634 Parameters
1635 ----------
1635 ----------
1636 parameter
1636 parameter
1637 Jedi's function `Param`
1637 Jedi's function `Param`
1638
1638
1639 Returns
1639 Returns
1640 -------
1640 -------
1641 A string like 'a', 'b=1', '*args', '**kwargs'
1641 A string like 'a', 'b=1', '*args', '**kwargs'
1642
1642
1643 """
1643 """
1644 description = parameter.description
1644 description = parameter.description
1645 if not description.startswith('param '):
1645 if not description.startswith('param '):
1646 raise ValueError('Jedi function parameter description have change format.'
1646 raise ValueError('Jedi function parameter description have change format.'
1647 'Expected "param ...", found %r".' % description)
1647 'Expected "param ...", found %r".' % description)
1648 return description[6:]
1648 return description[6:]
1649
1649
1650 def _make_signature(completion)-> str:
1650 def _make_signature(completion)-> str:
1651 """
1651 """
1652 Make the signature from a jedi completion
1652 Make the signature from a jedi completion
1653
1653
1654 Parameters
1654 Parameters
1655 ----------
1655 ----------
1656 completion : jedi.Completion
1656 completion : jedi.Completion
1657 object does not complete a function type
1657 object does not complete a function type
1658
1658
1659 Returns
1659 Returns
1660 -------
1660 -------
1661 a string consisting of the function signature, with the parenthesis but
1661 a string consisting of the function signature, with the parenthesis but
1662 without the function name. example:
1662 without the function name. example:
1663 `(a, *args, b=1, **kwargs)`
1663 `(a, *args, b=1, **kwargs)`
1664
1664
1665 """
1665 """
1666
1666
1667 # it looks like this might work on jedi 0.17
1667 # it looks like this might work on jedi 0.17
1668 if hasattr(completion, 'get_signatures'):
1668 if hasattr(completion, 'get_signatures'):
1669 signatures = completion.get_signatures()
1669 signatures = completion.get_signatures()
1670 if not signatures:
1670 if not signatures:
1671 return '(?)'
1671 return '(?)'
1672
1672
1673 c0 = completion.get_signatures()[0]
1673 c0 = completion.get_signatures()[0]
1674 return '('+c0.to_string().split('(', maxsplit=1)[1]
1674 return '('+c0.to_string().split('(', maxsplit=1)[1]
1675
1675
1676 return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1676 return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1677 for p in signature.defined_names()) if f])
1677 for p in signature.defined_names()) if f])
1678
1678
1679
1679
1680 _CompleteResult = Dict[str, MatcherResult]
1680 _CompleteResult = Dict[str, MatcherResult]
1681
1681
1682
1682
1683 DICT_MATCHER_REGEX = re.compile(
1683 DICT_MATCHER_REGEX = re.compile(
1684 r"""(?x)
1684 r"""(?x)
1685 ( # match dict-referring - or any get item object - expression
1685 ( # match dict-referring - or any get item object - expression
1686 .+
1686 .+
1687 )
1687 )
1688 \[ # open bracket
1688 \[ # open bracket
1689 \s* # and optional whitespace
1689 \s* # and optional whitespace
1690 # Capture any number of serializable objects (e.g. "a", "b", 'c')
1690 # Capture any number of serializable objects (e.g. "a", "b", 'c')
1691 # and slices
1691 # and slices
1692 ((?:(?:
1692 ((?:(?:
1693 (?: # closed string
1693 (?: # closed string
1694 [uUbB]? # string prefix (r not handled)
1694 [uUbB]? # string prefix (r not handled)
1695 (?:
1695 (?:
1696 '(?:[^']|(?<!\\)\\')*'
1696 '(?:[^']|(?<!\\)\\')*'
1697 |
1697 |
1698 "(?:[^"]|(?<!\\)\\")*"
1698 "(?:[^"]|(?<!\\)\\")*"
1699 )
1699 )
1700 )
1700 )
1701 |
1701 |
1702 # capture integers and slices
1702 # capture integers and slices
1703 (?:[-+]?\d+)?(?::(?:[-+]?\d+)?){0,2}
1703 (?:[-+]?\d+)?(?::(?:[-+]?\d+)?){0,2}
1704 |
1704 |
1705 # integer in bin/hex/oct notation
1705 # integer in bin/hex/oct notation
1706 0[bBxXoO]_?(?:\w|\d)+
1706 0[bBxXoO]_?(?:\w|\d)+
1707 )
1707 )
1708 \s*,\s*
1708 \s*,\s*
1709 )*)
1709 )*)
1710 ((?:
1710 ((?:
1711 (?: # unclosed string
1711 (?: # unclosed string
1712 [uUbB]? # string prefix (r not handled)
1712 [uUbB]? # string prefix (r not handled)
1713 (?:
1713 (?:
1714 '(?:[^']|(?<!\\)\\')*
1714 '(?:[^']|(?<!\\)\\')*
1715 |
1715 |
1716 "(?:[^"]|(?<!\\)\\")*
1716 "(?:[^"]|(?<!\\)\\")*
1717 )
1717 )
1718 )
1718 )
1719 |
1719 |
1720 # unfinished integer
1720 # unfinished integer
1721 (?:[-+]?\d+)
1721 (?:[-+]?\d+)
1722 |
1722 |
1723 # integer in bin/hex/oct notation
1723 # integer in bin/hex/oct notation
1724 0[bBxXoO]_?(?:\w|\d)+
1724 0[bBxXoO]_?(?:\w|\d)+
1725 )
1725 )
1726 )?
1726 )?
1727 $
1727 $
1728 """
1728 """
1729 )
1729 )
1730
1730
1731
1731
1732 def _convert_matcher_v1_result_to_v2(
1732 def _convert_matcher_v1_result_to_v2(
1733 matches: Sequence[str],
1733 matches: Sequence[str],
1734 type: str,
1734 type: str,
1735 fragment: Optional[str] = None,
1735 fragment: Optional[str] = None,
1736 suppress_if_matches: bool = False,
1736 suppress_if_matches: bool = False,
1737 ) -> SimpleMatcherResult:
1737 ) -> SimpleMatcherResult:
1738 """Utility to help with transition"""
1738 """Utility to help with transition"""
1739 result = {
1739 result = {
1740 "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1740 "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1741 "suppress": (True if matches else False) if suppress_if_matches else False,
1741 "suppress": (True if matches else False) if suppress_if_matches else False,
1742 }
1742 }
1743 if fragment is not None:
1743 if fragment is not None:
1744 result["matched_fragment"] = fragment
1744 result["matched_fragment"] = fragment
1745 return cast(SimpleMatcherResult, result)
1745 return cast(SimpleMatcherResult, result)
1746
1746
1747
1747
1748 class IPCompleter(Completer):
1748 class IPCompleter(Completer):
1749 """Extension of the completer class with IPython-specific features"""
1749 """Extension of the completer class with IPython-specific features"""
1750
1750
1751 @observe('greedy')
1751 @observe('greedy')
1752 def _greedy_changed(self, change):
1752 def _greedy_changed(self, change):
1753 """update the splitter and readline delims when greedy is changed"""
1753 """update the splitter and readline delims when greedy is changed"""
1754 if change["new"]:
1754 if change["new"]:
1755 self.evaluation = "unsafe"
1755 self.evaluation = "unsafe"
1756 self.auto_close_dict_keys = True
1756 self.auto_close_dict_keys = True
1757 self.splitter.delims = GREEDY_DELIMS
1757 self.splitter.delims = GREEDY_DELIMS
1758 else:
1758 else:
1759 self.evaluation = "limited"
1759 self.evaluation = "limited"
1760 self.auto_close_dict_keys = False
1760 self.auto_close_dict_keys = False
1761 self.splitter.delims = DELIMS
1761 self.splitter.delims = DELIMS
1762
1762
1763 dict_keys_only = Bool(
1763 dict_keys_only = Bool(
1764 False,
1764 False,
1765 help="""
1765 help="""
1766 Whether to show dict key matches only.
1766 Whether to show dict key matches only.
1767
1767
1768 (disables all matchers except for `IPCompleter.dict_key_matcher`).
1768 (disables all matchers except for `IPCompleter.dict_key_matcher`).
1769 """,
1769 """,
1770 )
1770 )
1771
1771
1772 suppress_competing_matchers = UnionTrait(
1772 suppress_competing_matchers = UnionTrait(
1773 [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1773 [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1774 default_value=None,
1774 default_value=None,
1775 help="""
1775 help="""
1776 Whether to suppress completions from other *Matchers*.
1776 Whether to suppress completions from other *Matchers*.
1777
1777
1778 When set to ``None`` (default) the matchers will attempt to auto-detect
1778 When set to ``None`` (default) the matchers will attempt to auto-detect
1779 whether suppression of other matchers is desirable. For example, at
1779 whether suppression of other matchers is desirable. For example, at
1780 the beginning of a line followed by `%` we expect a magic completion
1780 the beginning of a line followed by `%` we expect a magic completion
1781 to be the only applicable option, and after ``my_dict['`` we usually
1781 to be the only applicable option, and after ``my_dict['`` we usually
1782 expect a completion with an existing dictionary key.
1782 expect a completion with an existing dictionary key.
1783
1783
1784 If you want to disable this heuristic and see completions from all matchers,
1784 If you want to disable this heuristic and see completions from all matchers,
1785 set ``IPCompleter.suppress_competing_matchers = False``.
1785 set ``IPCompleter.suppress_competing_matchers = False``.
1786 To disable the heuristic for specific matchers provide a dictionary mapping:
1786 To disable the heuristic for specific matchers provide a dictionary mapping:
1787 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1787 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1788
1788
1789 Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1789 Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1790 completions to the set of matchers with the highest priority;
1790 completions to the set of matchers with the highest priority;
1791 this is equivalent to ``IPCompleter.merge_completions`` and
1791 this is equivalent to ``IPCompleter.merge_completions`` and
1792 can be beneficial for performance, but will sometimes omit relevant
1792 can be beneficial for performance, but will sometimes omit relevant
1793 candidates from matchers further down the priority list.
1793 candidates from matchers further down the priority list.
1794 """,
1794 """,
1795 ).tag(config=True)
1795 ).tag(config=True)
1796
1796
1797 merge_completions = Bool(
1797 merge_completions = Bool(
1798 True,
1798 True,
1799 help="""Whether to merge completion results into a single list
1799 help="""Whether to merge completion results into a single list
1800
1800
1801 If False, only the completion results from the first non-empty
1801 If False, only the completion results from the first non-empty
1802 completer will be returned.
1802 completer will be returned.
1803
1803
1804 As of version 8.6.0, setting the value to ``False`` is an alias for:
1804 As of version 8.6.0, setting the value to ``False`` is an alias for:
1805 ``IPCompleter.suppress_competing_matchers = True.``.
1805 ``IPCompleter.suppress_competing_matchers = True.``.
1806 """,
1806 """,
1807 ).tag(config=True)
1807 ).tag(config=True)
1808
1808
1809 disable_matchers = ListTrait(
1809 disable_matchers = ListTrait(
1810 Unicode(),
1810 Unicode(),
1811 help="""List of matchers to disable.
1811 help="""List of matchers to disable.
1812
1812
1813 The list should contain matcher identifiers (see :any:`completion_matcher`).
1813 The list should contain matcher identifiers (see :any:`completion_matcher`).
1814 """,
1814 """,
1815 ).tag(config=True)
1815 ).tag(config=True)
1816
1816
1817 omit__names = Enum(
1817 omit__names = Enum(
1818 (0, 1, 2),
1818 (0, 1, 2),
1819 default_value=2,
1819 default_value=2,
1820 help="""Instruct the completer to omit private method names
1820 help="""Instruct the completer to omit private method names
1821
1821
1822 Specifically, when completing on ``object.<tab>``.
1822 Specifically, when completing on ``object.<tab>``.
1823
1823
1824 When 2 [default]: all names that start with '_' will be excluded.
1824 When 2 [default]: all names that start with '_' will be excluded.
1825
1825
1826 When 1: all 'magic' names (``__foo__``) will be excluded.
1826 When 1: all 'magic' names (``__foo__``) will be excluded.
1827
1827
1828 When 0: nothing will be excluded.
1828 When 0: nothing will be excluded.
1829 """
1829 """
1830 ).tag(config=True)
1830 ).tag(config=True)
1831 limit_to__all__ = Bool(False,
1831 limit_to__all__ = Bool(False,
1832 help="""
1832 help="""
1833 DEPRECATED as of version 5.0.
1833 DEPRECATED as of version 5.0.
1834
1834
1835 Instruct the completer to use __all__ for the completion
1835 Instruct the completer to use __all__ for the completion
1836
1836
1837 Specifically, when completing on ``object.<tab>``.
1837 Specifically, when completing on ``object.<tab>``.
1838
1838
1839 When True: only those names in obj.__all__ will be included.
1839 When True: only those names in obj.__all__ will be included.
1840
1840
1841 When False [default]: the __all__ attribute is ignored
1841 When False [default]: the __all__ attribute is ignored
1842 """,
1842 """,
1843 ).tag(config=True)
1843 ).tag(config=True)
1844
1844
1845 profile_completions = Bool(
1845 profile_completions = Bool(
1846 default_value=False,
1846 default_value=False,
1847 help="If True, emit profiling data for completion subsystem using cProfile."
1847 help="If True, emit profiling data for completion subsystem using cProfile."
1848 ).tag(config=True)
1848 ).tag(config=True)
1849
1849
1850 profiler_output_dir = Unicode(
1850 profiler_output_dir = Unicode(
1851 default_value=".completion_profiles",
1851 default_value=".completion_profiles",
1852 help="Template for path at which to output profile data for completions."
1852 help="Template for path at which to output profile data for completions."
1853 ).tag(config=True)
1853 ).tag(config=True)
1854
1854
1855 @observe('limit_to__all__')
1855 @observe('limit_to__all__')
1856 def _limit_to_all_changed(self, change):
1856 def _limit_to_all_changed(self, change):
1857 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1857 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1858 'value has been deprecated since IPython 5.0, will be made to have '
1858 'value has been deprecated since IPython 5.0, will be made to have '
1859 'no effects and then removed in future version of IPython.',
1859 'no effects and then removed in future version of IPython.',
1860 UserWarning)
1860 UserWarning)
1861
1861
1862 def __init__(
1862 def __init__(
1863 self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1863 self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1864 ):
1864 ):
1865 """IPCompleter() -> completer
1865 """IPCompleter() -> completer
1866
1866
1867 Return a completer object.
1867 Return a completer object.
1868
1868
1869 Parameters
1869 Parameters
1870 ----------
1870 ----------
1871 shell
1871 shell
1872 a pointer to the ipython shell itself. This is needed
1872 a pointer to the ipython shell itself. This is needed
1873 because this completer knows about magic functions, and those can
1873 because this completer knows about magic functions, and those can
1874 only be accessed via the ipython instance.
1874 only be accessed via the ipython instance.
1875 namespace : dict, optional
1875 namespace : dict, optional
1876 an optional dict where completions are performed.
1876 an optional dict where completions are performed.
1877 global_namespace : dict, optional
1877 global_namespace : dict, optional
1878 secondary optional dict for completions, to
1878 secondary optional dict for completions, to
1879 handle cases (such as IPython embedded inside functions) where
1879 handle cases (such as IPython embedded inside functions) where
1880 both Python scopes are visible.
1880 both Python scopes are visible.
1881 config : Config
1881 config : Config
1882 traitlet's config object
1882 traitlet's config object
1883 **kwargs
1883 **kwargs
1884 passed to super class unmodified.
1884 passed to super class unmodified.
1885 """
1885 """
1886
1886
1887 self.magic_escape = ESC_MAGIC
1887 self.magic_escape = ESC_MAGIC
1888 self.splitter = CompletionSplitter()
1888 self.splitter = CompletionSplitter()
1889
1889
1890 # _greedy_changed() depends on splitter and readline being defined:
1890 # _greedy_changed() depends on splitter and readline being defined:
1891 super().__init__(
1891 super().__init__(
1892 namespace=namespace,
1892 namespace=namespace,
1893 global_namespace=global_namespace,
1893 global_namespace=global_namespace,
1894 config=config,
1894 config=config,
1895 **kwargs,
1895 **kwargs,
1896 )
1896 )
1897
1897
1898 # List where completion matches will be stored
1898 # List where completion matches will be stored
1899 self.matches = []
1899 self.matches = []
1900 self.shell = shell
1900 self.shell = shell
1901 # Regexp to split filenames with spaces in them
1901 # Regexp to split filenames with spaces in them
1902 self.space_name_re = re.compile(r'([^\\] )')
1902 self.space_name_re = re.compile(r'([^\\] )')
1903 # Hold a local ref. to glob.glob for speed
1903 # Hold a local ref. to glob.glob for speed
1904 self.glob = glob.glob
1904 self.glob = glob.glob
1905
1905
1906 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1906 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1907 # buffers, to avoid completion problems.
1907 # buffers, to avoid completion problems.
1908 term = os.environ.get('TERM','xterm')
1908 term = os.environ.get('TERM','xterm')
1909 self.dumb_terminal = term in ['dumb','emacs']
1909 self.dumb_terminal = term in ['dumb','emacs']
1910
1910
1911 # Special handling of backslashes needed in win32 platforms
1911 # Special handling of backslashes needed in win32 platforms
1912 if sys.platform == "win32":
1912 if sys.platform == "win32":
1913 self.clean_glob = self._clean_glob_win32
1913 self.clean_glob = self._clean_glob_win32
1914 else:
1914 else:
1915 self.clean_glob = self._clean_glob
1915 self.clean_glob = self._clean_glob
1916
1916
1917 #regexp to parse docstring for function signature
1917 #regexp to parse docstring for function signature
1918 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1918 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1919 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1919 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1920 #use this if positional argument name is also needed
1920 #use this if positional argument name is also needed
1921 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1921 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1922
1922
1923 self.magic_arg_matchers = [
1923 self.magic_arg_matchers = [
1924 self.magic_config_matcher,
1924 self.magic_config_matcher,
1925 self.magic_color_matcher,
1925 self.magic_color_matcher,
1926 ]
1926 ]
1927
1927
1928 # This is set externally by InteractiveShell
1928 # This is set externally by InteractiveShell
1929 self.custom_completers = None
1929 self.custom_completers = None
1930
1930
1931 # This is a list of names of unicode characters that can be completed
1931 # This is a list of names of unicode characters that can be completed
1932 # into their corresponding unicode value. The list is large, so we
1932 # into their corresponding unicode value. The list is large, so we
1933 # lazily initialize it on first use. Consuming code should access this
1933 # lazily initialize it on first use. Consuming code should access this
1934 # attribute through the `@unicode_names` property.
1934 # attribute through the `@unicode_names` property.
1935 self._unicode_names = None
1935 self._unicode_names = None
1936
1936
1937 self._backslash_combining_matchers = [
1937 self._backslash_combining_matchers = [
1938 self.latex_name_matcher,
1938 self.latex_name_matcher,
1939 self.unicode_name_matcher,
1939 self.unicode_name_matcher,
1940 back_latex_name_matcher,
1940 back_latex_name_matcher,
1941 back_unicode_name_matcher,
1941 back_unicode_name_matcher,
1942 self.fwd_unicode_matcher,
1942 self.fwd_unicode_matcher,
1943 ]
1943 ]
1944
1944
1945 if not self.backslash_combining_completions:
1945 if not self.backslash_combining_completions:
1946 for matcher in self._backslash_combining_matchers:
1946 for matcher in self._backslash_combining_matchers:
1947 self.disable_matchers.append(_get_matcher_id(matcher))
1947 self.disable_matchers.append(_get_matcher_id(matcher))
1948
1948
1949 if not self.merge_completions:
1949 if not self.merge_completions:
1950 self.suppress_competing_matchers = True
1950 self.suppress_competing_matchers = True
1951
1951
1952 @property
1952 @property
1953 def matchers(self) -> List[Matcher]:
1953 def matchers(self) -> List[Matcher]:
1954 """All active matcher routines for completion"""
1954 """All active matcher routines for completion"""
1955 if self.dict_keys_only:
1955 if self.dict_keys_only:
1956 return [self.dict_key_matcher]
1956 return [self.dict_key_matcher]
1957
1957
1958 if self.use_jedi:
1958 if self.use_jedi:
1959 return [
1959 return [
1960 *self.custom_matchers,
1960 *self.custom_matchers,
1961 *self._backslash_combining_matchers,
1961 *self._backslash_combining_matchers,
1962 *self.magic_arg_matchers,
1962 *self.magic_arg_matchers,
1963 self.custom_completer_matcher,
1963 self.custom_completer_matcher,
1964 self.magic_matcher,
1964 self.magic_matcher,
1965 self._jedi_matcher,
1965 self._jedi_matcher,
1966 self.dict_key_matcher,
1966 self.dict_key_matcher,
1967 self.file_matcher,
1967 self.file_matcher,
1968 ]
1968 ]
1969 else:
1969 else:
1970 return [
1970 return [
1971 *self.custom_matchers,
1971 *self.custom_matchers,
1972 *self._backslash_combining_matchers,
1972 *self._backslash_combining_matchers,
1973 *self.magic_arg_matchers,
1973 *self.magic_arg_matchers,
1974 self.custom_completer_matcher,
1974 self.custom_completer_matcher,
1975 self.dict_key_matcher,
1975 self.dict_key_matcher,
1976 # TODO: convert python_matches to v2 API
1976 # TODO: convert python_matches to v2 API
1977 self.magic_matcher,
1977 self.magic_matcher,
1978 self.python_matches,
1978 self.python_matches,
1979 self.file_matcher,
1979 self.file_matcher,
1980 self.python_func_kw_matcher,
1980 self.python_func_kw_matcher,
1981 ]
1981 ]
1982
1982
1983 def all_completions(self, text:str) -> List[str]:
1983 def all_completions(self, text:str) -> List[str]:
1984 """
1984 """
1985 Wrapper around the completion methods for the benefit of emacs.
1985 Wrapper around the completion methods for the benefit of emacs.
1986 """
1986 """
1987 prefix = text.rpartition('.')[0]
1987 prefix = text.rpartition('.')[0]
1988 with provisionalcompleter():
1988 with provisionalcompleter():
1989 return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
1989 return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
1990 for c in self.completions(text, len(text))]
1990 for c in self.completions(text, len(text))]
1991
1991
1992 return self.complete(text)[1]
1992 return self.complete(text)[1]
1993
1993
1994 def _clean_glob(self, text:str):
1994 def _clean_glob(self, text:str):
1995 return self.glob("%s*" % text)
1995 return self.glob("%s*" % text)
1996
1996
1997 def _clean_glob_win32(self, text:str):
1997 def _clean_glob_win32(self, text:str):
1998 return [f.replace("\\","/")
1998 return [f.replace("\\","/")
1999 for f in self.glob("%s*" % text)]
1999 for f in self.glob("%s*" % text)]
2000
2000
2001 @context_matcher()
2001 @context_matcher()
2002 def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2002 def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2003 """Same as :any:`file_matches`, but adopted to new Matcher API."""
2003 """Same as :any:`file_matches`, but adopted to new Matcher API."""
2004 matches = self.file_matches(context.token)
2004 matches = self.file_matches(context.token)
2005 # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
2005 # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
2006 # starts with `/home/`, `C:\`, etc)
2006 # starts with `/home/`, `C:\`, etc)
2007 return _convert_matcher_v1_result_to_v2(matches, type="path")
2007 return _convert_matcher_v1_result_to_v2(matches, type="path")
2008
2008
2009 def file_matches(self, text: str) -> List[str]:
2009 def file_matches(self, text: str) -> List[str]:
2010 """Match filenames, expanding ~USER type strings.
2010 """Match filenames, expanding ~USER type strings.
2011
2011
2012 Most of the seemingly convoluted logic in this completer is an
2012 Most of the seemingly convoluted logic in this completer is an
2013 attempt to handle filenames with spaces in them. And yet it's not
2013 attempt to handle filenames with spaces in them. And yet it's not
2014 quite perfect, because Python's readline doesn't expose all of the
2014 quite perfect, because Python's readline doesn't expose all of the
2015 GNU readline details needed for this to be done correctly.
2015 GNU readline details needed for this to be done correctly.
2016
2016
2017 For a filename with a space in it, the printed completions will be
2017 For a filename with a space in it, the printed completions will be
2018 only the parts after what's already been typed (instead of the
2018 only the parts after what's already been typed (instead of the
2019 full completions, as is normally done). I don't think with the
2019 full completions, as is normally done). I don't think with the
2020 current (as of Python 2.3) Python readline it's possible to do
2020 current (as of Python 2.3) Python readline it's possible to do
2021 better.
2021 better.
2022
2022
2023 .. deprecated:: 8.6
2023 .. deprecated:: 8.6
2024 You can use :meth:`file_matcher` instead.
2024 You can use :meth:`file_matcher` instead.
2025 """
2025 """
2026
2026
2027 # chars that require escaping with backslash - i.e. chars
2027 # chars that require escaping with backslash - i.e. chars
2028 # that readline treats incorrectly as delimiters, but we
2028 # that readline treats incorrectly as delimiters, but we
2029 # don't want to treat as delimiters in filename matching
2029 # don't want to treat as delimiters in filename matching
2030 # when escaped with backslash
2030 # when escaped with backslash
2031 if text.startswith('!'):
2031 if text.startswith('!'):
2032 text = text[1:]
2032 text = text[1:]
2033 text_prefix = u'!'
2033 text_prefix = u'!'
2034 else:
2034 else:
2035 text_prefix = u''
2035 text_prefix = u''
2036
2036
2037 text_until_cursor = self.text_until_cursor
2037 text_until_cursor = self.text_until_cursor
2038 # track strings with open quotes
2038 # track strings with open quotes
2039 open_quotes = has_open_quotes(text_until_cursor)
2039 open_quotes = has_open_quotes(text_until_cursor)
2040
2040
2041 if '(' in text_until_cursor or '[' in text_until_cursor:
2041 if '(' in text_until_cursor or '[' in text_until_cursor:
2042 lsplit = text
2042 lsplit = text
2043 else:
2043 else:
2044 try:
2044 try:
2045 # arg_split ~ shlex.split, but with unicode bugs fixed by us
2045 # arg_split ~ shlex.split, but with unicode bugs fixed by us
2046 lsplit = arg_split(text_until_cursor)[-1]
2046 lsplit = arg_split(text_until_cursor)[-1]
2047 except ValueError:
2047 except ValueError:
2048 # typically an unmatched ", or backslash without escaped char.
2048 # typically an unmatched ", or backslash without escaped char.
2049 if open_quotes:
2049 if open_quotes:
2050 lsplit = text_until_cursor.split(open_quotes)[-1]
2050 lsplit = text_until_cursor.split(open_quotes)[-1]
2051 else:
2051 else:
2052 return []
2052 return []
2053 except IndexError:
2053 except IndexError:
2054 # tab pressed on empty line
2054 # tab pressed on empty line
2055 lsplit = ""
2055 lsplit = ""
2056
2056
2057 if not open_quotes and lsplit != protect_filename(lsplit):
2057 if not open_quotes and lsplit != protect_filename(lsplit):
2058 # if protectables are found, do matching on the whole escaped name
2058 # if protectables are found, do matching on the whole escaped name
2059 has_protectables = True
2059 has_protectables = True
2060 text0,text = text,lsplit
2060 text0,text = text,lsplit
2061 else:
2061 else:
2062 has_protectables = False
2062 has_protectables = False
2063 text = os.path.expanduser(text)
2063 text = os.path.expanduser(text)
2064
2064
2065 if text == "":
2065 if text == "":
2066 return [text_prefix + protect_filename(f) for f in self.glob("*")]
2066 return [text_prefix + protect_filename(f) for f in self.glob("*")]
2067
2067
2068 # Compute the matches from the filesystem
2068 # Compute the matches from the filesystem
2069 if sys.platform == 'win32':
2069 if sys.platform == 'win32':
2070 m0 = self.clean_glob(text)
2070 m0 = self.clean_glob(text)
2071 else:
2071 else:
2072 m0 = self.clean_glob(text.replace('\\', ''))
2072 m0 = self.clean_glob(text.replace('\\', ''))
2073
2073
2074 if has_protectables:
2074 if has_protectables:
2075 # If we had protectables, we need to revert our changes to the
2075 # If we had protectables, we need to revert our changes to the
2076 # beginning of filename so that we don't double-write the part
2076 # beginning of filename so that we don't double-write the part
2077 # of the filename we have so far
2077 # of the filename we have so far
2078 len_lsplit = len(lsplit)
2078 len_lsplit = len(lsplit)
2079 matches = [text_prefix + text0 +
2079 matches = [text_prefix + text0 +
2080 protect_filename(f[len_lsplit:]) for f in m0]
2080 protect_filename(f[len_lsplit:]) for f in m0]
2081 else:
2081 else:
2082 if open_quotes:
2082 if open_quotes:
2083 # if we have a string with an open quote, we don't need to
2083 # if we have a string with an open quote, we don't need to
2084 # protect the names beyond the quote (and we _shouldn't_, as
2084 # protect the names beyond the quote (and we _shouldn't_, as
2085 # it would cause bugs when the filesystem call is made).
2085 # it would cause bugs when the filesystem call is made).
2086 matches = m0 if sys.platform == "win32" else\
2086 matches = m0 if sys.platform == "win32" else\
2087 [protect_filename(f, open_quotes) for f in m0]
2087 [protect_filename(f, open_quotes) for f in m0]
2088 else:
2088 else:
2089 matches = [text_prefix +
2089 matches = [text_prefix +
2090 protect_filename(f) for f in m0]
2090 protect_filename(f) for f in m0]
2091
2091
2092 # Mark directories in input list by appending '/' to their names.
2092 # Mark directories in input list by appending '/' to their names.
2093 return [x+'/' if os.path.isdir(x) else x for x in matches]
2093 return [x+'/' if os.path.isdir(x) else x for x in matches]
2094
2094
2095 @context_matcher()
2095 @context_matcher()
2096 def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2096 def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2097 """Match magics."""
2097 """Match magics."""
2098 text = context.token
2098 text = context.token
2099 matches = self.magic_matches(text)
2099 matches = self.magic_matches(text)
2100 result = _convert_matcher_v1_result_to_v2(matches, type="magic")
2100 result = _convert_matcher_v1_result_to_v2(matches, type="magic")
2101 is_magic_prefix = len(text) > 0 and text[0] == "%"
2101 is_magic_prefix = len(text) > 0 and text[0] == "%"
2102 result["suppress"] = is_magic_prefix and bool(result["completions"])
2102 result["suppress"] = is_magic_prefix and bool(result["completions"])
2103 return result
2103 return result
2104
2104
2105 def magic_matches(self, text: str):
2105 def magic_matches(self, text: str):
2106 """Match magics.
2106 """Match magics.
2107
2107
2108 .. deprecated:: 8.6
2108 .. deprecated:: 8.6
2109 You can use :meth:`magic_matcher` instead.
2109 You can use :meth:`magic_matcher` instead.
2110 """
2110 """
2111 # Get all shell magics now rather than statically, so magics loaded at
2111 # Get all shell magics now rather than statically, so magics loaded at
2112 # runtime show up too.
2112 # runtime show up too.
2113 lsm = self.shell.magics_manager.lsmagic()
2113 lsm = self.shell.magics_manager.lsmagic()
2114 line_magics = lsm['line']
2114 line_magics = lsm['line']
2115 cell_magics = lsm['cell']
2115 cell_magics = lsm['cell']
2116 pre = self.magic_escape
2116 pre = self.magic_escape
2117 pre2 = pre+pre
2117 pre2 = pre+pre
2118
2118
2119 explicit_magic = text.startswith(pre)
2119 explicit_magic = text.startswith(pre)
2120
2120
2121 # Completion logic:
2121 # Completion logic:
2122 # - user gives %%: only do cell magics
2122 # - user gives %%: only do cell magics
2123 # - user gives %: do both line and cell magics
2123 # - user gives %: do both line and cell magics
2124 # - no prefix: do both
2124 # - no prefix: do both
2125 # In other words, line magics are skipped if the user gives %% explicitly
2125 # In other words, line magics are skipped if the user gives %% explicitly
2126 #
2126 #
2127 # We also exclude magics that match any currently visible names:
2127 # We also exclude magics that match any currently visible names:
2128 # https://github.com/ipython/ipython/issues/4877, unless the user has
2128 # https://github.com/ipython/ipython/issues/4877, unless the user has
2129 # typed a %:
2129 # typed a %:
2130 # https://github.com/ipython/ipython/issues/10754
2130 # https://github.com/ipython/ipython/issues/10754
2131 bare_text = text.lstrip(pre)
2131 bare_text = text.lstrip(pre)
2132 global_matches = self.global_matches(bare_text)
2132 global_matches = self.global_matches(bare_text)
2133 if not explicit_magic:
2133 if not explicit_magic:
2134 def matches(magic):
2134 def matches(magic):
2135 """
2135 """
2136 Filter magics, in particular remove magics that match
2136 Filter magics, in particular remove magics that match
2137 a name present in global namespace.
2137 a name present in global namespace.
2138 """
2138 """
2139 return ( magic.startswith(bare_text) and
2139 return ( magic.startswith(bare_text) and
2140 magic not in global_matches )
2140 magic not in global_matches )
2141 else:
2141 else:
2142 def matches(magic):
2142 def matches(magic):
2143 return magic.startswith(bare_text)
2143 return magic.startswith(bare_text)
2144
2144
2145 comp = [ pre2+m for m in cell_magics if matches(m)]
2145 comp = [ pre2+m for m in cell_magics if matches(m)]
2146 if not text.startswith(pre2):
2146 if not text.startswith(pre2):
2147 comp += [ pre+m for m in line_magics if matches(m)]
2147 comp += [ pre+m for m in line_magics if matches(m)]
2148
2148
2149 return comp
2149 return comp
2150
2150
2151 @context_matcher()
2151 @context_matcher()
2152 def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2152 def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2153 """Match class names and attributes for %config magic."""
2153 """Match class names and attributes for %config magic."""
2154 # NOTE: uses `line_buffer` equivalent for compatibility
2154 # NOTE: uses `line_buffer` equivalent for compatibility
2155 matches = self.magic_config_matches(context.line_with_cursor)
2155 matches = self.magic_config_matches(context.line_with_cursor)
2156 return _convert_matcher_v1_result_to_v2(matches, type="param")
2156 return _convert_matcher_v1_result_to_v2(matches, type="param")
2157
2157
2158 def magic_config_matches(self, text: str) -> List[str]:
2158 def magic_config_matches(self, text: str) -> List[str]:
2159 """Match class names and attributes for %config magic.
2159 """Match class names and attributes for %config magic.
2160
2160
2161 .. deprecated:: 8.6
2161 .. deprecated:: 8.6
2162 You can use :meth:`magic_config_matcher` instead.
2162 You can use :meth:`magic_config_matcher` instead.
2163 """
2163 """
2164 texts = text.strip().split()
2164 texts = text.strip().split()
2165
2165
2166 if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
2166 if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
2167 # get all configuration classes
2167 # get all configuration classes
2168 classes = sorted(set([ c for c in self.shell.configurables
2168 classes = sorted(set([ c for c in self.shell.configurables
2169 if c.__class__.class_traits(config=True)
2169 if c.__class__.class_traits(config=True)
2170 ]), key=lambda x: x.__class__.__name__)
2170 ]), key=lambda x: x.__class__.__name__)
2171 classnames = [ c.__class__.__name__ for c in classes ]
2171 classnames = [ c.__class__.__name__ for c in classes ]
2172
2172
2173 # return all classnames if config or %config is given
2173 # return all classnames if config or %config is given
2174 if len(texts) == 1:
2174 if len(texts) == 1:
2175 return classnames
2175 return classnames
2176
2176
2177 # match classname
2177 # match classname
2178 classname_texts = texts[1].split('.')
2178 classname_texts = texts[1].split('.')
2179 classname = classname_texts[0]
2179 classname = classname_texts[0]
2180 classname_matches = [ c for c in classnames
2180 classname_matches = [ c for c in classnames
2181 if c.startswith(classname) ]
2181 if c.startswith(classname) ]
2182
2182
2183 # return matched classes or the matched class with attributes
2183 # return matched classes or the matched class with attributes
2184 if texts[1].find('.') < 0:
2184 if texts[1].find('.') < 0:
2185 return classname_matches
2185 return classname_matches
2186 elif len(classname_matches) == 1 and \
2186 elif len(classname_matches) == 1 and \
2187 classname_matches[0] == classname:
2187 classname_matches[0] == classname:
2188 cls = classes[classnames.index(classname)].__class__
2188 cls = classes[classnames.index(classname)].__class__
2189 help = cls.class_get_help()
2189 help = cls.class_get_help()
2190 # strip leading '--' from cl-args:
2190 # strip leading '--' from cl-args:
2191 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
2191 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
2192 return [ attr.split('=')[0]
2192 return [ attr.split('=')[0]
2193 for attr in help.strip().splitlines()
2193 for attr in help.strip().splitlines()
2194 if attr.startswith(texts[1]) ]
2194 if attr.startswith(texts[1]) ]
2195 return []
2195 return []
2196
2196
2197 @context_matcher()
2197 @context_matcher()
2198 def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2198 def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2199 """Match color schemes for %colors magic."""
2199 """Match color schemes for %colors magic."""
2200 # NOTE: uses `line_buffer` equivalent for compatibility
2200 # NOTE: uses `line_buffer` equivalent for compatibility
2201 matches = self.magic_color_matches(context.line_with_cursor)
2201 matches = self.magic_color_matches(context.line_with_cursor)
2202 return _convert_matcher_v1_result_to_v2(matches, type="param")
2202 return _convert_matcher_v1_result_to_v2(matches, type="param")
2203
2203
2204 def magic_color_matches(self, text: str) -> List[str]:
2204 def magic_color_matches(self, text: str) -> List[str]:
2205 """Match color schemes for %colors magic.
2205 """Match color schemes for %colors magic.
2206
2206
2207 .. deprecated:: 8.6
2207 .. deprecated:: 8.6
2208 You can use :meth:`magic_color_matcher` instead.
2208 You can use :meth:`magic_color_matcher` instead.
2209 """
2209 """
2210 texts = text.split()
2210 texts = text.split()
2211 if text.endswith(' '):
2211 if text.endswith(' '):
2212 # .split() strips off the trailing whitespace. Add '' back
2212 # .split() strips off the trailing whitespace. Add '' back
2213 # so that: '%colors ' -> ['%colors', '']
2213 # so that: '%colors ' -> ['%colors', '']
2214 texts.append('')
2214 texts.append('')
2215
2215
2216 if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
2216 if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
2217 prefix = texts[1]
2217 prefix = texts[1]
2218 return [ color for color in InspectColors.keys()
2218 return [ color for color in InspectColors.keys()
2219 if color.startswith(prefix) ]
2219 if color.startswith(prefix) ]
2220 return []
2220 return []
2221
2221
2222 @context_matcher(identifier="IPCompleter.jedi_matcher")
2222 @context_matcher(identifier="IPCompleter.jedi_matcher")
2223 def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
2223 def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
2224 matches = self._jedi_matches(
2224 matches = self._jedi_matches(
2225 cursor_column=context.cursor_position,
2225 cursor_column=context.cursor_position,
2226 cursor_line=context.cursor_line,
2226 cursor_line=context.cursor_line,
2227 text=context.full_text,
2227 text=context.full_text,
2228 )
2228 )
2229 return {
2229 return {
2230 "completions": matches,
2230 "completions": matches,
2231 # static analysis should not suppress other matchers
2231 # static analysis should not suppress other matchers
2232 "suppress": False,
2232 "suppress": False,
2233 }
2233 }
2234
2234
2235 def _jedi_matches(
2235 def _jedi_matches(
2236 self, cursor_column: int, cursor_line: int, text: str
2236 self, cursor_column: int, cursor_line: int, text: str
2237 ) -> Iterator[_JediCompletionLike]:
2237 ) -> Iterator[_JediCompletionLike]:
2238 """
2238 """
2239 Return a list of :any:`jedi.api.Completion`s object from a ``text`` and
2239 Return a list of :any:`jedi.api.Completion`\\s object from a ``text`` and
2240 cursor position.
2240 cursor position.
2241
2241
2242 Parameters
2242 Parameters
2243 ----------
2243 ----------
2244 cursor_column : int
2244 cursor_column : int
2245 column position of the cursor in ``text``, 0-indexed.
2245 column position of the cursor in ``text``, 0-indexed.
2246 cursor_line : int
2246 cursor_line : int
2247 line position of the cursor in ``text``, 0-indexed
2247 line position of the cursor in ``text``, 0-indexed
2248 text : str
2248 text : str
2249 text to complete
2249 text to complete
2250
2250
2251 Notes
2251 Notes
2252 -----
2252 -----
2253 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
2253 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
2254 object containing a string with the Jedi debug information attached.
2254 object containing a string with the Jedi debug information attached.
2255
2255
2256 .. deprecated:: 8.6
2256 .. deprecated:: 8.6
2257 You can use :meth:`_jedi_matcher` instead.
2257 You can use :meth:`_jedi_matcher` instead.
2258 """
2258 """
2259 namespaces = [self.namespace]
2259 namespaces = [self.namespace]
2260 if self.global_namespace is not None:
2260 if self.global_namespace is not None:
2261 namespaces.append(self.global_namespace)
2261 namespaces.append(self.global_namespace)
2262
2262
2263 completion_filter = lambda x:x
2263 completion_filter = lambda x:x
2264 offset = cursor_to_position(text, cursor_line, cursor_column)
2264 offset = cursor_to_position(text, cursor_line, cursor_column)
2265 # filter output if we are completing for object members
2265 # filter output if we are completing for object members
2266 if offset:
2266 if offset:
2267 pre = text[offset-1]
2267 pre = text[offset-1]
2268 if pre == '.':
2268 if pre == '.':
2269 if self.omit__names == 2:
2269 if self.omit__names == 2:
2270 completion_filter = lambda c:not c.name.startswith('_')
2270 completion_filter = lambda c:not c.name.startswith('_')
2271 elif self.omit__names == 1:
2271 elif self.omit__names == 1:
2272 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
2272 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
2273 elif self.omit__names == 0:
2273 elif self.omit__names == 0:
2274 completion_filter = lambda x:x
2274 completion_filter = lambda x:x
2275 else:
2275 else:
2276 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
2276 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
2277
2277
2278 interpreter = jedi.Interpreter(text[:offset], namespaces)
2278 interpreter = jedi.Interpreter(text[:offset], namespaces)
2279 try_jedi = True
2279 try_jedi = True
2280
2280
2281 try:
2281 try:
2282 # find the first token in the current tree -- if it is a ' or " then we are in a string
2282 # find the first token in the current tree -- if it is a ' or " then we are in a string
2283 completing_string = False
2283 completing_string = False
2284 try:
2284 try:
2285 first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
2285 first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
2286 except StopIteration:
2286 except StopIteration:
2287 pass
2287 pass
2288 else:
2288 else:
2289 # note the value may be ', ", or it may also be ''' or """, or
2289 # note the value may be ', ", or it may also be ''' or """, or
2290 # in some cases, """what/you/typed..., but all of these are
2290 # in some cases, """what/you/typed..., but all of these are
2291 # strings.
2291 # strings.
2292 completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
2292 completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
2293
2293
2294 # if we are in a string jedi is likely not the right candidate for
2294 # if we are in a string jedi is likely not the right candidate for
2295 # now. Skip it.
2295 # now. Skip it.
2296 try_jedi = not completing_string
2296 try_jedi = not completing_string
2297 except Exception as e:
2297 except Exception as e:
2298 # many of things can go wrong, we are using private API just don't crash.
2298 # many of things can go wrong, we are using private API just don't crash.
2299 if self.debug:
2299 if self.debug:
2300 print("Error detecting if completing a non-finished string :", e, '|')
2300 print("Error detecting if completing a non-finished string :", e, '|')
2301
2301
2302 if not try_jedi:
2302 if not try_jedi:
2303 return iter([])
2303 return iter([])
2304 try:
2304 try:
2305 return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
2305 return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
2306 except Exception as e:
2306 except Exception as e:
2307 if self.debug:
2307 if self.debug:
2308 return iter(
2308 return iter(
2309 [
2309 [
2310 _FakeJediCompletion(
2310 _FakeJediCompletion(
2311 'Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""'
2311 'Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""'
2312 % (e)
2312 % (e)
2313 )
2313 )
2314 ]
2314 ]
2315 )
2315 )
2316 else:
2316 else:
2317 return iter([])
2317 return iter([])
2318
2318
2319 @completion_matcher(api_version=1)
2319 @completion_matcher(api_version=1)
2320 def python_matches(self, text: str) -> Iterable[str]:
2320 def python_matches(self, text: str) -> Iterable[str]:
2321 """Match attributes or global python names"""
2321 """Match attributes or global python names"""
2322 if "." in text:
2322 if "." in text:
2323 try:
2323 try:
2324 matches = self.attr_matches(text)
2324 matches = self.attr_matches(text)
2325 if text.endswith('.') and self.omit__names:
2325 if text.endswith('.') and self.omit__names:
2326 if self.omit__names == 1:
2326 if self.omit__names == 1:
2327 # true if txt is _not_ a __ name, false otherwise:
2327 # true if txt is _not_ a __ name, false otherwise:
2328 no__name = (lambda txt:
2328 no__name = (lambda txt:
2329 re.match(r'.*\.__.*?__',txt) is None)
2329 re.match(r'.*\.__.*?__',txt) is None)
2330 else:
2330 else:
2331 # true if txt is _not_ a _ name, false otherwise:
2331 # true if txt is _not_ a _ name, false otherwise:
2332 no__name = (lambda txt:
2332 no__name = (lambda txt:
2333 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
2333 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
2334 matches = filter(no__name, matches)
2334 matches = filter(no__name, matches)
2335 except NameError:
2335 except NameError:
2336 # catches <undefined attributes>.<tab>
2336 # catches <undefined attributes>.<tab>
2337 matches = []
2337 matches = []
2338 else:
2338 else:
2339 matches = self.global_matches(text)
2339 matches = self.global_matches(text)
2340 return matches
2340 return matches
2341
2341
2342 def _default_arguments_from_docstring(self, doc):
2342 def _default_arguments_from_docstring(self, doc):
2343 """Parse the first line of docstring for call signature.
2343 """Parse the first line of docstring for call signature.
2344
2344
2345 Docstring should be of the form 'min(iterable[, key=func])\n'.
2345 Docstring should be of the form 'min(iterable[, key=func])\n'.
2346 It can also parse cython docstring of the form
2346 It can also parse cython docstring of the form
2347 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
2347 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
2348 """
2348 """
2349 if doc is None:
2349 if doc is None:
2350 return []
2350 return []
2351
2351
2352 #care only the firstline
2352 #care only the firstline
2353 line = doc.lstrip().splitlines()[0]
2353 line = doc.lstrip().splitlines()[0]
2354
2354
2355 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
2355 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
2356 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
2356 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
2357 sig = self.docstring_sig_re.search(line)
2357 sig = self.docstring_sig_re.search(line)
2358 if sig is None:
2358 if sig is None:
2359 return []
2359 return []
2360 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
2360 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
2361 sig = sig.groups()[0].split(',')
2361 sig = sig.groups()[0].split(',')
2362 ret = []
2362 ret = []
2363 for s in sig:
2363 for s in sig:
2364 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
2364 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
2365 ret += self.docstring_kwd_re.findall(s)
2365 ret += self.docstring_kwd_re.findall(s)
2366 return ret
2366 return ret
2367
2367
2368 def _default_arguments(self, obj):
2368 def _default_arguments(self, obj):
2369 """Return the list of default arguments of obj if it is callable,
2369 """Return the list of default arguments of obj if it is callable,
2370 or empty list otherwise."""
2370 or empty list otherwise."""
2371 call_obj = obj
2371 call_obj = obj
2372 ret = []
2372 ret = []
2373 if inspect.isbuiltin(obj):
2373 if inspect.isbuiltin(obj):
2374 pass
2374 pass
2375 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
2375 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
2376 if inspect.isclass(obj):
2376 if inspect.isclass(obj):
2377 #for cython embedsignature=True the constructor docstring
2377 #for cython embedsignature=True the constructor docstring
2378 #belongs to the object itself not __init__
2378 #belongs to the object itself not __init__
2379 ret += self._default_arguments_from_docstring(
2379 ret += self._default_arguments_from_docstring(
2380 getattr(obj, '__doc__', ''))
2380 getattr(obj, '__doc__', ''))
2381 # for classes, check for __init__,__new__
2381 # for classes, check for __init__,__new__
2382 call_obj = (getattr(obj, '__init__', None) or
2382 call_obj = (getattr(obj, '__init__', None) or
2383 getattr(obj, '__new__', None))
2383 getattr(obj, '__new__', None))
2384 # for all others, check if they are __call__able
2384 # for all others, check if they are __call__able
2385 elif hasattr(obj, '__call__'):
2385 elif hasattr(obj, '__call__'):
2386 call_obj = obj.__call__
2386 call_obj = obj.__call__
2387 ret += self._default_arguments_from_docstring(
2387 ret += self._default_arguments_from_docstring(
2388 getattr(call_obj, '__doc__', ''))
2388 getattr(call_obj, '__doc__', ''))
2389
2389
2390 _keeps = (inspect.Parameter.KEYWORD_ONLY,
2390 _keeps = (inspect.Parameter.KEYWORD_ONLY,
2391 inspect.Parameter.POSITIONAL_OR_KEYWORD)
2391 inspect.Parameter.POSITIONAL_OR_KEYWORD)
2392
2392
2393 try:
2393 try:
2394 sig = inspect.signature(obj)
2394 sig = inspect.signature(obj)
2395 ret.extend(k for k, v in sig.parameters.items() if
2395 ret.extend(k for k, v in sig.parameters.items() if
2396 v.kind in _keeps)
2396 v.kind in _keeps)
2397 except ValueError:
2397 except ValueError:
2398 pass
2398 pass
2399
2399
2400 return list(set(ret))
2400 return list(set(ret))
2401
2401
2402 @context_matcher()
2402 @context_matcher()
2403 def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2403 def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2404 """Match named parameters (kwargs) of the last open function."""
2404 """Match named parameters (kwargs) of the last open function."""
2405 matches = self.python_func_kw_matches(context.token)
2405 matches = self.python_func_kw_matches(context.token)
2406 return _convert_matcher_v1_result_to_v2(matches, type="param")
2406 return _convert_matcher_v1_result_to_v2(matches, type="param")
2407
2407
2408 def python_func_kw_matches(self, text):
2408 def python_func_kw_matches(self, text):
2409 """Match named parameters (kwargs) of the last open function.
2409 """Match named parameters (kwargs) of the last open function.
2410
2410
2411 .. deprecated:: 8.6
2411 .. deprecated:: 8.6
2412 You can use :meth:`python_func_kw_matcher` instead.
2412 You can use :meth:`python_func_kw_matcher` instead.
2413 """
2413 """
2414
2414
2415 if "." in text: # a parameter cannot be dotted
2415 if "." in text: # a parameter cannot be dotted
2416 return []
2416 return []
2417 try: regexp = self.__funcParamsRegex
2417 try: regexp = self.__funcParamsRegex
2418 except AttributeError:
2418 except AttributeError:
2419 regexp = self.__funcParamsRegex = re.compile(r'''
2419 regexp = self.__funcParamsRegex = re.compile(r'''
2420 '.*?(?<!\\)' | # single quoted strings or
2420 '.*?(?<!\\)' | # single quoted strings or
2421 ".*?(?<!\\)" | # double quoted strings or
2421 ".*?(?<!\\)" | # double quoted strings or
2422 \w+ | # identifier
2422 \w+ | # identifier
2423 \S # other characters
2423 \S # other characters
2424 ''', re.VERBOSE | re.DOTALL)
2424 ''', re.VERBOSE | re.DOTALL)
2425 # 1. find the nearest identifier that comes before an unclosed
2425 # 1. find the nearest identifier that comes before an unclosed
2426 # parenthesis before the cursor
2426 # parenthesis before the cursor
2427 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
2427 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
2428 tokens = regexp.findall(self.text_until_cursor)
2428 tokens = regexp.findall(self.text_until_cursor)
2429 iterTokens = reversed(tokens); openPar = 0
2429 iterTokens = reversed(tokens); openPar = 0
2430
2430
2431 for token in iterTokens:
2431 for token in iterTokens:
2432 if token == ')':
2432 if token == ')':
2433 openPar -= 1
2433 openPar -= 1
2434 elif token == '(':
2434 elif token == '(':
2435 openPar += 1
2435 openPar += 1
2436 if openPar > 0:
2436 if openPar > 0:
2437 # found the last unclosed parenthesis
2437 # found the last unclosed parenthesis
2438 break
2438 break
2439 else:
2439 else:
2440 return []
2440 return []
2441 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
2441 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
2442 ids = []
2442 ids = []
2443 isId = re.compile(r'\w+$').match
2443 isId = re.compile(r'\w+$').match
2444
2444
2445 while True:
2445 while True:
2446 try:
2446 try:
2447 ids.append(next(iterTokens))
2447 ids.append(next(iterTokens))
2448 if not isId(ids[-1]):
2448 if not isId(ids[-1]):
2449 ids.pop(); break
2449 ids.pop(); break
2450 if not next(iterTokens) == '.':
2450 if not next(iterTokens) == '.':
2451 break
2451 break
2452 except StopIteration:
2452 except StopIteration:
2453 break
2453 break
2454
2454
2455 # Find all named arguments already assigned to, as to avoid suggesting
2455 # Find all named arguments already assigned to, as to avoid suggesting
2456 # them again
2456 # them again
2457 usedNamedArgs = set()
2457 usedNamedArgs = set()
2458 par_level = -1
2458 par_level = -1
2459 for token, next_token in zip(tokens, tokens[1:]):
2459 for token, next_token in zip(tokens, tokens[1:]):
2460 if token == '(':
2460 if token == '(':
2461 par_level += 1
2461 par_level += 1
2462 elif token == ')':
2462 elif token == ')':
2463 par_level -= 1
2463 par_level -= 1
2464
2464
2465 if par_level != 0:
2465 if par_level != 0:
2466 continue
2466 continue
2467
2467
2468 if next_token != '=':
2468 if next_token != '=':
2469 continue
2469 continue
2470
2470
2471 usedNamedArgs.add(token)
2471 usedNamedArgs.add(token)
2472
2472
2473 argMatches = []
2473 argMatches = []
2474 try:
2474 try:
2475 callableObj = '.'.join(ids[::-1])
2475 callableObj = '.'.join(ids[::-1])
2476 namedArgs = self._default_arguments(eval(callableObj,
2476 namedArgs = self._default_arguments(eval(callableObj,
2477 self.namespace))
2477 self.namespace))
2478
2478
2479 # Remove used named arguments from the list, no need to show twice
2479 # Remove used named arguments from the list, no need to show twice
2480 for namedArg in set(namedArgs) - usedNamedArgs:
2480 for namedArg in set(namedArgs) - usedNamedArgs:
2481 if namedArg.startswith(text):
2481 if namedArg.startswith(text):
2482 argMatches.append("%s=" %namedArg)
2482 argMatches.append("%s=" %namedArg)
2483 except:
2483 except:
2484 pass
2484 pass
2485
2485
2486 return argMatches
2486 return argMatches
2487
2487
2488 @staticmethod
2488 @staticmethod
2489 def _get_keys(obj: Any) -> List[Any]:
2489 def _get_keys(obj: Any) -> List[Any]:
2490 # Objects can define their own completions by defining an
2490 # Objects can define their own completions by defining an
2491 # _ipy_key_completions_() method.
2491 # _ipy_key_completions_() method.
2492 method = get_real_method(obj, '_ipython_key_completions_')
2492 method = get_real_method(obj, '_ipython_key_completions_')
2493 if method is not None:
2493 if method is not None:
2494 return method()
2494 return method()
2495
2495
2496 # Special case some common in-memory dict-like types
2496 # Special case some common in-memory dict-like types
2497 if isinstance(obj, dict) or _safe_isinstance(obj, "pandas", "DataFrame"):
2497 if isinstance(obj, dict) or _safe_isinstance(obj, "pandas", "DataFrame"):
2498 try:
2498 try:
2499 return list(obj.keys())
2499 return list(obj.keys())
2500 except Exception:
2500 except Exception:
2501 return []
2501 return []
2502 elif _safe_isinstance(obj, "pandas", "core", "indexing", "_LocIndexer"):
2502 elif _safe_isinstance(obj, "pandas", "core", "indexing", "_LocIndexer"):
2503 try:
2503 try:
2504 return list(obj.obj.keys())
2504 return list(obj.obj.keys())
2505 except Exception:
2505 except Exception:
2506 return []
2506 return []
2507 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
2507 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
2508 _safe_isinstance(obj, 'numpy', 'void'):
2508 _safe_isinstance(obj, 'numpy', 'void'):
2509 return obj.dtype.names or []
2509 return obj.dtype.names or []
2510 return []
2510 return []
2511
2511
2512 @context_matcher()
2512 @context_matcher()
2513 def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2513 def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2514 """Match string keys in a dictionary, after e.g. ``foo[``."""
2514 """Match string keys in a dictionary, after e.g. ``foo[``."""
2515 matches = self.dict_key_matches(context.token)
2515 matches = self.dict_key_matches(context.token)
2516 return _convert_matcher_v1_result_to_v2(
2516 return _convert_matcher_v1_result_to_v2(
2517 matches, type="dict key", suppress_if_matches=True
2517 matches, type="dict key", suppress_if_matches=True
2518 )
2518 )
2519
2519
2520 def dict_key_matches(self, text: str) -> List[str]:
2520 def dict_key_matches(self, text: str) -> List[str]:
2521 """Match string keys in a dictionary, after e.g. ``foo[``.
2521 """Match string keys in a dictionary, after e.g. ``foo[``.
2522
2522
2523 .. deprecated:: 8.6
2523 .. deprecated:: 8.6
2524 You can use :meth:`dict_key_matcher` instead.
2524 You can use :meth:`dict_key_matcher` instead.
2525 """
2525 """
2526
2526
2527 # Short-circuit on closed dictionary (regular expression would
2527 # Short-circuit on closed dictionary (regular expression would
2528 # not match anyway, but would take quite a while).
2528 # not match anyway, but would take quite a while).
2529 if self.text_until_cursor.strip().endswith("]"):
2529 if self.text_until_cursor.strip().endswith("]"):
2530 return []
2530 return []
2531
2531
2532 match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
2532 match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
2533
2533
2534 if match is None:
2534 if match is None:
2535 return []
2535 return []
2536
2536
2537 expr, prior_tuple_keys, key_prefix = match.groups()
2537 expr, prior_tuple_keys, key_prefix = match.groups()
2538
2538
2539 obj = self._evaluate_expr(expr)
2539 obj = self._evaluate_expr(expr)
2540
2540
2541 if obj is not_found:
2541 if obj is not_found:
2542 return []
2542 return []
2543
2543
2544 keys = self._get_keys(obj)
2544 keys = self._get_keys(obj)
2545 if not keys:
2545 if not keys:
2546 return keys
2546 return keys
2547
2547
2548 tuple_prefix = guarded_eval(
2548 tuple_prefix = guarded_eval(
2549 prior_tuple_keys,
2549 prior_tuple_keys,
2550 EvaluationContext(
2550 EvaluationContext(
2551 globals=self.global_namespace,
2551 globals=self.global_namespace,
2552 locals=self.namespace,
2552 locals=self.namespace,
2553 evaluation=self.evaluation,
2553 evaluation=self.evaluation,
2554 in_subscript=True,
2554 in_subscript=True,
2555 ),
2555 ),
2556 )
2556 )
2557
2557
2558 closing_quote, token_offset, matches = match_dict_keys(
2558 closing_quote, token_offset, matches = match_dict_keys(
2559 keys, key_prefix, self.splitter.delims, extra_prefix=tuple_prefix
2559 keys, key_prefix, self.splitter.delims, extra_prefix=tuple_prefix
2560 )
2560 )
2561 if not matches:
2561 if not matches:
2562 return []
2562 return []
2563
2563
2564 # get the cursor position of
2564 # get the cursor position of
2565 # - the text being completed
2565 # - the text being completed
2566 # - the start of the key text
2566 # - the start of the key text
2567 # - the start of the completion
2567 # - the start of the completion
2568 text_start = len(self.text_until_cursor) - len(text)
2568 text_start = len(self.text_until_cursor) - len(text)
2569 if key_prefix:
2569 if key_prefix:
2570 key_start = match.start(3)
2570 key_start = match.start(3)
2571 completion_start = key_start + token_offset
2571 completion_start = key_start + token_offset
2572 else:
2572 else:
2573 key_start = completion_start = match.end()
2573 key_start = completion_start = match.end()
2574
2574
2575 # grab the leading prefix, to make sure all completions start with `text`
2575 # grab the leading prefix, to make sure all completions start with `text`
2576 if text_start > key_start:
2576 if text_start > key_start:
2577 leading = ''
2577 leading = ''
2578 else:
2578 else:
2579 leading = text[text_start:completion_start]
2579 leading = text[text_start:completion_start]
2580
2580
2581 # append closing quote and bracket as appropriate
2581 # append closing quote and bracket as appropriate
2582 # this is *not* appropriate if the opening quote or bracket is outside
2582 # this is *not* appropriate if the opening quote or bracket is outside
2583 # the text given to this method, e.g. `d["""a\nt
2583 # the text given to this method, e.g. `d["""a\nt
2584 can_close_quote = False
2584 can_close_quote = False
2585 can_close_bracket = False
2585 can_close_bracket = False
2586
2586
2587 continuation = self.line_buffer[len(self.text_until_cursor) :].strip()
2587 continuation = self.line_buffer[len(self.text_until_cursor) :].strip()
2588
2588
2589 if continuation.startswith(closing_quote):
2589 if continuation.startswith(closing_quote):
2590 # do not close if already closed, e.g. `d['a<tab>'`
2590 # do not close if already closed, e.g. `d['a<tab>'`
2591 continuation = continuation[len(closing_quote) :]
2591 continuation = continuation[len(closing_quote) :]
2592 else:
2592 else:
2593 can_close_quote = True
2593 can_close_quote = True
2594
2594
2595 continuation = continuation.strip()
2595 continuation = continuation.strip()
2596
2596
2597 # e.g. `pandas.DataFrame` has different tuple indexer behaviour,
2597 # e.g. `pandas.DataFrame` has different tuple indexer behaviour,
2598 # handling it is out of scope, so let's avoid appending suffixes.
2598 # handling it is out of scope, so let's avoid appending suffixes.
2599 has_known_tuple_handling = isinstance(obj, dict)
2599 has_known_tuple_handling = isinstance(obj, dict)
2600
2600
2601 can_close_bracket = (
2601 can_close_bracket = (
2602 not continuation.startswith("]") and self.auto_close_dict_keys
2602 not continuation.startswith("]") and self.auto_close_dict_keys
2603 )
2603 )
2604 can_close_tuple_item = (
2604 can_close_tuple_item = (
2605 not continuation.startswith(",")
2605 not continuation.startswith(",")
2606 and has_known_tuple_handling
2606 and has_known_tuple_handling
2607 and self.auto_close_dict_keys
2607 and self.auto_close_dict_keys
2608 )
2608 )
2609 can_close_quote = can_close_quote and self.auto_close_dict_keys
2609 can_close_quote = can_close_quote and self.auto_close_dict_keys
2610
2610
2611 # fast path if closing qoute should be appended but not suffix is allowed
2611 # fast path if closing qoute should be appended but not suffix is allowed
2612 if not can_close_quote and not can_close_bracket and closing_quote:
2612 if not can_close_quote and not can_close_bracket and closing_quote:
2613 return [leading + k for k in matches]
2613 return [leading + k for k in matches]
2614
2614
2615 results = []
2615 results = []
2616
2616
2617 end_of_tuple_or_item = _DictKeyState.END_OF_TUPLE | _DictKeyState.END_OF_ITEM
2617 end_of_tuple_or_item = _DictKeyState.END_OF_TUPLE | _DictKeyState.END_OF_ITEM
2618
2618
2619 for k, state_flag in matches.items():
2619 for k, state_flag in matches.items():
2620 result = leading + k
2620 result = leading + k
2621 if can_close_quote and closing_quote:
2621 if can_close_quote and closing_quote:
2622 result += closing_quote
2622 result += closing_quote
2623
2623
2624 if state_flag == end_of_tuple_or_item:
2624 if state_flag == end_of_tuple_or_item:
2625 # We do not know which suffix to add,
2625 # We do not know which suffix to add,
2626 # e.g. both tuple item and string
2626 # e.g. both tuple item and string
2627 # match this item.
2627 # match this item.
2628 pass
2628 pass
2629
2629
2630 if state_flag in end_of_tuple_or_item and can_close_bracket:
2630 if state_flag in end_of_tuple_or_item and can_close_bracket:
2631 result += "]"
2631 result += "]"
2632 if state_flag == _DictKeyState.IN_TUPLE and can_close_tuple_item:
2632 if state_flag == _DictKeyState.IN_TUPLE and can_close_tuple_item:
2633 result += ", "
2633 result += ", "
2634 results.append(result)
2634 results.append(result)
2635 return results
2635 return results
2636
2636
2637 @context_matcher()
2637 @context_matcher()
2638 def unicode_name_matcher(self, context: CompletionContext):
2638 def unicode_name_matcher(self, context: CompletionContext):
2639 """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2639 """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2640 fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2640 fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2641 return _convert_matcher_v1_result_to_v2(
2641 return _convert_matcher_v1_result_to_v2(
2642 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2642 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2643 )
2643 )
2644
2644
2645 @staticmethod
2645 @staticmethod
2646 def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
2646 def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
2647 """Match Latex-like syntax for unicode characters base
2647 """Match Latex-like syntax for unicode characters base
2648 on the name of the character.
2648 on the name of the character.
2649
2649
2650 This does ``\\GREEK SMALL LETTER ETA`` -> ``Ξ·``
2650 This does ``\\GREEK SMALL LETTER ETA`` -> ``Ξ·``
2651
2651
2652 Works only on valid python 3 identifier, or on combining characters that
2652 Works only on valid python 3 identifier, or on combining characters that
2653 will combine to form a valid identifier.
2653 will combine to form a valid identifier.
2654 """
2654 """
2655 slashpos = text.rfind('\\')
2655 slashpos = text.rfind('\\')
2656 if slashpos > -1:
2656 if slashpos > -1:
2657 s = text[slashpos+1:]
2657 s = text[slashpos+1:]
2658 try :
2658 try :
2659 unic = unicodedata.lookup(s)
2659 unic = unicodedata.lookup(s)
2660 # allow combining chars
2660 # allow combining chars
2661 if ('a'+unic).isidentifier():
2661 if ('a'+unic).isidentifier():
2662 return '\\'+s,[unic]
2662 return '\\'+s,[unic]
2663 except KeyError:
2663 except KeyError:
2664 pass
2664 pass
2665 return '', []
2665 return '', []
2666
2666
2667 @context_matcher()
2667 @context_matcher()
2668 def latex_name_matcher(self, context: CompletionContext):
2668 def latex_name_matcher(self, context: CompletionContext):
2669 """Match Latex syntax for unicode characters.
2669 """Match Latex syntax for unicode characters.
2670
2670
2671 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2671 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2672 """
2672 """
2673 fragment, matches = self.latex_matches(context.text_until_cursor)
2673 fragment, matches = self.latex_matches(context.text_until_cursor)
2674 return _convert_matcher_v1_result_to_v2(
2674 return _convert_matcher_v1_result_to_v2(
2675 matches, type="latex", fragment=fragment, suppress_if_matches=True
2675 matches, type="latex", fragment=fragment, suppress_if_matches=True
2676 )
2676 )
2677
2677
2678 def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
2678 def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
2679 """Match Latex syntax for unicode characters.
2679 """Match Latex syntax for unicode characters.
2680
2680
2681 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2681 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2682
2682
2683 .. deprecated:: 8.6
2683 .. deprecated:: 8.6
2684 You can use :meth:`latex_name_matcher` instead.
2684 You can use :meth:`latex_name_matcher` instead.
2685 """
2685 """
2686 slashpos = text.rfind('\\')
2686 slashpos = text.rfind('\\')
2687 if slashpos > -1:
2687 if slashpos > -1:
2688 s = text[slashpos:]
2688 s = text[slashpos:]
2689 if s in latex_symbols:
2689 if s in latex_symbols:
2690 # Try to complete a full latex symbol to unicode
2690 # Try to complete a full latex symbol to unicode
2691 # \\alpha -> Ξ±
2691 # \\alpha -> Ξ±
2692 return s, [latex_symbols[s]]
2692 return s, [latex_symbols[s]]
2693 else:
2693 else:
2694 # If a user has partially typed a latex symbol, give them
2694 # If a user has partially typed a latex symbol, give them
2695 # a full list of options \al -> [\aleph, \alpha]
2695 # a full list of options \al -> [\aleph, \alpha]
2696 matches = [k for k in latex_symbols if k.startswith(s)]
2696 matches = [k for k in latex_symbols if k.startswith(s)]
2697 if matches:
2697 if matches:
2698 return s, matches
2698 return s, matches
2699 return '', ()
2699 return '', ()
2700
2700
2701 @context_matcher()
2701 @context_matcher()
2702 def custom_completer_matcher(self, context):
2702 def custom_completer_matcher(self, context):
2703 """Dispatch custom completer.
2703 """Dispatch custom completer.
2704
2704
2705 If a match is found, suppresses all other matchers except for Jedi.
2705 If a match is found, suppresses all other matchers except for Jedi.
2706 """
2706 """
2707 matches = self.dispatch_custom_completer(context.token) or []
2707 matches = self.dispatch_custom_completer(context.token) or []
2708 result = _convert_matcher_v1_result_to_v2(
2708 result = _convert_matcher_v1_result_to_v2(
2709 matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2709 matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2710 )
2710 )
2711 result["ordered"] = True
2711 result["ordered"] = True
2712 result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2712 result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2713 return result
2713 return result
2714
2714
2715 def dispatch_custom_completer(self, text):
2715 def dispatch_custom_completer(self, text):
2716 """
2716 """
2717 .. deprecated:: 8.6
2717 .. deprecated:: 8.6
2718 You can use :meth:`custom_completer_matcher` instead.
2718 You can use :meth:`custom_completer_matcher` instead.
2719 """
2719 """
2720 if not self.custom_completers:
2720 if not self.custom_completers:
2721 return
2721 return
2722
2722
2723 line = self.line_buffer
2723 line = self.line_buffer
2724 if not line.strip():
2724 if not line.strip():
2725 return None
2725 return None
2726
2726
2727 # Create a little structure to pass all the relevant information about
2727 # Create a little structure to pass all the relevant information about
2728 # the current completion to any custom completer.
2728 # the current completion to any custom completer.
2729 event = SimpleNamespace()
2729 event = SimpleNamespace()
2730 event.line = line
2730 event.line = line
2731 event.symbol = text
2731 event.symbol = text
2732 cmd = line.split(None,1)[0]
2732 cmd = line.split(None,1)[0]
2733 event.command = cmd
2733 event.command = cmd
2734 event.text_until_cursor = self.text_until_cursor
2734 event.text_until_cursor = self.text_until_cursor
2735
2735
2736 # for foo etc, try also to find completer for %foo
2736 # for foo etc, try also to find completer for %foo
2737 if not cmd.startswith(self.magic_escape):
2737 if not cmd.startswith(self.magic_escape):
2738 try_magic = self.custom_completers.s_matches(
2738 try_magic = self.custom_completers.s_matches(
2739 self.magic_escape + cmd)
2739 self.magic_escape + cmd)
2740 else:
2740 else:
2741 try_magic = []
2741 try_magic = []
2742
2742
2743 for c in itertools.chain(self.custom_completers.s_matches(cmd),
2743 for c in itertools.chain(self.custom_completers.s_matches(cmd),
2744 try_magic,
2744 try_magic,
2745 self.custom_completers.flat_matches(self.text_until_cursor)):
2745 self.custom_completers.flat_matches(self.text_until_cursor)):
2746 try:
2746 try:
2747 res = c(event)
2747 res = c(event)
2748 if res:
2748 if res:
2749 # first, try case sensitive match
2749 # first, try case sensitive match
2750 withcase = [r for r in res if r.startswith(text)]
2750 withcase = [r for r in res if r.startswith(text)]
2751 if withcase:
2751 if withcase:
2752 return withcase
2752 return withcase
2753 # if none, then case insensitive ones are ok too
2753 # if none, then case insensitive ones are ok too
2754 text_low = text.lower()
2754 text_low = text.lower()
2755 return [r for r in res if r.lower().startswith(text_low)]
2755 return [r for r in res if r.lower().startswith(text_low)]
2756 except TryNext:
2756 except TryNext:
2757 pass
2757 pass
2758 except KeyboardInterrupt:
2758 except KeyboardInterrupt:
2759 """
2759 """
2760 If custom completer take too long,
2760 If custom completer take too long,
2761 let keyboard interrupt abort and return nothing.
2761 let keyboard interrupt abort and return nothing.
2762 """
2762 """
2763 break
2763 break
2764
2764
2765 return None
2765 return None
2766
2766
2767 def completions(self, text: str, offset: int)->Iterator[Completion]:
2767 def completions(self, text: str, offset: int)->Iterator[Completion]:
2768 """
2768 """
2769 Returns an iterator over the possible completions
2769 Returns an iterator over the possible completions
2770
2770
2771 .. warning::
2771 .. warning::
2772
2772
2773 Unstable
2773 Unstable
2774
2774
2775 This function is unstable, API may change without warning.
2775 This function is unstable, API may change without warning.
2776 It will also raise unless use in proper context manager.
2776 It will also raise unless use in proper context manager.
2777
2777
2778 Parameters
2778 Parameters
2779 ----------
2779 ----------
2780 text : str
2780 text : str
2781 Full text of the current input, multi line string.
2781 Full text of the current input, multi line string.
2782 offset : int
2782 offset : int
2783 Integer representing the position of the cursor in ``text``. Offset
2783 Integer representing the position of the cursor in ``text``. Offset
2784 is 0-based indexed.
2784 is 0-based indexed.
2785
2785
2786 Yields
2786 Yields
2787 ------
2787 ------
2788 Completion
2788 Completion
2789
2789
2790 Notes
2790 Notes
2791 -----
2791 -----
2792 The cursor on a text can either be seen as being "in between"
2792 The cursor on a text can either be seen as being "in between"
2793 characters or "On" a character depending on the interface visible to
2793 characters or "On" a character depending on the interface visible to
2794 the user. For consistency the cursor being on "in between" characters X
2794 the user. For consistency the cursor being on "in between" characters X
2795 and Y is equivalent to the cursor being "on" character Y, that is to say
2795 and Y is equivalent to the cursor being "on" character Y, that is to say
2796 the character the cursor is on is considered as being after the cursor.
2796 the character the cursor is on is considered as being after the cursor.
2797
2797
2798 Combining characters may span more that one position in the
2798 Combining characters may span more that one position in the
2799 text.
2799 text.
2800
2800
2801 .. note::
2801 .. note::
2802
2802
2803 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
2803 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
2804 fake Completion token to distinguish completion returned by Jedi
2804 fake Completion token to distinguish completion returned by Jedi
2805 and usual IPython completion.
2805 and usual IPython completion.
2806
2806
2807 .. note::
2807 .. note::
2808
2808
2809 Completions are not completely deduplicated yet. If identical
2809 Completions are not completely deduplicated yet. If identical
2810 completions are coming from different sources this function does not
2810 completions are coming from different sources this function does not
2811 ensure that each completion object will only be present once.
2811 ensure that each completion object will only be present once.
2812 """
2812 """
2813 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
2813 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
2814 "It may change without warnings. "
2814 "It may change without warnings. "
2815 "Use in corresponding context manager.",
2815 "Use in corresponding context manager.",
2816 category=ProvisionalCompleterWarning, stacklevel=2)
2816 category=ProvisionalCompleterWarning, stacklevel=2)
2817
2817
2818 seen = set()
2818 seen = set()
2819 profiler:Optional[cProfile.Profile]
2819 profiler:Optional[cProfile.Profile]
2820 try:
2820 try:
2821 if self.profile_completions:
2821 if self.profile_completions:
2822 import cProfile
2822 import cProfile
2823 profiler = cProfile.Profile()
2823 profiler = cProfile.Profile()
2824 profiler.enable()
2824 profiler.enable()
2825 else:
2825 else:
2826 profiler = None
2826 profiler = None
2827
2827
2828 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
2828 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
2829 if c and (c in seen):
2829 if c and (c in seen):
2830 continue
2830 continue
2831 yield c
2831 yield c
2832 seen.add(c)
2832 seen.add(c)
2833 except KeyboardInterrupt:
2833 except KeyboardInterrupt:
2834 """if completions take too long and users send keyboard interrupt,
2834 """if completions take too long and users send keyboard interrupt,
2835 do not crash and return ASAP. """
2835 do not crash and return ASAP. """
2836 pass
2836 pass
2837 finally:
2837 finally:
2838 if profiler is not None:
2838 if profiler is not None:
2839 profiler.disable()
2839 profiler.disable()
2840 ensure_dir_exists(self.profiler_output_dir)
2840 ensure_dir_exists(self.profiler_output_dir)
2841 output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
2841 output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
2842 print("Writing profiler output to", output_path)
2842 print("Writing profiler output to", output_path)
2843 profiler.dump_stats(output_path)
2843 profiler.dump_stats(output_path)
2844
2844
2845 def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
2845 def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
2846 """
2846 """
2847 Core completion module.Same signature as :any:`completions`, with the
2847 Core completion module.Same signature as :any:`completions`, with the
2848 extra `timeout` parameter (in seconds).
2848 extra `timeout` parameter (in seconds).
2849
2849
2850 Computing jedi's completion ``.type`` can be quite expensive (it is a
2850 Computing jedi's completion ``.type`` can be quite expensive (it is a
2851 lazy property) and can require some warm-up, more warm up than just
2851 lazy property) and can require some warm-up, more warm up than just
2852 computing the ``name`` of a completion. The warm-up can be :
2852 computing the ``name`` of a completion. The warm-up can be :
2853
2853
2854 - Long warm-up the first time a module is encountered after
2854 - Long warm-up the first time a module is encountered after
2855 install/update: actually build parse/inference tree.
2855 install/update: actually build parse/inference tree.
2856
2856
2857 - first time the module is encountered in a session: load tree from
2857 - first time the module is encountered in a session: load tree from
2858 disk.
2858 disk.
2859
2859
2860 We don't want to block completions for tens of seconds so we give the
2860 We don't want to block completions for tens of seconds so we give the
2861 completer a "budget" of ``_timeout`` seconds per invocation to compute
2861 completer a "budget" of ``_timeout`` seconds per invocation to compute
2862 completions types, the completions that have not yet been computed will
2862 completions types, the completions that have not yet been computed will
2863 be marked as "unknown" an will have a chance to be computed next round
2863 be marked as "unknown" an will have a chance to be computed next round
2864 are things get cached.
2864 are things get cached.
2865
2865
2866 Keep in mind that Jedi is not the only thing treating the completion so
2866 Keep in mind that Jedi is not the only thing treating the completion so
2867 keep the timeout short-ish as if we take more than 0.3 second we still
2867 keep the timeout short-ish as if we take more than 0.3 second we still
2868 have lots of processing to do.
2868 have lots of processing to do.
2869
2869
2870 """
2870 """
2871 deadline = time.monotonic() + _timeout
2871 deadline = time.monotonic() + _timeout
2872
2872
2873 before = full_text[:offset]
2873 before = full_text[:offset]
2874 cursor_line, cursor_column = position_to_cursor(full_text, offset)
2874 cursor_line, cursor_column = position_to_cursor(full_text, offset)
2875
2875
2876 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2876 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2877
2877
2878 def is_non_jedi_result(
2878 def is_non_jedi_result(
2879 result: MatcherResult, identifier: str
2879 result: MatcherResult, identifier: str
2880 ) -> TypeGuard[SimpleMatcherResult]:
2880 ) -> TypeGuard[SimpleMatcherResult]:
2881 return identifier != jedi_matcher_id
2881 return identifier != jedi_matcher_id
2882
2882
2883 results = self._complete(
2883 results = self._complete(
2884 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
2884 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
2885 )
2885 )
2886
2886
2887 non_jedi_results: Dict[str, SimpleMatcherResult] = {
2887 non_jedi_results: Dict[str, SimpleMatcherResult] = {
2888 identifier: result
2888 identifier: result
2889 for identifier, result in results.items()
2889 for identifier, result in results.items()
2890 if is_non_jedi_result(result, identifier)
2890 if is_non_jedi_result(result, identifier)
2891 }
2891 }
2892
2892
2893 jedi_matches = (
2893 jedi_matches = (
2894 cast(_JediMatcherResult, results[jedi_matcher_id])["completions"]
2894 cast(_JediMatcherResult, results[jedi_matcher_id])["completions"]
2895 if jedi_matcher_id in results
2895 if jedi_matcher_id in results
2896 else ()
2896 else ()
2897 )
2897 )
2898
2898
2899 iter_jm = iter(jedi_matches)
2899 iter_jm = iter(jedi_matches)
2900 if _timeout:
2900 if _timeout:
2901 for jm in iter_jm:
2901 for jm in iter_jm:
2902 try:
2902 try:
2903 type_ = jm.type
2903 type_ = jm.type
2904 except Exception:
2904 except Exception:
2905 if self.debug:
2905 if self.debug:
2906 print("Error in Jedi getting type of ", jm)
2906 print("Error in Jedi getting type of ", jm)
2907 type_ = None
2907 type_ = None
2908 delta = len(jm.name_with_symbols) - len(jm.complete)
2908 delta = len(jm.name_with_symbols) - len(jm.complete)
2909 if type_ == 'function':
2909 if type_ == 'function':
2910 signature = _make_signature(jm)
2910 signature = _make_signature(jm)
2911 else:
2911 else:
2912 signature = ''
2912 signature = ''
2913 yield Completion(start=offset - delta,
2913 yield Completion(start=offset - delta,
2914 end=offset,
2914 end=offset,
2915 text=jm.name_with_symbols,
2915 text=jm.name_with_symbols,
2916 type=type_,
2916 type=type_,
2917 signature=signature,
2917 signature=signature,
2918 _origin='jedi')
2918 _origin='jedi')
2919
2919
2920 if time.monotonic() > deadline:
2920 if time.monotonic() > deadline:
2921 break
2921 break
2922
2922
2923 for jm in iter_jm:
2923 for jm in iter_jm:
2924 delta = len(jm.name_with_symbols) - len(jm.complete)
2924 delta = len(jm.name_with_symbols) - len(jm.complete)
2925 yield Completion(
2925 yield Completion(
2926 start=offset - delta,
2926 start=offset - delta,
2927 end=offset,
2927 end=offset,
2928 text=jm.name_with_symbols,
2928 text=jm.name_with_symbols,
2929 type=_UNKNOWN_TYPE, # don't compute type for speed
2929 type=_UNKNOWN_TYPE, # don't compute type for speed
2930 _origin="jedi",
2930 _origin="jedi",
2931 signature="",
2931 signature="",
2932 )
2932 )
2933
2933
2934 # TODO:
2934 # TODO:
2935 # Suppress this, right now just for debug.
2935 # Suppress this, right now just for debug.
2936 if jedi_matches and non_jedi_results and self.debug:
2936 if jedi_matches and non_jedi_results and self.debug:
2937 some_start_offset = before.rfind(
2937 some_start_offset = before.rfind(
2938 next(iter(non_jedi_results.values()))["matched_fragment"]
2938 next(iter(non_jedi_results.values()))["matched_fragment"]
2939 )
2939 )
2940 yield Completion(
2940 yield Completion(
2941 start=some_start_offset,
2941 start=some_start_offset,
2942 end=offset,
2942 end=offset,
2943 text="--jedi/ipython--",
2943 text="--jedi/ipython--",
2944 _origin="debug",
2944 _origin="debug",
2945 type="none",
2945 type="none",
2946 signature="",
2946 signature="",
2947 )
2947 )
2948
2948
2949 ordered: List[Completion] = []
2949 ordered: List[Completion] = []
2950 sortable: List[Completion] = []
2950 sortable: List[Completion] = []
2951
2951
2952 for origin, result in non_jedi_results.items():
2952 for origin, result in non_jedi_results.items():
2953 matched_text = result["matched_fragment"]
2953 matched_text = result["matched_fragment"]
2954 start_offset = before.rfind(matched_text)
2954 start_offset = before.rfind(matched_text)
2955 is_ordered = result.get("ordered", False)
2955 is_ordered = result.get("ordered", False)
2956 container = ordered if is_ordered else sortable
2956 container = ordered if is_ordered else sortable
2957
2957
2958 # I'm unsure if this is always true, so let's assert and see if it
2958 # I'm unsure if this is always true, so let's assert and see if it
2959 # crash
2959 # crash
2960 assert before.endswith(matched_text)
2960 assert before.endswith(matched_text)
2961
2961
2962 for simple_completion in result["completions"]:
2962 for simple_completion in result["completions"]:
2963 completion = Completion(
2963 completion = Completion(
2964 start=start_offset,
2964 start=start_offset,
2965 end=offset,
2965 end=offset,
2966 text=simple_completion.text,
2966 text=simple_completion.text,
2967 _origin=origin,
2967 _origin=origin,
2968 signature="",
2968 signature="",
2969 type=simple_completion.type or _UNKNOWN_TYPE,
2969 type=simple_completion.type or _UNKNOWN_TYPE,
2970 )
2970 )
2971 container.append(completion)
2971 container.append(completion)
2972
2972
2973 yield from list(self._deduplicate(ordered + self._sort(sortable)))[
2973 yield from list(self._deduplicate(ordered + self._sort(sortable)))[
2974 :MATCHES_LIMIT
2974 :MATCHES_LIMIT
2975 ]
2975 ]
2976
2976
2977 def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
2977 def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
2978 """Find completions for the given text and line context.
2978 """Find completions for the given text and line context.
2979
2979
2980 Note that both the text and the line_buffer are optional, but at least
2980 Note that both the text and the line_buffer are optional, but at least
2981 one of them must be given.
2981 one of them must be given.
2982
2982
2983 Parameters
2983 Parameters
2984 ----------
2984 ----------
2985 text : string, optional
2985 text : string, optional
2986 Text to perform the completion on. If not given, the line buffer
2986 Text to perform the completion on. If not given, the line buffer
2987 is split using the instance's CompletionSplitter object.
2987 is split using the instance's CompletionSplitter object.
2988 line_buffer : string, optional
2988 line_buffer : string, optional
2989 If not given, the completer attempts to obtain the current line
2989 If not given, the completer attempts to obtain the current line
2990 buffer via readline. This keyword allows clients which are
2990 buffer via readline. This keyword allows clients which are
2991 requesting for text completions in non-readline contexts to inform
2991 requesting for text completions in non-readline contexts to inform
2992 the completer of the entire text.
2992 the completer of the entire text.
2993 cursor_pos : int, optional
2993 cursor_pos : int, optional
2994 Index of the cursor in the full line buffer. Should be provided by
2994 Index of the cursor in the full line buffer. Should be provided by
2995 remote frontends where kernel has no access to frontend state.
2995 remote frontends where kernel has no access to frontend state.
2996
2996
2997 Returns
2997 Returns
2998 -------
2998 -------
2999 Tuple of two items:
2999 Tuple of two items:
3000 text : str
3000 text : str
3001 Text that was actually used in the completion.
3001 Text that was actually used in the completion.
3002 matches : list
3002 matches : list
3003 A list of completion matches.
3003 A list of completion matches.
3004
3004
3005 Notes
3005 Notes
3006 -----
3006 -----
3007 This API is likely to be deprecated and replaced by
3007 This API is likely to be deprecated and replaced by
3008 :any:`IPCompleter.completions` in the future.
3008 :any:`IPCompleter.completions` in the future.
3009
3009
3010 """
3010 """
3011 warnings.warn('`Completer.complete` is pending deprecation since '
3011 warnings.warn('`Completer.complete` is pending deprecation since '
3012 'IPython 6.0 and will be replaced by `Completer.completions`.',
3012 'IPython 6.0 and will be replaced by `Completer.completions`.',
3013 PendingDeprecationWarning)
3013 PendingDeprecationWarning)
3014 # potential todo, FOLD the 3rd throw away argument of _complete
3014 # potential todo, FOLD the 3rd throw away argument of _complete
3015 # into the first 2 one.
3015 # into the first 2 one.
3016 # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
3016 # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
3017 # TODO: should we deprecate now, or does it stay?
3017 # TODO: should we deprecate now, or does it stay?
3018
3018
3019 results = self._complete(
3019 results = self._complete(
3020 line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
3020 line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
3021 )
3021 )
3022
3022
3023 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3023 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3024
3024
3025 return self._arrange_and_extract(
3025 return self._arrange_and_extract(
3026 results,
3026 results,
3027 # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
3027 # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
3028 skip_matchers={jedi_matcher_id},
3028 skip_matchers={jedi_matcher_id},
3029 # this API does not support different start/end positions (fragments of token).
3029 # this API does not support different start/end positions (fragments of token).
3030 abort_if_offset_changes=True,
3030 abort_if_offset_changes=True,
3031 )
3031 )
3032
3032
3033 def _arrange_and_extract(
3033 def _arrange_and_extract(
3034 self,
3034 self,
3035 results: Dict[str, MatcherResult],
3035 results: Dict[str, MatcherResult],
3036 skip_matchers: Set[str],
3036 skip_matchers: Set[str],
3037 abort_if_offset_changes: bool,
3037 abort_if_offset_changes: bool,
3038 ):
3038 ):
3039 sortable: List[AnyMatcherCompletion] = []
3039 sortable: List[AnyMatcherCompletion] = []
3040 ordered: List[AnyMatcherCompletion] = []
3040 ordered: List[AnyMatcherCompletion] = []
3041 most_recent_fragment = None
3041 most_recent_fragment = None
3042 for identifier, result in results.items():
3042 for identifier, result in results.items():
3043 if identifier in skip_matchers:
3043 if identifier in skip_matchers:
3044 continue
3044 continue
3045 if not result["completions"]:
3045 if not result["completions"]:
3046 continue
3046 continue
3047 if not most_recent_fragment:
3047 if not most_recent_fragment:
3048 most_recent_fragment = result["matched_fragment"]
3048 most_recent_fragment = result["matched_fragment"]
3049 if (
3049 if (
3050 abort_if_offset_changes
3050 abort_if_offset_changes
3051 and result["matched_fragment"] != most_recent_fragment
3051 and result["matched_fragment"] != most_recent_fragment
3052 ):
3052 ):
3053 break
3053 break
3054 if result.get("ordered", False):
3054 if result.get("ordered", False):
3055 ordered.extend(result["completions"])
3055 ordered.extend(result["completions"])
3056 else:
3056 else:
3057 sortable.extend(result["completions"])
3057 sortable.extend(result["completions"])
3058
3058
3059 if not most_recent_fragment:
3059 if not most_recent_fragment:
3060 most_recent_fragment = "" # to satisfy typechecker (and just in case)
3060 most_recent_fragment = "" # to satisfy typechecker (and just in case)
3061
3061
3062 return most_recent_fragment, [
3062 return most_recent_fragment, [
3063 m.text for m in self._deduplicate(ordered + self._sort(sortable))
3063 m.text for m in self._deduplicate(ordered + self._sort(sortable))
3064 ]
3064 ]
3065
3065
3066 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
3066 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
3067 full_text=None) -> _CompleteResult:
3067 full_text=None) -> _CompleteResult:
3068 """
3068 """
3069 Like complete but can also returns raw jedi completions as well as the
3069 Like complete but can also returns raw jedi completions as well as the
3070 origin of the completion text. This could (and should) be made much
3070 origin of the completion text. This could (and should) be made much
3071 cleaner but that will be simpler once we drop the old (and stateful)
3071 cleaner but that will be simpler once we drop the old (and stateful)
3072 :any:`complete` API.
3072 :any:`complete` API.
3073
3073
3074 With current provisional API, cursor_pos act both (depending on the
3074 With current provisional API, cursor_pos act both (depending on the
3075 caller) as the offset in the ``text`` or ``line_buffer``, or as the
3075 caller) as the offset in the ``text`` or ``line_buffer``, or as the
3076 ``column`` when passing multiline strings this could/should be renamed
3076 ``column`` when passing multiline strings this could/should be renamed
3077 but would add extra noise.
3077 but would add extra noise.
3078
3078
3079 Parameters
3079 Parameters
3080 ----------
3080 ----------
3081 cursor_line
3081 cursor_line
3082 Index of the line the cursor is on. 0 indexed.
3082 Index of the line the cursor is on. 0 indexed.
3083 cursor_pos
3083 cursor_pos
3084 Position of the cursor in the current line/line_buffer/text. 0
3084 Position of the cursor in the current line/line_buffer/text. 0
3085 indexed.
3085 indexed.
3086 line_buffer : optional, str
3086 line_buffer : optional, str
3087 The current line the cursor is in, this is mostly due to legacy
3087 The current line the cursor is in, this is mostly due to legacy
3088 reason that readline could only give a us the single current line.
3088 reason that readline could only give a us the single current line.
3089 Prefer `full_text`.
3089 Prefer `full_text`.
3090 text : str
3090 text : str
3091 The current "token" the cursor is in, mostly also for historical
3091 The current "token" the cursor is in, mostly also for historical
3092 reasons. as the completer would trigger only after the current line
3092 reasons. as the completer would trigger only after the current line
3093 was parsed.
3093 was parsed.
3094 full_text : str
3094 full_text : str
3095 Full text of the current cell.
3095 Full text of the current cell.
3096
3096
3097 Returns
3097 Returns
3098 -------
3098 -------
3099 An ordered dictionary where keys are identifiers of completion
3099 An ordered dictionary where keys are identifiers of completion
3100 matchers and values are ``MatcherResult``s.
3100 matchers and values are ``MatcherResult``s.
3101 """
3101 """
3102
3102
3103 # if the cursor position isn't given, the only sane assumption we can
3103 # if the cursor position isn't given, the only sane assumption we can
3104 # make is that it's at the end of the line (the common case)
3104 # make is that it's at the end of the line (the common case)
3105 if cursor_pos is None:
3105 if cursor_pos is None:
3106 cursor_pos = len(line_buffer) if text is None else len(text)
3106 cursor_pos = len(line_buffer) if text is None else len(text)
3107
3107
3108 if self.use_main_ns:
3108 if self.use_main_ns:
3109 self.namespace = __main__.__dict__
3109 self.namespace = __main__.__dict__
3110
3110
3111 # if text is either None or an empty string, rely on the line buffer
3111 # if text is either None or an empty string, rely on the line buffer
3112 if (not line_buffer) and full_text:
3112 if (not line_buffer) and full_text:
3113 line_buffer = full_text.split('\n')[cursor_line]
3113 line_buffer = full_text.split('\n')[cursor_line]
3114 if not text: # issue #11508: check line_buffer before calling split_line
3114 if not text: # issue #11508: check line_buffer before calling split_line
3115 text = (
3115 text = (
3116 self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
3116 self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
3117 )
3117 )
3118
3118
3119 # If no line buffer is given, assume the input text is all there was
3119 # If no line buffer is given, assume the input text is all there was
3120 if line_buffer is None:
3120 if line_buffer is None:
3121 line_buffer = text
3121 line_buffer = text
3122
3122
3123 # deprecated - do not use `line_buffer` in new code.
3123 # deprecated - do not use `line_buffer` in new code.
3124 self.line_buffer = line_buffer
3124 self.line_buffer = line_buffer
3125 self.text_until_cursor = self.line_buffer[:cursor_pos]
3125 self.text_until_cursor = self.line_buffer[:cursor_pos]
3126
3126
3127 if not full_text:
3127 if not full_text:
3128 full_text = line_buffer
3128 full_text = line_buffer
3129
3129
3130 context = CompletionContext(
3130 context = CompletionContext(
3131 full_text=full_text,
3131 full_text=full_text,
3132 cursor_position=cursor_pos,
3132 cursor_position=cursor_pos,
3133 cursor_line=cursor_line,
3133 cursor_line=cursor_line,
3134 token=text,
3134 token=text,
3135 limit=MATCHES_LIMIT,
3135 limit=MATCHES_LIMIT,
3136 )
3136 )
3137
3137
3138 # Start with a clean slate of completions
3138 # Start with a clean slate of completions
3139 results: Dict[str, MatcherResult] = {}
3139 results: Dict[str, MatcherResult] = {}
3140
3140
3141 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3141 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3142
3142
3143 suppressed_matchers: Set[str] = set()
3143 suppressed_matchers: Set[str] = set()
3144
3144
3145 matchers = {
3145 matchers = {
3146 _get_matcher_id(matcher): matcher
3146 _get_matcher_id(matcher): matcher
3147 for matcher in sorted(
3147 for matcher in sorted(
3148 self.matchers, key=_get_matcher_priority, reverse=True
3148 self.matchers, key=_get_matcher_priority, reverse=True
3149 )
3149 )
3150 }
3150 }
3151
3151
3152 for matcher_id, matcher in matchers.items():
3152 for matcher_id, matcher in matchers.items():
3153 matcher_id = _get_matcher_id(matcher)
3153 matcher_id = _get_matcher_id(matcher)
3154
3154
3155 if matcher_id in self.disable_matchers:
3155 if matcher_id in self.disable_matchers:
3156 continue
3156 continue
3157
3157
3158 if matcher_id in results:
3158 if matcher_id in results:
3159 warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
3159 warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
3160
3160
3161 if matcher_id in suppressed_matchers:
3161 if matcher_id in suppressed_matchers:
3162 continue
3162 continue
3163
3163
3164 result: MatcherResult
3164 result: MatcherResult
3165 try:
3165 try:
3166 if _is_matcher_v1(matcher):
3166 if _is_matcher_v1(matcher):
3167 result = _convert_matcher_v1_result_to_v2(
3167 result = _convert_matcher_v1_result_to_v2(
3168 matcher(text), type=_UNKNOWN_TYPE
3168 matcher(text), type=_UNKNOWN_TYPE
3169 )
3169 )
3170 elif _is_matcher_v2(matcher):
3170 elif _is_matcher_v2(matcher):
3171 result = matcher(context)
3171 result = matcher(context)
3172 else:
3172 else:
3173 api_version = _get_matcher_api_version(matcher)
3173 api_version = _get_matcher_api_version(matcher)
3174 raise ValueError(f"Unsupported API version {api_version}")
3174 raise ValueError(f"Unsupported API version {api_version}")
3175 except:
3175 except:
3176 # Show the ugly traceback if the matcher causes an
3176 # Show the ugly traceback if the matcher causes an
3177 # exception, but do NOT crash the kernel!
3177 # exception, but do NOT crash the kernel!
3178 sys.excepthook(*sys.exc_info())
3178 sys.excepthook(*sys.exc_info())
3179 continue
3179 continue
3180
3180
3181 # set default value for matched fragment if suffix was not selected.
3181 # set default value for matched fragment if suffix was not selected.
3182 result["matched_fragment"] = result.get("matched_fragment", context.token)
3182 result["matched_fragment"] = result.get("matched_fragment", context.token)
3183
3183
3184 if not suppressed_matchers:
3184 if not suppressed_matchers:
3185 suppression_recommended: Union[bool, Set[str]] = result.get(
3185 suppression_recommended: Union[bool, Set[str]] = result.get(
3186 "suppress", False
3186 "suppress", False
3187 )
3187 )
3188
3188
3189 suppression_config = (
3189 suppression_config = (
3190 self.suppress_competing_matchers.get(matcher_id, None)
3190 self.suppress_competing_matchers.get(matcher_id, None)
3191 if isinstance(self.suppress_competing_matchers, dict)
3191 if isinstance(self.suppress_competing_matchers, dict)
3192 else self.suppress_competing_matchers
3192 else self.suppress_competing_matchers
3193 )
3193 )
3194 should_suppress = (
3194 should_suppress = (
3195 (suppression_config is True)
3195 (suppression_config is True)
3196 or (suppression_recommended and (suppression_config is not False))
3196 or (suppression_recommended and (suppression_config is not False))
3197 ) and has_any_completions(result)
3197 ) and has_any_completions(result)
3198
3198
3199 if should_suppress:
3199 if should_suppress:
3200 suppression_exceptions: Set[str] = result.get(
3200 suppression_exceptions: Set[str] = result.get(
3201 "do_not_suppress", set()
3201 "do_not_suppress", set()
3202 )
3202 )
3203 if isinstance(suppression_recommended, Iterable):
3203 if isinstance(suppression_recommended, Iterable):
3204 to_suppress = set(suppression_recommended)
3204 to_suppress = set(suppression_recommended)
3205 else:
3205 else:
3206 to_suppress = set(matchers)
3206 to_suppress = set(matchers)
3207 suppressed_matchers = to_suppress - suppression_exceptions
3207 suppressed_matchers = to_suppress - suppression_exceptions
3208
3208
3209 new_results = {}
3209 new_results = {}
3210 for previous_matcher_id, previous_result in results.items():
3210 for previous_matcher_id, previous_result in results.items():
3211 if previous_matcher_id not in suppressed_matchers:
3211 if previous_matcher_id not in suppressed_matchers:
3212 new_results[previous_matcher_id] = previous_result
3212 new_results[previous_matcher_id] = previous_result
3213 results = new_results
3213 results = new_results
3214
3214
3215 results[matcher_id] = result
3215 results[matcher_id] = result
3216
3216
3217 _, matches = self._arrange_and_extract(
3217 _, matches = self._arrange_and_extract(
3218 results,
3218 results,
3219 # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
3219 # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
3220 # if it was omission, we can remove the filtering step, otherwise remove this comment.
3220 # if it was omission, we can remove the filtering step, otherwise remove this comment.
3221 skip_matchers={jedi_matcher_id},
3221 skip_matchers={jedi_matcher_id},
3222 abort_if_offset_changes=False,
3222 abort_if_offset_changes=False,
3223 )
3223 )
3224
3224
3225 # populate legacy stateful API
3225 # populate legacy stateful API
3226 self.matches = matches
3226 self.matches = matches
3227
3227
3228 return results
3228 return results
3229
3229
3230 @staticmethod
3230 @staticmethod
3231 def _deduplicate(
3231 def _deduplicate(
3232 matches: Sequence[AnyCompletion],
3232 matches: Sequence[AnyCompletion],
3233 ) -> Iterable[AnyCompletion]:
3233 ) -> Iterable[AnyCompletion]:
3234 filtered_matches: Dict[str, AnyCompletion] = {}
3234 filtered_matches: Dict[str, AnyCompletion] = {}
3235 for match in matches:
3235 for match in matches:
3236 text = match.text
3236 text = match.text
3237 if (
3237 if (
3238 text not in filtered_matches
3238 text not in filtered_matches
3239 or filtered_matches[text].type == _UNKNOWN_TYPE
3239 or filtered_matches[text].type == _UNKNOWN_TYPE
3240 ):
3240 ):
3241 filtered_matches[text] = match
3241 filtered_matches[text] = match
3242
3242
3243 return filtered_matches.values()
3243 return filtered_matches.values()
3244
3244
3245 @staticmethod
3245 @staticmethod
3246 def _sort(matches: Sequence[AnyCompletion]):
3246 def _sort(matches: Sequence[AnyCompletion]):
3247 return sorted(matches, key=lambda x: completions_sorting_key(x.text))
3247 return sorted(matches, key=lambda x: completions_sorting_key(x.text))
3248
3248
3249 @context_matcher()
3249 @context_matcher()
3250 def fwd_unicode_matcher(self, context: CompletionContext):
3250 def fwd_unicode_matcher(self, context: CompletionContext):
3251 """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
3251 """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
3252 # TODO: use `context.limit` to terminate early once we matched the maximum
3252 # TODO: use `context.limit` to terminate early once we matched the maximum
3253 # number that will be used downstream; can be added as an optional to
3253 # number that will be used downstream; can be added as an optional to
3254 # `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
3254 # `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
3255 fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
3255 fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
3256 return _convert_matcher_v1_result_to_v2(
3256 return _convert_matcher_v1_result_to_v2(
3257 matches, type="unicode", fragment=fragment, suppress_if_matches=True
3257 matches, type="unicode", fragment=fragment, suppress_if_matches=True
3258 )
3258 )
3259
3259
3260 def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
3260 def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
3261 """
3261 """
3262 Forward match a string starting with a backslash with a list of
3262 Forward match a string starting with a backslash with a list of
3263 potential Unicode completions.
3263 potential Unicode completions.
3264
3264
3265 Will compute list of Unicode character names on first call and cache it.
3265 Will compute list of Unicode character names on first call and cache it.
3266
3266
3267 .. deprecated:: 8.6
3267 .. deprecated:: 8.6
3268 You can use :meth:`fwd_unicode_matcher` instead.
3268 You can use :meth:`fwd_unicode_matcher` instead.
3269
3269
3270 Returns
3270 Returns
3271 -------
3271 -------
3272 At tuple with:
3272 At tuple with:
3273 - matched text (empty if no matches)
3273 - matched text (empty if no matches)
3274 - list of potential completions, empty tuple otherwise)
3274 - list of potential completions, empty tuple otherwise)
3275 """
3275 """
3276 # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
3276 # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
3277 # We could do a faster match using a Trie.
3277 # We could do a faster match using a Trie.
3278
3278
3279 # Using pygtrie the following seem to work:
3279 # Using pygtrie the following seem to work:
3280
3280
3281 # s = PrefixSet()
3281 # s = PrefixSet()
3282
3282
3283 # for c in range(0,0x10FFFF + 1):
3283 # for c in range(0,0x10FFFF + 1):
3284 # try:
3284 # try:
3285 # s.add(unicodedata.name(chr(c)))
3285 # s.add(unicodedata.name(chr(c)))
3286 # except ValueError:
3286 # except ValueError:
3287 # pass
3287 # pass
3288 # [''.join(k) for k in s.iter(prefix)]
3288 # [''.join(k) for k in s.iter(prefix)]
3289
3289
3290 # But need to be timed and adds an extra dependency.
3290 # But need to be timed and adds an extra dependency.
3291
3291
3292 slashpos = text.rfind('\\')
3292 slashpos = text.rfind('\\')
3293 # if text starts with slash
3293 # if text starts with slash
3294 if slashpos > -1:
3294 if slashpos > -1:
3295 # PERF: It's important that we don't access self._unicode_names
3295 # PERF: It's important that we don't access self._unicode_names
3296 # until we're inside this if-block. _unicode_names is lazily
3296 # until we're inside this if-block. _unicode_names is lazily
3297 # initialized, and it takes a user-noticeable amount of time to
3297 # initialized, and it takes a user-noticeable amount of time to
3298 # initialize it, so we don't want to initialize it unless we're
3298 # initialize it, so we don't want to initialize it unless we're
3299 # actually going to use it.
3299 # actually going to use it.
3300 s = text[slashpos + 1 :]
3300 s = text[slashpos + 1 :]
3301 sup = s.upper()
3301 sup = s.upper()
3302 candidates = [x for x in self.unicode_names if x.startswith(sup)]
3302 candidates = [x for x in self.unicode_names if x.startswith(sup)]
3303 if candidates:
3303 if candidates:
3304 return s, candidates
3304 return s, candidates
3305 candidates = [x for x in self.unicode_names if sup in x]
3305 candidates = [x for x in self.unicode_names if sup in x]
3306 if candidates:
3306 if candidates:
3307 return s, candidates
3307 return s, candidates
3308 splitsup = sup.split(" ")
3308 splitsup = sup.split(" ")
3309 candidates = [
3309 candidates = [
3310 x for x in self.unicode_names if all(u in x for u in splitsup)
3310 x for x in self.unicode_names if all(u in x for u in splitsup)
3311 ]
3311 ]
3312 if candidates:
3312 if candidates:
3313 return s, candidates
3313 return s, candidates
3314
3314
3315 return "", ()
3315 return "", ()
3316
3316
3317 # if text does not start with slash
3317 # if text does not start with slash
3318 else:
3318 else:
3319 return '', ()
3319 return '', ()
3320
3320
3321 @property
3321 @property
3322 def unicode_names(self) -> List[str]:
3322 def unicode_names(self) -> List[str]:
3323 """List of names of unicode code points that can be completed.
3323 """List of names of unicode code points that can be completed.
3324
3324
3325 The list is lazily initialized on first access.
3325 The list is lazily initialized on first access.
3326 """
3326 """
3327 if self._unicode_names is None:
3327 if self._unicode_names is None:
3328 names = []
3328 names = []
3329 for c in range(0,0x10FFFF + 1):
3329 for c in range(0,0x10FFFF + 1):
3330 try:
3330 try:
3331 names.append(unicodedata.name(chr(c)))
3331 names.append(unicodedata.name(chr(c)))
3332 except ValueError:
3332 except ValueError:
3333 pass
3333 pass
3334 self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
3334 self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
3335
3335
3336 return self._unicode_names
3336 return self._unicode_names
3337
3337
3338 def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
3338 def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
3339 names = []
3339 names = []
3340 for start,stop in ranges:
3340 for start,stop in ranges:
3341 for c in range(start, stop) :
3341 for c in range(start, stop) :
3342 try:
3342 try:
3343 names.append(unicodedata.name(chr(c)))
3343 names.append(unicodedata.name(chr(c)))
3344 except ValueError:
3344 except ValueError:
3345 pass
3345 pass
3346 return names
3346 return names
General Comments 0
You need to be logged in to leave comments. Login now