##// END OF EJS Templates
Fix exception on missing attribute matches
krassowski -
Show More
@@ -1,3388 +1,3388 b''
1 1 """Completion for IPython.
2 2
3 3 This module started as fork of the rlcompleter module in the Python standard
4 4 library. The original enhancements made to rlcompleter have been sent
5 5 upstream and were accepted as of Python 2.3,
6 6
7 7 This module now support a wide variety of completion mechanism both available
8 8 for normal classic Python code, as well as completer for IPython specific
9 9 Syntax like magics.
10 10
11 11 Latex and Unicode completion
12 12 ============================
13 13
14 14 IPython and compatible frontends not only can complete your code, but can help
15 15 you to input a wide range of characters. In particular we allow you to insert
16 16 a unicode character using the tab completion mechanism.
17 17
18 18 Forward latex/unicode completion
19 19 --------------------------------
20 20
21 21 Forward completion allows you to easily type a unicode character using its latex
22 22 name, or unicode long description. To do so type a backslash follow by the
23 23 relevant name and press tab:
24 24
25 25
26 26 Using latex completion:
27 27
28 28 .. code::
29 29
30 30 \\alpha<tab>
31 31 Ξ±
32 32
33 33 or using unicode completion:
34 34
35 35
36 36 .. code::
37 37
38 38 \\GREEK SMALL LETTER ALPHA<tab>
39 39 Ξ±
40 40
41 41
42 42 Only valid Python identifiers will complete. Combining characters (like arrow or
43 43 dots) are also available, unlike latex they need to be put after the their
44 44 counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
45 45
46 46 Some browsers are known to display combining characters incorrectly.
47 47
48 48 Backward latex completion
49 49 -------------------------
50 50
51 51 It is sometime challenging to know how to type a character, if you are using
52 52 IPython, or any compatible frontend you can prepend backslash to the character
53 53 and press :kbd:`Tab` to expand it to its latex form.
54 54
55 55 .. code::
56 56
57 57 \\Ξ±<tab>
58 58 \\alpha
59 59
60 60
61 61 Both forward and backward completions can be deactivated by setting the
62 62 :std:configtrait:`Completer.backslash_combining_completions` option to
63 63 ``False``.
64 64
65 65
66 66 Experimental
67 67 ============
68 68
69 69 Starting with IPython 6.0, this module can make use of the Jedi library to
70 70 generate completions both using static analysis of the code, and dynamically
71 71 inspecting multiple namespaces. Jedi is an autocompletion and static analysis
72 72 for Python. The APIs attached to this new mechanism is unstable and will
73 73 raise unless use in an :any:`provisionalcompleter` context manager.
74 74
75 75 You will find that the following are experimental:
76 76
77 77 - :any:`provisionalcompleter`
78 78 - :any:`IPCompleter.completions`
79 79 - :any:`Completion`
80 80 - :any:`rectify_completions`
81 81
82 82 .. note::
83 83
84 84 better name for :any:`rectify_completions` ?
85 85
86 86 We welcome any feedback on these new API, and we also encourage you to try this
87 87 module in debug mode (start IPython with ``--Completer.debug=True``) in order
88 88 to have extra logging information if :any:`jedi` is crashing, or if current
89 89 IPython completer pending deprecations are returning results not yet handled
90 90 by :any:`jedi`
91 91
92 92 Using Jedi for tab completion allow snippets like the following to work without
93 93 having to execute any code:
94 94
95 95 >>> myvar = ['hello', 42]
96 96 ... myvar[1].bi<tab>
97 97
98 98 Tab completion will be able to infer that ``myvar[1]`` is a real number without
99 99 executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
100 100 option.
101 101
102 102 Be sure to update :any:`jedi` to the latest stable version or to try the
103 103 current development version to get better completions.
104 104
105 105 Matchers
106 106 ========
107 107
108 108 All completions routines are implemented using unified *Matchers* API.
109 109 The matchers API is provisional and subject to change without notice.
110 110
111 111 The built-in matchers include:
112 112
113 113 - :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
114 114 - :any:`IPCompleter.magic_matcher`: completions for magics,
115 115 - :any:`IPCompleter.unicode_name_matcher`,
116 116 :any:`IPCompleter.fwd_unicode_matcher`
117 117 and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
118 118 - :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
119 119 - :any:`IPCompleter.file_matcher`: paths to files and directories,
120 120 - :any:`IPCompleter.python_func_kw_matcher` - function keywords,
121 121 - :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
122 122 - ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
123 123 - :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
124 124 implementation in :any:`InteractiveShell` which uses IPython hooks system
125 125 (`complete_command`) with string dispatch (including regular expressions).
126 126 Differently to other matchers, ``custom_completer_matcher`` will not suppress
127 127 Jedi results to match behaviour in earlier IPython versions.
128 128
129 129 Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
130 130
131 131 Matcher API
132 132 -----------
133 133
134 134 Simplifying some details, the ``Matcher`` interface can described as
135 135
136 136 .. code-block::
137 137
138 138 MatcherAPIv1 = Callable[[str], list[str]]
139 139 MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
140 140
141 141 Matcher = MatcherAPIv1 | MatcherAPIv2
142 142
143 143 The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
144 144 and remains supported as a simplest way for generating completions. This is also
145 145 currently the only API supported by the IPython hooks system `complete_command`.
146 146
147 147 To distinguish between matcher versions ``matcher_api_version`` attribute is used.
148 148 More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
149 149 and requires a literal ``2`` for v2 Matchers.
150 150
151 151 Once the API stabilises future versions may relax the requirement for specifying
152 152 ``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
153 153 please do not rely on the presence of ``matcher_api_version`` for any purposes.
154 154
155 155 Suppression of competing matchers
156 156 ---------------------------------
157 157
158 158 By default results from all matchers are combined, in the order determined by
159 159 their priority. Matchers can request to suppress results from subsequent
160 160 matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
161 161
162 162 When multiple matchers simultaneously request surpression, the results from of
163 163 the matcher with higher priority will be returned.
164 164
165 165 Sometimes it is desirable to suppress most but not all other matchers;
166 166 this can be achieved by adding a set of identifiers of matchers which
167 167 should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
168 168
169 169 The suppression behaviour can is user-configurable via
170 170 :std:configtrait:`IPCompleter.suppress_competing_matchers`.
171 171 """
172 172
173 173
174 174 # Copyright (c) IPython Development Team.
175 175 # Distributed under the terms of the Modified BSD License.
176 176 #
177 177 # Some of this code originated from rlcompleter in the Python standard library
178 178 # Copyright (C) 2001 Python Software Foundation, www.python.org
179 179
180 180 from __future__ import annotations
181 181 import builtins as builtin_mod
182 182 import enum
183 183 import glob
184 184 import inspect
185 185 import itertools
186 186 import keyword
187 187 import os
188 188 import re
189 189 import string
190 190 import sys
191 191 import tokenize
192 192 import time
193 193 import unicodedata
194 194 import uuid
195 195 import warnings
196 196 from ast import literal_eval
197 197 from collections import defaultdict
198 198 from contextlib import contextmanager
199 199 from dataclasses import dataclass
200 200 from functools import cached_property, partial
201 201 from types import SimpleNamespace
202 202 from typing import (
203 203 Iterable,
204 204 Iterator,
205 205 List,
206 206 Tuple,
207 207 Union,
208 208 Any,
209 209 Sequence,
210 210 Dict,
211 211 Optional,
212 212 TYPE_CHECKING,
213 213 Set,
214 214 Sized,
215 215 TypeVar,
216 216 Literal,
217 217 )
218 218
219 219 from IPython.core.guarded_eval import guarded_eval, EvaluationContext
220 220 from IPython.core.error import TryNext
221 221 from IPython.core.inputtransformer2 import ESC_MAGIC
222 222 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
223 223 from IPython.core.oinspect import InspectColors
224 224 from IPython.testing.skipdoctest import skip_doctest
225 225 from IPython.utils import generics
226 226 from IPython.utils.decorators import sphinx_options
227 227 from IPython.utils.dir2 import dir2, get_real_method
228 228 from IPython.utils.docs import GENERATING_DOCUMENTATION
229 229 from IPython.utils.path import ensure_dir_exists
230 230 from IPython.utils.process import arg_split
231 231 from traitlets import (
232 232 Bool,
233 233 Enum,
234 234 Int,
235 235 List as ListTrait,
236 236 Unicode,
237 237 Dict as DictTrait,
238 238 Union as UnionTrait,
239 239 observe,
240 240 )
241 241 from traitlets.config.configurable import Configurable
242 242
243 243 import __main__
244 244
245 245 # skip module docstests
246 246 __skip_doctest__ = True
247 247
248 248
249 249 try:
250 250 import jedi
251 251 jedi.settings.case_insensitive_completion = False
252 252 import jedi.api.helpers
253 253 import jedi.api.classes
254 254 JEDI_INSTALLED = True
255 255 except ImportError:
256 256 JEDI_INSTALLED = False
257 257
258 258
259 259 if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
260 260 from typing import cast
261 261 from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
262 262 else:
263 263 from typing import Generic
264 264
265 265 def cast(type_, obj):
266 266 """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
267 267 return obj
268 268
269 269 # do not require on runtime
270 270 NotRequired = Tuple # requires Python >=3.11
271 271 TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
272 272 Protocol = object # requires Python >=3.8
273 273 TypeAlias = Any # requires Python >=3.10
274 274 TypeGuard = Generic # requires Python >=3.10
275 275 if GENERATING_DOCUMENTATION:
276 276 from typing import TypedDict
277 277
278 278 # -----------------------------------------------------------------------------
279 279 # Globals
280 280 #-----------------------------------------------------------------------------
281 281
282 282 # ranges where we have most of the valid unicode names. We could be more finer
283 283 # grained but is it worth it for performance While unicode have character in the
284 284 # range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
285 285 # write this). With below range we cover them all, with a density of ~67%
286 286 # biggest next gap we consider only adds up about 1% density and there are 600
287 287 # gaps that would need hard coding.
288 288 _UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
289 289
290 290 # Public API
291 291 __all__ = ["Completer", "IPCompleter"]
292 292
293 293 if sys.platform == 'win32':
294 294 PROTECTABLES = ' '
295 295 else:
296 296 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
297 297
298 298 # Protect against returning an enormous number of completions which the frontend
299 299 # may have trouble processing.
300 300 MATCHES_LIMIT = 500
301 301
302 302 # Completion type reported when no type can be inferred.
303 303 _UNKNOWN_TYPE = "<unknown>"
304 304
305 305 # sentinel value to signal lack of a match
306 306 not_found = object()
307 307
308 308 class ProvisionalCompleterWarning(FutureWarning):
309 309 """
310 310 Exception raise by an experimental feature in this module.
311 311
312 312 Wrap code in :any:`provisionalcompleter` context manager if you
313 313 are certain you want to use an unstable feature.
314 314 """
315 315 pass
316 316
317 317 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
318 318
319 319
320 320 @skip_doctest
321 321 @contextmanager
322 322 def provisionalcompleter(action='ignore'):
323 323 """
324 324 This context manager has to be used in any place where unstable completer
325 325 behavior and API may be called.
326 326
327 327 >>> with provisionalcompleter():
328 328 ... completer.do_experimental_things() # works
329 329
330 330 >>> completer.do_experimental_things() # raises.
331 331
332 332 .. note::
333 333
334 334 Unstable
335 335
336 336 By using this context manager you agree that the API in use may change
337 337 without warning, and that you won't complain if they do so.
338 338
339 339 You also understand that, if the API is not to your liking, you should report
340 340 a bug to explain your use case upstream.
341 341
342 342 We'll be happy to get your feedback, feature requests, and improvements on
343 343 any of the unstable APIs!
344 344 """
345 345 with warnings.catch_warnings():
346 346 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
347 347 yield
348 348
349 349
350 350 def has_open_quotes(s):
351 351 """Return whether a string has open quotes.
352 352
353 353 This simply counts whether the number of quote characters of either type in
354 354 the string is odd.
355 355
356 356 Returns
357 357 -------
358 358 If there is an open quote, the quote character is returned. Else, return
359 359 False.
360 360 """
361 361 # We check " first, then ', so complex cases with nested quotes will get
362 362 # the " to take precedence.
363 363 if s.count('"') % 2:
364 364 return '"'
365 365 elif s.count("'") % 2:
366 366 return "'"
367 367 else:
368 368 return False
369 369
370 370
371 371 def protect_filename(s, protectables=PROTECTABLES):
372 372 """Escape a string to protect certain characters."""
373 373 if set(s) & set(protectables):
374 374 if sys.platform == "win32":
375 375 return '"' + s + '"'
376 376 else:
377 377 return "".join(("\\" + c if c in protectables else c) for c in s)
378 378 else:
379 379 return s
380 380
381 381
382 382 def expand_user(path:str) -> Tuple[str, bool, str]:
383 383 """Expand ``~``-style usernames in strings.
384 384
385 385 This is similar to :func:`os.path.expanduser`, but it computes and returns
386 386 extra information that will be useful if the input was being used in
387 387 computing completions, and you wish to return the completions with the
388 388 original '~' instead of its expanded value.
389 389
390 390 Parameters
391 391 ----------
392 392 path : str
393 393 String to be expanded. If no ~ is present, the output is the same as the
394 394 input.
395 395
396 396 Returns
397 397 -------
398 398 newpath : str
399 399 Result of ~ expansion in the input path.
400 400 tilde_expand : bool
401 401 Whether any expansion was performed or not.
402 402 tilde_val : str
403 403 The value that ~ was replaced with.
404 404 """
405 405 # Default values
406 406 tilde_expand = False
407 407 tilde_val = ''
408 408 newpath = path
409 409
410 410 if path.startswith('~'):
411 411 tilde_expand = True
412 412 rest = len(path)-1
413 413 newpath = os.path.expanduser(path)
414 414 if rest:
415 415 tilde_val = newpath[:-rest]
416 416 else:
417 417 tilde_val = newpath
418 418
419 419 return newpath, tilde_expand, tilde_val
420 420
421 421
422 422 def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
423 423 """Does the opposite of expand_user, with its outputs.
424 424 """
425 425 if tilde_expand:
426 426 return path.replace(tilde_val, '~')
427 427 else:
428 428 return path
429 429
430 430
431 431 def completions_sorting_key(word):
432 432 """key for sorting completions
433 433
434 434 This does several things:
435 435
436 436 - Demote any completions starting with underscores to the end
437 437 - Insert any %magic and %%cellmagic completions in the alphabetical order
438 438 by their name
439 439 """
440 440 prio1, prio2 = 0, 0
441 441
442 442 if word.startswith('__'):
443 443 prio1 = 2
444 444 elif word.startswith('_'):
445 445 prio1 = 1
446 446
447 447 if word.endswith('='):
448 448 prio1 = -1
449 449
450 450 if word.startswith('%%'):
451 451 # If there's another % in there, this is something else, so leave it alone
452 452 if not "%" in word[2:]:
453 453 word = word[2:]
454 454 prio2 = 2
455 455 elif word.startswith('%'):
456 456 if not "%" in word[1:]:
457 457 word = word[1:]
458 458 prio2 = 1
459 459
460 460 return prio1, word, prio2
461 461
462 462
463 463 class _FakeJediCompletion:
464 464 """
465 465 This is a workaround to communicate to the UI that Jedi has crashed and to
466 466 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
467 467
468 468 Added in IPython 6.0 so should likely be removed for 7.0
469 469
470 470 """
471 471
472 472 def __init__(self, name):
473 473
474 474 self.name = name
475 475 self.complete = name
476 476 self.type = 'crashed'
477 477 self.name_with_symbols = name
478 478 self.signature = ""
479 479 self._origin = "fake"
480 480 self.text = "crashed"
481 481
482 482 def __repr__(self):
483 483 return '<Fake completion object jedi has crashed>'
484 484
485 485
486 486 _JediCompletionLike = Union["jedi.api.Completion", _FakeJediCompletion]
487 487
488 488
489 489 class Completion:
490 490 """
491 491 Completion object used and returned by IPython completers.
492 492
493 493 .. warning::
494 494
495 495 Unstable
496 496
497 497 This function is unstable, API may change without warning.
498 498 It will also raise unless use in proper context manager.
499 499
500 500 This act as a middle ground :any:`Completion` object between the
501 501 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
502 502 object. While Jedi need a lot of information about evaluator and how the
503 503 code should be ran/inspected, PromptToolkit (and other frontend) mostly
504 504 need user facing information.
505 505
506 506 - Which range should be replaced replaced by what.
507 507 - Some metadata (like completion type), or meta information to displayed to
508 508 the use user.
509 509
510 510 For debugging purpose we can also store the origin of the completion (``jedi``,
511 511 ``IPython.python_matches``, ``IPython.magics_matches``...).
512 512 """
513 513
514 514 __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
515 515
516 516 def __init__(
517 517 self,
518 518 start: int,
519 519 end: int,
520 520 text: str,
521 521 *,
522 522 type: Optional[str] = None,
523 523 _origin="",
524 524 signature="",
525 525 ) -> None:
526 526 warnings.warn(
527 527 "``Completion`` is a provisional API (as of IPython 6.0). "
528 528 "It may change without warnings. "
529 529 "Use in corresponding context manager.",
530 530 category=ProvisionalCompleterWarning,
531 531 stacklevel=2,
532 532 )
533 533
534 534 self.start = start
535 535 self.end = end
536 536 self.text = text
537 537 self.type = type
538 538 self.signature = signature
539 539 self._origin = _origin
540 540
541 541 def __repr__(self):
542 542 return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
543 543 (self.start, self.end, self.text, self.type or '?', self.signature or '?')
544 544
545 545 def __eq__(self, other) -> bool:
546 546 """
547 547 Equality and hash do not hash the type (as some completer may not be
548 548 able to infer the type), but are use to (partially) de-duplicate
549 549 completion.
550 550
551 551 Completely de-duplicating completion is a bit tricker that just
552 552 comparing as it depends on surrounding text, which Completions are not
553 553 aware of.
554 554 """
555 555 return self.start == other.start and \
556 556 self.end == other.end and \
557 557 self.text == other.text
558 558
559 559 def __hash__(self):
560 560 return hash((self.start, self.end, self.text))
561 561
562 562
563 563 class SimpleCompletion:
564 564 """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
565 565
566 566 .. warning::
567 567
568 568 Provisional
569 569
570 570 This class is used to describe the currently supported attributes of
571 571 simple completion items, and any additional implementation details
572 572 should not be relied on. Additional attributes may be included in
573 573 future versions, and meaning of text disambiguated from the current
574 574 dual meaning of "text to insert" and "text to used as a label".
575 575 """
576 576
577 577 __slots__ = ["text", "type"]
578 578
579 579 def __init__(self, text: str, *, type: Optional[str] = None):
580 580 self.text = text
581 581 self.type = type
582 582
583 583 def __repr__(self):
584 584 return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
585 585
586 586
587 587 class _MatcherResultBase(TypedDict):
588 588 """Definition of dictionary to be returned by new-style Matcher (API v2)."""
589 589
590 590 #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
591 591 matched_fragment: NotRequired[str]
592 592
593 593 #: Whether to suppress results from all other matchers (True), some
594 594 #: matchers (set of identifiers) or none (False); default is False.
595 595 suppress: NotRequired[Union[bool, Set[str]]]
596 596
597 597 #: Identifiers of matchers which should NOT be suppressed when this matcher
598 598 #: requests to suppress all other matchers; defaults to an empty set.
599 599 do_not_suppress: NotRequired[Set[str]]
600 600
601 601 #: Are completions already ordered and should be left as-is? default is False.
602 602 ordered: NotRequired[bool]
603 603
604 604
605 605 @sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
606 606 class SimpleMatcherResult(_MatcherResultBase, TypedDict):
607 607 """Result of new-style completion matcher."""
608 608
609 609 # note: TypedDict is added again to the inheritance chain
610 610 # in order to get __orig_bases__ for documentation
611 611
612 612 #: List of candidate completions
613 613 completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
614 614
615 615
616 616 class _JediMatcherResult(_MatcherResultBase):
617 617 """Matching result returned by Jedi (will be processed differently)"""
618 618
619 619 #: list of candidate completions
620 620 completions: Iterator[_JediCompletionLike]
621 621
622 622
623 623 AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
624 624 AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
625 625
626 626
627 627 @dataclass
628 628 class CompletionContext:
629 629 """Completion context provided as an argument to matchers in the Matcher API v2."""
630 630
631 631 # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
632 632 # which was not explicitly visible as an argument of the matcher, making any refactor
633 633 # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
634 634 # from the completer, and make substituting them in sub-classes easier.
635 635
636 636 #: Relevant fragment of code directly preceding the cursor.
637 637 #: The extraction of token is implemented via splitter heuristic
638 638 #: (following readline behaviour for legacy reasons), which is user configurable
639 639 #: (by switching the greedy mode).
640 640 token: str
641 641
642 642 #: The full available content of the editor or buffer
643 643 full_text: str
644 644
645 645 #: Cursor position in the line (the same for ``full_text`` and ``text``).
646 646 cursor_position: int
647 647
648 648 #: Cursor line in ``full_text``.
649 649 cursor_line: int
650 650
651 651 #: The maximum number of completions that will be used downstream.
652 652 #: Matchers can use this information to abort early.
653 653 #: The built-in Jedi matcher is currently excepted from this limit.
654 654 # If not given, return all possible completions.
655 655 limit: Optional[int]
656 656
657 657 @cached_property
658 658 def text_until_cursor(self) -> str:
659 659 return self.line_with_cursor[: self.cursor_position]
660 660
661 661 @cached_property
662 662 def line_with_cursor(self) -> str:
663 663 return self.full_text.split("\n")[self.cursor_line]
664 664
665 665
666 666 #: Matcher results for API v2.
667 667 MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
668 668
669 669
670 670 class _MatcherAPIv1Base(Protocol):
671 671 def __call__(self, text: str) -> List[str]:
672 672 """Call signature."""
673 673 ...
674 674
675 675 #: Used to construct the default matcher identifier
676 676 __qualname__: str
677 677
678 678
679 679 class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
680 680 #: API version
681 681 matcher_api_version: Optional[Literal[1]]
682 682
683 683 def __call__(self, text: str) -> List[str]:
684 684 """Call signature."""
685 685 ...
686 686
687 687
688 688 #: Protocol describing Matcher API v1.
689 689 MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
690 690
691 691
692 692 class MatcherAPIv2(Protocol):
693 693 """Protocol describing Matcher API v2."""
694 694
695 695 #: API version
696 696 matcher_api_version: Literal[2] = 2
697 697
698 698 def __call__(self, context: CompletionContext) -> MatcherResult:
699 699 """Call signature."""
700 700 ...
701 701
702 702 #: Used to construct the default matcher identifier
703 703 __qualname__: str
704 704
705 705
706 706 Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
707 707
708 708
709 709 def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
710 710 api_version = _get_matcher_api_version(matcher)
711 711 return api_version == 1
712 712
713 713
714 714 def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
715 715 api_version = _get_matcher_api_version(matcher)
716 716 return api_version == 2
717 717
718 718
719 719 def _is_sizable(value: Any) -> TypeGuard[Sized]:
720 720 """Determines whether objects is sizable"""
721 721 return hasattr(value, "__len__")
722 722
723 723
724 724 def _is_iterator(value: Any) -> TypeGuard[Iterator]:
725 725 """Determines whether objects is sizable"""
726 726 return hasattr(value, "__next__")
727 727
728 728
729 729 def has_any_completions(result: MatcherResult) -> bool:
730 730 """Check if any result includes any completions."""
731 731 completions = result["completions"]
732 732 if _is_sizable(completions):
733 733 return len(completions) != 0
734 734 if _is_iterator(completions):
735 735 try:
736 736 old_iterator = completions
737 737 first = next(old_iterator)
738 738 result["completions"] = cast(
739 739 Iterator[SimpleCompletion],
740 740 itertools.chain([first], old_iterator),
741 741 )
742 742 return True
743 743 except StopIteration:
744 744 return False
745 745 raise ValueError(
746 746 "Completions returned by matcher need to be an Iterator or a Sizable"
747 747 )
748 748
749 749
750 750 def completion_matcher(
751 751 *,
752 752 priority: Optional[float] = None,
753 753 identifier: Optional[str] = None,
754 754 api_version: int = 1,
755 755 ):
756 756 """Adds attributes describing the matcher.
757 757
758 758 Parameters
759 759 ----------
760 760 priority : Optional[float]
761 761 The priority of the matcher, determines the order of execution of matchers.
762 762 Higher priority means that the matcher will be executed first. Defaults to 0.
763 763 identifier : Optional[str]
764 764 identifier of the matcher allowing users to modify the behaviour via traitlets,
765 765 and also used to for debugging (will be passed as ``origin`` with the completions).
766 766
767 767 Defaults to matcher function's ``__qualname__`` (for example,
768 768 ``IPCompleter.file_matcher`` for the built-in matched defined
769 769 as a ``file_matcher`` method of the ``IPCompleter`` class).
770 770 api_version: Optional[int]
771 771 version of the Matcher API used by this matcher.
772 772 Currently supported values are 1 and 2.
773 773 Defaults to 1.
774 774 """
775 775
776 776 def wrapper(func: Matcher):
777 777 func.matcher_priority = priority or 0 # type: ignore
778 778 func.matcher_identifier = identifier or func.__qualname__ # type: ignore
779 779 func.matcher_api_version = api_version # type: ignore
780 780 if TYPE_CHECKING:
781 781 if api_version == 1:
782 782 func = cast(MatcherAPIv1, func)
783 783 elif api_version == 2:
784 784 func = cast(MatcherAPIv2, func)
785 785 return func
786 786
787 787 return wrapper
788 788
789 789
790 790 def _get_matcher_priority(matcher: Matcher):
791 791 return getattr(matcher, "matcher_priority", 0)
792 792
793 793
794 794 def _get_matcher_id(matcher: Matcher):
795 795 return getattr(matcher, "matcher_identifier", matcher.__qualname__)
796 796
797 797
798 798 def _get_matcher_api_version(matcher):
799 799 return getattr(matcher, "matcher_api_version", 1)
800 800
801 801
802 802 context_matcher = partial(completion_matcher, api_version=2)
803 803
804 804
805 805 _IC = Iterable[Completion]
806 806
807 807
808 808 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
809 809 """
810 810 Deduplicate a set of completions.
811 811
812 812 .. warning::
813 813
814 814 Unstable
815 815
816 816 This function is unstable, API may change without warning.
817 817
818 818 Parameters
819 819 ----------
820 820 text : str
821 821 text that should be completed.
822 822 completions : Iterator[Completion]
823 823 iterator over the completions to deduplicate
824 824
825 825 Yields
826 826 ------
827 827 `Completions` objects
828 828 Completions coming from multiple sources, may be different but end up having
829 829 the same effect when applied to ``text``. If this is the case, this will
830 830 consider completions as equal and only emit the first encountered.
831 831 Not folded in `completions()` yet for debugging purpose, and to detect when
832 832 the IPython completer does return things that Jedi does not, but should be
833 833 at some point.
834 834 """
835 835 completions = list(completions)
836 836 if not completions:
837 837 return
838 838
839 839 new_start = min(c.start for c in completions)
840 840 new_end = max(c.end for c in completions)
841 841
842 842 seen = set()
843 843 for c in completions:
844 844 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
845 845 if new_text not in seen:
846 846 yield c
847 847 seen.add(new_text)
848 848
849 849
850 850 def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
851 851 """
852 852 Rectify a set of completions to all have the same ``start`` and ``end``
853 853
854 854 .. warning::
855 855
856 856 Unstable
857 857
858 858 This function is unstable, API may change without warning.
859 859 It will also raise unless use in proper context manager.
860 860
861 861 Parameters
862 862 ----------
863 863 text : str
864 864 text that should be completed.
865 865 completions : Iterator[Completion]
866 866 iterator over the completions to rectify
867 867 _debug : bool
868 868 Log failed completion
869 869
870 870 Notes
871 871 -----
872 872 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
873 873 the Jupyter Protocol requires them to behave like so. This will readjust
874 874 the completion to have the same ``start`` and ``end`` by padding both
875 875 extremities with surrounding text.
876 876
877 877 During stabilisation should support a ``_debug`` option to log which
878 878 completion are return by the IPython completer and not found in Jedi in
879 879 order to make upstream bug report.
880 880 """
881 881 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
882 882 "It may change without warnings. "
883 883 "Use in corresponding context manager.",
884 884 category=ProvisionalCompleterWarning, stacklevel=2)
885 885
886 886 completions = list(completions)
887 887 if not completions:
888 888 return
889 889 starts = (c.start for c in completions)
890 890 ends = (c.end for c in completions)
891 891
892 892 new_start = min(starts)
893 893 new_end = max(ends)
894 894
895 895 seen_jedi = set()
896 896 seen_python_matches = set()
897 897 for c in completions:
898 898 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
899 899 if c._origin == 'jedi':
900 900 seen_jedi.add(new_text)
901 901 elif c._origin == "IPCompleter.python_matcher":
902 902 seen_python_matches.add(new_text)
903 903 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
904 904 diff = seen_python_matches.difference(seen_jedi)
905 905 if diff and _debug:
906 906 print('IPython.python matches have extras:', diff)
907 907
908 908
909 909 if sys.platform == 'win32':
910 910 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
911 911 else:
912 912 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
913 913
914 914 GREEDY_DELIMS = ' =\r\n'
915 915
916 916
917 917 class CompletionSplitter(object):
918 918 """An object to split an input line in a manner similar to readline.
919 919
920 920 By having our own implementation, we can expose readline-like completion in
921 921 a uniform manner to all frontends. This object only needs to be given the
922 922 line of text to be split and the cursor position on said line, and it
923 923 returns the 'word' to be completed on at the cursor after splitting the
924 924 entire line.
925 925
926 926 What characters are used as splitting delimiters can be controlled by
927 927 setting the ``delims`` attribute (this is a property that internally
928 928 automatically builds the necessary regular expression)"""
929 929
930 930 # Private interface
931 931
932 932 # A string of delimiter characters. The default value makes sense for
933 933 # IPython's most typical usage patterns.
934 934 _delims = DELIMS
935 935
936 936 # The expression (a normal string) to be compiled into a regular expression
937 937 # for actual splitting. We store it as an attribute mostly for ease of
938 938 # debugging, since this type of code can be so tricky to debug.
939 939 _delim_expr = None
940 940
941 941 # The regular expression that does the actual splitting
942 942 _delim_re = None
943 943
944 944 def __init__(self, delims=None):
945 945 delims = CompletionSplitter._delims if delims is None else delims
946 946 self.delims = delims
947 947
948 948 @property
949 949 def delims(self):
950 950 """Return the string of delimiter characters."""
951 951 return self._delims
952 952
953 953 @delims.setter
954 954 def delims(self, delims):
955 955 """Set the delimiters for line splitting."""
956 956 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
957 957 self._delim_re = re.compile(expr)
958 958 self._delims = delims
959 959 self._delim_expr = expr
960 960
961 961 def split_line(self, line, cursor_pos=None):
962 962 """Split a line of text with a cursor at the given position.
963 963 """
964 964 l = line if cursor_pos is None else line[:cursor_pos]
965 965 return self._delim_re.split(l)[-1]
966 966
967 967
968 968
969 969 class Completer(Configurable):
970 970
971 971 greedy = Bool(
972 972 False,
973 973 help="""Activate greedy completion.
974 974
975 975 .. deprecated:: 8.8
976 976 Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
977 977
978 978 When enabled in IPython 8.8 or newer, changes configuration as follows:
979 979
980 980 - ``Completer.evaluation = 'unsafe'``
981 981 - ``Completer.auto_close_dict_keys = True``
982 982 """,
983 983 ).tag(config=True)
984 984
985 985 evaluation = Enum(
986 986 ("forbidden", "minimal", "limited", "unsafe", "dangerous"),
987 987 default_value="limited",
988 988 help="""Policy for code evaluation under completion.
989 989
990 990 Successive options allow to enable more eager evaluation for better
991 991 completion suggestions, including for nested dictionaries, nested lists,
992 992 or even results of function calls.
993 993 Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
994 994 code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
995 995
996 996 Allowed values are:
997 997
998 998 - ``forbidden``: no evaluation of code is permitted,
999 999 - ``minimal``: evaluation of literals and access to built-in namespace;
1000 1000 no item/attribute evaluationm no access to locals/globals,
1001 1001 no evaluation of any operations or comparisons.
1002 1002 - ``limited``: access to all namespaces, evaluation of hard-coded methods
1003 1003 (for example: :any:`dict.keys`, :any:`object.__getattr__`,
1004 1004 :any:`object.__getitem__`) on allow-listed objects (for example:
1005 1005 :any:`dict`, :any:`list`, :any:`tuple`, ``pandas.Series``),
1006 1006 - ``unsafe``: evaluation of all methods and function calls but not of
1007 1007 syntax with side-effects like `del x`,
1008 1008 - ``dangerous``: completely arbitrary evaluation.
1009 1009 """,
1010 1010 ).tag(config=True)
1011 1011
1012 1012 use_jedi = Bool(default_value=JEDI_INSTALLED,
1013 1013 help="Experimental: Use Jedi to generate autocompletions. "
1014 1014 "Default to True if jedi is installed.").tag(config=True)
1015 1015
1016 1016 jedi_compute_type_timeout = Int(default_value=400,
1017 1017 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
1018 1018 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
1019 1019 performance by preventing jedi to build its cache.
1020 1020 """).tag(config=True)
1021 1021
1022 1022 debug = Bool(default_value=False,
1023 1023 help='Enable debug for the Completer. Mostly print extra '
1024 1024 'information for experimental jedi integration.')\
1025 1025 .tag(config=True)
1026 1026
1027 1027 backslash_combining_completions = Bool(True,
1028 1028 help="Enable unicode completions, e.g. \\alpha<tab> . "
1029 1029 "Includes completion of latex commands, unicode names, and expanding "
1030 1030 "unicode characters back to latex commands.").tag(config=True)
1031 1031
1032 1032 auto_close_dict_keys = Bool(
1033 1033 False,
1034 1034 help="""
1035 1035 Enable auto-closing dictionary keys.
1036 1036
1037 1037 When enabled string keys will be suffixed with a final quote
1038 1038 (matching the opening quote), tuple keys will also receive a
1039 1039 separating comma if needed, and keys which are final will
1040 1040 receive a closing bracket (``]``).
1041 1041 """,
1042 1042 ).tag(config=True)
1043 1043
1044 1044 def __init__(self, namespace=None, global_namespace=None, **kwargs):
1045 1045 """Create a new completer for the command line.
1046 1046
1047 1047 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
1048 1048
1049 1049 If unspecified, the default namespace where completions are performed
1050 1050 is __main__ (technically, __main__.__dict__). Namespaces should be
1051 1051 given as dictionaries.
1052 1052
1053 1053 An optional second namespace can be given. This allows the completer
1054 1054 to handle cases where both the local and global scopes need to be
1055 1055 distinguished.
1056 1056 """
1057 1057
1058 1058 # Don't bind to namespace quite yet, but flag whether the user wants a
1059 1059 # specific namespace or to use __main__.__dict__. This will allow us
1060 1060 # to bind to __main__.__dict__ at completion time, not now.
1061 1061 if namespace is None:
1062 1062 self.use_main_ns = True
1063 1063 else:
1064 1064 self.use_main_ns = False
1065 1065 self.namespace = namespace
1066 1066
1067 1067 # The global namespace, if given, can be bound directly
1068 1068 if global_namespace is None:
1069 1069 self.global_namespace = {}
1070 1070 else:
1071 1071 self.global_namespace = global_namespace
1072 1072
1073 1073 self.custom_matchers = []
1074 1074
1075 1075 super(Completer, self).__init__(**kwargs)
1076 1076
1077 1077 def complete(self, text, state):
1078 1078 """Return the next possible completion for 'text'.
1079 1079
1080 1080 This is called successively with state == 0, 1, 2, ... until it
1081 1081 returns None. The completion should begin with 'text'.
1082 1082
1083 1083 """
1084 1084 if self.use_main_ns:
1085 1085 self.namespace = __main__.__dict__
1086 1086
1087 1087 if state == 0:
1088 1088 if "." in text:
1089 1089 self.matches = self.attr_matches(text)
1090 1090 else:
1091 1091 self.matches = self.global_matches(text)
1092 1092 try:
1093 1093 return self.matches[state]
1094 1094 except IndexError:
1095 1095 return None
1096 1096
1097 1097 def global_matches(self, text):
1098 1098 """Compute matches when text is a simple name.
1099 1099
1100 1100 Return a list of all keywords, built-in functions and names currently
1101 1101 defined in self.namespace or self.global_namespace that match.
1102 1102
1103 1103 """
1104 1104 matches = []
1105 1105 match_append = matches.append
1106 1106 n = len(text)
1107 1107 for lst in [
1108 1108 keyword.kwlist,
1109 1109 builtin_mod.__dict__.keys(),
1110 1110 list(self.namespace.keys()),
1111 1111 list(self.global_namespace.keys()),
1112 1112 ]:
1113 1113 for word in lst:
1114 1114 if word[:n] == text and word != "__builtins__":
1115 1115 match_append(word)
1116 1116
1117 1117 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
1118 1118 for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
1119 1119 shortened = {
1120 1120 "_".join([sub[0] for sub in word.split("_")]): word
1121 1121 for word in lst
1122 1122 if snake_case_re.match(word)
1123 1123 }
1124 1124 for word in shortened.keys():
1125 1125 if word[:n] == text and word != "__builtins__":
1126 1126 match_append(shortened[word])
1127 1127 return matches
1128 1128
1129 1129 def attr_matches(self, text):
1130 1130 """Compute matches when text contains a dot.
1131 1131
1132 1132 Assuming the text is of the form NAME.NAME....[NAME], and is
1133 1133 evaluatable in self.namespace or self.global_namespace, it will be
1134 1134 evaluated and its attributes (as revealed by dir()) are used as
1135 1135 possible completions. (For class instances, class members are
1136 1136 also considered.)
1137 1137
1138 1138 WARNING: this can still invoke arbitrary C code, if an object
1139 1139 with a __getattr__ hook is evaluated.
1140 1140
1141 1141 """
1142 1142 return self._attr_matches(text)[0]
1143 1143
1144 def _attr_matches(self, text, include_prefix=True):
1144 def _attr_matches(self, text, include_prefix=True) -> Tuple[Sequence[str], str]:
1145 1145 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
1146 1146 if not m2:
1147 return []
1147 return [], ""
1148 1148 expr, attr = m2.group(1, 2)
1149 1149
1150 1150 obj = self._evaluate_expr(expr)
1151 1151
1152 1152 if obj is not_found:
1153 return []
1153 return [], ""
1154 1154
1155 1155 if self.limit_to__all__ and hasattr(obj, '__all__'):
1156 1156 words = get__all__entries(obj)
1157 1157 else:
1158 1158 words = dir2(obj)
1159 1159
1160 1160 try:
1161 1161 words = generics.complete_object(obj, words)
1162 1162 except TryNext:
1163 1163 pass
1164 1164 except AssertionError:
1165 1165 raise
1166 1166 except Exception:
1167 1167 # Silence errors from completion function
1168 1168 pass
1169 1169 # Build match list to return
1170 1170 n = len(attr)
1171 1171
1172 1172 # Note: ideally we would just return words here and the prefix
1173 1173 # reconciliator would know that we intend to append to rather than
1174 1174 # replace the input text; this requires refactoring to return range
1175 1175 # which ought to be replaced (as does jedi).
1176 1176 if include_prefix:
1177 1177 tokens = _parse_tokens(expr)
1178 1178 rev_tokens = reversed(tokens)
1179 1179 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1180 1180 name_turn = True
1181 1181
1182 1182 parts = []
1183 1183 for token in rev_tokens:
1184 1184 if token.type in skip_over:
1185 1185 continue
1186 1186 if token.type == tokenize.NAME and name_turn:
1187 1187 parts.append(token.string)
1188 1188 name_turn = False
1189 1189 elif (
1190 1190 token.type == tokenize.OP and token.string == "." and not name_turn
1191 1191 ):
1192 1192 parts.append(token.string)
1193 1193 name_turn = True
1194 1194 else:
1195 1195 # short-circuit if not empty nor name token
1196 1196 break
1197 1197
1198 1198 prefix_after_space = "".join(reversed(parts))
1199 1199 else:
1200 1200 prefix_after_space = ""
1201 1201
1202 1202 return (
1203 1203 ["%s.%s" % (prefix_after_space, w) for w in words if w[:n] == attr],
1204 1204 "." + attr,
1205 1205 )
1206 1206
1207 1207 def _evaluate_expr(self, expr):
1208 1208 obj = not_found
1209 1209 done = False
1210 1210 while not done and expr:
1211 1211 try:
1212 1212 obj = guarded_eval(
1213 1213 expr,
1214 1214 EvaluationContext(
1215 1215 globals=self.global_namespace,
1216 1216 locals=self.namespace,
1217 1217 evaluation=self.evaluation,
1218 1218 ),
1219 1219 )
1220 1220 done = True
1221 1221 except Exception as e:
1222 1222 if self.debug:
1223 1223 print("Evaluation exception", e)
1224 1224 # trim the expression to remove any invalid prefix
1225 1225 # e.g. user starts `(d[`, so we get `expr = '(d'`,
1226 1226 # where parenthesis is not closed.
1227 1227 # TODO: make this faster by reusing parts of the computation?
1228 1228 expr = expr[1:]
1229 1229 return obj
1230 1230
1231 1231 def get__all__entries(obj):
1232 1232 """returns the strings in the __all__ attribute"""
1233 1233 try:
1234 1234 words = getattr(obj, '__all__')
1235 1235 except:
1236 1236 return []
1237 1237
1238 1238 return [w for w in words if isinstance(w, str)]
1239 1239
1240 1240
1241 1241 class _DictKeyState(enum.Flag):
1242 1242 """Represent state of the key match in context of other possible matches.
1243 1243
1244 1244 - given `d1 = {'a': 1}` completion on `d1['<tab>` will yield `{'a': END_OF_ITEM}` as there is no tuple.
1245 1245 - given `d2 = {('a', 'b'): 1}`: `d2['a', '<tab>` will yield `{'b': END_OF_TUPLE}` as there is no tuple members to add beyond `'b'`.
1246 1246 - given `d3 = {('a', 'b'): 1}`: `d3['<tab>` will yield `{'a': IN_TUPLE}` as `'a'` can be added.
1247 1247 - given `d4 = {'a': 1, ('a', 'b'): 2}`: `d4['<tab>` will yield `{'a': END_OF_ITEM & END_OF_TUPLE}`
1248 1248 """
1249 1249
1250 1250 BASELINE = 0
1251 1251 END_OF_ITEM = enum.auto()
1252 1252 END_OF_TUPLE = enum.auto()
1253 1253 IN_TUPLE = enum.auto()
1254 1254
1255 1255
1256 1256 def _parse_tokens(c):
1257 1257 """Parse tokens even if there is an error."""
1258 1258 tokens = []
1259 1259 token_generator = tokenize.generate_tokens(iter(c.splitlines()).__next__)
1260 1260 while True:
1261 1261 try:
1262 1262 tokens.append(next(token_generator))
1263 1263 except tokenize.TokenError:
1264 1264 return tokens
1265 1265 except StopIteration:
1266 1266 return tokens
1267 1267
1268 1268
1269 1269 def _match_number_in_dict_key_prefix(prefix: str) -> Union[str, None]:
1270 1270 """Match any valid Python numeric literal in a prefix of dictionary keys.
1271 1271
1272 1272 References:
1273 1273 - https://docs.python.org/3/reference/lexical_analysis.html#numeric-literals
1274 1274 - https://docs.python.org/3/library/tokenize.html
1275 1275 """
1276 1276 if prefix[-1].isspace():
1277 1277 # if user typed a space we do not have anything to complete
1278 1278 # even if there was a valid number token before
1279 1279 return None
1280 1280 tokens = _parse_tokens(prefix)
1281 1281 rev_tokens = reversed(tokens)
1282 1282 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1283 1283 number = None
1284 1284 for token in rev_tokens:
1285 1285 if token.type in skip_over:
1286 1286 continue
1287 1287 if number is None:
1288 1288 if token.type == tokenize.NUMBER:
1289 1289 number = token.string
1290 1290 continue
1291 1291 else:
1292 1292 # we did not match a number
1293 1293 return None
1294 1294 if token.type == tokenize.OP:
1295 1295 if token.string == ",":
1296 1296 break
1297 1297 if token.string in {"+", "-"}:
1298 1298 number = token.string + number
1299 1299 else:
1300 1300 return None
1301 1301 return number
1302 1302
1303 1303
1304 1304 _INT_FORMATS = {
1305 1305 "0b": bin,
1306 1306 "0o": oct,
1307 1307 "0x": hex,
1308 1308 }
1309 1309
1310 1310
1311 1311 def match_dict_keys(
1312 1312 keys: List[Union[str, bytes, Tuple[Union[str, bytes], ...]]],
1313 1313 prefix: str,
1314 1314 delims: str,
1315 1315 extra_prefix: Optional[Tuple[Union[str, bytes], ...]] = None,
1316 1316 ) -> Tuple[str, int, Dict[str, _DictKeyState]]:
1317 1317 """Used by dict_key_matches, matching the prefix to a list of keys
1318 1318
1319 1319 Parameters
1320 1320 ----------
1321 1321 keys
1322 1322 list of keys in dictionary currently being completed.
1323 1323 prefix
1324 1324 Part of the text already typed by the user. E.g. `mydict[b'fo`
1325 1325 delims
1326 1326 String of delimiters to consider when finding the current key.
1327 1327 extra_prefix : optional
1328 1328 Part of the text already typed in multi-key index cases. E.g. for
1329 1329 `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
1330 1330
1331 1331 Returns
1332 1332 -------
1333 1333 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
1334 1334 ``quote`` being the quote that need to be used to close current string.
1335 1335 ``token_start`` the position where the replacement should start occurring,
1336 1336 ``matches`` a dictionary of replacement/completion keys on keys and values
1337 1337 indicating whether the state.
1338 1338 """
1339 1339 prefix_tuple = extra_prefix if extra_prefix else ()
1340 1340
1341 1341 prefix_tuple_size = sum(
1342 1342 [
1343 1343 # for pandas, do not count slices as taking space
1344 1344 not isinstance(k, slice)
1345 1345 for k in prefix_tuple
1346 1346 ]
1347 1347 )
1348 1348 text_serializable_types = (str, bytes, int, float, slice)
1349 1349
1350 1350 def filter_prefix_tuple(key):
1351 1351 # Reject too short keys
1352 1352 if len(key) <= prefix_tuple_size:
1353 1353 return False
1354 1354 # Reject keys which cannot be serialised to text
1355 1355 for k in key:
1356 1356 if not isinstance(k, text_serializable_types):
1357 1357 return False
1358 1358 # Reject keys that do not match the prefix
1359 1359 for k, pt in zip(key, prefix_tuple):
1360 1360 if k != pt and not isinstance(pt, slice):
1361 1361 return False
1362 1362 # All checks passed!
1363 1363 return True
1364 1364
1365 1365 filtered_key_is_final: Dict[
1366 1366 Union[str, bytes, int, float], _DictKeyState
1367 1367 ] = defaultdict(lambda: _DictKeyState.BASELINE)
1368 1368
1369 1369 for k in keys:
1370 1370 # If at least one of the matches is not final, mark as undetermined.
1371 1371 # This can happen with `d = {111: 'b', (111, 222): 'a'}` where
1372 1372 # `111` appears final on first match but is not final on the second.
1373 1373
1374 1374 if isinstance(k, tuple):
1375 1375 if filter_prefix_tuple(k):
1376 1376 key_fragment = k[prefix_tuple_size]
1377 1377 filtered_key_is_final[key_fragment] |= (
1378 1378 _DictKeyState.END_OF_TUPLE
1379 1379 if len(k) == prefix_tuple_size + 1
1380 1380 else _DictKeyState.IN_TUPLE
1381 1381 )
1382 1382 elif prefix_tuple_size > 0:
1383 1383 # we are completing a tuple but this key is not a tuple,
1384 1384 # so we should ignore it
1385 1385 pass
1386 1386 else:
1387 1387 if isinstance(k, text_serializable_types):
1388 1388 filtered_key_is_final[k] |= _DictKeyState.END_OF_ITEM
1389 1389
1390 1390 filtered_keys = filtered_key_is_final.keys()
1391 1391
1392 1392 if not prefix:
1393 1393 return "", 0, {repr(k): v for k, v in filtered_key_is_final.items()}
1394 1394
1395 1395 quote_match = re.search("(?:\"|')", prefix)
1396 1396 is_user_prefix_numeric = False
1397 1397
1398 1398 if quote_match:
1399 1399 quote = quote_match.group()
1400 1400 valid_prefix = prefix + quote
1401 1401 try:
1402 1402 prefix_str = literal_eval(valid_prefix)
1403 1403 except Exception:
1404 1404 return "", 0, {}
1405 1405 else:
1406 1406 # If it does not look like a string, let's assume
1407 1407 # we are dealing with a number or variable.
1408 1408 number_match = _match_number_in_dict_key_prefix(prefix)
1409 1409
1410 1410 # We do not want the key matcher to suggest variable names so we yield:
1411 1411 if number_match is None:
1412 1412 # The alternative would be to assume that user forgort the quote
1413 1413 # and if the substring matches, suggest adding it at the start.
1414 1414 return "", 0, {}
1415 1415
1416 1416 prefix_str = number_match
1417 1417 is_user_prefix_numeric = True
1418 1418 quote = ""
1419 1419
1420 1420 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
1421 1421 token_match = re.search(pattern, prefix, re.UNICODE)
1422 1422 assert token_match is not None # silence mypy
1423 1423 token_start = token_match.start()
1424 1424 token_prefix = token_match.group()
1425 1425
1426 1426 matched: Dict[str, _DictKeyState] = {}
1427 1427
1428 1428 str_key: Union[str, bytes]
1429 1429
1430 1430 for key in filtered_keys:
1431 1431 if isinstance(key, (int, float)):
1432 1432 # User typed a number but this key is not a number.
1433 1433 if not is_user_prefix_numeric:
1434 1434 continue
1435 1435 str_key = str(key)
1436 1436 if isinstance(key, int):
1437 1437 int_base = prefix_str[:2].lower()
1438 1438 # if user typed integer using binary/oct/hex notation:
1439 1439 if int_base in _INT_FORMATS:
1440 1440 int_format = _INT_FORMATS[int_base]
1441 1441 str_key = int_format(key)
1442 1442 else:
1443 1443 # User typed a string but this key is a number.
1444 1444 if is_user_prefix_numeric:
1445 1445 continue
1446 1446 str_key = key
1447 1447 try:
1448 1448 if not str_key.startswith(prefix_str):
1449 1449 continue
1450 1450 except (AttributeError, TypeError, UnicodeError) as e:
1451 1451 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
1452 1452 continue
1453 1453
1454 1454 # reformat remainder of key to begin with prefix
1455 1455 rem = str_key[len(prefix_str) :]
1456 1456 # force repr wrapped in '
1457 1457 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
1458 1458 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
1459 1459 if quote == '"':
1460 1460 # The entered prefix is quoted with ",
1461 1461 # but the match is quoted with '.
1462 1462 # A contained " hence needs escaping for comparison:
1463 1463 rem_repr = rem_repr.replace('"', '\\"')
1464 1464
1465 1465 # then reinsert prefix from start of token
1466 1466 match = "%s%s" % (token_prefix, rem_repr)
1467 1467
1468 1468 matched[match] = filtered_key_is_final[key]
1469 1469 return quote, token_start, matched
1470 1470
1471 1471
1472 1472 def cursor_to_position(text:str, line:int, column:int)->int:
1473 1473 """
1474 1474 Convert the (line,column) position of the cursor in text to an offset in a
1475 1475 string.
1476 1476
1477 1477 Parameters
1478 1478 ----------
1479 1479 text : str
1480 1480 The text in which to calculate the cursor offset
1481 1481 line : int
1482 1482 Line of the cursor; 0-indexed
1483 1483 column : int
1484 1484 Column of the cursor 0-indexed
1485 1485
1486 1486 Returns
1487 1487 -------
1488 1488 Position of the cursor in ``text``, 0-indexed.
1489 1489
1490 1490 See Also
1491 1491 --------
1492 1492 position_to_cursor : reciprocal of this function
1493 1493
1494 1494 """
1495 1495 lines = text.split('\n')
1496 1496 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
1497 1497
1498 1498 return sum(len(l) + 1 for l in lines[:line]) + column
1499 1499
1500 1500 def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
1501 1501 """
1502 1502 Convert the position of the cursor in text (0 indexed) to a line
1503 1503 number(0-indexed) and a column number (0-indexed) pair
1504 1504
1505 1505 Position should be a valid position in ``text``.
1506 1506
1507 1507 Parameters
1508 1508 ----------
1509 1509 text : str
1510 1510 The text in which to calculate the cursor offset
1511 1511 offset : int
1512 1512 Position of the cursor in ``text``, 0-indexed.
1513 1513
1514 1514 Returns
1515 1515 -------
1516 1516 (line, column) : (int, int)
1517 1517 Line of the cursor; 0-indexed, column of the cursor 0-indexed
1518 1518
1519 1519 See Also
1520 1520 --------
1521 1521 cursor_to_position : reciprocal of this function
1522 1522
1523 1523 """
1524 1524
1525 1525 assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
1526 1526
1527 1527 before = text[:offset]
1528 1528 blines = before.split('\n') # ! splitnes trim trailing \n
1529 1529 line = before.count('\n')
1530 1530 col = len(blines[-1])
1531 1531 return line, col
1532 1532
1533 1533
1534 1534 def _safe_isinstance(obj, module, class_name, *attrs):
1535 1535 """Checks if obj is an instance of module.class_name if loaded
1536 1536 """
1537 1537 if module in sys.modules:
1538 1538 m = sys.modules[module]
1539 1539 for attr in [class_name, *attrs]:
1540 1540 m = getattr(m, attr)
1541 1541 return isinstance(obj, m)
1542 1542
1543 1543
1544 1544 @context_matcher()
1545 1545 def back_unicode_name_matcher(context: CompletionContext):
1546 1546 """Match Unicode characters back to Unicode name
1547 1547
1548 1548 Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1549 1549 """
1550 1550 fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1551 1551 return _convert_matcher_v1_result_to_v2(
1552 1552 matches, type="unicode", fragment=fragment, suppress_if_matches=True
1553 1553 )
1554 1554
1555 1555
1556 1556 def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1557 1557 """Match Unicode characters back to Unicode name
1558 1558
1559 1559 This does ``β˜ƒ`` -> ``\\snowman``
1560 1560
1561 1561 Note that snowman is not a valid python3 combining character but will be expanded.
1562 1562 Though it will not recombine back to the snowman character by the completion machinery.
1563 1563
1564 1564 This will not either back-complete standard sequences like \\n, \\b ...
1565 1565
1566 1566 .. deprecated:: 8.6
1567 1567 You can use :meth:`back_unicode_name_matcher` instead.
1568 1568
1569 1569 Returns
1570 1570 =======
1571 1571
1572 1572 Return a tuple with two elements:
1573 1573
1574 1574 - The Unicode character that was matched (preceded with a backslash), or
1575 1575 empty string,
1576 1576 - a sequence (of 1), name for the match Unicode character, preceded by
1577 1577 backslash, or empty if no match.
1578 1578 """
1579 1579 if len(text)<2:
1580 1580 return '', ()
1581 1581 maybe_slash = text[-2]
1582 1582 if maybe_slash != '\\':
1583 1583 return '', ()
1584 1584
1585 1585 char = text[-1]
1586 1586 # no expand on quote for completion in strings.
1587 1587 # nor backcomplete standard ascii keys
1588 1588 if char in string.ascii_letters or char in ('"',"'"):
1589 1589 return '', ()
1590 1590 try :
1591 1591 unic = unicodedata.name(char)
1592 1592 return '\\'+char,('\\'+unic,)
1593 1593 except KeyError:
1594 1594 pass
1595 1595 return '', ()
1596 1596
1597 1597
1598 1598 @context_matcher()
1599 1599 def back_latex_name_matcher(context: CompletionContext):
1600 1600 """Match latex characters back to unicode name
1601 1601
1602 1602 Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1603 1603 """
1604 1604 fragment, matches = back_latex_name_matches(context.text_until_cursor)
1605 1605 return _convert_matcher_v1_result_to_v2(
1606 1606 matches, type="latex", fragment=fragment, suppress_if_matches=True
1607 1607 )
1608 1608
1609 1609
1610 1610 def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1611 1611 """Match latex characters back to unicode name
1612 1612
1613 1613 This does ``\\β„΅`` -> ``\\aleph``
1614 1614
1615 1615 .. deprecated:: 8.6
1616 1616 You can use :meth:`back_latex_name_matcher` instead.
1617 1617 """
1618 1618 if len(text)<2:
1619 1619 return '', ()
1620 1620 maybe_slash = text[-2]
1621 1621 if maybe_slash != '\\':
1622 1622 return '', ()
1623 1623
1624 1624
1625 1625 char = text[-1]
1626 1626 # no expand on quote for completion in strings.
1627 1627 # nor backcomplete standard ascii keys
1628 1628 if char in string.ascii_letters or char in ('"',"'"):
1629 1629 return '', ()
1630 1630 try :
1631 1631 latex = reverse_latex_symbol[char]
1632 1632 # '\\' replace the \ as well
1633 1633 return '\\'+char,[latex]
1634 1634 except KeyError:
1635 1635 pass
1636 1636 return '', ()
1637 1637
1638 1638
1639 1639 def _formatparamchildren(parameter) -> str:
1640 1640 """
1641 1641 Get parameter name and value from Jedi Private API
1642 1642
1643 1643 Jedi does not expose a simple way to get `param=value` from its API.
1644 1644
1645 1645 Parameters
1646 1646 ----------
1647 1647 parameter
1648 1648 Jedi's function `Param`
1649 1649
1650 1650 Returns
1651 1651 -------
1652 1652 A string like 'a', 'b=1', '*args', '**kwargs'
1653 1653
1654 1654 """
1655 1655 description = parameter.description
1656 1656 if not description.startswith('param '):
1657 1657 raise ValueError('Jedi function parameter description have change format.'
1658 1658 'Expected "param ...", found %r".' % description)
1659 1659 return description[6:]
1660 1660
1661 1661 def _make_signature(completion)-> str:
1662 1662 """
1663 1663 Make the signature from a jedi completion
1664 1664
1665 1665 Parameters
1666 1666 ----------
1667 1667 completion : jedi.Completion
1668 1668 object does not complete a function type
1669 1669
1670 1670 Returns
1671 1671 -------
1672 1672 a string consisting of the function signature, with the parenthesis but
1673 1673 without the function name. example:
1674 1674 `(a, *args, b=1, **kwargs)`
1675 1675
1676 1676 """
1677 1677
1678 1678 # it looks like this might work on jedi 0.17
1679 1679 if hasattr(completion, 'get_signatures'):
1680 1680 signatures = completion.get_signatures()
1681 1681 if not signatures:
1682 1682 return '(?)'
1683 1683
1684 1684 c0 = completion.get_signatures()[0]
1685 1685 return '('+c0.to_string().split('(', maxsplit=1)[1]
1686 1686
1687 1687 return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1688 1688 for p in signature.defined_names()) if f])
1689 1689
1690 1690
1691 1691 _CompleteResult = Dict[str, MatcherResult]
1692 1692
1693 1693
1694 1694 DICT_MATCHER_REGEX = re.compile(
1695 1695 r"""(?x)
1696 1696 ( # match dict-referring - or any get item object - expression
1697 1697 .+
1698 1698 )
1699 1699 \[ # open bracket
1700 1700 \s* # and optional whitespace
1701 1701 # Capture any number of serializable objects (e.g. "a", "b", 'c')
1702 1702 # and slices
1703 1703 ((?:(?:
1704 1704 (?: # closed string
1705 1705 [uUbB]? # string prefix (r not handled)
1706 1706 (?:
1707 1707 '(?:[^']|(?<!\\)\\')*'
1708 1708 |
1709 1709 "(?:[^"]|(?<!\\)\\")*"
1710 1710 )
1711 1711 )
1712 1712 |
1713 1713 # capture integers and slices
1714 1714 (?:[-+]?\d+)?(?::(?:[-+]?\d+)?){0,2}
1715 1715 |
1716 1716 # integer in bin/hex/oct notation
1717 1717 0[bBxXoO]_?(?:\w|\d)+
1718 1718 )
1719 1719 \s*,\s*
1720 1720 )*)
1721 1721 ((?:
1722 1722 (?: # unclosed string
1723 1723 [uUbB]? # string prefix (r not handled)
1724 1724 (?:
1725 1725 '(?:[^']|(?<!\\)\\')*
1726 1726 |
1727 1727 "(?:[^"]|(?<!\\)\\")*
1728 1728 )
1729 1729 )
1730 1730 |
1731 1731 # unfinished integer
1732 1732 (?:[-+]?\d+)
1733 1733 |
1734 1734 # integer in bin/hex/oct notation
1735 1735 0[bBxXoO]_?(?:\w|\d)+
1736 1736 )
1737 1737 )?
1738 1738 $
1739 1739 """
1740 1740 )
1741 1741
1742 1742
1743 1743 def _convert_matcher_v1_result_to_v2(
1744 1744 matches: Sequence[str],
1745 1745 type: str,
1746 1746 fragment: Optional[str] = None,
1747 1747 suppress_if_matches: bool = False,
1748 1748 ) -> SimpleMatcherResult:
1749 1749 """Utility to help with transition"""
1750 1750 result = {
1751 1751 "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1752 1752 "suppress": (True if matches else False) if suppress_if_matches else False,
1753 1753 }
1754 1754 if fragment is not None:
1755 1755 result["matched_fragment"] = fragment
1756 1756 return cast(SimpleMatcherResult, result)
1757 1757
1758 1758
1759 1759 class IPCompleter(Completer):
1760 1760 """Extension of the completer class with IPython-specific features"""
1761 1761
1762 1762 @observe('greedy')
1763 1763 def _greedy_changed(self, change):
1764 1764 """update the splitter and readline delims when greedy is changed"""
1765 1765 if change["new"]:
1766 1766 self.evaluation = "unsafe"
1767 1767 self.auto_close_dict_keys = True
1768 1768 self.splitter.delims = GREEDY_DELIMS
1769 1769 else:
1770 1770 self.evaluation = "limited"
1771 1771 self.auto_close_dict_keys = False
1772 1772 self.splitter.delims = DELIMS
1773 1773
1774 1774 dict_keys_only = Bool(
1775 1775 False,
1776 1776 help="""
1777 1777 Whether to show dict key matches only.
1778 1778
1779 1779 (disables all matchers except for `IPCompleter.dict_key_matcher`).
1780 1780 """,
1781 1781 )
1782 1782
1783 1783 suppress_competing_matchers = UnionTrait(
1784 1784 [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1785 1785 default_value=None,
1786 1786 help="""
1787 1787 Whether to suppress completions from other *Matchers*.
1788 1788
1789 1789 When set to ``None`` (default) the matchers will attempt to auto-detect
1790 1790 whether suppression of other matchers is desirable. For example, at
1791 1791 the beginning of a line followed by `%` we expect a magic completion
1792 1792 to be the only applicable option, and after ``my_dict['`` we usually
1793 1793 expect a completion with an existing dictionary key.
1794 1794
1795 1795 If you want to disable this heuristic and see completions from all matchers,
1796 1796 set ``IPCompleter.suppress_competing_matchers = False``.
1797 1797 To disable the heuristic for specific matchers provide a dictionary mapping:
1798 1798 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1799 1799
1800 1800 Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1801 1801 completions to the set of matchers with the highest priority;
1802 1802 this is equivalent to ``IPCompleter.merge_completions`` and
1803 1803 can be beneficial for performance, but will sometimes omit relevant
1804 1804 candidates from matchers further down the priority list.
1805 1805 """,
1806 1806 ).tag(config=True)
1807 1807
1808 1808 merge_completions = Bool(
1809 1809 True,
1810 1810 help="""Whether to merge completion results into a single list
1811 1811
1812 1812 If False, only the completion results from the first non-empty
1813 1813 completer will be returned.
1814 1814
1815 1815 As of version 8.6.0, setting the value to ``False`` is an alias for:
1816 1816 ``IPCompleter.suppress_competing_matchers = True.``.
1817 1817 """,
1818 1818 ).tag(config=True)
1819 1819
1820 1820 disable_matchers = ListTrait(
1821 1821 Unicode(),
1822 1822 help="""List of matchers to disable.
1823 1823
1824 1824 The list should contain matcher identifiers (see :any:`completion_matcher`).
1825 1825 """,
1826 1826 ).tag(config=True)
1827 1827
1828 1828 omit__names = Enum(
1829 1829 (0, 1, 2),
1830 1830 default_value=2,
1831 1831 help="""Instruct the completer to omit private method names
1832 1832
1833 1833 Specifically, when completing on ``object.<tab>``.
1834 1834
1835 1835 When 2 [default]: all names that start with '_' will be excluded.
1836 1836
1837 1837 When 1: all 'magic' names (``__foo__``) will be excluded.
1838 1838
1839 1839 When 0: nothing will be excluded.
1840 1840 """
1841 1841 ).tag(config=True)
1842 1842 limit_to__all__ = Bool(False,
1843 1843 help="""
1844 1844 DEPRECATED as of version 5.0.
1845 1845
1846 1846 Instruct the completer to use __all__ for the completion
1847 1847
1848 1848 Specifically, when completing on ``object.<tab>``.
1849 1849
1850 1850 When True: only those names in obj.__all__ will be included.
1851 1851
1852 1852 When False [default]: the __all__ attribute is ignored
1853 1853 """,
1854 1854 ).tag(config=True)
1855 1855
1856 1856 profile_completions = Bool(
1857 1857 default_value=False,
1858 1858 help="If True, emit profiling data for completion subsystem using cProfile."
1859 1859 ).tag(config=True)
1860 1860
1861 1861 profiler_output_dir = Unicode(
1862 1862 default_value=".completion_profiles",
1863 1863 help="Template for path at which to output profile data for completions."
1864 1864 ).tag(config=True)
1865 1865
1866 1866 @observe('limit_to__all__')
1867 1867 def _limit_to_all_changed(self, change):
1868 1868 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1869 1869 'value has been deprecated since IPython 5.0, will be made to have '
1870 1870 'no effects and then removed in future version of IPython.',
1871 1871 UserWarning)
1872 1872
1873 1873 def __init__(
1874 1874 self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1875 1875 ):
1876 1876 """IPCompleter() -> completer
1877 1877
1878 1878 Return a completer object.
1879 1879
1880 1880 Parameters
1881 1881 ----------
1882 1882 shell
1883 1883 a pointer to the ipython shell itself. This is needed
1884 1884 because this completer knows about magic functions, and those can
1885 1885 only be accessed via the ipython instance.
1886 1886 namespace : dict, optional
1887 1887 an optional dict where completions are performed.
1888 1888 global_namespace : dict, optional
1889 1889 secondary optional dict for completions, to
1890 1890 handle cases (such as IPython embedded inside functions) where
1891 1891 both Python scopes are visible.
1892 1892 config : Config
1893 1893 traitlet's config object
1894 1894 **kwargs
1895 1895 passed to super class unmodified.
1896 1896 """
1897 1897
1898 1898 self.magic_escape = ESC_MAGIC
1899 1899 self.splitter = CompletionSplitter()
1900 1900
1901 1901 # _greedy_changed() depends on splitter and readline being defined:
1902 1902 super().__init__(
1903 1903 namespace=namespace,
1904 1904 global_namespace=global_namespace,
1905 1905 config=config,
1906 1906 **kwargs,
1907 1907 )
1908 1908
1909 1909 # List where completion matches will be stored
1910 1910 self.matches = []
1911 1911 self.shell = shell
1912 1912 # Regexp to split filenames with spaces in them
1913 1913 self.space_name_re = re.compile(r'([^\\] )')
1914 1914 # Hold a local ref. to glob.glob for speed
1915 1915 self.glob = glob.glob
1916 1916
1917 1917 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1918 1918 # buffers, to avoid completion problems.
1919 1919 term = os.environ.get('TERM','xterm')
1920 1920 self.dumb_terminal = term in ['dumb','emacs']
1921 1921
1922 1922 # Special handling of backslashes needed in win32 platforms
1923 1923 if sys.platform == "win32":
1924 1924 self.clean_glob = self._clean_glob_win32
1925 1925 else:
1926 1926 self.clean_glob = self._clean_glob
1927 1927
1928 1928 #regexp to parse docstring for function signature
1929 1929 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1930 1930 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1931 1931 #use this if positional argument name is also needed
1932 1932 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1933 1933
1934 1934 self.magic_arg_matchers = [
1935 1935 self.magic_config_matcher,
1936 1936 self.magic_color_matcher,
1937 1937 ]
1938 1938
1939 1939 # This is set externally by InteractiveShell
1940 1940 self.custom_completers = None
1941 1941
1942 1942 # This is a list of names of unicode characters that can be completed
1943 1943 # into their corresponding unicode value. The list is large, so we
1944 1944 # lazily initialize it on first use. Consuming code should access this
1945 1945 # attribute through the `@unicode_names` property.
1946 1946 self._unicode_names = None
1947 1947
1948 1948 self._backslash_combining_matchers = [
1949 1949 self.latex_name_matcher,
1950 1950 self.unicode_name_matcher,
1951 1951 back_latex_name_matcher,
1952 1952 back_unicode_name_matcher,
1953 1953 self.fwd_unicode_matcher,
1954 1954 ]
1955 1955
1956 1956 if not self.backslash_combining_completions:
1957 1957 for matcher in self._backslash_combining_matchers:
1958 1958 self.disable_matchers.append(_get_matcher_id(matcher))
1959 1959
1960 1960 if not self.merge_completions:
1961 1961 self.suppress_competing_matchers = True
1962 1962
1963 1963 @property
1964 1964 def matchers(self) -> List[Matcher]:
1965 1965 """All active matcher routines for completion"""
1966 1966 if self.dict_keys_only:
1967 1967 return [self.dict_key_matcher]
1968 1968
1969 1969 if self.use_jedi:
1970 1970 return [
1971 1971 *self.custom_matchers,
1972 1972 *self._backslash_combining_matchers,
1973 1973 *self.magic_arg_matchers,
1974 1974 self.custom_completer_matcher,
1975 1975 self.magic_matcher,
1976 1976 self._jedi_matcher,
1977 1977 self.dict_key_matcher,
1978 1978 self.file_matcher,
1979 1979 ]
1980 1980 else:
1981 1981 return [
1982 1982 *self.custom_matchers,
1983 1983 *self._backslash_combining_matchers,
1984 1984 *self.magic_arg_matchers,
1985 1985 self.custom_completer_matcher,
1986 1986 self.dict_key_matcher,
1987 1987 self.magic_matcher,
1988 1988 self.python_matcher,
1989 1989 self.file_matcher,
1990 1990 self.python_func_kw_matcher,
1991 1991 ]
1992 1992
1993 1993 def all_completions(self, text:str) -> List[str]:
1994 1994 """
1995 1995 Wrapper around the completion methods for the benefit of emacs.
1996 1996 """
1997 1997 prefix = text.rpartition('.')[0]
1998 1998 with provisionalcompleter():
1999 1999 return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
2000 2000 for c in self.completions(text, len(text))]
2001 2001
2002 2002 return self.complete(text)[1]
2003 2003
2004 2004 def _clean_glob(self, text:str):
2005 2005 return self.glob("%s*" % text)
2006 2006
2007 2007 def _clean_glob_win32(self, text:str):
2008 2008 return [f.replace("\\","/")
2009 2009 for f in self.glob("%s*" % text)]
2010 2010
2011 2011 @context_matcher()
2012 2012 def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2013 2013 """Same as :any:`file_matches`, but adopted to new Matcher API."""
2014 2014 matches = self.file_matches(context.token)
2015 2015 # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
2016 2016 # starts with `/home/`, `C:\`, etc)
2017 2017 return _convert_matcher_v1_result_to_v2(matches, type="path")
2018 2018
2019 2019 def file_matches(self, text: str) -> List[str]:
2020 2020 """Match filenames, expanding ~USER type strings.
2021 2021
2022 2022 Most of the seemingly convoluted logic in this completer is an
2023 2023 attempt to handle filenames with spaces in them. And yet it's not
2024 2024 quite perfect, because Python's readline doesn't expose all of the
2025 2025 GNU readline details needed for this to be done correctly.
2026 2026
2027 2027 For a filename with a space in it, the printed completions will be
2028 2028 only the parts after what's already been typed (instead of the
2029 2029 full completions, as is normally done). I don't think with the
2030 2030 current (as of Python 2.3) Python readline it's possible to do
2031 2031 better.
2032 2032
2033 2033 .. deprecated:: 8.6
2034 2034 You can use :meth:`file_matcher` instead.
2035 2035 """
2036 2036
2037 2037 # chars that require escaping with backslash - i.e. chars
2038 2038 # that readline treats incorrectly as delimiters, but we
2039 2039 # don't want to treat as delimiters in filename matching
2040 2040 # when escaped with backslash
2041 2041 if text.startswith('!'):
2042 2042 text = text[1:]
2043 2043 text_prefix = u'!'
2044 2044 else:
2045 2045 text_prefix = u''
2046 2046
2047 2047 text_until_cursor = self.text_until_cursor
2048 2048 # track strings with open quotes
2049 2049 open_quotes = has_open_quotes(text_until_cursor)
2050 2050
2051 2051 if '(' in text_until_cursor or '[' in text_until_cursor:
2052 2052 lsplit = text
2053 2053 else:
2054 2054 try:
2055 2055 # arg_split ~ shlex.split, but with unicode bugs fixed by us
2056 2056 lsplit = arg_split(text_until_cursor)[-1]
2057 2057 except ValueError:
2058 2058 # typically an unmatched ", or backslash without escaped char.
2059 2059 if open_quotes:
2060 2060 lsplit = text_until_cursor.split(open_quotes)[-1]
2061 2061 else:
2062 2062 return []
2063 2063 except IndexError:
2064 2064 # tab pressed on empty line
2065 2065 lsplit = ""
2066 2066
2067 2067 if not open_quotes and lsplit != protect_filename(lsplit):
2068 2068 # if protectables are found, do matching on the whole escaped name
2069 2069 has_protectables = True
2070 2070 text0,text = text,lsplit
2071 2071 else:
2072 2072 has_protectables = False
2073 2073 text = os.path.expanduser(text)
2074 2074
2075 2075 if text == "":
2076 2076 return [text_prefix + protect_filename(f) for f in self.glob("*")]
2077 2077
2078 2078 # Compute the matches from the filesystem
2079 2079 if sys.platform == 'win32':
2080 2080 m0 = self.clean_glob(text)
2081 2081 else:
2082 2082 m0 = self.clean_glob(text.replace('\\', ''))
2083 2083
2084 2084 if has_protectables:
2085 2085 # If we had protectables, we need to revert our changes to the
2086 2086 # beginning of filename so that we don't double-write the part
2087 2087 # of the filename we have so far
2088 2088 len_lsplit = len(lsplit)
2089 2089 matches = [text_prefix + text0 +
2090 2090 protect_filename(f[len_lsplit:]) for f in m0]
2091 2091 else:
2092 2092 if open_quotes:
2093 2093 # if we have a string with an open quote, we don't need to
2094 2094 # protect the names beyond the quote (and we _shouldn't_, as
2095 2095 # it would cause bugs when the filesystem call is made).
2096 2096 matches = m0 if sys.platform == "win32" else\
2097 2097 [protect_filename(f, open_quotes) for f in m0]
2098 2098 else:
2099 2099 matches = [text_prefix +
2100 2100 protect_filename(f) for f in m0]
2101 2101
2102 2102 # Mark directories in input list by appending '/' to their names.
2103 2103 return [x+'/' if os.path.isdir(x) else x for x in matches]
2104 2104
2105 2105 @context_matcher()
2106 2106 def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2107 2107 """Match magics."""
2108 2108 text = context.token
2109 2109 matches = self.magic_matches(text)
2110 2110 result = _convert_matcher_v1_result_to_v2(matches, type="magic")
2111 2111 is_magic_prefix = len(text) > 0 and text[0] == "%"
2112 2112 result["suppress"] = is_magic_prefix and bool(result["completions"])
2113 2113 return result
2114 2114
2115 2115 def magic_matches(self, text: str):
2116 2116 """Match magics.
2117 2117
2118 2118 .. deprecated:: 8.6
2119 2119 You can use :meth:`magic_matcher` instead.
2120 2120 """
2121 2121 # Get all shell magics now rather than statically, so magics loaded at
2122 2122 # runtime show up too.
2123 2123 lsm = self.shell.magics_manager.lsmagic()
2124 2124 line_magics = lsm['line']
2125 2125 cell_magics = lsm['cell']
2126 2126 pre = self.magic_escape
2127 2127 pre2 = pre+pre
2128 2128
2129 2129 explicit_magic = text.startswith(pre)
2130 2130
2131 2131 # Completion logic:
2132 2132 # - user gives %%: only do cell magics
2133 2133 # - user gives %: do both line and cell magics
2134 2134 # - no prefix: do both
2135 2135 # In other words, line magics are skipped if the user gives %% explicitly
2136 2136 #
2137 2137 # We also exclude magics that match any currently visible names:
2138 2138 # https://github.com/ipython/ipython/issues/4877, unless the user has
2139 2139 # typed a %:
2140 2140 # https://github.com/ipython/ipython/issues/10754
2141 2141 bare_text = text.lstrip(pre)
2142 2142 global_matches = self.global_matches(bare_text)
2143 2143 if not explicit_magic:
2144 2144 def matches(magic):
2145 2145 """
2146 2146 Filter magics, in particular remove magics that match
2147 2147 a name present in global namespace.
2148 2148 """
2149 2149 return ( magic.startswith(bare_text) and
2150 2150 magic not in global_matches )
2151 2151 else:
2152 2152 def matches(magic):
2153 2153 return magic.startswith(bare_text)
2154 2154
2155 2155 comp = [ pre2+m for m in cell_magics if matches(m)]
2156 2156 if not text.startswith(pre2):
2157 2157 comp += [ pre+m for m in line_magics if matches(m)]
2158 2158
2159 2159 return comp
2160 2160
2161 2161 @context_matcher()
2162 2162 def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2163 2163 """Match class names and attributes for %config magic."""
2164 2164 # NOTE: uses `line_buffer` equivalent for compatibility
2165 2165 matches = self.magic_config_matches(context.line_with_cursor)
2166 2166 return _convert_matcher_v1_result_to_v2(matches, type="param")
2167 2167
2168 2168 def magic_config_matches(self, text: str) -> List[str]:
2169 2169 """Match class names and attributes for %config magic.
2170 2170
2171 2171 .. deprecated:: 8.6
2172 2172 You can use :meth:`magic_config_matcher` instead.
2173 2173 """
2174 2174 texts = text.strip().split()
2175 2175
2176 2176 if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
2177 2177 # get all configuration classes
2178 2178 classes = sorted(set([ c for c in self.shell.configurables
2179 2179 if c.__class__.class_traits(config=True)
2180 2180 ]), key=lambda x: x.__class__.__name__)
2181 2181 classnames = [ c.__class__.__name__ for c in classes ]
2182 2182
2183 2183 # return all classnames if config or %config is given
2184 2184 if len(texts) == 1:
2185 2185 return classnames
2186 2186
2187 2187 # match classname
2188 2188 classname_texts = texts[1].split('.')
2189 2189 classname = classname_texts[0]
2190 2190 classname_matches = [ c for c in classnames
2191 2191 if c.startswith(classname) ]
2192 2192
2193 2193 # return matched classes or the matched class with attributes
2194 2194 if texts[1].find('.') < 0:
2195 2195 return classname_matches
2196 2196 elif len(classname_matches) == 1 and \
2197 2197 classname_matches[0] == classname:
2198 2198 cls = classes[classnames.index(classname)].__class__
2199 2199 help = cls.class_get_help()
2200 2200 # strip leading '--' from cl-args:
2201 2201 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
2202 2202 return [ attr.split('=')[0]
2203 2203 for attr in help.strip().splitlines()
2204 2204 if attr.startswith(texts[1]) ]
2205 2205 return []
2206 2206
2207 2207 @context_matcher()
2208 2208 def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2209 2209 """Match color schemes for %colors magic."""
2210 2210 # NOTE: uses `line_buffer` equivalent for compatibility
2211 2211 matches = self.magic_color_matches(context.line_with_cursor)
2212 2212 return _convert_matcher_v1_result_to_v2(matches, type="param")
2213 2213
2214 2214 def magic_color_matches(self, text: str) -> List[str]:
2215 2215 """Match color schemes for %colors magic.
2216 2216
2217 2217 .. deprecated:: 8.6
2218 2218 You can use :meth:`magic_color_matcher` instead.
2219 2219 """
2220 2220 texts = text.split()
2221 2221 if text.endswith(' '):
2222 2222 # .split() strips off the trailing whitespace. Add '' back
2223 2223 # so that: '%colors ' -> ['%colors', '']
2224 2224 texts.append('')
2225 2225
2226 2226 if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
2227 2227 prefix = texts[1]
2228 2228 return [ color for color in InspectColors.keys()
2229 2229 if color.startswith(prefix) ]
2230 2230 return []
2231 2231
2232 2232 @context_matcher(identifier="IPCompleter.jedi_matcher")
2233 2233 def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
2234 2234 matches = self._jedi_matches(
2235 2235 cursor_column=context.cursor_position,
2236 2236 cursor_line=context.cursor_line,
2237 2237 text=context.full_text,
2238 2238 )
2239 2239 return {
2240 2240 "completions": matches,
2241 2241 # static analysis should not suppress other matchers
2242 2242 "suppress": False,
2243 2243 }
2244 2244
2245 2245 def _jedi_matches(
2246 2246 self, cursor_column: int, cursor_line: int, text: str
2247 2247 ) -> Iterator[_JediCompletionLike]:
2248 2248 """
2249 2249 Return a list of :any:`jedi.api.Completion`\\s object from a ``text`` and
2250 2250 cursor position.
2251 2251
2252 2252 Parameters
2253 2253 ----------
2254 2254 cursor_column : int
2255 2255 column position of the cursor in ``text``, 0-indexed.
2256 2256 cursor_line : int
2257 2257 line position of the cursor in ``text``, 0-indexed
2258 2258 text : str
2259 2259 text to complete
2260 2260
2261 2261 Notes
2262 2262 -----
2263 2263 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
2264 2264 object containing a string with the Jedi debug information attached.
2265 2265
2266 2266 .. deprecated:: 8.6
2267 2267 You can use :meth:`_jedi_matcher` instead.
2268 2268 """
2269 2269 namespaces = [self.namespace]
2270 2270 if self.global_namespace is not None:
2271 2271 namespaces.append(self.global_namespace)
2272 2272
2273 2273 completion_filter = lambda x:x
2274 2274 offset = cursor_to_position(text, cursor_line, cursor_column)
2275 2275 # filter output if we are completing for object members
2276 2276 if offset:
2277 2277 pre = text[offset-1]
2278 2278 if pre == '.':
2279 2279 if self.omit__names == 2:
2280 2280 completion_filter = lambda c:not c.name.startswith('_')
2281 2281 elif self.omit__names == 1:
2282 2282 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
2283 2283 elif self.omit__names == 0:
2284 2284 completion_filter = lambda x:x
2285 2285 else:
2286 2286 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
2287 2287
2288 2288 interpreter = jedi.Interpreter(text[:offset], namespaces)
2289 2289 try_jedi = True
2290 2290
2291 2291 try:
2292 2292 # find the first token in the current tree -- if it is a ' or " then we are in a string
2293 2293 completing_string = False
2294 2294 try:
2295 2295 first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
2296 2296 except StopIteration:
2297 2297 pass
2298 2298 else:
2299 2299 # note the value may be ', ", or it may also be ''' or """, or
2300 2300 # in some cases, """what/you/typed..., but all of these are
2301 2301 # strings.
2302 2302 completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
2303 2303
2304 2304 # if we are in a string jedi is likely not the right candidate for
2305 2305 # now. Skip it.
2306 2306 try_jedi = not completing_string
2307 2307 except Exception as e:
2308 2308 # many of things can go wrong, we are using private API just don't crash.
2309 2309 if self.debug:
2310 2310 print("Error detecting if completing a non-finished string :", e, '|')
2311 2311
2312 2312 if not try_jedi:
2313 2313 return iter([])
2314 2314 try:
2315 2315 return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
2316 2316 except Exception as e:
2317 2317 if self.debug:
2318 2318 return iter(
2319 2319 [
2320 2320 _FakeJediCompletion(
2321 2321 'Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""'
2322 2322 % (e)
2323 2323 )
2324 2324 ]
2325 2325 )
2326 2326 else:
2327 2327 return iter([])
2328 2328
2329 2329 @context_matcher()
2330 2330 def python_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2331 2331 """Match attributes or global python names"""
2332 2332 text = context.line_with_cursor
2333 2333 if "." in text:
2334 2334 try:
2335 2335 matches, fragment = self._attr_matches(text, include_prefix=False)
2336 2336 if text.endswith(".") and self.omit__names:
2337 2337 if self.omit__names == 1:
2338 2338 # true if txt is _not_ a __ name, false otherwise:
2339 2339 no__name = lambda txt: re.match(r".*\.__.*?__", txt) is None
2340 2340 else:
2341 2341 # true if txt is _not_ a _ name, false otherwise:
2342 2342 no__name = (
2343 2343 lambda txt: re.match(r"\._.*?", txt[txt.rindex(".") :])
2344 2344 is None
2345 2345 )
2346 2346 matches = filter(no__name, matches)
2347 2347 return _convert_matcher_v1_result_to_v2(
2348 2348 matches, type="attribute", fragment=fragment
2349 2349 )
2350 2350 except NameError:
2351 2351 # catches <undefined attributes>.<tab>
2352 2352 matches = []
2353 2353 return _convert_matcher_v1_result_to_v2(matches, type="attribute")
2354 2354 else:
2355 2355 matches = self.global_matches(context.token)
2356 2356 return _convert_matcher_v1_result_to_v2(matches, type="variable")
2357 2357
2358 2358 @completion_matcher(api_version=1)
2359 2359 def python_matches(self, text: str) -> Iterable[str]:
2360 2360 """Match attributes or global python names.
2361 2361
2362 2362 .. deprecated:: 8.27
2363 2363 You can use :meth:`python_matcher` instead."""
2364 2364 if "." in text:
2365 2365 try:
2366 2366 matches = self.attr_matches(text)
2367 2367 if text.endswith('.') and self.omit__names:
2368 2368 if self.omit__names == 1:
2369 2369 # true if txt is _not_ a __ name, false otherwise:
2370 2370 no__name = (lambda txt:
2371 2371 re.match(r'.*\.__.*?__',txt) is None)
2372 2372 else:
2373 2373 # true if txt is _not_ a _ name, false otherwise:
2374 2374 no__name = (lambda txt:
2375 2375 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
2376 2376 matches = filter(no__name, matches)
2377 2377 except NameError:
2378 2378 # catches <undefined attributes>.<tab>
2379 2379 matches = []
2380 2380 else:
2381 2381 matches = self.global_matches(text)
2382 2382 return matches
2383 2383
2384 2384 def _default_arguments_from_docstring(self, doc):
2385 2385 """Parse the first line of docstring for call signature.
2386 2386
2387 2387 Docstring should be of the form 'min(iterable[, key=func])\n'.
2388 2388 It can also parse cython docstring of the form
2389 2389 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
2390 2390 """
2391 2391 if doc is None:
2392 2392 return []
2393 2393
2394 2394 #care only the firstline
2395 2395 line = doc.lstrip().splitlines()[0]
2396 2396
2397 2397 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
2398 2398 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
2399 2399 sig = self.docstring_sig_re.search(line)
2400 2400 if sig is None:
2401 2401 return []
2402 2402 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
2403 2403 sig = sig.groups()[0].split(',')
2404 2404 ret = []
2405 2405 for s in sig:
2406 2406 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
2407 2407 ret += self.docstring_kwd_re.findall(s)
2408 2408 return ret
2409 2409
2410 2410 def _default_arguments(self, obj):
2411 2411 """Return the list of default arguments of obj if it is callable,
2412 2412 or empty list otherwise."""
2413 2413 call_obj = obj
2414 2414 ret = []
2415 2415 if inspect.isbuiltin(obj):
2416 2416 pass
2417 2417 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
2418 2418 if inspect.isclass(obj):
2419 2419 #for cython embedsignature=True the constructor docstring
2420 2420 #belongs to the object itself not __init__
2421 2421 ret += self._default_arguments_from_docstring(
2422 2422 getattr(obj, '__doc__', ''))
2423 2423 # for classes, check for __init__,__new__
2424 2424 call_obj = (getattr(obj, '__init__', None) or
2425 2425 getattr(obj, '__new__', None))
2426 2426 # for all others, check if they are __call__able
2427 2427 elif hasattr(obj, '__call__'):
2428 2428 call_obj = obj.__call__
2429 2429 ret += self._default_arguments_from_docstring(
2430 2430 getattr(call_obj, '__doc__', ''))
2431 2431
2432 2432 _keeps = (inspect.Parameter.KEYWORD_ONLY,
2433 2433 inspect.Parameter.POSITIONAL_OR_KEYWORD)
2434 2434
2435 2435 try:
2436 2436 sig = inspect.signature(obj)
2437 2437 ret.extend(k for k, v in sig.parameters.items() if
2438 2438 v.kind in _keeps)
2439 2439 except ValueError:
2440 2440 pass
2441 2441
2442 2442 return list(set(ret))
2443 2443
2444 2444 @context_matcher()
2445 2445 def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2446 2446 """Match named parameters (kwargs) of the last open function."""
2447 2447 matches = self.python_func_kw_matches(context.token)
2448 2448 return _convert_matcher_v1_result_to_v2(matches, type="param")
2449 2449
2450 2450 def python_func_kw_matches(self, text):
2451 2451 """Match named parameters (kwargs) of the last open function.
2452 2452
2453 2453 .. deprecated:: 8.6
2454 2454 You can use :meth:`python_func_kw_matcher` instead.
2455 2455 """
2456 2456
2457 2457 if "." in text: # a parameter cannot be dotted
2458 2458 return []
2459 2459 try: regexp = self.__funcParamsRegex
2460 2460 except AttributeError:
2461 2461 regexp = self.__funcParamsRegex = re.compile(r'''
2462 2462 '.*?(?<!\\)' | # single quoted strings or
2463 2463 ".*?(?<!\\)" | # double quoted strings or
2464 2464 \w+ | # identifier
2465 2465 \S # other characters
2466 2466 ''', re.VERBOSE | re.DOTALL)
2467 2467 # 1. find the nearest identifier that comes before an unclosed
2468 2468 # parenthesis before the cursor
2469 2469 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
2470 2470 tokens = regexp.findall(self.text_until_cursor)
2471 2471 iterTokens = reversed(tokens); openPar = 0
2472 2472
2473 2473 for token in iterTokens:
2474 2474 if token == ')':
2475 2475 openPar -= 1
2476 2476 elif token == '(':
2477 2477 openPar += 1
2478 2478 if openPar > 0:
2479 2479 # found the last unclosed parenthesis
2480 2480 break
2481 2481 else:
2482 2482 return []
2483 2483 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
2484 2484 ids = []
2485 2485 isId = re.compile(r'\w+$').match
2486 2486
2487 2487 while True:
2488 2488 try:
2489 2489 ids.append(next(iterTokens))
2490 2490 if not isId(ids[-1]):
2491 2491 ids.pop(); break
2492 2492 if not next(iterTokens) == '.':
2493 2493 break
2494 2494 except StopIteration:
2495 2495 break
2496 2496
2497 2497 # Find all named arguments already assigned to, as to avoid suggesting
2498 2498 # them again
2499 2499 usedNamedArgs = set()
2500 2500 par_level = -1
2501 2501 for token, next_token in zip(tokens, tokens[1:]):
2502 2502 if token == '(':
2503 2503 par_level += 1
2504 2504 elif token == ')':
2505 2505 par_level -= 1
2506 2506
2507 2507 if par_level != 0:
2508 2508 continue
2509 2509
2510 2510 if next_token != '=':
2511 2511 continue
2512 2512
2513 2513 usedNamedArgs.add(token)
2514 2514
2515 2515 argMatches = []
2516 2516 try:
2517 2517 callableObj = '.'.join(ids[::-1])
2518 2518 namedArgs = self._default_arguments(eval(callableObj,
2519 2519 self.namespace))
2520 2520
2521 2521 # Remove used named arguments from the list, no need to show twice
2522 2522 for namedArg in set(namedArgs) - usedNamedArgs:
2523 2523 if namedArg.startswith(text):
2524 2524 argMatches.append("%s=" %namedArg)
2525 2525 except:
2526 2526 pass
2527 2527
2528 2528 return argMatches
2529 2529
2530 2530 @staticmethod
2531 2531 def _get_keys(obj: Any) -> List[Any]:
2532 2532 # Objects can define their own completions by defining an
2533 2533 # _ipy_key_completions_() method.
2534 2534 method = get_real_method(obj, '_ipython_key_completions_')
2535 2535 if method is not None:
2536 2536 return method()
2537 2537
2538 2538 # Special case some common in-memory dict-like types
2539 2539 if isinstance(obj, dict) or _safe_isinstance(obj, "pandas", "DataFrame"):
2540 2540 try:
2541 2541 return list(obj.keys())
2542 2542 except Exception:
2543 2543 return []
2544 2544 elif _safe_isinstance(obj, "pandas", "core", "indexing", "_LocIndexer"):
2545 2545 try:
2546 2546 return list(obj.obj.keys())
2547 2547 except Exception:
2548 2548 return []
2549 2549 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
2550 2550 _safe_isinstance(obj, 'numpy', 'void'):
2551 2551 return obj.dtype.names or []
2552 2552 return []
2553 2553
2554 2554 @context_matcher()
2555 2555 def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2556 2556 """Match string keys in a dictionary, after e.g. ``foo[``."""
2557 2557 matches = self.dict_key_matches(context.token)
2558 2558 return _convert_matcher_v1_result_to_v2(
2559 2559 matches, type="dict key", suppress_if_matches=True
2560 2560 )
2561 2561
2562 2562 def dict_key_matches(self, text: str) -> List[str]:
2563 2563 """Match string keys in a dictionary, after e.g. ``foo[``.
2564 2564
2565 2565 .. deprecated:: 8.6
2566 2566 You can use :meth:`dict_key_matcher` instead.
2567 2567 """
2568 2568
2569 2569 # Short-circuit on closed dictionary (regular expression would
2570 2570 # not match anyway, but would take quite a while).
2571 2571 if self.text_until_cursor.strip().endswith("]"):
2572 2572 return []
2573 2573
2574 2574 match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
2575 2575
2576 2576 if match is None:
2577 2577 return []
2578 2578
2579 2579 expr, prior_tuple_keys, key_prefix = match.groups()
2580 2580
2581 2581 obj = self._evaluate_expr(expr)
2582 2582
2583 2583 if obj is not_found:
2584 2584 return []
2585 2585
2586 2586 keys = self._get_keys(obj)
2587 2587 if not keys:
2588 2588 return keys
2589 2589
2590 2590 tuple_prefix = guarded_eval(
2591 2591 prior_tuple_keys,
2592 2592 EvaluationContext(
2593 2593 globals=self.global_namespace,
2594 2594 locals=self.namespace,
2595 2595 evaluation=self.evaluation, # type: ignore
2596 2596 in_subscript=True,
2597 2597 ),
2598 2598 )
2599 2599
2600 2600 closing_quote, token_offset, matches = match_dict_keys(
2601 2601 keys, key_prefix, self.splitter.delims, extra_prefix=tuple_prefix
2602 2602 )
2603 2603 if not matches:
2604 2604 return []
2605 2605
2606 2606 # get the cursor position of
2607 2607 # - the text being completed
2608 2608 # - the start of the key text
2609 2609 # - the start of the completion
2610 2610 text_start = len(self.text_until_cursor) - len(text)
2611 2611 if key_prefix:
2612 2612 key_start = match.start(3)
2613 2613 completion_start = key_start + token_offset
2614 2614 else:
2615 2615 key_start = completion_start = match.end()
2616 2616
2617 2617 # grab the leading prefix, to make sure all completions start with `text`
2618 2618 if text_start > key_start:
2619 2619 leading = ''
2620 2620 else:
2621 2621 leading = text[text_start:completion_start]
2622 2622
2623 2623 # append closing quote and bracket as appropriate
2624 2624 # this is *not* appropriate if the opening quote or bracket is outside
2625 2625 # the text given to this method, e.g. `d["""a\nt
2626 2626 can_close_quote = False
2627 2627 can_close_bracket = False
2628 2628
2629 2629 continuation = self.line_buffer[len(self.text_until_cursor) :].strip()
2630 2630
2631 2631 if continuation.startswith(closing_quote):
2632 2632 # do not close if already closed, e.g. `d['a<tab>'`
2633 2633 continuation = continuation[len(closing_quote) :]
2634 2634 else:
2635 2635 can_close_quote = True
2636 2636
2637 2637 continuation = continuation.strip()
2638 2638
2639 2639 # e.g. `pandas.DataFrame` has different tuple indexer behaviour,
2640 2640 # handling it is out of scope, so let's avoid appending suffixes.
2641 2641 has_known_tuple_handling = isinstance(obj, dict)
2642 2642
2643 2643 can_close_bracket = (
2644 2644 not continuation.startswith("]") and self.auto_close_dict_keys
2645 2645 )
2646 2646 can_close_tuple_item = (
2647 2647 not continuation.startswith(",")
2648 2648 and has_known_tuple_handling
2649 2649 and self.auto_close_dict_keys
2650 2650 )
2651 2651 can_close_quote = can_close_quote and self.auto_close_dict_keys
2652 2652
2653 2653 # fast path if closing qoute should be appended but not suffix is allowed
2654 2654 if not can_close_quote and not can_close_bracket and closing_quote:
2655 2655 return [leading + k for k in matches]
2656 2656
2657 2657 results = []
2658 2658
2659 2659 end_of_tuple_or_item = _DictKeyState.END_OF_TUPLE | _DictKeyState.END_OF_ITEM
2660 2660
2661 2661 for k, state_flag in matches.items():
2662 2662 result = leading + k
2663 2663 if can_close_quote and closing_quote:
2664 2664 result += closing_quote
2665 2665
2666 2666 if state_flag == end_of_tuple_or_item:
2667 2667 # We do not know which suffix to add,
2668 2668 # e.g. both tuple item and string
2669 2669 # match this item.
2670 2670 pass
2671 2671
2672 2672 if state_flag in end_of_tuple_or_item and can_close_bracket:
2673 2673 result += "]"
2674 2674 if state_flag == _DictKeyState.IN_TUPLE and can_close_tuple_item:
2675 2675 result += ", "
2676 2676 results.append(result)
2677 2677 return results
2678 2678
2679 2679 @context_matcher()
2680 2680 def unicode_name_matcher(self, context: CompletionContext):
2681 2681 """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2682 2682 fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2683 2683 return _convert_matcher_v1_result_to_v2(
2684 2684 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2685 2685 )
2686 2686
2687 2687 @staticmethod
2688 2688 def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
2689 2689 """Match Latex-like syntax for unicode characters base
2690 2690 on the name of the character.
2691 2691
2692 2692 This does ``\\GREEK SMALL LETTER ETA`` -> ``Ξ·``
2693 2693
2694 2694 Works only on valid python 3 identifier, or on combining characters that
2695 2695 will combine to form a valid identifier.
2696 2696 """
2697 2697 slashpos = text.rfind('\\')
2698 2698 if slashpos > -1:
2699 2699 s = text[slashpos+1:]
2700 2700 try :
2701 2701 unic = unicodedata.lookup(s)
2702 2702 # allow combining chars
2703 2703 if ('a'+unic).isidentifier():
2704 2704 return '\\'+s,[unic]
2705 2705 except KeyError:
2706 2706 pass
2707 2707 return '', []
2708 2708
2709 2709 @context_matcher()
2710 2710 def latex_name_matcher(self, context: CompletionContext):
2711 2711 """Match Latex syntax for unicode characters.
2712 2712
2713 2713 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2714 2714 """
2715 2715 fragment, matches = self.latex_matches(context.text_until_cursor)
2716 2716 return _convert_matcher_v1_result_to_v2(
2717 2717 matches, type="latex", fragment=fragment, suppress_if_matches=True
2718 2718 )
2719 2719
2720 2720 def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
2721 2721 """Match Latex syntax for unicode characters.
2722 2722
2723 2723 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2724 2724
2725 2725 .. deprecated:: 8.6
2726 2726 You can use :meth:`latex_name_matcher` instead.
2727 2727 """
2728 2728 slashpos = text.rfind('\\')
2729 2729 if slashpos > -1:
2730 2730 s = text[slashpos:]
2731 2731 if s in latex_symbols:
2732 2732 # Try to complete a full latex symbol to unicode
2733 2733 # \\alpha -> Ξ±
2734 2734 return s, [latex_symbols[s]]
2735 2735 else:
2736 2736 # If a user has partially typed a latex symbol, give them
2737 2737 # a full list of options \al -> [\aleph, \alpha]
2738 2738 matches = [k for k in latex_symbols if k.startswith(s)]
2739 2739 if matches:
2740 2740 return s, matches
2741 2741 return '', ()
2742 2742
2743 2743 @context_matcher()
2744 2744 def custom_completer_matcher(self, context):
2745 2745 """Dispatch custom completer.
2746 2746
2747 2747 If a match is found, suppresses all other matchers except for Jedi.
2748 2748 """
2749 2749 matches = self.dispatch_custom_completer(context.token) or []
2750 2750 result = _convert_matcher_v1_result_to_v2(
2751 2751 matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2752 2752 )
2753 2753 result["ordered"] = True
2754 2754 result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2755 2755 return result
2756 2756
2757 2757 def dispatch_custom_completer(self, text):
2758 2758 """
2759 2759 .. deprecated:: 8.6
2760 2760 You can use :meth:`custom_completer_matcher` instead.
2761 2761 """
2762 2762 if not self.custom_completers:
2763 2763 return
2764 2764
2765 2765 line = self.line_buffer
2766 2766 if not line.strip():
2767 2767 return None
2768 2768
2769 2769 # Create a little structure to pass all the relevant information about
2770 2770 # the current completion to any custom completer.
2771 2771 event = SimpleNamespace()
2772 2772 event.line = line
2773 2773 event.symbol = text
2774 2774 cmd = line.split(None,1)[0]
2775 2775 event.command = cmd
2776 2776 event.text_until_cursor = self.text_until_cursor
2777 2777
2778 2778 # for foo etc, try also to find completer for %foo
2779 2779 if not cmd.startswith(self.magic_escape):
2780 2780 try_magic = self.custom_completers.s_matches(
2781 2781 self.magic_escape + cmd)
2782 2782 else:
2783 2783 try_magic = []
2784 2784
2785 2785 for c in itertools.chain(self.custom_completers.s_matches(cmd),
2786 2786 try_magic,
2787 2787 self.custom_completers.flat_matches(self.text_until_cursor)):
2788 2788 try:
2789 2789 res = c(event)
2790 2790 if res:
2791 2791 # first, try case sensitive match
2792 2792 withcase = [r for r in res if r.startswith(text)]
2793 2793 if withcase:
2794 2794 return withcase
2795 2795 # if none, then case insensitive ones are ok too
2796 2796 text_low = text.lower()
2797 2797 return [r for r in res if r.lower().startswith(text_low)]
2798 2798 except TryNext:
2799 2799 pass
2800 2800 except KeyboardInterrupt:
2801 2801 """
2802 2802 If custom completer take too long,
2803 2803 let keyboard interrupt abort and return nothing.
2804 2804 """
2805 2805 break
2806 2806
2807 2807 return None
2808 2808
2809 2809 def completions(self, text: str, offset: int)->Iterator[Completion]:
2810 2810 """
2811 2811 Returns an iterator over the possible completions
2812 2812
2813 2813 .. warning::
2814 2814
2815 2815 Unstable
2816 2816
2817 2817 This function is unstable, API may change without warning.
2818 2818 It will also raise unless use in proper context manager.
2819 2819
2820 2820 Parameters
2821 2821 ----------
2822 2822 text : str
2823 2823 Full text of the current input, multi line string.
2824 2824 offset : int
2825 2825 Integer representing the position of the cursor in ``text``. Offset
2826 2826 is 0-based indexed.
2827 2827
2828 2828 Yields
2829 2829 ------
2830 2830 Completion
2831 2831
2832 2832 Notes
2833 2833 -----
2834 2834 The cursor on a text can either be seen as being "in between"
2835 2835 characters or "On" a character depending on the interface visible to
2836 2836 the user. For consistency the cursor being on "in between" characters X
2837 2837 and Y is equivalent to the cursor being "on" character Y, that is to say
2838 2838 the character the cursor is on is considered as being after the cursor.
2839 2839
2840 2840 Combining characters may span more that one position in the
2841 2841 text.
2842 2842
2843 2843 .. note::
2844 2844
2845 2845 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
2846 2846 fake Completion token to distinguish completion returned by Jedi
2847 2847 and usual IPython completion.
2848 2848
2849 2849 .. note::
2850 2850
2851 2851 Completions are not completely deduplicated yet. If identical
2852 2852 completions are coming from different sources this function does not
2853 2853 ensure that each completion object will only be present once.
2854 2854 """
2855 2855 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
2856 2856 "It may change without warnings. "
2857 2857 "Use in corresponding context manager.",
2858 2858 category=ProvisionalCompleterWarning, stacklevel=2)
2859 2859
2860 2860 seen = set()
2861 2861 profiler:Optional[cProfile.Profile]
2862 2862 try:
2863 2863 if self.profile_completions:
2864 2864 import cProfile
2865 2865 profiler = cProfile.Profile()
2866 2866 profiler.enable()
2867 2867 else:
2868 2868 profiler = None
2869 2869
2870 2870 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
2871 2871 if c and (c in seen):
2872 2872 continue
2873 2873 yield c
2874 2874 seen.add(c)
2875 2875 except KeyboardInterrupt:
2876 2876 """if completions take too long and users send keyboard interrupt,
2877 2877 do not crash and return ASAP. """
2878 2878 pass
2879 2879 finally:
2880 2880 if profiler is not None:
2881 2881 profiler.disable()
2882 2882 ensure_dir_exists(self.profiler_output_dir)
2883 2883 output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
2884 2884 print("Writing profiler output to", output_path)
2885 2885 profiler.dump_stats(output_path)
2886 2886
2887 2887 def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
2888 2888 """
2889 2889 Core completion module.Same signature as :any:`completions`, with the
2890 2890 extra `timeout` parameter (in seconds).
2891 2891
2892 2892 Computing jedi's completion ``.type`` can be quite expensive (it is a
2893 2893 lazy property) and can require some warm-up, more warm up than just
2894 2894 computing the ``name`` of a completion. The warm-up can be :
2895 2895
2896 2896 - Long warm-up the first time a module is encountered after
2897 2897 install/update: actually build parse/inference tree.
2898 2898
2899 2899 - first time the module is encountered in a session: load tree from
2900 2900 disk.
2901 2901
2902 2902 We don't want to block completions for tens of seconds so we give the
2903 2903 completer a "budget" of ``_timeout`` seconds per invocation to compute
2904 2904 completions types, the completions that have not yet been computed will
2905 2905 be marked as "unknown" an will have a chance to be computed next round
2906 2906 are things get cached.
2907 2907
2908 2908 Keep in mind that Jedi is not the only thing treating the completion so
2909 2909 keep the timeout short-ish as if we take more than 0.3 second we still
2910 2910 have lots of processing to do.
2911 2911
2912 2912 """
2913 2913 deadline = time.monotonic() + _timeout
2914 2914
2915 2915 before = full_text[:offset]
2916 2916 cursor_line, cursor_column = position_to_cursor(full_text, offset)
2917 2917
2918 2918 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2919 2919
2920 2920 def is_non_jedi_result(
2921 2921 result: MatcherResult, identifier: str
2922 2922 ) -> TypeGuard[SimpleMatcherResult]:
2923 2923 return identifier != jedi_matcher_id
2924 2924
2925 2925 results = self._complete(
2926 2926 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
2927 2927 )
2928 2928
2929 2929 non_jedi_results: Dict[str, SimpleMatcherResult] = {
2930 2930 identifier: result
2931 2931 for identifier, result in results.items()
2932 2932 if is_non_jedi_result(result, identifier)
2933 2933 }
2934 2934
2935 2935 jedi_matches = (
2936 2936 cast(_JediMatcherResult, results[jedi_matcher_id])["completions"]
2937 2937 if jedi_matcher_id in results
2938 2938 else ()
2939 2939 )
2940 2940
2941 2941 iter_jm = iter(jedi_matches)
2942 2942 if _timeout:
2943 2943 for jm in iter_jm:
2944 2944 try:
2945 2945 type_ = jm.type
2946 2946 except Exception:
2947 2947 if self.debug:
2948 2948 print("Error in Jedi getting type of ", jm)
2949 2949 type_ = None
2950 2950 delta = len(jm.name_with_symbols) - len(jm.complete)
2951 2951 if type_ == 'function':
2952 2952 signature = _make_signature(jm)
2953 2953 else:
2954 2954 signature = ''
2955 2955 yield Completion(start=offset - delta,
2956 2956 end=offset,
2957 2957 text=jm.name_with_symbols,
2958 2958 type=type_,
2959 2959 signature=signature,
2960 2960 _origin='jedi')
2961 2961
2962 2962 if time.monotonic() > deadline:
2963 2963 break
2964 2964
2965 2965 for jm in iter_jm:
2966 2966 delta = len(jm.name_with_symbols) - len(jm.complete)
2967 2967 yield Completion(
2968 2968 start=offset - delta,
2969 2969 end=offset,
2970 2970 text=jm.name_with_symbols,
2971 2971 type=_UNKNOWN_TYPE, # don't compute type for speed
2972 2972 _origin="jedi",
2973 2973 signature="",
2974 2974 )
2975 2975
2976 2976 # TODO:
2977 2977 # Suppress this, right now just for debug.
2978 2978 if jedi_matches and non_jedi_results and self.debug:
2979 2979 some_start_offset = before.rfind(
2980 2980 next(iter(non_jedi_results.values()))["matched_fragment"]
2981 2981 )
2982 2982 yield Completion(
2983 2983 start=some_start_offset,
2984 2984 end=offset,
2985 2985 text="--jedi/ipython--",
2986 2986 _origin="debug",
2987 2987 type="none",
2988 2988 signature="",
2989 2989 )
2990 2990
2991 2991 ordered: List[Completion] = []
2992 2992 sortable: List[Completion] = []
2993 2993
2994 2994 for origin, result in non_jedi_results.items():
2995 2995 matched_text = result["matched_fragment"]
2996 2996 start_offset = before.rfind(matched_text)
2997 2997 is_ordered = result.get("ordered", False)
2998 2998 container = ordered if is_ordered else sortable
2999 2999
3000 3000 # I'm unsure if this is always true, so let's assert and see if it
3001 3001 # crash
3002 3002 assert before.endswith(matched_text)
3003 3003
3004 3004 for simple_completion in result["completions"]:
3005 3005 completion = Completion(
3006 3006 start=start_offset,
3007 3007 end=offset,
3008 3008 text=simple_completion.text,
3009 3009 _origin=origin,
3010 3010 signature="",
3011 3011 type=simple_completion.type or _UNKNOWN_TYPE,
3012 3012 )
3013 3013 container.append(completion)
3014 3014
3015 3015 yield from list(self._deduplicate(ordered + self._sort(sortable)))[
3016 3016 :MATCHES_LIMIT
3017 3017 ]
3018 3018
3019 3019 def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
3020 3020 """Find completions for the given text and line context.
3021 3021
3022 3022 Note that both the text and the line_buffer are optional, but at least
3023 3023 one of them must be given.
3024 3024
3025 3025 Parameters
3026 3026 ----------
3027 3027 text : string, optional
3028 3028 Text to perform the completion on. If not given, the line buffer
3029 3029 is split using the instance's CompletionSplitter object.
3030 3030 line_buffer : string, optional
3031 3031 If not given, the completer attempts to obtain the current line
3032 3032 buffer via readline. This keyword allows clients which are
3033 3033 requesting for text completions in non-readline contexts to inform
3034 3034 the completer of the entire text.
3035 3035 cursor_pos : int, optional
3036 3036 Index of the cursor in the full line buffer. Should be provided by
3037 3037 remote frontends where kernel has no access to frontend state.
3038 3038
3039 3039 Returns
3040 3040 -------
3041 3041 Tuple of two items:
3042 3042 text : str
3043 3043 Text that was actually used in the completion.
3044 3044 matches : list
3045 3045 A list of completion matches.
3046 3046
3047 3047 Notes
3048 3048 -----
3049 3049 This API is likely to be deprecated and replaced by
3050 3050 :any:`IPCompleter.completions` in the future.
3051 3051
3052 3052 """
3053 3053 warnings.warn('`Completer.complete` is pending deprecation since '
3054 3054 'IPython 6.0 and will be replaced by `Completer.completions`.',
3055 3055 PendingDeprecationWarning)
3056 3056 # potential todo, FOLD the 3rd throw away argument of _complete
3057 3057 # into the first 2 one.
3058 3058 # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
3059 3059 # TODO: should we deprecate now, or does it stay?
3060 3060
3061 3061 results = self._complete(
3062 3062 line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
3063 3063 )
3064 3064
3065 3065 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3066 3066
3067 3067 return self._arrange_and_extract(
3068 3068 results,
3069 3069 # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
3070 3070 skip_matchers={jedi_matcher_id},
3071 3071 # this API does not support different start/end positions (fragments of token).
3072 3072 abort_if_offset_changes=True,
3073 3073 )
3074 3074
3075 3075 def _arrange_and_extract(
3076 3076 self,
3077 3077 results: Dict[str, MatcherResult],
3078 3078 skip_matchers: Set[str],
3079 3079 abort_if_offset_changes: bool,
3080 3080 ):
3081 3081 sortable: List[AnyMatcherCompletion] = []
3082 3082 ordered: List[AnyMatcherCompletion] = []
3083 3083 most_recent_fragment = None
3084 3084 for identifier, result in results.items():
3085 3085 if identifier in skip_matchers:
3086 3086 continue
3087 3087 if not result["completions"]:
3088 3088 continue
3089 3089 if not most_recent_fragment:
3090 3090 most_recent_fragment = result["matched_fragment"]
3091 3091 if (
3092 3092 abort_if_offset_changes
3093 3093 and result["matched_fragment"] != most_recent_fragment
3094 3094 ):
3095 3095 break
3096 3096 if result.get("ordered", False):
3097 3097 ordered.extend(result["completions"])
3098 3098 else:
3099 3099 sortable.extend(result["completions"])
3100 3100
3101 3101 if not most_recent_fragment:
3102 3102 most_recent_fragment = "" # to satisfy typechecker (and just in case)
3103 3103
3104 3104 return most_recent_fragment, [
3105 3105 m.text for m in self._deduplicate(ordered + self._sort(sortable))
3106 3106 ]
3107 3107
3108 3108 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
3109 3109 full_text=None) -> _CompleteResult:
3110 3110 """
3111 3111 Like complete but can also returns raw jedi completions as well as the
3112 3112 origin of the completion text. This could (and should) be made much
3113 3113 cleaner but that will be simpler once we drop the old (and stateful)
3114 3114 :any:`complete` API.
3115 3115
3116 3116 With current provisional API, cursor_pos act both (depending on the
3117 3117 caller) as the offset in the ``text`` or ``line_buffer``, or as the
3118 3118 ``column`` when passing multiline strings this could/should be renamed
3119 3119 but would add extra noise.
3120 3120
3121 3121 Parameters
3122 3122 ----------
3123 3123 cursor_line
3124 3124 Index of the line the cursor is on. 0 indexed.
3125 3125 cursor_pos
3126 3126 Position of the cursor in the current line/line_buffer/text. 0
3127 3127 indexed.
3128 3128 line_buffer : optional, str
3129 3129 The current line the cursor is in, this is mostly due to legacy
3130 3130 reason that readline could only give a us the single current line.
3131 3131 Prefer `full_text`.
3132 3132 text : str
3133 3133 The current "token" the cursor is in, mostly also for historical
3134 3134 reasons. as the completer would trigger only after the current line
3135 3135 was parsed.
3136 3136 full_text : str
3137 3137 Full text of the current cell.
3138 3138
3139 3139 Returns
3140 3140 -------
3141 3141 An ordered dictionary where keys are identifiers of completion
3142 3142 matchers and values are ``MatcherResult``s.
3143 3143 """
3144 3144
3145 3145 # if the cursor position isn't given, the only sane assumption we can
3146 3146 # make is that it's at the end of the line (the common case)
3147 3147 if cursor_pos is None:
3148 3148 cursor_pos = len(line_buffer) if text is None else len(text)
3149 3149
3150 3150 if self.use_main_ns:
3151 3151 self.namespace = __main__.__dict__
3152 3152
3153 3153 # if text is either None or an empty string, rely on the line buffer
3154 3154 if (not line_buffer) and full_text:
3155 3155 line_buffer = full_text.split('\n')[cursor_line]
3156 3156 if not text: # issue #11508: check line_buffer before calling split_line
3157 3157 text = (
3158 3158 self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
3159 3159 )
3160 3160
3161 3161 # If no line buffer is given, assume the input text is all there was
3162 3162 if line_buffer is None:
3163 3163 line_buffer = text
3164 3164
3165 3165 # deprecated - do not use `line_buffer` in new code.
3166 3166 self.line_buffer = line_buffer
3167 3167 self.text_until_cursor = self.line_buffer[:cursor_pos]
3168 3168
3169 3169 if not full_text:
3170 3170 full_text = line_buffer
3171 3171
3172 3172 context = CompletionContext(
3173 3173 full_text=full_text,
3174 3174 cursor_position=cursor_pos,
3175 3175 cursor_line=cursor_line,
3176 3176 token=text,
3177 3177 limit=MATCHES_LIMIT,
3178 3178 )
3179 3179
3180 3180 # Start with a clean slate of completions
3181 3181 results: Dict[str, MatcherResult] = {}
3182 3182
3183 3183 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3184 3184
3185 3185 suppressed_matchers: Set[str] = set()
3186 3186
3187 3187 matchers = {
3188 3188 _get_matcher_id(matcher): matcher
3189 3189 for matcher in sorted(
3190 3190 self.matchers, key=_get_matcher_priority, reverse=True
3191 3191 )
3192 3192 }
3193 3193
3194 3194 for matcher_id, matcher in matchers.items():
3195 3195 matcher_id = _get_matcher_id(matcher)
3196 3196
3197 3197 if matcher_id in self.disable_matchers:
3198 3198 continue
3199 3199
3200 3200 if matcher_id in results:
3201 3201 warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
3202 3202
3203 3203 if matcher_id in suppressed_matchers:
3204 3204 continue
3205 3205
3206 3206 result: MatcherResult
3207 3207 try:
3208 3208 if _is_matcher_v1(matcher):
3209 3209 result = _convert_matcher_v1_result_to_v2(
3210 3210 matcher(text), type=_UNKNOWN_TYPE
3211 3211 )
3212 3212 elif _is_matcher_v2(matcher):
3213 3213 result = matcher(context)
3214 3214 else:
3215 3215 api_version = _get_matcher_api_version(matcher)
3216 3216 raise ValueError(f"Unsupported API version {api_version}")
3217 3217 except:
3218 3218 # Show the ugly traceback if the matcher causes an
3219 3219 # exception, but do NOT crash the kernel!
3220 3220 sys.excepthook(*sys.exc_info())
3221 3221 continue
3222 3222
3223 3223 # set default value for matched fragment if suffix was not selected.
3224 3224 result["matched_fragment"] = result.get("matched_fragment", context.token)
3225 3225
3226 3226 if not suppressed_matchers:
3227 3227 suppression_recommended: Union[bool, Set[str]] = result.get(
3228 3228 "suppress", False
3229 3229 )
3230 3230
3231 3231 suppression_config = (
3232 3232 self.suppress_competing_matchers.get(matcher_id, None)
3233 3233 if isinstance(self.suppress_competing_matchers, dict)
3234 3234 else self.suppress_competing_matchers
3235 3235 )
3236 3236 should_suppress = (
3237 3237 (suppression_config is True)
3238 3238 or (suppression_recommended and (suppression_config is not False))
3239 3239 ) and has_any_completions(result)
3240 3240
3241 3241 if should_suppress:
3242 3242 suppression_exceptions: Set[str] = result.get(
3243 3243 "do_not_suppress", set()
3244 3244 )
3245 3245 if isinstance(suppression_recommended, Iterable):
3246 3246 to_suppress = set(suppression_recommended)
3247 3247 else:
3248 3248 to_suppress = set(matchers)
3249 3249 suppressed_matchers = to_suppress - suppression_exceptions
3250 3250
3251 3251 new_results = {}
3252 3252 for previous_matcher_id, previous_result in results.items():
3253 3253 if previous_matcher_id not in suppressed_matchers:
3254 3254 new_results[previous_matcher_id] = previous_result
3255 3255 results = new_results
3256 3256
3257 3257 results[matcher_id] = result
3258 3258
3259 3259 _, matches = self._arrange_and_extract(
3260 3260 results,
3261 3261 # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
3262 3262 # if it was omission, we can remove the filtering step, otherwise remove this comment.
3263 3263 skip_matchers={jedi_matcher_id},
3264 3264 abort_if_offset_changes=False,
3265 3265 )
3266 3266
3267 3267 # populate legacy stateful API
3268 3268 self.matches = matches
3269 3269
3270 3270 return results
3271 3271
3272 3272 @staticmethod
3273 3273 def _deduplicate(
3274 3274 matches: Sequence[AnyCompletion],
3275 3275 ) -> Iterable[AnyCompletion]:
3276 3276 filtered_matches: Dict[str, AnyCompletion] = {}
3277 3277 for match in matches:
3278 3278 text = match.text
3279 3279 if (
3280 3280 text not in filtered_matches
3281 3281 or filtered_matches[text].type == _UNKNOWN_TYPE
3282 3282 ):
3283 3283 filtered_matches[text] = match
3284 3284
3285 3285 return filtered_matches.values()
3286 3286
3287 3287 @staticmethod
3288 3288 def _sort(matches: Sequence[AnyCompletion]):
3289 3289 return sorted(matches, key=lambda x: completions_sorting_key(x.text))
3290 3290
3291 3291 @context_matcher()
3292 3292 def fwd_unicode_matcher(self, context: CompletionContext):
3293 3293 """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
3294 3294 # TODO: use `context.limit` to terminate early once we matched the maximum
3295 3295 # number that will be used downstream; can be added as an optional to
3296 3296 # `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
3297 3297 fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
3298 3298 return _convert_matcher_v1_result_to_v2(
3299 3299 matches, type="unicode", fragment=fragment, suppress_if_matches=True
3300 3300 )
3301 3301
3302 3302 def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
3303 3303 """
3304 3304 Forward match a string starting with a backslash with a list of
3305 3305 potential Unicode completions.
3306 3306
3307 3307 Will compute list of Unicode character names on first call and cache it.
3308 3308
3309 3309 .. deprecated:: 8.6
3310 3310 You can use :meth:`fwd_unicode_matcher` instead.
3311 3311
3312 3312 Returns
3313 3313 -------
3314 3314 At tuple with:
3315 3315 - matched text (empty if no matches)
3316 3316 - list of potential completions, empty tuple otherwise)
3317 3317 """
3318 3318 # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
3319 3319 # We could do a faster match using a Trie.
3320 3320
3321 3321 # Using pygtrie the following seem to work:
3322 3322
3323 3323 # s = PrefixSet()
3324 3324
3325 3325 # for c in range(0,0x10FFFF + 1):
3326 3326 # try:
3327 3327 # s.add(unicodedata.name(chr(c)))
3328 3328 # except ValueError:
3329 3329 # pass
3330 3330 # [''.join(k) for k in s.iter(prefix)]
3331 3331
3332 3332 # But need to be timed and adds an extra dependency.
3333 3333
3334 3334 slashpos = text.rfind('\\')
3335 3335 # if text starts with slash
3336 3336 if slashpos > -1:
3337 3337 # PERF: It's important that we don't access self._unicode_names
3338 3338 # until we're inside this if-block. _unicode_names is lazily
3339 3339 # initialized, and it takes a user-noticeable amount of time to
3340 3340 # initialize it, so we don't want to initialize it unless we're
3341 3341 # actually going to use it.
3342 3342 s = text[slashpos + 1 :]
3343 3343 sup = s.upper()
3344 3344 candidates = [x for x in self.unicode_names if x.startswith(sup)]
3345 3345 if candidates:
3346 3346 return s, candidates
3347 3347 candidates = [x for x in self.unicode_names if sup in x]
3348 3348 if candidates:
3349 3349 return s, candidates
3350 3350 splitsup = sup.split(" ")
3351 3351 candidates = [
3352 3352 x for x in self.unicode_names if all(u in x for u in splitsup)
3353 3353 ]
3354 3354 if candidates:
3355 3355 return s, candidates
3356 3356
3357 3357 return "", ()
3358 3358
3359 3359 # if text does not start with slash
3360 3360 else:
3361 3361 return '', ()
3362 3362
3363 3363 @property
3364 3364 def unicode_names(self) -> List[str]:
3365 3365 """List of names of unicode code points that can be completed.
3366 3366
3367 3367 The list is lazily initialized on first access.
3368 3368 """
3369 3369 if self._unicode_names is None:
3370 3370 names = []
3371 3371 for c in range(0,0x10FFFF + 1):
3372 3372 try:
3373 3373 names.append(unicodedata.name(chr(c)))
3374 3374 except ValueError:
3375 3375 pass
3376 3376 self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
3377 3377
3378 3378 return self._unicode_names
3379 3379
3380 3380 def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
3381 3381 names = []
3382 3382 for start,stop in ranges:
3383 3383 for c in range(start, stop) :
3384 3384 try:
3385 3385 names.append(unicodedata.name(chr(c)))
3386 3386 except ValueError:
3387 3387 pass
3388 3388 return names
General Comments 0
You need to be logged in to leave comments. Login now