##// END OF EJS Templates
Strip prefix in `attr_matches` result
krassowski -
Show More
@@ -1,3322 +1,3347 b''
1 1 """Completion for IPython.
2 2
3 3 This module started as fork of the rlcompleter module in the Python standard
4 4 library. The original enhancements made to rlcompleter have been sent
5 5 upstream and were accepted as of Python 2.3,
6 6
7 7 This module now support a wide variety of completion mechanism both available
8 8 for normal classic Python code, as well as completer for IPython specific
9 9 Syntax like magics.
10 10
11 11 Latex and Unicode completion
12 12 ============================
13 13
14 14 IPython and compatible frontends not only can complete your code, but can help
15 15 you to input a wide range of characters. In particular we allow you to insert
16 16 a unicode character using the tab completion mechanism.
17 17
18 18 Forward latex/unicode completion
19 19 --------------------------------
20 20
21 21 Forward completion allows you to easily type a unicode character using its latex
22 22 name, or unicode long description. To do so type a backslash follow by the
23 23 relevant name and press tab:
24 24
25 25
26 26 Using latex completion:
27 27
28 28 .. code::
29 29
30 30 \\alpha<tab>
31 31 Ξ±
32 32
33 33 or using unicode completion:
34 34
35 35
36 36 .. code::
37 37
38 38 \\GREEK SMALL LETTER ALPHA<tab>
39 39 Ξ±
40 40
41 41
42 42 Only valid Python identifiers will complete. Combining characters (like arrow or
43 43 dots) are also available, unlike latex they need to be put after the their
44 44 counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
45 45
46 46 Some browsers are known to display combining characters incorrectly.
47 47
48 48 Backward latex completion
49 49 -------------------------
50 50
51 51 It is sometime challenging to know how to type a character, if you are using
52 52 IPython, or any compatible frontend you can prepend backslash to the character
53 53 and press :kbd:`Tab` to expand it to its latex form.
54 54
55 55 .. code::
56 56
57 57 \\Ξ±<tab>
58 58 \\alpha
59 59
60 60
61 61 Both forward and backward completions can be deactivated by setting the
62 62 :std:configtrait:`Completer.backslash_combining_completions` option to
63 63 ``False``.
64 64
65 65
66 66 Experimental
67 67 ============
68 68
69 69 Starting with IPython 6.0, this module can make use of the Jedi library to
70 70 generate completions both using static analysis of the code, and dynamically
71 71 inspecting multiple namespaces. Jedi is an autocompletion and static analysis
72 72 for Python. The APIs attached to this new mechanism is unstable and will
73 73 raise unless use in an :any:`provisionalcompleter` context manager.
74 74
75 75 You will find that the following are experimental:
76 76
77 77 - :any:`provisionalcompleter`
78 78 - :any:`IPCompleter.completions`
79 79 - :any:`Completion`
80 80 - :any:`rectify_completions`
81 81
82 82 .. note::
83 83
84 84 better name for :any:`rectify_completions` ?
85 85
86 86 We welcome any feedback on these new API, and we also encourage you to try this
87 87 module in debug mode (start IPython with ``--Completer.debug=True``) in order
88 88 to have extra logging information if :any:`jedi` is crashing, or if current
89 89 IPython completer pending deprecations are returning results not yet handled
90 90 by :any:`jedi`
91 91
92 92 Using Jedi for tab completion allow snippets like the following to work without
93 93 having to execute any code:
94 94
95 95 >>> myvar = ['hello', 42]
96 96 ... myvar[1].bi<tab>
97 97
98 98 Tab completion will be able to infer that ``myvar[1]`` is a real number without
99 99 executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
100 100 option.
101 101
102 102 Be sure to update :any:`jedi` to the latest stable version or to try the
103 103 current development version to get better completions.
104 104
105 105 Matchers
106 106 ========
107 107
108 108 All completions routines are implemented using unified *Matchers* API.
109 109 The matchers API is provisional and subject to change without notice.
110 110
111 111 The built-in matchers include:
112 112
113 113 - :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
114 114 - :any:`IPCompleter.magic_matcher`: completions for magics,
115 115 - :any:`IPCompleter.unicode_name_matcher`,
116 116 :any:`IPCompleter.fwd_unicode_matcher`
117 117 and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
118 118 - :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
119 119 - :any:`IPCompleter.file_matcher`: paths to files and directories,
120 120 - :any:`IPCompleter.python_func_kw_matcher` - function keywords,
121 121 - :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
122 122 - ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
123 123 - :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
124 124 implementation in :any:`InteractiveShell` which uses IPython hooks system
125 125 (`complete_command`) with string dispatch (including regular expressions).
126 126 Differently to other matchers, ``custom_completer_matcher`` will not suppress
127 127 Jedi results to match behaviour in earlier IPython versions.
128 128
129 129 Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
130 130
131 131 Matcher API
132 132 -----------
133 133
134 134 Simplifying some details, the ``Matcher`` interface can described as
135 135
136 136 .. code-block::
137 137
138 138 MatcherAPIv1 = Callable[[str], list[str]]
139 139 MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
140 140
141 141 Matcher = MatcherAPIv1 | MatcherAPIv2
142 142
143 143 The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
144 144 and remains supported as a simplest way for generating completions. This is also
145 145 currently the only API supported by the IPython hooks system `complete_command`.
146 146
147 147 To distinguish between matcher versions ``matcher_api_version`` attribute is used.
148 148 More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
149 149 and requires a literal ``2`` for v2 Matchers.
150 150
151 151 Once the API stabilises future versions may relax the requirement for specifying
152 152 ``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
153 153 please do not rely on the presence of ``matcher_api_version`` for any purposes.
154 154
155 155 Suppression of competing matchers
156 156 ---------------------------------
157 157
158 158 By default results from all matchers are combined, in the order determined by
159 159 their priority. Matchers can request to suppress results from subsequent
160 160 matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
161 161
162 162 When multiple matchers simultaneously request surpression, the results from of
163 163 the matcher with higher priority will be returned.
164 164
165 165 Sometimes it is desirable to suppress most but not all other matchers;
166 166 this can be achieved by adding a list of identifiers of matchers which
167 167 should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
168 168
169 169 The suppression behaviour can is user-configurable via
170 170 :std:configtrait:`IPCompleter.suppress_competing_matchers`.
171 171 """
172 172
173 173
174 174 # Copyright (c) IPython Development Team.
175 175 # Distributed under the terms of the Modified BSD License.
176 176 #
177 177 # Some of this code originated from rlcompleter in the Python standard library
178 178 # Copyright (C) 2001 Python Software Foundation, www.python.org
179 179
180 180 from __future__ import annotations
181 181 import builtins as builtin_mod
182 182 import enum
183 183 import glob
184 184 import inspect
185 185 import itertools
186 186 import keyword
187 187 import os
188 188 import re
189 189 import string
190 190 import sys
191 191 import tokenize
192 192 import time
193 193 import unicodedata
194 194 import uuid
195 195 import warnings
196 196 from ast import literal_eval
197 197 from collections import defaultdict
198 198 from contextlib import contextmanager
199 199 from dataclasses import dataclass
200 200 from functools import cached_property, partial
201 201 from types import SimpleNamespace
202 202 from typing import (
203 203 Iterable,
204 204 Iterator,
205 205 List,
206 206 Tuple,
207 207 Union,
208 208 Any,
209 209 Sequence,
210 210 Dict,
211 211 Optional,
212 212 TYPE_CHECKING,
213 213 Set,
214 214 Sized,
215 215 TypeVar,
216 216 Literal,
217 217 )
218 218
219 219 from IPython.core.guarded_eval import guarded_eval, EvaluationContext
220 220 from IPython.core.error import TryNext
221 221 from IPython.core.inputtransformer2 import ESC_MAGIC
222 222 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
223 223 from IPython.core.oinspect import InspectColors
224 224 from IPython.testing.skipdoctest import skip_doctest
225 225 from IPython.utils import generics
226 226 from IPython.utils.decorators import sphinx_options
227 227 from IPython.utils.dir2 import dir2, get_real_method
228 228 from IPython.utils.docs import GENERATING_DOCUMENTATION
229 229 from IPython.utils.path import ensure_dir_exists
230 230 from IPython.utils.process import arg_split
231 231 from traitlets import (
232 232 Bool,
233 233 Enum,
234 234 Int,
235 235 List as ListTrait,
236 236 Unicode,
237 237 Dict as DictTrait,
238 238 Union as UnionTrait,
239 239 observe,
240 240 )
241 241 from traitlets.config.configurable import Configurable
242 242
243 243 import __main__
244 244
245 245 # skip module docstests
246 246 __skip_doctest__ = True
247 247
248 248
249 249 try:
250 250 import jedi
251 251 jedi.settings.case_insensitive_completion = False
252 252 import jedi.api.helpers
253 253 import jedi.api.classes
254 254 JEDI_INSTALLED = True
255 255 except ImportError:
256 256 JEDI_INSTALLED = False
257 257
258 258
259 259 if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
260 260 from typing import cast
261 261 from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
262 262 else:
263 263 from typing import Generic
264 264
265 265 def cast(type_, obj):
266 266 """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
267 267 return obj
268 268
269 269 # do not require on runtime
270 270 NotRequired = Tuple # requires Python >=3.11
271 271 TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
272 272 Protocol = object # requires Python >=3.8
273 273 TypeAlias = Any # requires Python >=3.10
274 274 TypeGuard = Generic # requires Python >=3.10
275 275 if GENERATING_DOCUMENTATION:
276 276 from typing import TypedDict
277 277
278 278 # -----------------------------------------------------------------------------
279 279 # Globals
280 280 #-----------------------------------------------------------------------------
281 281
282 282 # ranges where we have most of the valid unicode names. We could be more finer
283 283 # grained but is it worth it for performance While unicode have character in the
284 284 # range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
285 285 # write this). With below range we cover them all, with a density of ~67%
286 286 # biggest next gap we consider only adds up about 1% density and there are 600
287 287 # gaps that would need hard coding.
288 288 _UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
289 289
290 290 # Public API
291 291 __all__ = ["Completer", "IPCompleter"]
292 292
293 293 if sys.platform == 'win32':
294 294 PROTECTABLES = ' '
295 295 else:
296 296 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
297 297
298 298 # Protect against returning an enormous number of completions which the frontend
299 299 # may have trouble processing.
300 300 MATCHES_LIMIT = 500
301 301
302 302 # Completion type reported when no type can be inferred.
303 303 _UNKNOWN_TYPE = "<unknown>"
304 304
305 305 # sentinel value to signal lack of a match
306 306 not_found = object()
307 307
308 308 class ProvisionalCompleterWarning(FutureWarning):
309 309 """
310 310 Exception raise by an experimental feature in this module.
311 311
312 312 Wrap code in :any:`provisionalcompleter` context manager if you
313 313 are certain you want to use an unstable feature.
314 314 """
315 315 pass
316 316
317 317 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
318 318
319 319
320 320 @skip_doctest
321 321 @contextmanager
322 322 def provisionalcompleter(action='ignore'):
323 323 """
324 324 This context manager has to be used in any place where unstable completer
325 325 behavior and API may be called.
326 326
327 327 >>> with provisionalcompleter():
328 328 ... completer.do_experimental_things() # works
329 329
330 330 >>> completer.do_experimental_things() # raises.
331 331
332 332 .. note::
333 333
334 334 Unstable
335 335
336 336 By using this context manager you agree that the API in use may change
337 337 without warning, and that you won't complain if they do so.
338 338
339 339 You also understand that, if the API is not to your liking, you should report
340 340 a bug to explain your use case upstream.
341 341
342 342 We'll be happy to get your feedback, feature requests, and improvements on
343 343 any of the unstable APIs!
344 344 """
345 345 with warnings.catch_warnings():
346 346 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
347 347 yield
348 348
349 349
350 350 def has_open_quotes(s):
351 351 """Return whether a string has open quotes.
352 352
353 353 This simply counts whether the number of quote characters of either type in
354 354 the string is odd.
355 355
356 356 Returns
357 357 -------
358 358 If there is an open quote, the quote character is returned. Else, return
359 359 False.
360 360 """
361 361 # We check " first, then ', so complex cases with nested quotes will get
362 362 # the " to take precedence.
363 363 if s.count('"') % 2:
364 364 return '"'
365 365 elif s.count("'") % 2:
366 366 return "'"
367 367 else:
368 368 return False
369 369
370 370
371 371 def protect_filename(s, protectables=PROTECTABLES):
372 372 """Escape a string to protect certain characters."""
373 373 if set(s) & set(protectables):
374 374 if sys.platform == "win32":
375 375 return '"' + s + '"'
376 376 else:
377 377 return "".join(("\\" + c if c in protectables else c) for c in s)
378 378 else:
379 379 return s
380 380
381 381
382 382 def expand_user(path:str) -> Tuple[str, bool, str]:
383 383 """Expand ``~``-style usernames in strings.
384 384
385 385 This is similar to :func:`os.path.expanduser`, but it computes and returns
386 386 extra information that will be useful if the input was being used in
387 387 computing completions, and you wish to return the completions with the
388 388 original '~' instead of its expanded value.
389 389
390 390 Parameters
391 391 ----------
392 392 path : str
393 393 String to be expanded. If no ~ is present, the output is the same as the
394 394 input.
395 395
396 396 Returns
397 397 -------
398 398 newpath : str
399 399 Result of ~ expansion in the input path.
400 400 tilde_expand : bool
401 401 Whether any expansion was performed or not.
402 402 tilde_val : str
403 403 The value that ~ was replaced with.
404 404 """
405 405 # Default values
406 406 tilde_expand = False
407 407 tilde_val = ''
408 408 newpath = path
409 409
410 410 if path.startswith('~'):
411 411 tilde_expand = True
412 412 rest = len(path)-1
413 413 newpath = os.path.expanduser(path)
414 414 if rest:
415 415 tilde_val = newpath[:-rest]
416 416 else:
417 417 tilde_val = newpath
418 418
419 419 return newpath, tilde_expand, tilde_val
420 420
421 421
422 422 def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
423 423 """Does the opposite of expand_user, with its outputs.
424 424 """
425 425 if tilde_expand:
426 426 return path.replace(tilde_val, '~')
427 427 else:
428 428 return path
429 429
430 430
431 431 def completions_sorting_key(word):
432 432 """key for sorting completions
433 433
434 434 This does several things:
435 435
436 436 - Demote any completions starting with underscores to the end
437 437 - Insert any %magic and %%cellmagic completions in the alphabetical order
438 438 by their name
439 439 """
440 440 prio1, prio2 = 0, 0
441 441
442 442 if word.startswith('__'):
443 443 prio1 = 2
444 444 elif word.startswith('_'):
445 445 prio1 = 1
446 446
447 447 if word.endswith('='):
448 448 prio1 = -1
449 449
450 450 if word.startswith('%%'):
451 451 # If there's another % in there, this is something else, so leave it alone
452 452 if not "%" in word[2:]:
453 453 word = word[2:]
454 454 prio2 = 2
455 455 elif word.startswith('%'):
456 456 if not "%" in word[1:]:
457 457 word = word[1:]
458 458 prio2 = 1
459 459
460 460 return prio1, word, prio2
461 461
462 462
463 463 class _FakeJediCompletion:
464 464 """
465 465 This is a workaround to communicate to the UI that Jedi has crashed and to
466 466 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
467 467
468 468 Added in IPython 6.0 so should likely be removed for 7.0
469 469
470 470 """
471 471
472 472 def __init__(self, name):
473 473
474 474 self.name = name
475 475 self.complete = name
476 476 self.type = 'crashed'
477 477 self.name_with_symbols = name
478 478 self.signature = ""
479 479 self._origin = "fake"
480 480 self.text = "crashed"
481 481
482 482 def __repr__(self):
483 483 return '<Fake completion object jedi has crashed>'
484 484
485 485
486 486 _JediCompletionLike = Union[jedi.api.Completion, _FakeJediCompletion]
487 487
488 488
489 489 class Completion:
490 490 """
491 491 Completion object used and returned by IPython completers.
492 492
493 493 .. warning::
494 494
495 495 Unstable
496 496
497 497 This function is unstable, API may change without warning.
498 498 It will also raise unless use in proper context manager.
499 499
500 500 This act as a middle ground :any:`Completion` object between the
501 501 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
502 502 object. While Jedi need a lot of information about evaluator and how the
503 503 code should be ran/inspected, PromptToolkit (and other frontend) mostly
504 504 need user facing information.
505 505
506 506 - Which range should be replaced replaced by what.
507 507 - Some metadata (like completion type), or meta information to displayed to
508 508 the use user.
509 509
510 510 For debugging purpose we can also store the origin of the completion (``jedi``,
511 511 ``IPython.python_matches``, ``IPython.magics_matches``...).
512 512 """
513 513
514 514 __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
515 515
516 516 def __init__(
517 517 self,
518 518 start: int,
519 519 end: int,
520 520 text: str,
521 521 *,
522 522 type: Optional[str] = None,
523 523 _origin="",
524 524 signature="",
525 525 ) -> None:
526 526 warnings.warn(
527 527 "``Completion`` is a provisional API (as of IPython 6.0). "
528 528 "It may change without warnings. "
529 529 "Use in corresponding context manager.",
530 530 category=ProvisionalCompleterWarning,
531 531 stacklevel=2,
532 532 )
533 533
534 534 self.start = start
535 535 self.end = end
536 536 self.text = text
537 537 self.type = type
538 538 self.signature = signature
539 539 self._origin = _origin
540 540
541 541 def __repr__(self):
542 542 return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
543 543 (self.start, self.end, self.text, self.type or '?', self.signature or '?')
544 544
545 545 def __eq__(self, other) -> bool:
546 546 """
547 547 Equality and hash do not hash the type (as some completer may not be
548 548 able to infer the type), but are use to (partially) de-duplicate
549 549 completion.
550 550
551 551 Completely de-duplicating completion is a bit tricker that just
552 552 comparing as it depends on surrounding text, which Completions are not
553 553 aware of.
554 554 """
555 555 return self.start == other.start and \
556 556 self.end == other.end and \
557 557 self.text == other.text
558 558
559 559 def __hash__(self):
560 560 return hash((self.start, self.end, self.text))
561 561
562 562
563 563 class SimpleCompletion:
564 564 """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
565 565
566 566 .. warning::
567 567
568 568 Provisional
569 569
570 570 This class is used to describe the currently supported attributes of
571 571 simple completion items, and any additional implementation details
572 572 should not be relied on. Additional attributes may be included in
573 573 future versions, and meaning of text disambiguated from the current
574 574 dual meaning of "text to insert" and "text to used as a label".
575 575 """
576 576
577 577 __slots__ = ["text", "type"]
578 578
579 579 def __init__(self, text: str, *, type: Optional[str] = None):
580 580 self.text = text
581 581 self.type = type
582 582
583 583 def __repr__(self):
584 584 return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
585 585
586 586
587 587 class _MatcherResultBase(TypedDict):
588 588 """Definition of dictionary to be returned by new-style Matcher (API v2)."""
589 589
590 590 #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
591 591 matched_fragment: NotRequired[str]
592 592
593 593 #: Whether to suppress results from all other matchers (True), some
594 594 #: matchers (set of identifiers) or none (False); default is False.
595 595 suppress: NotRequired[Union[bool, Set[str]]]
596 596
597 597 #: Identifiers of matchers which should NOT be suppressed when this matcher
598 598 #: requests to suppress all other matchers; defaults to an empty set.
599 599 do_not_suppress: NotRequired[Set[str]]
600 600
601 601 #: Are completions already ordered and should be left as-is? default is False.
602 602 ordered: NotRequired[bool]
603 603
604 604
605 605 @sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
606 606 class SimpleMatcherResult(_MatcherResultBase, TypedDict):
607 607 """Result of new-style completion matcher."""
608 608
609 609 # note: TypedDict is added again to the inheritance chain
610 610 # in order to get __orig_bases__ for documentation
611 611
612 612 #: List of candidate completions
613 613 completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
614 614
615 615
616 616 class _JediMatcherResult(_MatcherResultBase):
617 617 """Matching result returned by Jedi (will be processed differently)"""
618 618
619 619 #: list of candidate completions
620 620 completions: Iterator[_JediCompletionLike]
621 621
622 622
623 623 AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
624 624 AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
625 625
626 626
627 627 @dataclass
628 628 class CompletionContext:
629 629 """Completion context provided as an argument to matchers in the Matcher API v2."""
630 630
631 631 # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
632 632 # which was not explicitly visible as an argument of the matcher, making any refactor
633 633 # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
634 634 # from the completer, and make substituting them in sub-classes easier.
635 635
636 636 #: Relevant fragment of code directly preceding the cursor.
637 637 #: The extraction of token is implemented via splitter heuristic
638 638 #: (following readline behaviour for legacy reasons), which is user configurable
639 639 #: (by switching the greedy mode).
640 640 token: str
641 641
642 642 #: The full available content of the editor or buffer
643 643 full_text: str
644 644
645 645 #: Cursor position in the line (the same for ``full_text`` and ``text``).
646 646 cursor_position: int
647 647
648 648 #: Cursor line in ``full_text``.
649 649 cursor_line: int
650 650
651 651 #: The maximum number of completions that will be used downstream.
652 652 #: Matchers can use this information to abort early.
653 653 #: The built-in Jedi matcher is currently excepted from this limit.
654 654 # If not given, return all possible completions.
655 655 limit: Optional[int]
656 656
657 657 @cached_property
658 658 def text_until_cursor(self) -> str:
659 659 return self.line_with_cursor[: self.cursor_position]
660 660
661 661 @cached_property
662 662 def line_with_cursor(self) -> str:
663 663 return self.full_text.split("\n")[self.cursor_line]
664 664
665 665
666 666 #: Matcher results for API v2.
667 667 MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
668 668
669 669
670 670 class _MatcherAPIv1Base(Protocol):
671 671 def __call__(self, text: str) -> List[str]:
672 672 """Call signature."""
673 673 ...
674 674
675 675 #: Used to construct the default matcher identifier
676 676 __qualname__: str
677 677
678 678
679 679 class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
680 680 #: API version
681 681 matcher_api_version: Optional[Literal[1]]
682 682
683 683 def __call__(self, text: str) -> List[str]:
684 684 """Call signature."""
685 685 ...
686 686
687 687
688 688 #: Protocol describing Matcher API v1.
689 689 MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
690 690
691 691
692 692 class MatcherAPIv2(Protocol):
693 693 """Protocol describing Matcher API v2."""
694 694
695 695 #: API version
696 696 matcher_api_version: Literal[2] = 2
697 697
698 698 def __call__(self, context: CompletionContext) -> MatcherResult:
699 699 """Call signature."""
700 700 ...
701 701
702 702 #: Used to construct the default matcher identifier
703 703 __qualname__: str
704 704
705 705
706 706 Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
707 707
708 708
709 709 def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
710 710 api_version = _get_matcher_api_version(matcher)
711 711 return api_version == 1
712 712
713 713
714 714 def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
715 715 api_version = _get_matcher_api_version(matcher)
716 716 return api_version == 2
717 717
718 718
719 719 def _is_sizable(value: Any) -> TypeGuard[Sized]:
720 720 """Determines whether objects is sizable"""
721 721 return hasattr(value, "__len__")
722 722
723 723
724 724 def _is_iterator(value: Any) -> TypeGuard[Iterator]:
725 725 """Determines whether objects is sizable"""
726 726 return hasattr(value, "__next__")
727 727
728 728
729 729 def has_any_completions(result: MatcherResult) -> bool:
730 730 """Check if any result includes any completions."""
731 731 completions = result["completions"]
732 732 if _is_sizable(completions):
733 733 return len(completions) != 0
734 734 if _is_iterator(completions):
735 735 try:
736 736 old_iterator = completions
737 737 first = next(old_iterator)
738 738 result["completions"] = cast(
739 739 Iterator[SimpleCompletion],
740 740 itertools.chain([first], old_iterator),
741 741 )
742 742 return True
743 743 except StopIteration:
744 744 return False
745 745 raise ValueError(
746 746 "Completions returned by matcher need to be an Iterator or a Sizable"
747 747 )
748 748
749 749
750 750 def completion_matcher(
751 751 *,
752 752 priority: Optional[float] = None,
753 753 identifier: Optional[str] = None,
754 754 api_version: int = 1,
755 755 ):
756 756 """Adds attributes describing the matcher.
757 757
758 758 Parameters
759 759 ----------
760 760 priority : Optional[float]
761 761 The priority of the matcher, determines the order of execution of matchers.
762 762 Higher priority means that the matcher will be executed first. Defaults to 0.
763 763 identifier : Optional[str]
764 764 identifier of the matcher allowing users to modify the behaviour via traitlets,
765 765 and also used to for debugging (will be passed as ``origin`` with the completions).
766 766
767 767 Defaults to matcher function's ``__qualname__`` (for example,
768 768 ``IPCompleter.file_matcher`` for the built-in matched defined
769 769 as a ``file_matcher`` method of the ``IPCompleter`` class).
770 770 api_version: Optional[int]
771 771 version of the Matcher API used by this matcher.
772 772 Currently supported values are 1 and 2.
773 773 Defaults to 1.
774 774 """
775 775
776 776 def wrapper(func: Matcher):
777 777 func.matcher_priority = priority or 0 # type: ignore
778 778 func.matcher_identifier = identifier or func.__qualname__ # type: ignore
779 779 func.matcher_api_version = api_version # type: ignore
780 780 if TYPE_CHECKING:
781 781 if api_version == 1:
782 782 func = cast(MatcherAPIv1, func)
783 783 elif api_version == 2:
784 784 func = cast(MatcherAPIv2, func)
785 785 return func
786 786
787 787 return wrapper
788 788
789 789
790 790 def _get_matcher_priority(matcher: Matcher):
791 791 return getattr(matcher, "matcher_priority", 0)
792 792
793 793
794 794 def _get_matcher_id(matcher: Matcher):
795 795 return getattr(matcher, "matcher_identifier", matcher.__qualname__)
796 796
797 797
798 798 def _get_matcher_api_version(matcher):
799 799 return getattr(matcher, "matcher_api_version", 1)
800 800
801 801
802 802 context_matcher = partial(completion_matcher, api_version=2)
803 803
804 804
805 805 _IC = Iterable[Completion]
806 806
807 807
808 808 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
809 809 """
810 810 Deduplicate a set of completions.
811 811
812 812 .. warning::
813 813
814 814 Unstable
815 815
816 816 This function is unstable, API may change without warning.
817 817
818 818 Parameters
819 819 ----------
820 820 text : str
821 821 text that should be completed.
822 822 completions : Iterator[Completion]
823 823 iterator over the completions to deduplicate
824 824
825 825 Yields
826 826 ------
827 827 `Completions` objects
828 828 Completions coming from multiple sources, may be different but end up having
829 829 the same effect when applied to ``text``. If this is the case, this will
830 830 consider completions as equal and only emit the first encountered.
831 831 Not folded in `completions()` yet for debugging purpose, and to detect when
832 832 the IPython completer does return things that Jedi does not, but should be
833 833 at some point.
834 834 """
835 835 completions = list(completions)
836 836 if not completions:
837 837 return
838 838
839 839 new_start = min(c.start for c in completions)
840 840 new_end = max(c.end for c in completions)
841 841
842 842 seen = set()
843 843 for c in completions:
844 844 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
845 845 if new_text not in seen:
846 846 yield c
847 847 seen.add(new_text)
848 848
849 849
850 850 def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
851 851 """
852 852 Rectify a set of completions to all have the same ``start`` and ``end``
853 853
854 854 .. warning::
855 855
856 856 Unstable
857 857
858 858 This function is unstable, API may change without warning.
859 859 It will also raise unless use in proper context manager.
860 860
861 861 Parameters
862 862 ----------
863 863 text : str
864 864 text that should be completed.
865 865 completions : Iterator[Completion]
866 866 iterator over the completions to rectify
867 867 _debug : bool
868 868 Log failed completion
869 869
870 870 Notes
871 871 -----
872 872 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
873 873 the Jupyter Protocol requires them to behave like so. This will readjust
874 874 the completion to have the same ``start`` and ``end`` by padding both
875 875 extremities with surrounding text.
876 876
877 877 During stabilisation should support a ``_debug`` option to log which
878 878 completion are return by the IPython completer and not found in Jedi in
879 879 order to make upstream bug report.
880 880 """
881 881 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
882 882 "It may change without warnings. "
883 883 "Use in corresponding context manager.",
884 884 category=ProvisionalCompleterWarning, stacklevel=2)
885 885
886 886 completions = list(completions)
887 887 if not completions:
888 888 return
889 889 starts = (c.start for c in completions)
890 890 ends = (c.end for c in completions)
891 891
892 892 new_start = min(starts)
893 893 new_end = max(ends)
894 894
895 895 seen_jedi = set()
896 896 seen_python_matches = set()
897 897 for c in completions:
898 898 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
899 899 if c._origin == 'jedi':
900 900 seen_jedi.add(new_text)
901 901 elif c._origin == 'IPCompleter.python_matches':
902 902 seen_python_matches.add(new_text)
903 903 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
904 904 diff = seen_python_matches.difference(seen_jedi)
905 905 if diff and _debug:
906 906 print('IPython.python matches have extras:', diff)
907 907
908 908
909 909 if sys.platform == 'win32':
910 910 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
911 911 else:
912 912 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
913 913
914 914 GREEDY_DELIMS = ' =\r\n'
915 915
916 916
917 917 class CompletionSplitter(object):
918 918 """An object to split an input line in a manner similar to readline.
919 919
920 920 By having our own implementation, we can expose readline-like completion in
921 921 a uniform manner to all frontends. This object only needs to be given the
922 922 line of text to be split and the cursor position on said line, and it
923 923 returns the 'word' to be completed on at the cursor after splitting the
924 924 entire line.
925 925
926 926 What characters are used as splitting delimiters can be controlled by
927 927 setting the ``delims`` attribute (this is a property that internally
928 928 automatically builds the necessary regular expression)"""
929 929
930 930 # Private interface
931 931
932 932 # A string of delimiter characters. The default value makes sense for
933 933 # IPython's most typical usage patterns.
934 934 _delims = DELIMS
935 935
936 936 # The expression (a normal string) to be compiled into a regular expression
937 937 # for actual splitting. We store it as an attribute mostly for ease of
938 938 # debugging, since this type of code can be so tricky to debug.
939 939 _delim_expr = None
940 940
941 941 # The regular expression that does the actual splitting
942 942 _delim_re = None
943 943
944 944 def __init__(self, delims=None):
945 945 delims = CompletionSplitter._delims if delims is None else delims
946 946 self.delims = delims
947 947
948 948 @property
949 949 def delims(self):
950 950 """Return the string of delimiter characters."""
951 951 return self._delims
952 952
953 953 @delims.setter
954 954 def delims(self, delims):
955 955 """Set the delimiters for line splitting."""
956 956 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
957 957 self._delim_re = re.compile(expr)
958 958 self._delims = delims
959 959 self._delim_expr = expr
960 960
961 961 def split_line(self, line, cursor_pos=None):
962 962 """Split a line of text with a cursor at the given position.
963 963 """
964 964 l = line if cursor_pos is None else line[:cursor_pos]
965 965 return self._delim_re.split(l)[-1]
966 966
967 967
968 968
969 969 class Completer(Configurable):
970 970
971 971 greedy = Bool(
972 972 False,
973 973 help="""Activate greedy completion.
974 974
975 975 .. deprecated:: 8.8
976 976 Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
977 977
978 978 When enabled in IPython 8.8 or newer, changes configuration as follows:
979 979
980 980 - ``Completer.evaluation = 'unsafe'``
981 981 - ``Completer.auto_close_dict_keys = True``
982 982 """,
983 983 ).tag(config=True)
984 984
985 985 evaluation = Enum(
986 986 ("forbidden", "minimal", "limited", "unsafe", "dangerous"),
987 987 default_value="limited",
988 988 help="""Policy for code evaluation under completion.
989 989
990 990 Successive options allow to enable more eager evaluation for better
991 991 completion suggestions, including for nested dictionaries, nested lists,
992 992 or even results of function calls.
993 993 Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
994 994 code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
995 995
996 996 Allowed values are:
997 997
998 998 - ``forbidden``: no evaluation of code is permitted,
999 999 - ``minimal``: evaluation of literals and access to built-in namespace;
1000 1000 no item/attribute evaluationm no access to locals/globals,
1001 1001 no evaluation of any operations or comparisons.
1002 1002 - ``limited``: access to all namespaces, evaluation of hard-coded methods
1003 1003 (for example: :any:`dict.keys`, :any:`object.__getattr__`,
1004 1004 :any:`object.__getitem__`) on allow-listed objects (for example:
1005 1005 :any:`dict`, :any:`list`, :any:`tuple`, ``pandas.Series``),
1006 1006 - ``unsafe``: evaluation of all methods and function calls but not of
1007 1007 syntax with side-effects like `del x`,
1008 1008 - ``dangerous``: completely arbitrary evaluation.
1009 1009 """,
1010 1010 ).tag(config=True)
1011 1011
1012 1012 use_jedi = Bool(default_value=JEDI_INSTALLED,
1013 1013 help="Experimental: Use Jedi to generate autocompletions. "
1014 1014 "Default to True if jedi is installed.").tag(config=True)
1015 1015
1016 1016 jedi_compute_type_timeout = Int(default_value=400,
1017 1017 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
1018 1018 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
1019 1019 performance by preventing jedi to build its cache.
1020 1020 """).tag(config=True)
1021 1021
1022 1022 debug = Bool(default_value=False,
1023 1023 help='Enable debug for the Completer. Mostly print extra '
1024 1024 'information for experimental jedi integration.')\
1025 1025 .tag(config=True)
1026 1026
1027 1027 backslash_combining_completions = Bool(True,
1028 1028 help="Enable unicode completions, e.g. \\alpha<tab> . "
1029 1029 "Includes completion of latex commands, unicode names, and expanding "
1030 1030 "unicode characters back to latex commands.").tag(config=True)
1031 1031
1032 1032 auto_close_dict_keys = Bool(
1033 1033 False,
1034 1034 help="""
1035 1035 Enable auto-closing dictionary keys.
1036 1036
1037 1037 When enabled string keys will be suffixed with a final quote
1038 1038 (matching the opening quote), tuple keys will also receive a
1039 1039 separating comma if needed, and keys which are final will
1040 1040 receive a closing bracket (``]``).
1041 1041 """,
1042 1042 ).tag(config=True)
1043 1043
1044 1044 def __init__(self, namespace=None, global_namespace=None, **kwargs):
1045 1045 """Create a new completer for the command line.
1046 1046
1047 1047 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
1048 1048
1049 1049 If unspecified, the default namespace where completions are performed
1050 1050 is __main__ (technically, __main__.__dict__). Namespaces should be
1051 1051 given as dictionaries.
1052 1052
1053 1053 An optional second namespace can be given. This allows the completer
1054 1054 to handle cases where both the local and global scopes need to be
1055 1055 distinguished.
1056 1056 """
1057 1057
1058 1058 # Don't bind to namespace quite yet, but flag whether the user wants a
1059 1059 # specific namespace or to use __main__.__dict__. This will allow us
1060 1060 # to bind to __main__.__dict__ at completion time, not now.
1061 1061 if namespace is None:
1062 1062 self.use_main_ns = True
1063 1063 else:
1064 1064 self.use_main_ns = False
1065 1065 self.namespace = namespace
1066 1066
1067 1067 # The global namespace, if given, can be bound directly
1068 1068 if global_namespace is None:
1069 1069 self.global_namespace = {}
1070 1070 else:
1071 1071 self.global_namespace = global_namespace
1072 1072
1073 1073 self.custom_matchers = []
1074 1074
1075 1075 super(Completer, self).__init__(**kwargs)
1076 1076
1077 1077 def complete(self, text, state):
1078 1078 """Return the next possible completion for 'text'.
1079 1079
1080 1080 This is called successively with state == 0, 1, 2, ... until it
1081 1081 returns None. The completion should begin with 'text'.
1082 1082
1083 1083 """
1084 1084 if self.use_main_ns:
1085 1085 self.namespace = __main__.__dict__
1086 1086
1087 1087 if state == 0:
1088 1088 if "." in text:
1089 1089 self.matches = self.attr_matches(text)
1090 1090 else:
1091 1091 self.matches = self.global_matches(text)
1092 1092 try:
1093 1093 return self.matches[state]
1094 1094 except IndexError:
1095 1095 return None
1096 1096
1097 1097 def global_matches(self, text):
1098 1098 """Compute matches when text is a simple name.
1099 1099
1100 1100 Return a list of all keywords, built-in functions and names currently
1101 1101 defined in self.namespace or self.global_namespace that match.
1102 1102
1103 1103 """
1104 1104 matches = []
1105 1105 match_append = matches.append
1106 1106 n = len(text)
1107 1107 for lst in [
1108 1108 keyword.kwlist,
1109 1109 builtin_mod.__dict__.keys(),
1110 1110 list(self.namespace.keys()),
1111 1111 list(self.global_namespace.keys()),
1112 1112 ]:
1113 1113 for word in lst:
1114 1114 if word[:n] == text and word != "__builtins__":
1115 1115 match_append(word)
1116 1116
1117 1117 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
1118 1118 for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
1119 1119 shortened = {
1120 1120 "_".join([sub[0] for sub in word.split("_")]): word
1121 1121 for word in lst
1122 1122 if snake_case_re.match(word)
1123 1123 }
1124 1124 for word in shortened.keys():
1125 1125 if word[:n] == text and word != "__builtins__":
1126 1126 match_append(shortened[word])
1127 1127 return matches
1128 1128
1129 1129 def attr_matches(self, text):
1130 1130 """Compute matches when text contains a dot.
1131 1131
1132 1132 Assuming the text is of the form NAME.NAME....[NAME], and is
1133 1133 evaluatable in self.namespace or self.global_namespace, it will be
1134 1134 evaluated and its attributes (as revealed by dir()) are used as
1135 1135 possible completions. (For class instances, class members are
1136 1136 also considered.)
1137 1137
1138 1138 WARNING: this can still invoke arbitrary C code, if an object
1139 1139 with a __getattr__ hook is evaluated.
1140 1140
1141 1141 """
1142 1142 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
1143 1143 if not m2:
1144 1144 return []
1145 1145 expr, attr = m2.group(1, 2)
1146 1146
1147 1147 obj = self._evaluate_expr(expr)
1148 1148
1149 1149 if obj is not_found:
1150 1150 return []
1151 1151
1152 1152 if self.limit_to__all__ and hasattr(obj, '__all__'):
1153 1153 words = get__all__entries(obj)
1154 1154 else:
1155 1155 words = dir2(obj)
1156 1156
1157 1157 try:
1158 1158 words = generics.complete_object(obj, words)
1159 1159 except TryNext:
1160 1160 pass
1161 1161 except AssertionError:
1162 1162 raise
1163 1163 except Exception:
1164 1164 # Silence errors from completion function
1165 #raise # dbg
1166 1165 pass
1167 1166 # Build match list to return
1168 1167 n = len(attr)
1169 return ["%s.%s" % (expr, w) for w in words if w[:n] == attr]
1168
1169 # Note: ideally we would just return words here and the prefix
1170 # reconciliator would know that we intend to append to rather than
1171 # replace the input text; this requires refactoring to return range
1172 # which ought to be replaced (as does jedi).
1173 tokens = _parse_tokens(expr)
1174 rev_tokens = reversed(tokens)
1175 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1176 name_turn = True
1177
1178 parts = []
1179 for token in rev_tokens:
1180 if token.type in skip_over:
1181 continue
1182 if token.type == tokenize.NAME and name_turn:
1183 parts.append(token.string)
1184 name_turn = False
1185 elif token.type == tokenize.OP and token.string == "." and not name_turn:
1186 parts.append(token.string)
1187 name_turn = True
1188 else:
1189 # short-circuit if not empty nor name token
1190 break
1191
1192 prefix_after_space = "".join(reversed(parts))
1193
1194 return ["%s.%s" % (prefix_after_space, w) for w in words if w[:n] == attr]
1170 1195
1171 1196 def _evaluate_expr(self, expr):
1172 1197 obj = not_found
1173 1198 done = False
1174 1199 while not done and expr:
1175 1200 try:
1176 1201 obj = guarded_eval(
1177 1202 expr,
1178 1203 EvaluationContext(
1179 1204 globals=self.global_namespace,
1180 1205 locals=self.namespace,
1181 1206 evaluation=self.evaluation,
1182 1207 ),
1183 1208 )
1184 1209 done = True
1185 1210 except Exception as e:
1186 1211 if self.debug:
1187 1212 print("Evaluation exception", e)
1188 1213 # trim the expression to remove any invalid prefix
1189 1214 # e.g. user starts `(d[`, so we get `expr = '(d'`,
1190 1215 # where parenthesis is not closed.
1191 1216 # TODO: make this faster by reusing parts of the computation?
1192 1217 expr = expr[1:]
1193 1218 return obj
1194 1219
1195 1220 def get__all__entries(obj):
1196 1221 """returns the strings in the __all__ attribute"""
1197 1222 try:
1198 1223 words = getattr(obj, '__all__')
1199 1224 except:
1200 1225 return []
1201 1226
1202 1227 return [w for w in words if isinstance(w, str)]
1203 1228
1204 1229
1205 1230 class _DictKeyState(enum.Flag):
1206 1231 """Represent state of the key match in context of other possible matches.
1207 1232
1208 1233 - given `d1 = {'a': 1}` completion on `d1['<tab>` will yield `{'a': END_OF_ITEM}` as there is no tuple.
1209 1234 - given `d2 = {('a', 'b'): 1}`: `d2['a', '<tab>` will yield `{'b': END_OF_TUPLE}` as there is no tuple members to add beyond `'b'`.
1210 1235 - given `d3 = {('a', 'b'): 1}`: `d3['<tab>` will yield `{'a': IN_TUPLE}` as `'a'` can be added.
1211 1236 - given `d4 = {'a': 1, ('a', 'b'): 2}`: `d4['<tab>` will yield `{'a': END_OF_ITEM & END_OF_TUPLE}`
1212 1237 """
1213 1238
1214 1239 BASELINE = 0
1215 1240 END_OF_ITEM = enum.auto()
1216 1241 END_OF_TUPLE = enum.auto()
1217 1242 IN_TUPLE = enum.auto()
1218 1243
1219 1244
1220 1245 def _parse_tokens(c):
1221 1246 """Parse tokens even if there is an error."""
1222 1247 tokens = []
1223 1248 token_generator = tokenize.generate_tokens(iter(c.splitlines()).__next__)
1224 1249 while True:
1225 1250 try:
1226 1251 tokens.append(next(token_generator))
1227 1252 except tokenize.TokenError:
1228 1253 return tokens
1229 1254 except StopIteration:
1230 1255 return tokens
1231 1256
1232 1257
1233 1258 def _match_number_in_dict_key_prefix(prefix: str) -> Union[str, None]:
1234 1259 """Match any valid Python numeric literal in a prefix of dictionary keys.
1235 1260
1236 1261 References:
1237 1262 - https://docs.python.org/3/reference/lexical_analysis.html#numeric-literals
1238 1263 - https://docs.python.org/3/library/tokenize.html
1239 1264 """
1240 1265 if prefix[-1].isspace():
1241 1266 # if user typed a space we do not have anything to complete
1242 1267 # even if there was a valid number token before
1243 1268 return None
1244 1269 tokens = _parse_tokens(prefix)
1245 1270 rev_tokens = reversed(tokens)
1246 1271 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1247 1272 number = None
1248 1273 for token in rev_tokens:
1249 1274 if token.type in skip_over:
1250 1275 continue
1251 1276 if number is None:
1252 1277 if token.type == tokenize.NUMBER:
1253 1278 number = token.string
1254 1279 continue
1255 1280 else:
1256 1281 # we did not match a number
1257 1282 return None
1258 1283 if token.type == tokenize.OP:
1259 1284 if token.string == ",":
1260 1285 break
1261 1286 if token.string in {"+", "-"}:
1262 1287 number = token.string + number
1263 1288 else:
1264 1289 return None
1265 1290 return number
1266 1291
1267 1292
1268 1293 _INT_FORMATS = {
1269 1294 "0b": bin,
1270 1295 "0o": oct,
1271 1296 "0x": hex,
1272 1297 }
1273 1298
1274 1299
1275 1300 def match_dict_keys(
1276 1301 keys: List[Union[str, bytes, Tuple[Union[str, bytes], ...]]],
1277 1302 prefix: str,
1278 1303 delims: str,
1279 1304 extra_prefix: Optional[Tuple[Union[str, bytes], ...]] = None,
1280 1305 ) -> Tuple[str, int, Dict[str, _DictKeyState]]:
1281 1306 """Used by dict_key_matches, matching the prefix to a list of keys
1282 1307
1283 1308 Parameters
1284 1309 ----------
1285 1310 keys
1286 1311 list of keys in dictionary currently being completed.
1287 1312 prefix
1288 1313 Part of the text already typed by the user. E.g. `mydict[b'fo`
1289 1314 delims
1290 1315 String of delimiters to consider when finding the current key.
1291 1316 extra_prefix : optional
1292 1317 Part of the text already typed in multi-key index cases. E.g. for
1293 1318 `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
1294 1319
1295 1320 Returns
1296 1321 -------
1297 1322 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
1298 1323 ``quote`` being the quote that need to be used to close current string.
1299 1324 ``token_start`` the position where the replacement should start occurring,
1300 1325 ``matches`` a dictionary of replacement/completion keys on keys and values
1301 1326 indicating whether the state.
1302 1327 """
1303 1328 prefix_tuple = extra_prefix if extra_prefix else ()
1304 1329
1305 1330 prefix_tuple_size = sum(
1306 1331 [
1307 1332 # for pandas, do not count slices as taking space
1308 1333 not isinstance(k, slice)
1309 1334 for k in prefix_tuple
1310 1335 ]
1311 1336 )
1312 1337 text_serializable_types = (str, bytes, int, float, slice)
1313 1338
1314 1339 def filter_prefix_tuple(key):
1315 1340 # Reject too short keys
1316 1341 if len(key) <= prefix_tuple_size:
1317 1342 return False
1318 1343 # Reject keys which cannot be serialised to text
1319 1344 for k in key:
1320 1345 if not isinstance(k, text_serializable_types):
1321 1346 return False
1322 1347 # Reject keys that do not match the prefix
1323 1348 for k, pt in zip(key, prefix_tuple):
1324 1349 if k != pt and not isinstance(pt, slice):
1325 1350 return False
1326 1351 # All checks passed!
1327 1352 return True
1328 1353
1329 1354 filtered_key_is_final: Dict[
1330 1355 Union[str, bytes, int, float], _DictKeyState
1331 1356 ] = defaultdict(lambda: _DictKeyState.BASELINE)
1332 1357
1333 1358 for k in keys:
1334 1359 # If at least one of the matches is not final, mark as undetermined.
1335 1360 # This can happen with `d = {111: 'b', (111, 222): 'a'}` where
1336 1361 # `111` appears final on first match but is not final on the second.
1337 1362
1338 1363 if isinstance(k, tuple):
1339 1364 if filter_prefix_tuple(k):
1340 1365 key_fragment = k[prefix_tuple_size]
1341 1366 filtered_key_is_final[key_fragment] |= (
1342 1367 _DictKeyState.END_OF_TUPLE
1343 1368 if len(k) == prefix_tuple_size + 1
1344 1369 else _DictKeyState.IN_TUPLE
1345 1370 )
1346 1371 elif prefix_tuple_size > 0:
1347 1372 # we are completing a tuple but this key is not a tuple,
1348 1373 # so we should ignore it
1349 1374 pass
1350 1375 else:
1351 1376 if isinstance(k, text_serializable_types):
1352 1377 filtered_key_is_final[k] |= _DictKeyState.END_OF_ITEM
1353 1378
1354 1379 filtered_keys = filtered_key_is_final.keys()
1355 1380
1356 1381 if not prefix:
1357 1382 return "", 0, {repr(k): v for k, v in filtered_key_is_final.items()}
1358 1383
1359 1384 quote_match = re.search("(?:\"|')", prefix)
1360 1385 is_user_prefix_numeric = False
1361 1386
1362 1387 if quote_match:
1363 1388 quote = quote_match.group()
1364 1389 valid_prefix = prefix + quote
1365 1390 try:
1366 1391 prefix_str = literal_eval(valid_prefix)
1367 1392 except Exception:
1368 1393 return "", 0, {}
1369 1394 else:
1370 1395 # If it does not look like a string, let's assume
1371 1396 # we are dealing with a number or variable.
1372 1397 number_match = _match_number_in_dict_key_prefix(prefix)
1373 1398
1374 1399 # We do not want the key matcher to suggest variable names so we yield:
1375 1400 if number_match is None:
1376 1401 # The alternative would be to assume that user forgort the quote
1377 1402 # and if the substring matches, suggest adding it at the start.
1378 1403 return "", 0, {}
1379 1404
1380 1405 prefix_str = number_match
1381 1406 is_user_prefix_numeric = True
1382 1407 quote = ""
1383 1408
1384 1409 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
1385 1410 token_match = re.search(pattern, prefix, re.UNICODE)
1386 1411 assert token_match is not None # silence mypy
1387 1412 token_start = token_match.start()
1388 1413 token_prefix = token_match.group()
1389 1414
1390 1415 matched: Dict[str, _DictKeyState] = {}
1391 1416
1392 1417 str_key: Union[str, bytes]
1393 1418
1394 1419 for key in filtered_keys:
1395 1420 if isinstance(key, (int, float)):
1396 1421 # User typed a number but this key is not a number.
1397 1422 if not is_user_prefix_numeric:
1398 1423 continue
1399 1424 str_key = str(key)
1400 1425 if isinstance(key, int):
1401 1426 int_base = prefix_str[:2].lower()
1402 1427 # if user typed integer using binary/oct/hex notation:
1403 1428 if int_base in _INT_FORMATS:
1404 1429 int_format = _INT_FORMATS[int_base]
1405 1430 str_key = int_format(key)
1406 1431 else:
1407 1432 # User typed a string but this key is a number.
1408 1433 if is_user_prefix_numeric:
1409 1434 continue
1410 1435 str_key = key
1411 1436 try:
1412 1437 if not str_key.startswith(prefix_str):
1413 1438 continue
1414 1439 except (AttributeError, TypeError, UnicodeError) as e:
1415 1440 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
1416 1441 continue
1417 1442
1418 1443 # reformat remainder of key to begin with prefix
1419 1444 rem = str_key[len(prefix_str) :]
1420 1445 # force repr wrapped in '
1421 1446 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
1422 1447 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
1423 1448 if quote == '"':
1424 1449 # The entered prefix is quoted with ",
1425 1450 # but the match is quoted with '.
1426 1451 # A contained " hence needs escaping for comparison:
1427 1452 rem_repr = rem_repr.replace('"', '\\"')
1428 1453
1429 1454 # then reinsert prefix from start of token
1430 1455 match = "%s%s" % (token_prefix, rem_repr)
1431 1456
1432 1457 matched[match] = filtered_key_is_final[key]
1433 1458 return quote, token_start, matched
1434 1459
1435 1460
1436 1461 def cursor_to_position(text:str, line:int, column:int)->int:
1437 1462 """
1438 1463 Convert the (line,column) position of the cursor in text to an offset in a
1439 1464 string.
1440 1465
1441 1466 Parameters
1442 1467 ----------
1443 1468 text : str
1444 1469 The text in which to calculate the cursor offset
1445 1470 line : int
1446 1471 Line of the cursor; 0-indexed
1447 1472 column : int
1448 1473 Column of the cursor 0-indexed
1449 1474
1450 1475 Returns
1451 1476 -------
1452 1477 Position of the cursor in ``text``, 0-indexed.
1453 1478
1454 1479 See Also
1455 1480 --------
1456 1481 position_to_cursor : reciprocal of this function
1457 1482
1458 1483 """
1459 1484 lines = text.split('\n')
1460 1485 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
1461 1486
1462 1487 return sum(len(l) + 1 for l in lines[:line]) + column
1463 1488
1464 1489 def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
1465 1490 """
1466 1491 Convert the position of the cursor in text (0 indexed) to a line
1467 1492 number(0-indexed) and a column number (0-indexed) pair
1468 1493
1469 1494 Position should be a valid position in ``text``.
1470 1495
1471 1496 Parameters
1472 1497 ----------
1473 1498 text : str
1474 1499 The text in which to calculate the cursor offset
1475 1500 offset : int
1476 1501 Position of the cursor in ``text``, 0-indexed.
1477 1502
1478 1503 Returns
1479 1504 -------
1480 1505 (line, column) : (int, int)
1481 1506 Line of the cursor; 0-indexed, column of the cursor 0-indexed
1482 1507
1483 1508 See Also
1484 1509 --------
1485 1510 cursor_to_position : reciprocal of this function
1486 1511
1487 1512 """
1488 1513
1489 1514 assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
1490 1515
1491 1516 before = text[:offset]
1492 1517 blines = before.split('\n') # ! splitnes trim trailing \n
1493 1518 line = before.count('\n')
1494 1519 col = len(blines[-1])
1495 1520 return line, col
1496 1521
1497 1522
1498 1523 def _safe_isinstance(obj, module, class_name, *attrs):
1499 1524 """Checks if obj is an instance of module.class_name if loaded
1500 1525 """
1501 1526 if module in sys.modules:
1502 1527 m = sys.modules[module]
1503 1528 for attr in [class_name, *attrs]:
1504 1529 m = getattr(m, attr)
1505 1530 return isinstance(obj, m)
1506 1531
1507 1532
1508 1533 @context_matcher()
1509 1534 def back_unicode_name_matcher(context: CompletionContext):
1510 1535 """Match Unicode characters back to Unicode name
1511 1536
1512 1537 Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1513 1538 """
1514 1539 fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1515 1540 return _convert_matcher_v1_result_to_v2(
1516 1541 matches, type="unicode", fragment=fragment, suppress_if_matches=True
1517 1542 )
1518 1543
1519 1544
1520 1545 def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1521 1546 """Match Unicode characters back to Unicode name
1522 1547
1523 1548 This does ``β˜ƒ`` -> ``\\snowman``
1524 1549
1525 1550 Note that snowman is not a valid python3 combining character but will be expanded.
1526 1551 Though it will not recombine back to the snowman character by the completion machinery.
1527 1552
1528 1553 This will not either back-complete standard sequences like \\n, \\b ...
1529 1554
1530 1555 .. deprecated:: 8.6
1531 1556 You can use :meth:`back_unicode_name_matcher` instead.
1532 1557
1533 1558 Returns
1534 1559 =======
1535 1560
1536 1561 Return a tuple with two elements:
1537 1562
1538 1563 - The Unicode character that was matched (preceded with a backslash), or
1539 1564 empty string,
1540 1565 - a sequence (of 1), name for the match Unicode character, preceded by
1541 1566 backslash, or empty if no match.
1542 1567 """
1543 1568 if len(text)<2:
1544 1569 return '', ()
1545 1570 maybe_slash = text[-2]
1546 1571 if maybe_slash != '\\':
1547 1572 return '', ()
1548 1573
1549 1574 char = text[-1]
1550 1575 # no expand on quote for completion in strings.
1551 1576 # nor backcomplete standard ascii keys
1552 1577 if char in string.ascii_letters or char in ('"',"'"):
1553 1578 return '', ()
1554 1579 try :
1555 1580 unic = unicodedata.name(char)
1556 1581 return '\\'+char,('\\'+unic,)
1557 1582 except KeyError:
1558 1583 pass
1559 1584 return '', ()
1560 1585
1561 1586
1562 1587 @context_matcher()
1563 1588 def back_latex_name_matcher(context: CompletionContext):
1564 1589 """Match latex characters back to unicode name
1565 1590
1566 1591 Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1567 1592 """
1568 1593 fragment, matches = back_latex_name_matches(context.text_until_cursor)
1569 1594 return _convert_matcher_v1_result_to_v2(
1570 1595 matches, type="latex", fragment=fragment, suppress_if_matches=True
1571 1596 )
1572 1597
1573 1598
1574 1599 def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1575 1600 """Match latex characters back to unicode name
1576 1601
1577 1602 This does ``\\β„΅`` -> ``\\aleph``
1578 1603
1579 1604 .. deprecated:: 8.6
1580 1605 You can use :meth:`back_latex_name_matcher` instead.
1581 1606 """
1582 1607 if len(text)<2:
1583 1608 return '', ()
1584 1609 maybe_slash = text[-2]
1585 1610 if maybe_slash != '\\':
1586 1611 return '', ()
1587 1612
1588 1613
1589 1614 char = text[-1]
1590 1615 # no expand on quote for completion in strings.
1591 1616 # nor backcomplete standard ascii keys
1592 1617 if char in string.ascii_letters or char in ('"',"'"):
1593 1618 return '', ()
1594 1619 try :
1595 1620 latex = reverse_latex_symbol[char]
1596 1621 # '\\' replace the \ as well
1597 1622 return '\\'+char,[latex]
1598 1623 except KeyError:
1599 1624 pass
1600 1625 return '', ()
1601 1626
1602 1627
1603 1628 def _formatparamchildren(parameter) -> str:
1604 1629 """
1605 1630 Get parameter name and value from Jedi Private API
1606 1631
1607 1632 Jedi does not expose a simple way to get `param=value` from its API.
1608 1633
1609 1634 Parameters
1610 1635 ----------
1611 1636 parameter
1612 1637 Jedi's function `Param`
1613 1638
1614 1639 Returns
1615 1640 -------
1616 1641 A string like 'a', 'b=1', '*args', '**kwargs'
1617 1642
1618 1643 """
1619 1644 description = parameter.description
1620 1645 if not description.startswith('param '):
1621 1646 raise ValueError('Jedi function parameter description have change format.'
1622 1647 'Expected "param ...", found %r".' % description)
1623 1648 return description[6:]
1624 1649
1625 1650 def _make_signature(completion)-> str:
1626 1651 """
1627 1652 Make the signature from a jedi completion
1628 1653
1629 1654 Parameters
1630 1655 ----------
1631 1656 completion : jedi.Completion
1632 1657 object does not complete a function type
1633 1658
1634 1659 Returns
1635 1660 -------
1636 1661 a string consisting of the function signature, with the parenthesis but
1637 1662 without the function name. example:
1638 1663 `(a, *args, b=1, **kwargs)`
1639 1664
1640 1665 """
1641 1666
1642 1667 # it looks like this might work on jedi 0.17
1643 1668 if hasattr(completion, 'get_signatures'):
1644 1669 signatures = completion.get_signatures()
1645 1670 if not signatures:
1646 1671 return '(?)'
1647 1672
1648 1673 c0 = completion.get_signatures()[0]
1649 1674 return '('+c0.to_string().split('(', maxsplit=1)[1]
1650 1675
1651 1676 return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1652 1677 for p in signature.defined_names()) if f])
1653 1678
1654 1679
1655 1680 _CompleteResult = Dict[str, MatcherResult]
1656 1681
1657 1682
1658 1683 DICT_MATCHER_REGEX = re.compile(
1659 1684 r"""(?x)
1660 1685 ( # match dict-referring - or any get item object - expression
1661 1686 .+
1662 1687 )
1663 1688 \[ # open bracket
1664 1689 \s* # and optional whitespace
1665 1690 # Capture any number of serializable objects (e.g. "a", "b", 'c')
1666 1691 # and slices
1667 1692 ((?:(?:
1668 1693 (?: # closed string
1669 1694 [uUbB]? # string prefix (r not handled)
1670 1695 (?:
1671 1696 '(?:[^']|(?<!\\)\\')*'
1672 1697 |
1673 1698 "(?:[^"]|(?<!\\)\\")*"
1674 1699 )
1675 1700 )
1676 1701 |
1677 1702 # capture integers and slices
1678 1703 (?:[-+]?\d+)?(?::(?:[-+]?\d+)?){0,2}
1679 1704 |
1680 1705 # integer in bin/hex/oct notation
1681 1706 0[bBxXoO]_?(?:\w|\d)+
1682 1707 )
1683 1708 \s*,\s*
1684 1709 )*)
1685 1710 ((?:
1686 1711 (?: # unclosed string
1687 1712 [uUbB]? # string prefix (r not handled)
1688 1713 (?:
1689 1714 '(?:[^']|(?<!\\)\\')*
1690 1715 |
1691 1716 "(?:[^"]|(?<!\\)\\")*
1692 1717 )
1693 1718 )
1694 1719 |
1695 1720 # unfinished integer
1696 1721 (?:[-+]?\d+)
1697 1722 |
1698 1723 # integer in bin/hex/oct notation
1699 1724 0[bBxXoO]_?(?:\w|\d)+
1700 1725 )
1701 1726 )?
1702 1727 $
1703 1728 """
1704 1729 )
1705 1730
1706 1731
1707 1732 def _convert_matcher_v1_result_to_v2(
1708 1733 matches: Sequence[str],
1709 1734 type: str,
1710 1735 fragment: Optional[str] = None,
1711 1736 suppress_if_matches: bool = False,
1712 1737 ) -> SimpleMatcherResult:
1713 1738 """Utility to help with transition"""
1714 1739 result = {
1715 1740 "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1716 1741 "suppress": (True if matches else False) if suppress_if_matches else False,
1717 1742 }
1718 1743 if fragment is not None:
1719 1744 result["matched_fragment"] = fragment
1720 1745 return cast(SimpleMatcherResult, result)
1721 1746
1722 1747
1723 1748 class IPCompleter(Completer):
1724 1749 """Extension of the completer class with IPython-specific features"""
1725 1750
1726 1751 @observe('greedy')
1727 1752 def _greedy_changed(self, change):
1728 1753 """update the splitter and readline delims when greedy is changed"""
1729 1754 if change["new"]:
1730 1755 self.evaluation = "unsafe"
1731 1756 self.auto_close_dict_keys = True
1732 1757 self.splitter.delims = GREEDY_DELIMS
1733 1758 else:
1734 1759 self.evaluation = "limited"
1735 1760 self.auto_close_dict_keys = False
1736 1761 self.splitter.delims = DELIMS
1737 1762
1738 1763 dict_keys_only = Bool(
1739 1764 False,
1740 1765 help="""
1741 1766 Whether to show dict key matches only.
1742 1767
1743 1768 (disables all matchers except for `IPCompleter.dict_key_matcher`).
1744 1769 """,
1745 1770 )
1746 1771
1747 1772 suppress_competing_matchers = UnionTrait(
1748 1773 [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1749 1774 default_value=None,
1750 1775 help="""
1751 1776 Whether to suppress completions from other *Matchers*.
1752 1777
1753 1778 When set to ``None`` (default) the matchers will attempt to auto-detect
1754 1779 whether suppression of other matchers is desirable. For example, at
1755 1780 the beginning of a line followed by `%` we expect a magic completion
1756 1781 to be the only applicable option, and after ``my_dict['`` we usually
1757 1782 expect a completion with an existing dictionary key.
1758 1783
1759 1784 If you want to disable this heuristic and see completions from all matchers,
1760 1785 set ``IPCompleter.suppress_competing_matchers = False``.
1761 1786 To disable the heuristic for specific matchers provide a dictionary mapping:
1762 1787 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1763 1788
1764 1789 Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1765 1790 completions to the set of matchers with the highest priority;
1766 1791 this is equivalent to ``IPCompleter.merge_completions`` and
1767 1792 can be beneficial for performance, but will sometimes omit relevant
1768 1793 candidates from matchers further down the priority list.
1769 1794 """,
1770 1795 ).tag(config=True)
1771 1796
1772 1797 merge_completions = Bool(
1773 1798 True,
1774 1799 help="""Whether to merge completion results into a single list
1775 1800
1776 1801 If False, only the completion results from the first non-empty
1777 1802 completer will be returned.
1778 1803
1779 1804 As of version 8.6.0, setting the value to ``False`` is an alias for:
1780 1805 ``IPCompleter.suppress_competing_matchers = True.``.
1781 1806 """,
1782 1807 ).tag(config=True)
1783 1808
1784 1809 disable_matchers = ListTrait(
1785 1810 Unicode(),
1786 1811 help="""List of matchers to disable.
1787 1812
1788 1813 The list should contain matcher identifiers (see :any:`completion_matcher`).
1789 1814 """,
1790 1815 ).tag(config=True)
1791 1816
1792 1817 omit__names = Enum(
1793 1818 (0, 1, 2),
1794 1819 default_value=2,
1795 1820 help="""Instruct the completer to omit private method names
1796 1821
1797 1822 Specifically, when completing on ``object.<tab>``.
1798 1823
1799 1824 When 2 [default]: all names that start with '_' will be excluded.
1800 1825
1801 1826 When 1: all 'magic' names (``__foo__``) will be excluded.
1802 1827
1803 1828 When 0: nothing will be excluded.
1804 1829 """
1805 1830 ).tag(config=True)
1806 1831 limit_to__all__ = Bool(False,
1807 1832 help="""
1808 1833 DEPRECATED as of version 5.0.
1809 1834
1810 1835 Instruct the completer to use __all__ for the completion
1811 1836
1812 1837 Specifically, when completing on ``object.<tab>``.
1813 1838
1814 1839 When True: only those names in obj.__all__ will be included.
1815 1840
1816 1841 When False [default]: the __all__ attribute is ignored
1817 1842 """,
1818 1843 ).tag(config=True)
1819 1844
1820 1845 profile_completions = Bool(
1821 1846 default_value=False,
1822 1847 help="If True, emit profiling data for completion subsystem using cProfile."
1823 1848 ).tag(config=True)
1824 1849
1825 1850 profiler_output_dir = Unicode(
1826 1851 default_value=".completion_profiles",
1827 1852 help="Template for path at which to output profile data for completions."
1828 1853 ).tag(config=True)
1829 1854
1830 1855 @observe('limit_to__all__')
1831 1856 def _limit_to_all_changed(self, change):
1832 1857 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1833 1858 'value has been deprecated since IPython 5.0, will be made to have '
1834 1859 'no effects and then removed in future version of IPython.',
1835 1860 UserWarning)
1836 1861
1837 1862 def __init__(
1838 1863 self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1839 1864 ):
1840 1865 """IPCompleter() -> completer
1841 1866
1842 1867 Return a completer object.
1843 1868
1844 1869 Parameters
1845 1870 ----------
1846 1871 shell
1847 1872 a pointer to the ipython shell itself. This is needed
1848 1873 because this completer knows about magic functions, and those can
1849 1874 only be accessed via the ipython instance.
1850 1875 namespace : dict, optional
1851 1876 an optional dict where completions are performed.
1852 1877 global_namespace : dict, optional
1853 1878 secondary optional dict for completions, to
1854 1879 handle cases (such as IPython embedded inside functions) where
1855 1880 both Python scopes are visible.
1856 1881 config : Config
1857 1882 traitlet's config object
1858 1883 **kwargs
1859 1884 passed to super class unmodified.
1860 1885 """
1861 1886
1862 1887 self.magic_escape = ESC_MAGIC
1863 1888 self.splitter = CompletionSplitter()
1864 1889
1865 1890 # _greedy_changed() depends on splitter and readline being defined:
1866 1891 super().__init__(
1867 1892 namespace=namespace,
1868 1893 global_namespace=global_namespace,
1869 1894 config=config,
1870 1895 **kwargs,
1871 1896 )
1872 1897
1873 1898 # List where completion matches will be stored
1874 1899 self.matches = []
1875 1900 self.shell = shell
1876 1901 # Regexp to split filenames with spaces in them
1877 1902 self.space_name_re = re.compile(r'([^\\] )')
1878 1903 # Hold a local ref. to glob.glob for speed
1879 1904 self.glob = glob.glob
1880 1905
1881 1906 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1882 1907 # buffers, to avoid completion problems.
1883 1908 term = os.environ.get('TERM','xterm')
1884 1909 self.dumb_terminal = term in ['dumb','emacs']
1885 1910
1886 1911 # Special handling of backslashes needed in win32 platforms
1887 1912 if sys.platform == "win32":
1888 1913 self.clean_glob = self._clean_glob_win32
1889 1914 else:
1890 1915 self.clean_glob = self._clean_glob
1891 1916
1892 1917 #regexp to parse docstring for function signature
1893 1918 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1894 1919 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1895 1920 #use this if positional argument name is also needed
1896 1921 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1897 1922
1898 1923 self.magic_arg_matchers = [
1899 1924 self.magic_config_matcher,
1900 1925 self.magic_color_matcher,
1901 1926 ]
1902 1927
1903 1928 # This is set externally by InteractiveShell
1904 1929 self.custom_completers = None
1905 1930
1906 1931 # This is a list of names of unicode characters that can be completed
1907 1932 # into their corresponding unicode value. The list is large, so we
1908 1933 # lazily initialize it on first use. Consuming code should access this
1909 1934 # attribute through the `@unicode_names` property.
1910 1935 self._unicode_names = None
1911 1936
1912 1937 self._backslash_combining_matchers = [
1913 1938 self.latex_name_matcher,
1914 1939 self.unicode_name_matcher,
1915 1940 back_latex_name_matcher,
1916 1941 back_unicode_name_matcher,
1917 1942 self.fwd_unicode_matcher,
1918 1943 ]
1919 1944
1920 1945 if not self.backslash_combining_completions:
1921 1946 for matcher in self._backslash_combining_matchers:
1922 1947 self.disable_matchers.append(_get_matcher_id(matcher))
1923 1948
1924 1949 if not self.merge_completions:
1925 1950 self.suppress_competing_matchers = True
1926 1951
1927 1952 @property
1928 1953 def matchers(self) -> List[Matcher]:
1929 1954 """All active matcher routines for completion"""
1930 1955 if self.dict_keys_only:
1931 1956 return [self.dict_key_matcher]
1932 1957
1933 1958 if self.use_jedi:
1934 1959 return [
1935 1960 *self.custom_matchers,
1936 1961 *self._backslash_combining_matchers,
1937 1962 *self.magic_arg_matchers,
1938 1963 self.custom_completer_matcher,
1939 1964 self.magic_matcher,
1940 1965 self._jedi_matcher,
1941 1966 self.dict_key_matcher,
1942 1967 self.file_matcher,
1943 1968 ]
1944 1969 else:
1945 1970 return [
1946 1971 *self.custom_matchers,
1947 1972 *self._backslash_combining_matchers,
1948 1973 *self.magic_arg_matchers,
1949 1974 self.custom_completer_matcher,
1950 1975 self.dict_key_matcher,
1951 1976 # TODO: convert python_matches to v2 API
1952 1977 self.magic_matcher,
1953 1978 self.python_matches,
1954 1979 self.file_matcher,
1955 1980 self.python_func_kw_matcher,
1956 1981 ]
1957 1982
1958 1983 def all_completions(self, text:str) -> List[str]:
1959 1984 """
1960 1985 Wrapper around the completion methods for the benefit of emacs.
1961 1986 """
1962 1987 prefix = text.rpartition('.')[0]
1963 1988 with provisionalcompleter():
1964 1989 return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
1965 1990 for c in self.completions(text, len(text))]
1966 1991
1967 1992 return self.complete(text)[1]
1968 1993
1969 1994 def _clean_glob(self, text:str):
1970 1995 return self.glob("%s*" % text)
1971 1996
1972 1997 def _clean_glob_win32(self, text:str):
1973 1998 return [f.replace("\\","/")
1974 1999 for f in self.glob("%s*" % text)]
1975 2000
1976 2001 @context_matcher()
1977 2002 def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
1978 2003 """Same as :any:`file_matches`, but adopted to new Matcher API."""
1979 2004 matches = self.file_matches(context.token)
1980 2005 # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
1981 2006 # starts with `/home/`, `C:\`, etc)
1982 2007 return _convert_matcher_v1_result_to_v2(matches, type="path")
1983 2008
1984 2009 def file_matches(self, text: str) -> List[str]:
1985 2010 """Match filenames, expanding ~USER type strings.
1986 2011
1987 2012 Most of the seemingly convoluted logic in this completer is an
1988 2013 attempt to handle filenames with spaces in them. And yet it's not
1989 2014 quite perfect, because Python's readline doesn't expose all of the
1990 2015 GNU readline details needed for this to be done correctly.
1991 2016
1992 2017 For a filename with a space in it, the printed completions will be
1993 2018 only the parts after what's already been typed (instead of the
1994 2019 full completions, as is normally done). I don't think with the
1995 2020 current (as of Python 2.3) Python readline it's possible to do
1996 2021 better.
1997 2022
1998 2023 .. deprecated:: 8.6
1999 2024 You can use :meth:`file_matcher` instead.
2000 2025 """
2001 2026
2002 2027 # chars that require escaping with backslash - i.e. chars
2003 2028 # that readline treats incorrectly as delimiters, but we
2004 2029 # don't want to treat as delimiters in filename matching
2005 2030 # when escaped with backslash
2006 2031 if text.startswith('!'):
2007 2032 text = text[1:]
2008 2033 text_prefix = u'!'
2009 2034 else:
2010 2035 text_prefix = u''
2011 2036
2012 2037 text_until_cursor = self.text_until_cursor
2013 2038 # track strings with open quotes
2014 2039 open_quotes = has_open_quotes(text_until_cursor)
2015 2040
2016 2041 if '(' in text_until_cursor or '[' in text_until_cursor:
2017 2042 lsplit = text
2018 2043 else:
2019 2044 try:
2020 2045 # arg_split ~ shlex.split, but with unicode bugs fixed by us
2021 2046 lsplit = arg_split(text_until_cursor)[-1]
2022 2047 except ValueError:
2023 2048 # typically an unmatched ", or backslash without escaped char.
2024 2049 if open_quotes:
2025 2050 lsplit = text_until_cursor.split(open_quotes)[-1]
2026 2051 else:
2027 2052 return []
2028 2053 except IndexError:
2029 2054 # tab pressed on empty line
2030 2055 lsplit = ""
2031 2056
2032 2057 if not open_quotes and lsplit != protect_filename(lsplit):
2033 2058 # if protectables are found, do matching on the whole escaped name
2034 2059 has_protectables = True
2035 2060 text0,text = text,lsplit
2036 2061 else:
2037 2062 has_protectables = False
2038 2063 text = os.path.expanduser(text)
2039 2064
2040 2065 if text == "":
2041 2066 return [text_prefix + protect_filename(f) for f in self.glob("*")]
2042 2067
2043 2068 # Compute the matches from the filesystem
2044 2069 if sys.platform == 'win32':
2045 2070 m0 = self.clean_glob(text)
2046 2071 else:
2047 2072 m0 = self.clean_glob(text.replace('\\', ''))
2048 2073
2049 2074 if has_protectables:
2050 2075 # If we had protectables, we need to revert our changes to the
2051 2076 # beginning of filename so that we don't double-write the part
2052 2077 # of the filename we have so far
2053 2078 len_lsplit = len(lsplit)
2054 2079 matches = [text_prefix + text0 +
2055 2080 protect_filename(f[len_lsplit:]) for f in m0]
2056 2081 else:
2057 2082 if open_quotes:
2058 2083 # if we have a string with an open quote, we don't need to
2059 2084 # protect the names beyond the quote (and we _shouldn't_, as
2060 2085 # it would cause bugs when the filesystem call is made).
2061 2086 matches = m0 if sys.platform == "win32" else\
2062 2087 [protect_filename(f, open_quotes) for f in m0]
2063 2088 else:
2064 2089 matches = [text_prefix +
2065 2090 protect_filename(f) for f in m0]
2066 2091
2067 2092 # Mark directories in input list by appending '/' to their names.
2068 2093 return [x+'/' if os.path.isdir(x) else x for x in matches]
2069 2094
2070 2095 @context_matcher()
2071 2096 def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2072 2097 """Match magics."""
2073 2098 text = context.token
2074 2099 matches = self.magic_matches(text)
2075 2100 result = _convert_matcher_v1_result_to_v2(matches, type="magic")
2076 2101 is_magic_prefix = len(text) > 0 and text[0] == "%"
2077 2102 result["suppress"] = is_magic_prefix and bool(result["completions"])
2078 2103 return result
2079 2104
2080 2105 def magic_matches(self, text: str):
2081 2106 """Match magics.
2082 2107
2083 2108 .. deprecated:: 8.6
2084 2109 You can use :meth:`magic_matcher` instead.
2085 2110 """
2086 2111 # Get all shell magics now rather than statically, so magics loaded at
2087 2112 # runtime show up too.
2088 2113 lsm = self.shell.magics_manager.lsmagic()
2089 2114 line_magics = lsm['line']
2090 2115 cell_magics = lsm['cell']
2091 2116 pre = self.magic_escape
2092 2117 pre2 = pre+pre
2093 2118
2094 2119 explicit_magic = text.startswith(pre)
2095 2120
2096 2121 # Completion logic:
2097 2122 # - user gives %%: only do cell magics
2098 2123 # - user gives %: do both line and cell magics
2099 2124 # - no prefix: do both
2100 2125 # In other words, line magics are skipped if the user gives %% explicitly
2101 2126 #
2102 2127 # We also exclude magics that match any currently visible names:
2103 2128 # https://github.com/ipython/ipython/issues/4877, unless the user has
2104 2129 # typed a %:
2105 2130 # https://github.com/ipython/ipython/issues/10754
2106 2131 bare_text = text.lstrip(pre)
2107 2132 global_matches = self.global_matches(bare_text)
2108 2133 if not explicit_magic:
2109 2134 def matches(magic):
2110 2135 """
2111 2136 Filter magics, in particular remove magics that match
2112 2137 a name present in global namespace.
2113 2138 """
2114 2139 return ( magic.startswith(bare_text) and
2115 2140 magic not in global_matches )
2116 2141 else:
2117 2142 def matches(magic):
2118 2143 return magic.startswith(bare_text)
2119 2144
2120 2145 comp = [ pre2+m for m in cell_magics if matches(m)]
2121 2146 if not text.startswith(pre2):
2122 2147 comp += [ pre+m for m in line_magics if matches(m)]
2123 2148
2124 2149 return comp
2125 2150
2126 2151 @context_matcher()
2127 2152 def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2128 2153 """Match class names and attributes for %config magic."""
2129 2154 # NOTE: uses `line_buffer` equivalent for compatibility
2130 2155 matches = self.magic_config_matches(context.line_with_cursor)
2131 2156 return _convert_matcher_v1_result_to_v2(matches, type="param")
2132 2157
2133 2158 def magic_config_matches(self, text: str) -> List[str]:
2134 2159 """Match class names and attributes for %config magic.
2135 2160
2136 2161 .. deprecated:: 8.6
2137 2162 You can use :meth:`magic_config_matcher` instead.
2138 2163 """
2139 2164 texts = text.strip().split()
2140 2165
2141 2166 if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
2142 2167 # get all configuration classes
2143 2168 classes = sorted(set([ c for c in self.shell.configurables
2144 2169 if c.__class__.class_traits(config=True)
2145 2170 ]), key=lambda x: x.__class__.__name__)
2146 2171 classnames = [ c.__class__.__name__ for c in classes ]
2147 2172
2148 2173 # return all classnames if config or %config is given
2149 2174 if len(texts) == 1:
2150 2175 return classnames
2151 2176
2152 2177 # match classname
2153 2178 classname_texts = texts[1].split('.')
2154 2179 classname = classname_texts[0]
2155 2180 classname_matches = [ c for c in classnames
2156 2181 if c.startswith(classname) ]
2157 2182
2158 2183 # return matched classes or the matched class with attributes
2159 2184 if texts[1].find('.') < 0:
2160 2185 return classname_matches
2161 2186 elif len(classname_matches) == 1 and \
2162 2187 classname_matches[0] == classname:
2163 2188 cls = classes[classnames.index(classname)].__class__
2164 2189 help = cls.class_get_help()
2165 2190 # strip leading '--' from cl-args:
2166 2191 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
2167 2192 return [ attr.split('=')[0]
2168 2193 for attr in help.strip().splitlines()
2169 2194 if attr.startswith(texts[1]) ]
2170 2195 return []
2171 2196
2172 2197 @context_matcher()
2173 2198 def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2174 2199 """Match color schemes for %colors magic."""
2175 2200 # NOTE: uses `line_buffer` equivalent for compatibility
2176 2201 matches = self.magic_color_matches(context.line_with_cursor)
2177 2202 return _convert_matcher_v1_result_to_v2(matches, type="param")
2178 2203
2179 2204 def magic_color_matches(self, text: str) -> List[str]:
2180 2205 """Match color schemes for %colors magic.
2181 2206
2182 2207 .. deprecated:: 8.6
2183 2208 You can use :meth:`magic_color_matcher` instead.
2184 2209 """
2185 2210 texts = text.split()
2186 2211 if text.endswith(' '):
2187 2212 # .split() strips off the trailing whitespace. Add '' back
2188 2213 # so that: '%colors ' -> ['%colors', '']
2189 2214 texts.append('')
2190 2215
2191 2216 if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
2192 2217 prefix = texts[1]
2193 2218 return [ color for color in InspectColors.keys()
2194 2219 if color.startswith(prefix) ]
2195 2220 return []
2196 2221
2197 2222 @context_matcher(identifier="IPCompleter.jedi_matcher")
2198 2223 def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
2199 2224 matches = self._jedi_matches(
2200 2225 cursor_column=context.cursor_position,
2201 2226 cursor_line=context.cursor_line,
2202 2227 text=context.full_text,
2203 2228 )
2204 2229 return {
2205 2230 "completions": matches,
2206 2231 # static analysis should not suppress other matchers
2207 2232 "suppress": False,
2208 2233 }
2209 2234
2210 2235 def _jedi_matches(
2211 2236 self, cursor_column: int, cursor_line: int, text: str
2212 2237 ) -> Iterator[_JediCompletionLike]:
2213 2238 """
2214 2239 Return a list of :any:`jedi.api.Completion`s object from a ``text`` and
2215 2240 cursor position.
2216 2241
2217 2242 Parameters
2218 2243 ----------
2219 2244 cursor_column : int
2220 2245 column position of the cursor in ``text``, 0-indexed.
2221 2246 cursor_line : int
2222 2247 line position of the cursor in ``text``, 0-indexed
2223 2248 text : str
2224 2249 text to complete
2225 2250
2226 2251 Notes
2227 2252 -----
2228 2253 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
2229 2254 object containing a string with the Jedi debug information attached.
2230 2255
2231 2256 .. deprecated:: 8.6
2232 2257 You can use :meth:`_jedi_matcher` instead.
2233 2258 """
2234 2259 namespaces = [self.namespace]
2235 2260 if self.global_namespace is not None:
2236 2261 namespaces.append(self.global_namespace)
2237 2262
2238 2263 completion_filter = lambda x:x
2239 2264 offset = cursor_to_position(text, cursor_line, cursor_column)
2240 2265 # filter output if we are completing for object members
2241 2266 if offset:
2242 2267 pre = text[offset-1]
2243 2268 if pre == '.':
2244 2269 if self.omit__names == 2:
2245 2270 completion_filter = lambda c:not c.name.startswith('_')
2246 2271 elif self.omit__names == 1:
2247 2272 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
2248 2273 elif self.omit__names == 0:
2249 2274 completion_filter = lambda x:x
2250 2275 else:
2251 2276 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
2252 2277
2253 2278 interpreter = jedi.Interpreter(text[:offset], namespaces)
2254 2279 try_jedi = True
2255 2280
2256 2281 try:
2257 2282 # find the first token in the current tree -- if it is a ' or " then we are in a string
2258 2283 completing_string = False
2259 2284 try:
2260 2285 first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
2261 2286 except StopIteration:
2262 2287 pass
2263 2288 else:
2264 2289 # note the value may be ', ", or it may also be ''' or """, or
2265 2290 # in some cases, """what/you/typed..., but all of these are
2266 2291 # strings.
2267 2292 completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
2268 2293
2269 2294 # if we are in a string jedi is likely not the right candidate for
2270 2295 # now. Skip it.
2271 2296 try_jedi = not completing_string
2272 2297 except Exception as e:
2273 2298 # many of things can go wrong, we are using private API just don't crash.
2274 2299 if self.debug:
2275 2300 print("Error detecting if completing a non-finished string :", e, '|')
2276 2301
2277 2302 if not try_jedi:
2278 2303 return iter([])
2279 2304 try:
2280 2305 return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
2281 2306 except Exception as e:
2282 2307 if self.debug:
2283 2308 return iter(
2284 2309 [
2285 2310 _FakeJediCompletion(
2286 2311 'Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""'
2287 2312 % (e)
2288 2313 )
2289 2314 ]
2290 2315 )
2291 2316 else:
2292 2317 return iter([])
2293 2318
2294 2319 @completion_matcher(api_version=1)
2295 2320 def python_matches(self, text: str) -> Iterable[str]:
2296 2321 """Match attributes or global python names"""
2297 2322 if "." in text:
2298 2323 try:
2299 2324 matches = self.attr_matches(text)
2300 2325 if text.endswith('.') and self.omit__names:
2301 2326 if self.omit__names == 1:
2302 2327 # true if txt is _not_ a __ name, false otherwise:
2303 2328 no__name = (lambda txt:
2304 2329 re.match(r'.*\.__.*?__',txt) is None)
2305 2330 else:
2306 2331 # true if txt is _not_ a _ name, false otherwise:
2307 2332 no__name = (lambda txt:
2308 2333 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
2309 2334 matches = filter(no__name, matches)
2310 2335 except NameError:
2311 2336 # catches <undefined attributes>.<tab>
2312 2337 matches = []
2313 2338 else:
2314 2339 matches = self.global_matches(text)
2315 2340 return matches
2316 2341
2317 2342 def _default_arguments_from_docstring(self, doc):
2318 2343 """Parse the first line of docstring for call signature.
2319 2344
2320 2345 Docstring should be of the form 'min(iterable[, key=func])\n'.
2321 2346 It can also parse cython docstring of the form
2322 2347 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
2323 2348 """
2324 2349 if doc is None:
2325 2350 return []
2326 2351
2327 2352 #care only the firstline
2328 2353 line = doc.lstrip().splitlines()[0]
2329 2354
2330 2355 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
2331 2356 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
2332 2357 sig = self.docstring_sig_re.search(line)
2333 2358 if sig is None:
2334 2359 return []
2335 2360 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
2336 2361 sig = sig.groups()[0].split(',')
2337 2362 ret = []
2338 2363 for s in sig:
2339 2364 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
2340 2365 ret += self.docstring_kwd_re.findall(s)
2341 2366 return ret
2342 2367
2343 2368 def _default_arguments(self, obj):
2344 2369 """Return the list of default arguments of obj if it is callable,
2345 2370 or empty list otherwise."""
2346 2371 call_obj = obj
2347 2372 ret = []
2348 2373 if inspect.isbuiltin(obj):
2349 2374 pass
2350 2375 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
2351 2376 if inspect.isclass(obj):
2352 2377 #for cython embedsignature=True the constructor docstring
2353 2378 #belongs to the object itself not __init__
2354 2379 ret += self._default_arguments_from_docstring(
2355 2380 getattr(obj, '__doc__', ''))
2356 2381 # for classes, check for __init__,__new__
2357 2382 call_obj = (getattr(obj, '__init__', None) or
2358 2383 getattr(obj, '__new__', None))
2359 2384 # for all others, check if they are __call__able
2360 2385 elif hasattr(obj, '__call__'):
2361 2386 call_obj = obj.__call__
2362 2387 ret += self._default_arguments_from_docstring(
2363 2388 getattr(call_obj, '__doc__', ''))
2364 2389
2365 2390 _keeps = (inspect.Parameter.KEYWORD_ONLY,
2366 2391 inspect.Parameter.POSITIONAL_OR_KEYWORD)
2367 2392
2368 2393 try:
2369 2394 sig = inspect.signature(obj)
2370 2395 ret.extend(k for k, v in sig.parameters.items() if
2371 2396 v.kind in _keeps)
2372 2397 except ValueError:
2373 2398 pass
2374 2399
2375 2400 return list(set(ret))
2376 2401
2377 2402 @context_matcher()
2378 2403 def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2379 2404 """Match named parameters (kwargs) of the last open function."""
2380 2405 matches = self.python_func_kw_matches(context.token)
2381 2406 return _convert_matcher_v1_result_to_v2(matches, type="param")
2382 2407
2383 2408 def python_func_kw_matches(self, text):
2384 2409 """Match named parameters (kwargs) of the last open function.
2385 2410
2386 2411 .. deprecated:: 8.6
2387 2412 You can use :meth:`python_func_kw_matcher` instead.
2388 2413 """
2389 2414
2390 2415 if "." in text: # a parameter cannot be dotted
2391 2416 return []
2392 2417 try: regexp = self.__funcParamsRegex
2393 2418 except AttributeError:
2394 2419 regexp = self.__funcParamsRegex = re.compile(r'''
2395 2420 '.*?(?<!\\)' | # single quoted strings or
2396 2421 ".*?(?<!\\)" | # double quoted strings or
2397 2422 \w+ | # identifier
2398 2423 \S # other characters
2399 2424 ''', re.VERBOSE | re.DOTALL)
2400 2425 # 1. find the nearest identifier that comes before an unclosed
2401 2426 # parenthesis before the cursor
2402 2427 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
2403 2428 tokens = regexp.findall(self.text_until_cursor)
2404 2429 iterTokens = reversed(tokens); openPar = 0
2405 2430
2406 2431 for token in iterTokens:
2407 2432 if token == ')':
2408 2433 openPar -= 1
2409 2434 elif token == '(':
2410 2435 openPar += 1
2411 2436 if openPar > 0:
2412 2437 # found the last unclosed parenthesis
2413 2438 break
2414 2439 else:
2415 2440 return []
2416 2441 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
2417 2442 ids = []
2418 2443 isId = re.compile(r'\w+$').match
2419 2444
2420 2445 while True:
2421 2446 try:
2422 2447 ids.append(next(iterTokens))
2423 2448 if not isId(ids[-1]):
2424 2449 ids.pop(); break
2425 2450 if not next(iterTokens) == '.':
2426 2451 break
2427 2452 except StopIteration:
2428 2453 break
2429 2454
2430 2455 # Find all named arguments already assigned to, as to avoid suggesting
2431 2456 # them again
2432 2457 usedNamedArgs = set()
2433 2458 par_level = -1
2434 2459 for token, next_token in zip(tokens, tokens[1:]):
2435 2460 if token == '(':
2436 2461 par_level += 1
2437 2462 elif token == ')':
2438 2463 par_level -= 1
2439 2464
2440 2465 if par_level != 0:
2441 2466 continue
2442 2467
2443 2468 if next_token != '=':
2444 2469 continue
2445 2470
2446 2471 usedNamedArgs.add(token)
2447 2472
2448 2473 argMatches = []
2449 2474 try:
2450 2475 callableObj = '.'.join(ids[::-1])
2451 2476 namedArgs = self._default_arguments(eval(callableObj,
2452 2477 self.namespace))
2453 2478
2454 2479 # Remove used named arguments from the list, no need to show twice
2455 2480 for namedArg in set(namedArgs) - usedNamedArgs:
2456 2481 if namedArg.startswith(text):
2457 2482 argMatches.append("%s=" %namedArg)
2458 2483 except:
2459 2484 pass
2460 2485
2461 2486 return argMatches
2462 2487
2463 2488 @staticmethod
2464 2489 def _get_keys(obj: Any) -> List[Any]:
2465 2490 # Objects can define their own completions by defining an
2466 2491 # _ipy_key_completions_() method.
2467 2492 method = get_real_method(obj, '_ipython_key_completions_')
2468 2493 if method is not None:
2469 2494 return method()
2470 2495
2471 2496 # Special case some common in-memory dict-like types
2472 2497 if isinstance(obj, dict) or _safe_isinstance(obj, "pandas", "DataFrame"):
2473 2498 try:
2474 2499 return list(obj.keys())
2475 2500 except Exception:
2476 2501 return []
2477 2502 elif _safe_isinstance(obj, "pandas", "core", "indexing", "_LocIndexer"):
2478 2503 try:
2479 2504 return list(obj.obj.keys())
2480 2505 except Exception:
2481 2506 return []
2482 2507 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
2483 2508 _safe_isinstance(obj, 'numpy', 'void'):
2484 2509 return obj.dtype.names or []
2485 2510 return []
2486 2511
2487 2512 @context_matcher()
2488 2513 def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2489 2514 """Match string keys in a dictionary, after e.g. ``foo[``."""
2490 2515 matches = self.dict_key_matches(context.token)
2491 2516 return _convert_matcher_v1_result_to_v2(
2492 2517 matches, type="dict key", suppress_if_matches=True
2493 2518 )
2494 2519
2495 2520 def dict_key_matches(self, text: str) -> List[str]:
2496 2521 """Match string keys in a dictionary, after e.g. ``foo[``.
2497 2522
2498 2523 .. deprecated:: 8.6
2499 2524 You can use :meth:`dict_key_matcher` instead.
2500 2525 """
2501 2526
2502 2527 # Short-circuit on closed dictionary (regular expression would
2503 2528 # not match anyway, but would take quite a while).
2504 2529 if self.text_until_cursor.strip().endswith("]"):
2505 2530 return []
2506 2531
2507 2532 match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
2508 2533
2509 2534 if match is None:
2510 2535 return []
2511 2536
2512 2537 expr, prior_tuple_keys, key_prefix = match.groups()
2513 2538
2514 2539 obj = self._evaluate_expr(expr)
2515 2540
2516 2541 if obj is not_found:
2517 2542 return []
2518 2543
2519 2544 keys = self._get_keys(obj)
2520 2545 if not keys:
2521 2546 return keys
2522 2547
2523 2548 tuple_prefix = guarded_eval(
2524 2549 prior_tuple_keys,
2525 2550 EvaluationContext(
2526 2551 globals=self.global_namespace,
2527 2552 locals=self.namespace,
2528 2553 evaluation=self.evaluation,
2529 2554 in_subscript=True,
2530 2555 ),
2531 2556 )
2532 2557
2533 2558 closing_quote, token_offset, matches = match_dict_keys(
2534 2559 keys, key_prefix, self.splitter.delims, extra_prefix=tuple_prefix
2535 2560 )
2536 2561 if not matches:
2537 2562 return []
2538 2563
2539 2564 # get the cursor position of
2540 2565 # - the text being completed
2541 2566 # - the start of the key text
2542 2567 # - the start of the completion
2543 2568 text_start = len(self.text_until_cursor) - len(text)
2544 2569 if key_prefix:
2545 2570 key_start = match.start(3)
2546 2571 completion_start = key_start + token_offset
2547 2572 else:
2548 2573 key_start = completion_start = match.end()
2549 2574
2550 2575 # grab the leading prefix, to make sure all completions start with `text`
2551 2576 if text_start > key_start:
2552 2577 leading = ''
2553 2578 else:
2554 2579 leading = text[text_start:completion_start]
2555 2580
2556 2581 # append closing quote and bracket as appropriate
2557 2582 # this is *not* appropriate if the opening quote or bracket is outside
2558 2583 # the text given to this method, e.g. `d["""a\nt
2559 2584 can_close_quote = False
2560 2585 can_close_bracket = False
2561 2586
2562 2587 continuation = self.line_buffer[len(self.text_until_cursor) :].strip()
2563 2588
2564 2589 if continuation.startswith(closing_quote):
2565 2590 # do not close if already closed, e.g. `d['a<tab>'`
2566 2591 continuation = continuation[len(closing_quote) :]
2567 2592 else:
2568 2593 can_close_quote = True
2569 2594
2570 2595 continuation = continuation.strip()
2571 2596
2572 2597 # e.g. `pandas.DataFrame` has different tuple indexer behaviour,
2573 2598 # handling it is out of scope, so let's avoid appending suffixes.
2574 2599 has_known_tuple_handling = isinstance(obj, dict)
2575 2600
2576 2601 can_close_bracket = (
2577 2602 not continuation.startswith("]") and self.auto_close_dict_keys
2578 2603 )
2579 2604 can_close_tuple_item = (
2580 2605 not continuation.startswith(",")
2581 2606 and has_known_tuple_handling
2582 2607 and self.auto_close_dict_keys
2583 2608 )
2584 2609 can_close_quote = can_close_quote and self.auto_close_dict_keys
2585 2610
2586 2611 # fast path if closing qoute should be appended but not suffix is allowed
2587 2612 if not can_close_quote and not can_close_bracket and closing_quote:
2588 2613 return [leading + k for k in matches]
2589 2614
2590 2615 results = []
2591 2616
2592 2617 end_of_tuple_or_item = _DictKeyState.END_OF_TUPLE | _DictKeyState.END_OF_ITEM
2593 2618
2594 2619 for k, state_flag in matches.items():
2595 2620 result = leading + k
2596 2621 if can_close_quote and closing_quote:
2597 2622 result += closing_quote
2598 2623
2599 2624 if state_flag == end_of_tuple_or_item:
2600 2625 # We do not know which suffix to add,
2601 2626 # e.g. both tuple item and string
2602 2627 # match this item.
2603 2628 pass
2604 2629
2605 2630 if state_flag in end_of_tuple_or_item and can_close_bracket:
2606 2631 result += "]"
2607 2632 if state_flag == _DictKeyState.IN_TUPLE and can_close_tuple_item:
2608 2633 result += ", "
2609 2634 results.append(result)
2610 2635 return results
2611 2636
2612 2637 @context_matcher()
2613 2638 def unicode_name_matcher(self, context: CompletionContext):
2614 2639 """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2615 2640 fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2616 2641 return _convert_matcher_v1_result_to_v2(
2617 2642 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2618 2643 )
2619 2644
2620 2645 @staticmethod
2621 2646 def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
2622 2647 """Match Latex-like syntax for unicode characters base
2623 2648 on the name of the character.
2624 2649
2625 2650 This does ``\\GREEK SMALL LETTER ETA`` -> ``Ξ·``
2626 2651
2627 2652 Works only on valid python 3 identifier, or on combining characters that
2628 2653 will combine to form a valid identifier.
2629 2654 """
2630 2655 slashpos = text.rfind('\\')
2631 2656 if slashpos > -1:
2632 2657 s = text[slashpos+1:]
2633 2658 try :
2634 2659 unic = unicodedata.lookup(s)
2635 2660 # allow combining chars
2636 2661 if ('a'+unic).isidentifier():
2637 2662 return '\\'+s,[unic]
2638 2663 except KeyError:
2639 2664 pass
2640 2665 return '', []
2641 2666
2642 2667 @context_matcher()
2643 2668 def latex_name_matcher(self, context: CompletionContext):
2644 2669 """Match Latex syntax for unicode characters.
2645 2670
2646 2671 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2647 2672 """
2648 2673 fragment, matches = self.latex_matches(context.text_until_cursor)
2649 2674 return _convert_matcher_v1_result_to_v2(
2650 2675 matches, type="latex", fragment=fragment, suppress_if_matches=True
2651 2676 )
2652 2677
2653 2678 def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
2654 2679 """Match Latex syntax for unicode characters.
2655 2680
2656 2681 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
2657 2682
2658 2683 .. deprecated:: 8.6
2659 2684 You can use :meth:`latex_name_matcher` instead.
2660 2685 """
2661 2686 slashpos = text.rfind('\\')
2662 2687 if slashpos > -1:
2663 2688 s = text[slashpos:]
2664 2689 if s in latex_symbols:
2665 2690 # Try to complete a full latex symbol to unicode
2666 2691 # \\alpha -> Ξ±
2667 2692 return s, [latex_symbols[s]]
2668 2693 else:
2669 2694 # If a user has partially typed a latex symbol, give them
2670 2695 # a full list of options \al -> [\aleph, \alpha]
2671 2696 matches = [k for k in latex_symbols if k.startswith(s)]
2672 2697 if matches:
2673 2698 return s, matches
2674 2699 return '', ()
2675 2700
2676 2701 @context_matcher()
2677 2702 def custom_completer_matcher(self, context):
2678 2703 """Dispatch custom completer.
2679 2704
2680 2705 If a match is found, suppresses all other matchers except for Jedi.
2681 2706 """
2682 2707 matches = self.dispatch_custom_completer(context.token) or []
2683 2708 result = _convert_matcher_v1_result_to_v2(
2684 2709 matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2685 2710 )
2686 2711 result["ordered"] = True
2687 2712 result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2688 2713 return result
2689 2714
2690 2715 def dispatch_custom_completer(self, text):
2691 2716 """
2692 2717 .. deprecated:: 8.6
2693 2718 You can use :meth:`custom_completer_matcher` instead.
2694 2719 """
2695 2720 if not self.custom_completers:
2696 2721 return
2697 2722
2698 2723 line = self.line_buffer
2699 2724 if not line.strip():
2700 2725 return None
2701 2726
2702 2727 # Create a little structure to pass all the relevant information about
2703 2728 # the current completion to any custom completer.
2704 2729 event = SimpleNamespace()
2705 2730 event.line = line
2706 2731 event.symbol = text
2707 2732 cmd = line.split(None,1)[0]
2708 2733 event.command = cmd
2709 2734 event.text_until_cursor = self.text_until_cursor
2710 2735
2711 2736 # for foo etc, try also to find completer for %foo
2712 2737 if not cmd.startswith(self.magic_escape):
2713 2738 try_magic = self.custom_completers.s_matches(
2714 2739 self.magic_escape + cmd)
2715 2740 else:
2716 2741 try_magic = []
2717 2742
2718 2743 for c in itertools.chain(self.custom_completers.s_matches(cmd),
2719 2744 try_magic,
2720 2745 self.custom_completers.flat_matches(self.text_until_cursor)):
2721 2746 try:
2722 2747 res = c(event)
2723 2748 if res:
2724 2749 # first, try case sensitive match
2725 2750 withcase = [r for r in res if r.startswith(text)]
2726 2751 if withcase:
2727 2752 return withcase
2728 2753 # if none, then case insensitive ones are ok too
2729 2754 text_low = text.lower()
2730 2755 return [r for r in res if r.lower().startswith(text_low)]
2731 2756 except TryNext:
2732 2757 pass
2733 2758 except KeyboardInterrupt:
2734 2759 """
2735 2760 If custom completer take too long,
2736 2761 let keyboard interrupt abort and return nothing.
2737 2762 """
2738 2763 break
2739 2764
2740 2765 return None
2741 2766
2742 2767 def completions(self, text: str, offset: int)->Iterator[Completion]:
2743 2768 """
2744 2769 Returns an iterator over the possible completions
2745 2770
2746 2771 .. warning::
2747 2772
2748 2773 Unstable
2749 2774
2750 2775 This function is unstable, API may change without warning.
2751 2776 It will also raise unless use in proper context manager.
2752 2777
2753 2778 Parameters
2754 2779 ----------
2755 2780 text : str
2756 2781 Full text of the current input, multi line string.
2757 2782 offset : int
2758 2783 Integer representing the position of the cursor in ``text``. Offset
2759 2784 is 0-based indexed.
2760 2785
2761 2786 Yields
2762 2787 ------
2763 2788 Completion
2764 2789
2765 2790 Notes
2766 2791 -----
2767 2792 The cursor on a text can either be seen as being "in between"
2768 2793 characters or "On" a character depending on the interface visible to
2769 2794 the user. For consistency the cursor being on "in between" characters X
2770 2795 and Y is equivalent to the cursor being "on" character Y, that is to say
2771 2796 the character the cursor is on is considered as being after the cursor.
2772 2797
2773 2798 Combining characters may span more that one position in the
2774 2799 text.
2775 2800
2776 2801 .. note::
2777 2802
2778 2803 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
2779 2804 fake Completion token to distinguish completion returned by Jedi
2780 2805 and usual IPython completion.
2781 2806
2782 2807 .. note::
2783 2808
2784 2809 Completions are not completely deduplicated yet. If identical
2785 2810 completions are coming from different sources this function does not
2786 2811 ensure that each completion object will only be present once.
2787 2812 """
2788 2813 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
2789 2814 "It may change without warnings. "
2790 2815 "Use in corresponding context manager.",
2791 2816 category=ProvisionalCompleterWarning, stacklevel=2)
2792 2817
2793 2818 seen = set()
2794 2819 profiler:Optional[cProfile.Profile]
2795 2820 try:
2796 2821 if self.profile_completions:
2797 2822 import cProfile
2798 2823 profiler = cProfile.Profile()
2799 2824 profiler.enable()
2800 2825 else:
2801 2826 profiler = None
2802 2827
2803 2828 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
2804 2829 if c and (c in seen):
2805 2830 continue
2806 2831 yield c
2807 2832 seen.add(c)
2808 2833 except KeyboardInterrupt:
2809 2834 """if completions take too long and users send keyboard interrupt,
2810 2835 do not crash and return ASAP. """
2811 2836 pass
2812 2837 finally:
2813 2838 if profiler is not None:
2814 2839 profiler.disable()
2815 2840 ensure_dir_exists(self.profiler_output_dir)
2816 2841 output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
2817 2842 print("Writing profiler output to", output_path)
2818 2843 profiler.dump_stats(output_path)
2819 2844
2820 2845 def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
2821 2846 """
2822 2847 Core completion module.Same signature as :any:`completions`, with the
2823 2848 extra `timeout` parameter (in seconds).
2824 2849
2825 2850 Computing jedi's completion ``.type`` can be quite expensive (it is a
2826 2851 lazy property) and can require some warm-up, more warm up than just
2827 2852 computing the ``name`` of a completion. The warm-up can be :
2828 2853
2829 2854 - Long warm-up the first time a module is encountered after
2830 2855 install/update: actually build parse/inference tree.
2831 2856
2832 2857 - first time the module is encountered in a session: load tree from
2833 2858 disk.
2834 2859
2835 2860 We don't want to block completions for tens of seconds so we give the
2836 2861 completer a "budget" of ``_timeout`` seconds per invocation to compute
2837 2862 completions types, the completions that have not yet been computed will
2838 2863 be marked as "unknown" an will have a chance to be computed next round
2839 2864 are things get cached.
2840 2865
2841 2866 Keep in mind that Jedi is not the only thing treating the completion so
2842 2867 keep the timeout short-ish as if we take more than 0.3 second we still
2843 2868 have lots of processing to do.
2844 2869
2845 2870 """
2846 2871 deadline = time.monotonic() + _timeout
2847 2872
2848 2873 before = full_text[:offset]
2849 2874 cursor_line, cursor_column = position_to_cursor(full_text, offset)
2850 2875
2851 2876 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2852 2877
2853 2878 def is_non_jedi_result(
2854 2879 result: MatcherResult, identifier: str
2855 2880 ) -> TypeGuard[SimpleMatcherResult]:
2856 2881 return identifier != jedi_matcher_id
2857 2882
2858 2883 results = self._complete(
2859 2884 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
2860 2885 )
2861 2886
2862 2887 non_jedi_results: Dict[str, SimpleMatcherResult] = {
2863 2888 identifier: result
2864 2889 for identifier, result in results.items()
2865 2890 if is_non_jedi_result(result, identifier)
2866 2891 }
2867 2892
2868 2893 jedi_matches = (
2869 2894 cast(_JediMatcherResult, results[jedi_matcher_id])["completions"]
2870 2895 if jedi_matcher_id in results
2871 2896 else ()
2872 2897 )
2873 2898
2874 2899 iter_jm = iter(jedi_matches)
2875 2900 if _timeout:
2876 2901 for jm in iter_jm:
2877 2902 try:
2878 2903 type_ = jm.type
2879 2904 except Exception:
2880 2905 if self.debug:
2881 2906 print("Error in Jedi getting type of ", jm)
2882 2907 type_ = None
2883 2908 delta = len(jm.name_with_symbols) - len(jm.complete)
2884 2909 if type_ == 'function':
2885 2910 signature = _make_signature(jm)
2886 2911 else:
2887 2912 signature = ''
2888 2913 yield Completion(start=offset - delta,
2889 2914 end=offset,
2890 2915 text=jm.name_with_symbols,
2891 2916 type=type_,
2892 2917 signature=signature,
2893 2918 _origin='jedi')
2894 2919
2895 2920 if time.monotonic() > deadline:
2896 2921 break
2897 2922
2898 2923 for jm in iter_jm:
2899 2924 delta = len(jm.name_with_symbols) - len(jm.complete)
2900 2925 yield Completion(
2901 2926 start=offset - delta,
2902 2927 end=offset,
2903 2928 text=jm.name_with_symbols,
2904 2929 type=_UNKNOWN_TYPE, # don't compute type for speed
2905 2930 _origin="jedi",
2906 2931 signature="",
2907 2932 )
2908 2933
2909 2934 # TODO:
2910 2935 # Suppress this, right now just for debug.
2911 2936 if jedi_matches and non_jedi_results and self.debug:
2912 2937 some_start_offset = before.rfind(
2913 2938 next(iter(non_jedi_results.values()))["matched_fragment"]
2914 2939 )
2915 2940 yield Completion(
2916 2941 start=some_start_offset,
2917 2942 end=offset,
2918 2943 text="--jedi/ipython--",
2919 2944 _origin="debug",
2920 2945 type="none",
2921 2946 signature="",
2922 2947 )
2923 2948
2924 2949 ordered: List[Completion] = []
2925 2950 sortable: List[Completion] = []
2926 2951
2927 2952 for origin, result in non_jedi_results.items():
2928 2953 matched_text = result["matched_fragment"]
2929 2954 start_offset = before.rfind(matched_text)
2930 2955 is_ordered = result.get("ordered", False)
2931 2956 container = ordered if is_ordered else sortable
2932 2957
2933 2958 # I'm unsure if this is always true, so let's assert and see if it
2934 2959 # crash
2935 2960 assert before.endswith(matched_text)
2936 2961
2937 2962 for simple_completion in result["completions"]:
2938 2963 completion = Completion(
2939 2964 start=start_offset,
2940 2965 end=offset,
2941 2966 text=simple_completion.text,
2942 2967 _origin=origin,
2943 2968 signature="",
2944 2969 type=simple_completion.type or _UNKNOWN_TYPE,
2945 2970 )
2946 2971 container.append(completion)
2947 2972
2948 2973 yield from list(self._deduplicate(ordered + self._sort(sortable)))[
2949 2974 :MATCHES_LIMIT
2950 2975 ]
2951 2976
2952 2977 def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
2953 2978 """Find completions for the given text and line context.
2954 2979
2955 2980 Note that both the text and the line_buffer are optional, but at least
2956 2981 one of them must be given.
2957 2982
2958 2983 Parameters
2959 2984 ----------
2960 2985 text : string, optional
2961 2986 Text to perform the completion on. If not given, the line buffer
2962 2987 is split using the instance's CompletionSplitter object.
2963 2988 line_buffer : string, optional
2964 2989 If not given, the completer attempts to obtain the current line
2965 2990 buffer via readline. This keyword allows clients which are
2966 2991 requesting for text completions in non-readline contexts to inform
2967 2992 the completer of the entire text.
2968 2993 cursor_pos : int, optional
2969 2994 Index of the cursor in the full line buffer. Should be provided by
2970 2995 remote frontends where kernel has no access to frontend state.
2971 2996
2972 2997 Returns
2973 2998 -------
2974 2999 Tuple of two items:
2975 3000 text : str
2976 3001 Text that was actually used in the completion.
2977 3002 matches : list
2978 3003 A list of completion matches.
2979 3004
2980 3005 Notes
2981 3006 -----
2982 3007 This API is likely to be deprecated and replaced by
2983 3008 :any:`IPCompleter.completions` in the future.
2984 3009
2985 3010 """
2986 3011 warnings.warn('`Completer.complete` is pending deprecation since '
2987 3012 'IPython 6.0 and will be replaced by `Completer.completions`.',
2988 3013 PendingDeprecationWarning)
2989 3014 # potential todo, FOLD the 3rd throw away argument of _complete
2990 3015 # into the first 2 one.
2991 3016 # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
2992 3017 # TODO: should we deprecate now, or does it stay?
2993 3018
2994 3019 results = self._complete(
2995 3020 line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
2996 3021 )
2997 3022
2998 3023 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2999 3024
3000 3025 return self._arrange_and_extract(
3001 3026 results,
3002 3027 # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
3003 3028 skip_matchers={jedi_matcher_id},
3004 3029 # this API does not support different start/end positions (fragments of token).
3005 3030 abort_if_offset_changes=True,
3006 3031 )
3007 3032
3008 3033 def _arrange_and_extract(
3009 3034 self,
3010 3035 results: Dict[str, MatcherResult],
3011 3036 skip_matchers: Set[str],
3012 3037 abort_if_offset_changes: bool,
3013 3038 ):
3014 3039
3015 3040 sortable: List[AnyMatcherCompletion] = []
3016 3041 ordered: List[AnyMatcherCompletion] = []
3017 3042 most_recent_fragment = None
3018 3043 for identifier, result in results.items():
3019 3044 if identifier in skip_matchers:
3020 3045 continue
3021 3046 if not result["completions"]:
3022 3047 continue
3023 3048 if not most_recent_fragment:
3024 3049 most_recent_fragment = result["matched_fragment"]
3025 3050 if (
3026 3051 abort_if_offset_changes
3027 3052 and result["matched_fragment"] != most_recent_fragment
3028 3053 ):
3029 3054 break
3030 3055 if result.get("ordered", False):
3031 3056 ordered.extend(result["completions"])
3032 3057 else:
3033 3058 sortable.extend(result["completions"])
3034 3059
3035 3060 if not most_recent_fragment:
3036 3061 most_recent_fragment = "" # to satisfy typechecker (and just in case)
3037 3062
3038 3063 return most_recent_fragment, [
3039 3064 m.text for m in self._deduplicate(ordered + self._sort(sortable))
3040 3065 ]
3041 3066
3042 3067 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
3043 3068 full_text=None) -> _CompleteResult:
3044 3069 """
3045 3070 Like complete but can also returns raw jedi completions as well as the
3046 3071 origin of the completion text. This could (and should) be made much
3047 3072 cleaner but that will be simpler once we drop the old (and stateful)
3048 3073 :any:`complete` API.
3049 3074
3050 3075 With current provisional API, cursor_pos act both (depending on the
3051 3076 caller) as the offset in the ``text`` or ``line_buffer``, or as the
3052 3077 ``column`` when passing multiline strings this could/should be renamed
3053 3078 but would add extra noise.
3054 3079
3055 3080 Parameters
3056 3081 ----------
3057 3082 cursor_line
3058 3083 Index of the line the cursor is on. 0 indexed.
3059 3084 cursor_pos
3060 3085 Position of the cursor in the current line/line_buffer/text. 0
3061 3086 indexed.
3062 3087 line_buffer : optional, str
3063 3088 The current line the cursor is in, this is mostly due to legacy
3064 3089 reason that readline could only give a us the single current line.
3065 3090 Prefer `full_text`.
3066 3091 text : str
3067 3092 The current "token" the cursor is in, mostly also for historical
3068 3093 reasons. as the completer would trigger only after the current line
3069 3094 was parsed.
3070 3095 full_text : str
3071 3096 Full text of the current cell.
3072 3097
3073 3098 Returns
3074 3099 -------
3075 3100 An ordered dictionary where keys are identifiers of completion
3076 3101 matchers and values are ``MatcherResult``s.
3077 3102 """
3078 3103
3079 3104 # if the cursor position isn't given, the only sane assumption we can
3080 3105 # make is that it's at the end of the line (the common case)
3081 3106 if cursor_pos is None:
3082 3107 cursor_pos = len(line_buffer) if text is None else len(text)
3083 3108
3084 3109 if self.use_main_ns:
3085 3110 self.namespace = __main__.__dict__
3086 3111
3087 3112 # if text is either None or an empty string, rely on the line buffer
3088 3113 if (not line_buffer) and full_text:
3089 3114 line_buffer = full_text.split('\n')[cursor_line]
3090 3115 if not text: # issue #11508: check line_buffer before calling split_line
3091 3116 text = (
3092 3117 self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
3093 3118 )
3094 3119
3095 3120 # If no line buffer is given, assume the input text is all there was
3096 3121 if line_buffer is None:
3097 3122 line_buffer = text
3098 3123
3099 3124 # deprecated - do not use `line_buffer` in new code.
3100 3125 self.line_buffer = line_buffer
3101 3126 self.text_until_cursor = self.line_buffer[:cursor_pos]
3102 3127
3103 3128 if not full_text:
3104 3129 full_text = line_buffer
3105 3130
3106 3131 context = CompletionContext(
3107 3132 full_text=full_text,
3108 3133 cursor_position=cursor_pos,
3109 3134 cursor_line=cursor_line,
3110 3135 token=text,
3111 3136 limit=MATCHES_LIMIT,
3112 3137 )
3113 3138
3114 3139 # Start with a clean slate of completions
3115 3140 results: Dict[str, MatcherResult] = {}
3116 3141
3117 3142 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3118 3143
3119 3144 suppressed_matchers: Set[str] = set()
3120 3145
3121 3146 matchers = {
3122 3147 _get_matcher_id(matcher): matcher
3123 3148 for matcher in sorted(
3124 3149 self.matchers, key=_get_matcher_priority, reverse=True
3125 3150 )
3126 3151 }
3127 3152
3128 3153 for matcher_id, matcher in matchers.items():
3129 3154 matcher_id = _get_matcher_id(matcher)
3130 3155
3131 3156 if matcher_id in self.disable_matchers:
3132 3157 continue
3133 3158
3134 3159 if matcher_id in results:
3135 3160 warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
3136 3161
3137 3162 if matcher_id in suppressed_matchers:
3138 3163 continue
3139 3164
3140 3165 result: MatcherResult
3141 3166 try:
3142 3167 if _is_matcher_v1(matcher):
3143 3168 result = _convert_matcher_v1_result_to_v2(
3144 3169 matcher(text), type=_UNKNOWN_TYPE
3145 3170 )
3146 3171 elif _is_matcher_v2(matcher):
3147 3172 result = matcher(context)
3148 3173 else:
3149 3174 api_version = _get_matcher_api_version(matcher)
3150 3175 raise ValueError(f"Unsupported API version {api_version}")
3151 3176 except:
3152 3177 # Show the ugly traceback if the matcher causes an
3153 3178 # exception, but do NOT crash the kernel!
3154 3179 sys.excepthook(*sys.exc_info())
3155 3180 continue
3156 3181
3157 3182 # set default value for matched fragment if suffix was not selected.
3158 3183 result["matched_fragment"] = result.get("matched_fragment", context.token)
3159 3184
3160 3185 if not suppressed_matchers:
3161 3186 suppression_recommended: Union[bool, Set[str]] = result.get(
3162 3187 "suppress", False
3163 3188 )
3164 3189
3165 3190 suppression_config = (
3166 3191 self.suppress_competing_matchers.get(matcher_id, None)
3167 3192 if isinstance(self.suppress_competing_matchers, dict)
3168 3193 else self.suppress_competing_matchers
3169 3194 )
3170 3195 should_suppress = (
3171 3196 (suppression_config is True)
3172 3197 or (suppression_recommended and (suppression_config is not False))
3173 3198 ) and has_any_completions(result)
3174 3199
3175 3200 if should_suppress:
3176 3201 suppression_exceptions: Set[str] = result.get(
3177 3202 "do_not_suppress", set()
3178 3203 )
3179 3204 if isinstance(suppression_recommended, Iterable):
3180 3205 to_suppress = set(suppression_recommended)
3181 3206 else:
3182 3207 to_suppress = set(matchers)
3183 3208 suppressed_matchers = to_suppress - suppression_exceptions
3184 3209
3185 3210 new_results = {}
3186 3211 for previous_matcher_id, previous_result in results.items():
3187 3212 if previous_matcher_id not in suppressed_matchers:
3188 3213 new_results[previous_matcher_id] = previous_result
3189 3214 results = new_results
3190 3215
3191 3216 results[matcher_id] = result
3192 3217
3193 3218 _, matches = self._arrange_and_extract(
3194 3219 results,
3195 3220 # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
3196 3221 # if it was omission, we can remove the filtering step, otherwise remove this comment.
3197 3222 skip_matchers={jedi_matcher_id},
3198 3223 abort_if_offset_changes=False,
3199 3224 )
3200 3225
3201 3226 # populate legacy stateful API
3202 3227 self.matches = matches
3203 3228
3204 3229 return results
3205 3230
3206 3231 @staticmethod
3207 3232 def _deduplicate(
3208 3233 matches: Sequence[AnyCompletion],
3209 3234 ) -> Iterable[AnyCompletion]:
3210 3235 filtered_matches: Dict[str, AnyCompletion] = {}
3211 3236 for match in matches:
3212 3237 text = match.text
3213 3238 if (
3214 3239 text not in filtered_matches
3215 3240 or filtered_matches[text].type == _UNKNOWN_TYPE
3216 3241 ):
3217 3242 filtered_matches[text] = match
3218 3243
3219 3244 return filtered_matches.values()
3220 3245
3221 3246 @staticmethod
3222 3247 def _sort(matches: Sequence[AnyCompletion]):
3223 3248 return sorted(matches, key=lambda x: completions_sorting_key(x.text))
3224 3249
3225 3250 @context_matcher()
3226 3251 def fwd_unicode_matcher(self, context: CompletionContext):
3227 3252 """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
3228 3253 # TODO: use `context.limit` to terminate early once we matched the maximum
3229 3254 # number that will be used downstream; can be added as an optional to
3230 3255 # `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
3231 3256 fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
3232 3257 return _convert_matcher_v1_result_to_v2(
3233 3258 matches, type="unicode", fragment=fragment, suppress_if_matches=True
3234 3259 )
3235 3260
3236 3261 def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
3237 3262 """
3238 3263 Forward match a string starting with a backslash with a list of
3239 3264 potential Unicode completions.
3240 3265
3241 3266 Will compute list of Unicode character names on first call and cache it.
3242 3267
3243 3268 .. deprecated:: 8.6
3244 3269 You can use :meth:`fwd_unicode_matcher` instead.
3245 3270
3246 3271 Returns
3247 3272 -------
3248 3273 At tuple with:
3249 3274 - matched text (empty if no matches)
3250 3275 - list of potential completions, empty tuple otherwise)
3251 3276 """
3252 3277 # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
3253 3278 # We could do a faster match using a Trie.
3254 3279
3255 3280 # Using pygtrie the following seem to work:
3256 3281
3257 3282 # s = PrefixSet()
3258 3283
3259 3284 # for c in range(0,0x10FFFF + 1):
3260 3285 # try:
3261 3286 # s.add(unicodedata.name(chr(c)))
3262 3287 # except ValueError:
3263 3288 # pass
3264 3289 # [''.join(k) for k in s.iter(prefix)]
3265 3290
3266 3291 # But need to be timed and adds an extra dependency.
3267 3292
3268 3293 slashpos = text.rfind('\\')
3269 3294 # if text starts with slash
3270 3295 if slashpos > -1:
3271 3296 # PERF: It's important that we don't access self._unicode_names
3272 3297 # until we're inside this if-block. _unicode_names is lazily
3273 3298 # initialized, and it takes a user-noticeable amount of time to
3274 3299 # initialize it, so we don't want to initialize it unless we're
3275 3300 # actually going to use it.
3276 3301 s = text[slashpos + 1 :]
3277 3302 sup = s.upper()
3278 3303 candidates = [x for x in self.unicode_names if x.startswith(sup)]
3279 3304 if candidates:
3280 3305 return s, candidates
3281 3306 candidates = [x for x in self.unicode_names if sup in x]
3282 3307 if candidates:
3283 3308 return s, candidates
3284 3309 splitsup = sup.split(" ")
3285 3310 candidates = [
3286 3311 x for x in self.unicode_names if all(u in x for u in splitsup)
3287 3312 ]
3288 3313 if candidates:
3289 3314 return s, candidates
3290 3315
3291 3316 return "", ()
3292 3317
3293 3318 # if text does not start with slash
3294 3319 else:
3295 3320 return '', ()
3296 3321
3297 3322 @property
3298 3323 def unicode_names(self) -> List[str]:
3299 3324 """List of names of unicode code points that can be completed.
3300 3325
3301 3326 The list is lazily initialized on first access.
3302 3327 """
3303 3328 if self._unicode_names is None:
3304 3329 names = []
3305 3330 for c in range(0,0x10FFFF + 1):
3306 3331 try:
3307 3332 names.append(unicodedata.name(chr(c)))
3308 3333 except ValueError:
3309 3334 pass
3310 3335 self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
3311 3336
3312 3337 return self._unicode_names
3313 3338
3314 3339 def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
3315 3340 names = []
3316 3341 for start,stop in ranges:
3317 3342 for c in range(start, stop) :
3318 3343 try:
3319 3344 names.append(unicodedata.name(chr(c)))
3320 3345 except ValueError:
3321 3346 pass
3322 3347 return names
@@ -1,1702 +1,1717 b''
1 1 # encoding: utf-8
2 2 """Tests for the IPython tab-completion machinery."""
3 3
4 4 # Copyright (c) IPython Development Team.
5 5 # Distributed under the terms of the Modified BSD License.
6 6
7 7 import os
8 8 import pytest
9 9 import sys
10 10 import textwrap
11 11 import unittest
12 12
13 13 from contextlib import contextmanager
14 14
15 15 from traitlets.config.loader import Config
16 16 from IPython import get_ipython
17 17 from IPython.core import completer
18 18 from IPython.utils.tempdir import TemporaryDirectory, TemporaryWorkingDirectory
19 19 from IPython.utils.generics import complete_object
20 20 from IPython.testing import decorators as dec
21 21
22 22 from IPython.core.completer import (
23 23 Completion,
24 24 provisionalcompleter,
25 25 match_dict_keys,
26 26 _deduplicate_completions,
27 27 _match_number_in_dict_key_prefix,
28 28 completion_matcher,
29 29 SimpleCompletion,
30 30 CompletionContext,
31 31 )
32 32
33 33 # -----------------------------------------------------------------------------
34 34 # Test functions
35 35 # -----------------------------------------------------------------------------
36 36
37 37 def recompute_unicode_ranges():
38 38 """
39 39 utility to recompute the largest unicode range without any characters
40 40
41 41 use to recompute the gap in the global _UNICODE_RANGES of completer.py
42 42 """
43 43 import itertools
44 44 import unicodedata
45 45 valid = []
46 46 for c in range(0,0x10FFFF + 1):
47 47 try:
48 48 unicodedata.name(chr(c))
49 49 except ValueError:
50 50 continue
51 51 valid.append(c)
52 52
53 53 def ranges(i):
54 54 for a, b in itertools.groupby(enumerate(i), lambda pair: pair[1] - pair[0]):
55 55 b = list(b)
56 56 yield b[0][1], b[-1][1]
57 57
58 58 rg = list(ranges(valid))
59 59 lens = []
60 60 gap_lens = []
61 61 pstart, pstop = 0,0
62 62 for start, stop in rg:
63 63 lens.append(stop-start)
64 64 gap_lens.append((start - pstop, hex(pstop), hex(start), f'{round((start - pstop)/0xe01f0*100)}%'))
65 65 pstart, pstop = start, stop
66 66
67 67 return sorted(gap_lens)[-1]
68 68
69 69
70 70
71 71 def test_unicode_range():
72 72 """
73 73 Test that the ranges we test for unicode names give the same number of
74 74 results than testing the full length.
75 75 """
76 76 from IPython.core.completer import _unicode_name_compute, _UNICODE_RANGES
77 77
78 78 expected_list = _unicode_name_compute([(0, 0x110000)])
79 79 test = _unicode_name_compute(_UNICODE_RANGES)
80 80 len_exp = len(expected_list)
81 81 len_test = len(test)
82 82
83 83 # do not inline the len() or on error pytest will try to print the 130 000 +
84 84 # elements.
85 85 message = None
86 86 if len_exp != len_test or len_exp > 131808:
87 87 size, start, stop, prct = recompute_unicode_ranges()
88 88 message = f"""_UNICODE_RANGES likely wrong and need updating. This is
89 89 likely due to a new release of Python. We've find that the biggest gap
90 90 in unicode characters has reduces in size to be {size} characters
91 91 ({prct}), from {start}, to {stop}. In completer.py likely update to
92 92
93 93 _UNICODE_RANGES = [(32, {start}), ({stop}, 0xe01f0)]
94 94
95 95 And update the assertion below to use
96 96
97 97 len_exp <= {len_exp}
98 98 """
99 99 assert len_exp == len_test, message
100 100
101 101 # fail if new unicode symbols have been added.
102 102 assert len_exp <= 143041, message
103 103
104 104
105 105 @contextmanager
106 106 def greedy_completion():
107 107 ip = get_ipython()
108 108 greedy_original = ip.Completer.greedy
109 109 try:
110 110 ip.Completer.greedy = True
111 111 yield
112 112 finally:
113 113 ip.Completer.greedy = greedy_original
114 114
115 115
116 116 @contextmanager
117 117 def evaluation_policy(evaluation: str):
118 118 ip = get_ipython()
119 119 evaluation_original = ip.Completer.evaluation
120 120 try:
121 121 ip.Completer.evaluation = evaluation
122 122 yield
123 123 finally:
124 124 ip.Completer.evaluation = evaluation_original
125 125
126 126
127 127 @contextmanager
128 128 def custom_matchers(matchers):
129 129 ip = get_ipython()
130 130 try:
131 131 ip.Completer.custom_matchers.extend(matchers)
132 132 yield
133 133 finally:
134 134 ip.Completer.custom_matchers.clear()
135 135
136 136
137 137 def test_protect_filename():
138 138 if sys.platform == "win32":
139 139 pairs = [
140 140 ("abc", "abc"),
141 141 (" abc", '" abc"'),
142 142 ("a bc", '"a bc"'),
143 143 ("a bc", '"a bc"'),
144 144 (" bc", '" bc"'),
145 145 ]
146 146 else:
147 147 pairs = [
148 148 ("abc", "abc"),
149 149 (" abc", r"\ abc"),
150 150 ("a bc", r"a\ bc"),
151 151 ("a bc", r"a\ \ bc"),
152 152 (" bc", r"\ \ bc"),
153 153 # On posix, we also protect parens and other special characters.
154 154 ("a(bc", r"a\(bc"),
155 155 ("a)bc", r"a\)bc"),
156 156 ("a( )bc", r"a\(\ \)bc"),
157 157 ("a[1]bc", r"a\[1\]bc"),
158 158 ("a{1}bc", r"a\{1\}bc"),
159 159 ("a#bc", r"a\#bc"),
160 160 ("a?bc", r"a\?bc"),
161 161 ("a=bc", r"a\=bc"),
162 162 ("a\\bc", r"a\\bc"),
163 163 ("a|bc", r"a\|bc"),
164 164 ("a;bc", r"a\;bc"),
165 165 ("a:bc", r"a\:bc"),
166 166 ("a'bc", r"a\'bc"),
167 167 ("a*bc", r"a\*bc"),
168 168 ('a"bc', r"a\"bc"),
169 169 ("a^bc", r"a\^bc"),
170 170 ("a&bc", r"a\&bc"),
171 171 ]
172 172 # run the actual tests
173 173 for s1, s2 in pairs:
174 174 s1p = completer.protect_filename(s1)
175 175 assert s1p == s2
176 176
177 177
178 178 def check_line_split(splitter, test_specs):
179 179 for part1, part2, split in test_specs:
180 180 cursor_pos = len(part1)
181 181 line = part1 + part2
182 182 out = splitter.split_line(line, cursor_pos)
183 183 assert out == split
184 184
185 185 def test_line_split():
186 186 """Basic line splitter test with default specs."""
187 187 sp = completer.CompletionSplitter()
188 188 # The format of the test specs is: part1, part2, expected answer. Parts 1
189 189 # and 2 are joined into the 'line' sent to the splitter, as if the cursor
190 190 # was at the end of part1. So an empty part2 represents someone hitting
191 191 # tab at the end of the line, the most common case.
192 192 t = [
193 193 ("run some/scrip", "", "some/scrip"),
194 194 ("run scripts/er", "ror.py foo", "scripts/er"),
195 195 ("echo $HOM", "", "HOM"),
196 196 ("print sys.pa", "", "sys.pa"),
197 197 ("print(sys.pa", "", "sys.pa"),
198 198 ("execfile('scripts/er", "", "scripts/er"),
199 199 ("a[x.", "", "x."),
200 200 ("a[x.", "y", "x."),
201 201 ('cd "some_file/', "", "some_file/"),
202 202 ]
203 203 check_line_split(sp, t)
204 204 # Ensure splitting works OK with unicode by re-running the tests with
205 205 # all inputs turned into unicode
206 206 check_line_split(sp, [map(str, p) for p in t])
207 207
208 208
209 209 class NamedInstanceClass:
210 210 instances = {}
211 211
212 212 def __init__(self, name):
213 213 self.instances[name] = self
214 214
215 215 @classmethod
216 216 def _ipython_key_completions_(cls):
217 217 return cls.instances.keys()
218 218
219 219
220 220 class KeyCompletable:
221 221 def __init__(self, things=()):
222 222 self.things = things
223 223
224 224 def _ipython_key_completions_(self):
225 225 return list(self.things)
226 226
227 227
228 228 class TestCompleter(unittest.TestCase):
229 229 def setUp(self):
230 230 """
231 231 We want to silence all PendingDeprecationWarning when testing the completer
232 232 """
233 233 self._assertwarns = self.assertWarns(PendingDeprecationWarning)
234 234 self._assertwarns.__enter__()
235 235
236 236 def tearDown(self):
237 237 try:
238 238 self._assertwarns.__exit__(None, None, None)
239 239 except AssertionError:
240 240 pass
241 241
242 242 def test_custom_completion_error(self):
243 243 """Test that errors from custom attribute completers are silenced."""
244 244 ip = get_ipython()
245 245
246 246 class A:
247 247 pass
248 248
249 249 ip.user_ns["x"] = A()
250 250
251 251 @complete_object.register(A)
252 252 def complete_A(a, existing_completions):
253 253 raise TypeError("this should be silenced")
254 254
255 255 ip.complete("x.")
256 256
257 257 def test_custom_completion_ordering(self):
258 258 """Test that errors from custom attribute completers are silenced."""
259 259 ip = get_ipython()
260 260
261 261 _, matches = ip.complete('in')
262 262 assert matches.index('input') < matches.index('int')
263 263
264 264 def complete_example(a):
265 265 return ['example2', 'example1']
266 266
267 267 ip.Completer.custom_completers.add_re('ex*', complete_example)
268 268 _, matches = ip.complete('ex')
269 269 assert matches.index('example2') < matches.index('example1')
270 270
271 271 def test_unicode_completions(self):
272 272 ip = get_ipython()
273 273 # Some strings that trigger different types of completion. Check them both
274 274 # in str and unicode forms
275 275 s = ["ru", "%ru", "cd /", "floa", "float(x)/"]
276 276 for t in s + list(map(str, s)):
277 277 # We don't need to check exact completion values (they may change
278 278 # depending on the state of the namespace, but at least no exceptions
279 279 # should be thrown and the return value should be a pair of text, list
280 280 # values.
281 281 text, matches = ip.complete(t)
282 282 self.assertIsInstance(text, str)
283 283 self.assertIsInstance(matches, list)
284 284
285 285 def test_latex_completions(self):
286 286 from IPython.core.latex_symbols import latex_symbols
287 287 import random
288 288
289 289 ip = get_ipython()
290 290 # Test some random unicode symbols
291 291 keys = random.sample(sorted(latex_symbols), 10)
292 292 for k in keys:
293 293 text, matches = ip.complete(k)
294 294 self.assertEqual(text, k)
295 295 self.assertEqual(matches, [latex_symbols[k]])
296 296 # Test a more complex line
297 297 text, matches = ip.complete("print(\\alpha")
298 298 self.assertEqual(text, "\\alpha")
299 299 self.assertEqual(matches[0], latex_symbols["\\alpha"])
300 300 # Test multiple matching latex symbols
301 301 text, matches = ip.complete("\\al")
302 302 self.assertIn("\\alpha", matches)
303 303 self.assertIn("\\aleph", matches)
304 304
305 305 def test_latex_no_results(self):
306 306 """
307 307 forward latex should really return nothing in either field if nothing is found.
308 308 """
309 309 ip = get_ipython()
310 310 text, matches = ip.Completer.latex_matches("\\really_i_should_match_nothing")
311 311 self.assertEqual(text, "")
312 312 self.assertEqual(matches, ())
313 313
314 314 def test_back_latex_completion(self):
315 315 ip = get_ipython()
316 316
317 317 # do not return more than 1 matches for \beta, only the latex one.
318 318 name, matches = ip.complete("\\Ξ²")
319 319 self.assertEqual(matches, ["\\beta"])
320 320
321 321 def test_back_unicode_completion(self):
322 322 ip = get_ipython()
323 323
324 324 name, matches = ip.complete("\\β…€")
325 325 self.assertEqual(matches, ["\\ROMAN NUMERAL FIVE"])
326 326
327 327 def test_forward_unicode_completion(self):
328 328 ip = get_ipython()
329 329
330 330 name, matches = ip.complete("\\ROMAN NUMERAL FIVE")
331 331 self.assertEqual(matches, ["β…€"]) # This is not a V
332 332 self.assertEqual(matches, ["\u2164"]) # same as above but explicit.
333 333
334 334 def test_delim_setting(self):
335 335 sp = completer.CompletionSplitter()
336 336 sp.delims = " "
337 337 self.assertEqual(sp.delims, " ")
338 338 self.assertEqual(sp._delim_expr, r"[\ ]")
339 339
340 340 def test_spaces(self):
341 341 """Test with only spaces as split chars."""
342 342 sp = completer.CompletionSplitter()
343 343 sp.delims = " "
344 344 t = [("foo", "", "foo"), ("run foo", "", "foo"), ("run foo", "bar", "foo")]
345 345 check_line_split(sp, t)
346 346
347 347 def test_has_open_quotes1(self):
348 348 for s in ["'", "'''", "'hi' '"]:
349 349 self.assertEqual(completer.has_open_quotes(s), "'")
350 350
351 351 def test_has_open_quotes2(self):
352 352 for s in ['"', '"""', '"hi" "']:
353 353 self.assertEqual(completer.has_open_quotes(s), '"')
354 354
355 355 def test_has_open_quotes3(self):
356 356 for s in ["''", "''' '''", "'hi' 'ipython'"]:
357 357 self.assertFalse(completer.has_open_quotes(s))
358 358
359 359 def test_has_open_quotes4(self):
360 360 for s in ['""', '""" """', '"hi" "ipython"']:
361 361 self.assertFalse(completer.has_open_quotes(s))
362 362
363 363 @pytest.mark.xfail(
364 364 sys.platform == "win32", reason="abspath completions fail on Windows"
365 365 )
366 366 def test_abspath_file_completions(self):
367 367 ip = get_ipython()
368 368 with TemporaryDirectory() as tmpdir:
369 369 prefix = os.path.join(tmpdir, "foo")
370 370 suffixes = ["1", "2"]
371 371 names = [prefix + s for s in suffixes]
372 372 for n in names:
373 373 open(n, "w", encoding="utf-8").close()
374 374
375 375 # Check simple completion
376 376 c = ip.complete(prefix)[1]
377 377 self.assertEqual(c, names)
378 378
379 379 # Now check with a function call
380 380 cmd = 'a = f("%s' % prefix
381 381 c = ip.complete(prefix, cmd)[1]
382 382 comp = [prefix + s for s in suffixes]
383 383 self.assertEqual(c, comp)
384 384
385 385 def test_local_file_completions(self):
386 386 ip = get_ipython()
387 387 with TemporaryWorkingDirectory():
388 388 prefix = "./foo"
389 389 suffixes = ["1", "2"]
390 390 names = [prefix + s for s in suffixes]
391 391 for n in names:
392 392 open(n, "w", encoding="utf-8").close()
393 393
394 394 # Check simple completion
395 395 c = ip.complete(prefix)[1]
396 396 self.assertEqual(c, names)
397 397
398 398 # Now check with a function call
399 399 cmd = 'a = f("%s' % prefix
400 400 c = ip.complete(prefix, cmd)[1]
401 401 comp = {prefix + s for s in suffixes}
402 402 self.assertTrue(comp.issubset(set(c)))
403 403
404 404 def test_quoted_file_completions(self):
405 405 ip = get_ipython()
406 406
407 407 def _(text):
408 408 return ip.Completer._complete(
409 409 cursor_line=0, cursor_pos=len(text), full_text=text
410 410 )["IPCompleter.file_matcher"]["completions"]
411 411
412 412 with TemporaryWorkingDirectory():
413 413 name = "foo'bar"
414 414 open(name, "w", encoding="utf-8").close()
415 415
416 416 # Don't escape Windows
417 417 escaped = name if sys.platform == "win32" else "foo\\'bar"
418 418
419 419 # Single quote matches embedded single quote
420 420 c = _("open('foo")[0]
421 421 self.assertEqual(c.text, escaped)
422 422
423 423 # Double quote requires no escape
424 424 c = _('open("foo')[0]
425 425 self.assertEqual(c.text, name)
426 426
427 427 # No quote requires an escape
428 428 c = _("%ls foo")[0]
429 429 self.assertEqual(c.text, escaped)
430 430
431 431 def test_all_completions_dups(self):
432 432 """
433 433 Make sure the output of `IPCompleter.all_completions` does not have
434 434 duplicated prefixes.
435 435 """
436 436 ip = get_ipython()
437 437 c = ip.Completer
438 438 ip.ex("class TestClass():\n\ta=1\n\ta1=2")
439 439 for jedi_status in [True, False]:
440 440 with provisionalcompleter():
441 441 ip.Completer.use_jedi = jedi_status
442 442 matches = c.all_completions("TestCl")
443 443 assert matches == ["TestClass"], (jedi_status, matches)
444 444 matches = c.all_completions("TestClass.")
445 445 assert len(matches) > 2, (jedi_status, matches)
446 446 matches = c.all_completions("TestClass.a")
447 447 assert matches == ['TestClass.a', 'TestClass.a1'], jedi_status
448 448
449 449 def test_jedi(self):
450 450 """
451 451 A couple of issue we had with Jedi
452 452 """
453 453 ip = get_ipython()
454 454
455 455 def _test_complete(reason, s, comp, start=None, end=None):
456 456 l = len(s)
457 457 start = start if start is not None else l
458 458 end = end if end is not None else l
459 459 with provisionalcompleter():
460 460 ip.Completer.use_jedi = True
461 461 completions = set(ip.Completer.completions(s, l))
462 462 ip.Completer.use_jedi = False
463 463 assert Completion(start, end, comp) in completions, reason
464 464
465 465 def _test_not_complete(reason, s, comp):
466 466 l = len(s)
467 467 with provisionalcompleter():
468 468 ip.Completer.use_jedi = True
469 469 completions = set(ip.Completer.completions(s, l))
470 470 ip.Completer.use_jedi = False
471 471 assert Completion(l, l, comp) not in completions, reason
472 472
473 473 import jedi
474 474
475 475 jedi_version = tuple(int(i) for i in jedi.__version__.split(".")[:3])
476 476 if jedi_version > (0, 10):
477 477 _test_complete("jedi >0.9 should complete and not crash", "a=1;a.", "real")
478 478 _test_complete("can infer first argument", 'a=(1,"foo");a[0].', "real")
479 479 _test_complete("can infer second argument", 'a=(1,"foo");a[1].', "capitalize")
480 480 _test_complete("cover duplicate completions", "im", "import", 0, 2)
481 481
482 482 _test_not_complete("does not mix types", 'a=(1,"foo");a[0].', "capitalize")
483 483
484 484 def test_completion_have_signature(self):
485 485 """
486 486 Lets make sure jedi is capable of pulling out the signature of the function we are completing.
487 487 """
488 488 ip = get_ipython()
489 489 with provisionalcompleter():
490 490 ip.Completer.use_jedi = True
491 491 completions = ip.Completer.completions("ope", 3)
492 492 c = next(completions) # should be `open`
493 493 ip.Completer.use_jedi = False
494 494 assert "file" in c.signature, "Signature of function was not found by completer"
495 495 assert (
496 496 "encoding" in c.signature
497 497 ), "Signature of function was not found by completer"
498 498
499 499 def test_completions_have_type(self):
500 500 """
501 501 Lets make sure matchers provide completion type.
502 502 """
503 503 ip = get_ipython()
504 504 with provisionalcompleter():
505 505 ip.Completer.use_jedi = False
506 506 completions = ip.Completer.completions("%tim", 3)
507 507 c = next(completions) # should be `%time` or similar
508 508 assert c.type == "magic", "Type of magic was not assigned by completer"
509 509
510 510 @pytest.mark.xfail(reason="Known failure on jedi<=0.18.0")
511 511 def test_deduplicate_completions(self):
512 512 """
513 513 Test that completions are correctly deduplicated (even if ranges are not the same)
514 514 """
515 515 ip = get_ipython()
516 516 ip.ex(
517 517 textwrap.dedent(
518 518 """
519 519 class Z:
520 520 zoo = 1
521 521 """
522 522 )
523 523 )
524 524 with provisionalcompleter():
525 525 ip.Completer.use_jedi = True
526 526 l = list(
527 527 _deduplicate_completions("Z.z", ip.Completer.completions("Z.z", 3))
528 528 )
529 529 ip.Completer.use_jedi = False
530 530
531 531 assert len(l) == 1, "Completions (Z.z<tab>) correctly deduplicate: %s " % l
532 532 assert l[0].text == "zoo" # and not `it.accumulate`
533 533
534 534 def test_greedy_completions(self):
535 535 """
536 536 Test the capability of the Greedy completer.
537 537
538 538 Most of the test here does not really show off the greedy completer, for proof
539 539 each of the text below now pass with Jedi. The greedy completer is capable of more.
540 540
541 541 See the :any:`test_dict_key_completion_contexts`
542 542
543 543 """
544 544 ip = get_ipython()
545 545 ip.ex("a=list(range(5))")
546 ip.ex("d = {'a b': str}")
546 547 _, c = ip.complete(".", line="a[0].")
547 548 self.assertFalse(".real" in c, "Shouldn't have completed on a[0]: %s" % c)
548 549
549 550 def _(line, cursor_pos, expect, message, completion):
550 551 with greedy_completion(), provisionalcompleter():
551 552 ip.Completer.use_jedi = False
552 553 _, c = ip.complete(".", line=line, cursor_pos=cursor_pos)
553 554 self.assertIn(expect, c, message % c)
554 555
555 556 ip.Completer.use_jedi = True
556 557 with provisionalcompleter():
557 558 completions = ip.Completer.completions(line, cursor_pos)
558 559 self.assertIn(completion, completions)
559 560
560 561 with provisionalcompleter():
561 562 _(
562 563 "a[0].",
563 564 5,
564 "a[0].real",
565 ".real",
565 566 "Should have completed on a[0].: %s",
566 567 Completion(5, 5, "real"),
567 568 )
568 569 _(
569 570 "a[0].r",
570 571 6,
571 "a[0].real",
572 ".real",
572 573 "Should have completed on a[0].r: %s",
573 574 Completion(5, 6, "real"),
574 575 )
575 576
576 577 _(
577 578 "a[0].from_",
578 579 10,
579 "a[0].from_bytes",
580 ".from_bytes",
580 581 "Should have completed on a[0].from_: %s",
581 582 Completion(5, 10, "from_bytes"),
582 583 )
584 _(
585 "assert str.star",
586 14,
587 "str.startswith",
588 "Should have completed on `assert str.star`: %s",
589 Completion(11, 14, "startswith"),
590 )
591 _(
592 "d['a b'].str",
593 12,
594 ".strip",
595 "Should have completed on `d['a b'].str`: %s",
596 Completion(9, 12, "strip"),
597 )
583 598
584 599 def test_omit__names(self):
585 600 # also happens to test IPCompleter as a configurable
586 601 ip = get_ipython()
587 602 ip._hidden_attr = 1
588 603 ip._x = {}
589 604 c = ip.Completer
590 605 ip.ex("ip=get_ipython()")
591 606 cfg = Config()
592 607 cfg.IPCompleter.omit__names = 0
593 608 c.update_config(cfg)
594 609 with provisionalcompleter():
595 610 c.use_jedi = False
596 611 s, matches = c.complete("ip.")
597 612 self.assertIn("ip.__str__", matches)
598 613 self.assertIn("ip._hidden_attr", matches)
599 614
600 615 # c.use_jedi = True
601 616 # completions = set(c.completions('ip.', 3))
602 617 # self.assertIn(Completion(3, 3, '__str__'), completions)
603 618 # self.assertIn(Completion(3,3, "_hidden_attr"), completions)
604 619
605 620 cfg = Config()
606 621 cfg.IPCompleter.omit__names = 1
607 622 c.update_config(cfg)
608 623 with provisionalcompleter():
609 624 c.use_jedi = False
610 625 s, matches = c.complete("ip.")
611 626 self.assertNotIn("ip.__str__", matches)
612 627 # self.assertIn('ip._hidden_attr', matches)
613 628
614 629 # c.use_jedi = True
615 630 # completions = set(c.completions('ip.', 3))
616 631 # self.assertNotIn(Completion(3,3,'__str__'), completions)
617 632 # self.assertIn(Completion(3,3, "_hidden_attr"), completions)
618 633
619 634 cfg = Config()
620 635 cfg.IPCompleter.omit__names = 2
621 636 c.update_config(cfg)
622 637 with provisionalcompleter():
623 638 c.use_jedi = False
624 639 s, matches = c.complete("ip.")
625 640 self.assertNotIn("ip.__str__", matches)
626 641 self.assertNotIn("ip._hidden_attr", matches)
627 642
628 643 # c.use_jedi = True
629 644 # completions = set(c.completions('ip.', 3))
630 645 # self.assertNotIn(Completion(3,3,'__str__'), completions)
631 646 # self.assertNotIn(Completion(3,3, "_hidden_attr"), completions)
632 647
633 648 with provisionalcompleter():
634 649 c.use_jedi = False
635 650 s, matches = c.complete("ip._x.")
636 651 self.assertIn("ip._x.keys", matches)
637 652
638 653 # c.use_jedi = True
639 654 # completions = set(c.completions('ip._x.', 6))
640 655 # self.assertIn(Completion(6,6, "keys"), completions)
641 656
642 657 del ip._hidden_attr
643 658 del ip._x
644 659
645 660 def test_limit_to__all__False_ok(self):
646 661 """
647 662 Limit to all is deprecated, once we remove it this test can go away.
648 663 """
649 664 ip = get_ipython()
650 665 c = ip.Completer
651 666 c.use_jedi = False
652 667 ip.ex("class D: x=24")
653 668 ip.ex("d=D()")
654 669 cfg = Config()
655 670 cfg.IPCompleter.limit_to__all__ = False
656 671 c.update_config(cfg)
657 672 s, matches = c.complete("d.")
658 673 self.assertIn("d.x", matches)
659 674
660 675 def test_get__all__entries_ok(self):
661 676 class A:
662 677 __all__ = ["x", 1]
663 678
664 679 words = completer.get__all__entries(A())
665 680 self.assertEqual(words, ["x"])
666 681
667 682 def test_get__all__entries_no__all__ok(self):
668 683 class A:
669 684 pass
670 685
671 686 words = completer.get__all__entries(A())
672 687 self.assertEqual(words, [])
673 688
674 689 def test_func_kw_completions(self):
675 690 ip = get_ipython()
676 691 c = ip.Completer
677 692 c.use_jedi = False
678 693 ip.ex("def myfunc(a=1,b=2): return a+b")
679 694 s, matches = c.complete(None, "myfunc(1,b")
680 695 self.assertIn("b=", matches)
681 696 # Simulate completing with cursor right after b (pos==10):
682 697 s, matches = c.complete(None, "myfunc(1,b)", 10)
683 698 self.assertIn("b=", matches)
684 699 s, matches = c.complete(None, 'myfunc(a="escaped\\")string",b')
685 700 self.assertIn("b=", matches)
686 701 # builtin function
687 702 s, matches = c.complete(None, "min(k, k")
688 703 self.assertIn("key=", matches)
689 704
690 705 def test_default_arguments_from_docstring(self):
691 706 ip = get_ipython()
692 707 c = ip.Completer
693 708 kwd = c._default_arguments_from_docstring("min(iterable[, key=func]) -> value")
694 709 self.assertEqual(kwd, ["key"])
695 710 # with cython type etc
696 711 kwd = c._default_arguments_from_docstring(
697 712 "Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)\n"
698 713 )
699 714 self.assertEqual(kwd, ["ncall", "resume", "nsplit"])
700 715 # white spaces
701 716 kwd = c._default_arguments_from_docstring(
702 717 "\n Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)\n"
703 718 )
704 719 self.assertEqual(kwd, ["ncall", "resume", "nsplit"])
705 720
706 721 def test_line_magics(self):
707 722 ip = get_ipython()
708 723 c = ip.Completer
709 724 s, matches = c.complete(None, "lsmag")
710 725 self.assertIn("%lsmagic", matches)
711 726 s, matches = c.complete(None, "%lsmag")
712 727 self.assertIn("%lsmagic", matches)
713 728
714 729 def test_cell_magics(self):
715 730 from IPython.core.magic import register_cell_magic
716 731
717 732 @register_cell_magic
718 733 def _foo_cellm(line, cell):
719 734 pass
720 735
721 736 ip = get_ipython()
722 737 c = ip.Completer
723 738
724 739 s, matches = c.complete(None, "_foo_ce")
725 740 self.assertIn("%%_foo_cellm", matches)
726 741 s, matches = c.complete(None, "%%_foo_ce")
727 742 self.assertIn("%%_foo_cellm", matches)
728 743
729 744 def test_line_cell_magics(self):
730 745 from IPython.core.magic import register_line_cell_magic
731 746
732 747 @register_line_cell_magic
733 748 def _bar_cellm(line, cell):
734 749 pass
735 750
736 751 ip = get_ipython()
737 752 c = ip.Completer
738 753
739 754 # The policy here is trickier, see comments in completion code. The
740 755 # returned values depend on whether the user passes %% or not explicitly,
741 756 # and this will show a difference if the same name is both a line and cell
742 757 # magic.
743 758 s, matches = c.complete(None, "_bar_ce")
744 759 self.assertIn("%_bar_cellm", matches)
745 760 self.assertIn("%%_bar_cellm", matches)
746 761 s, matches = c.complete(None, "%_bar_ce")
747 762 self.assertIn("%_bar_cellm", matches)
748 763 self.assertIn("%%_bar_cellm", matches)
749 764 s, matches = c.complete(None, "%%_bar_ce")
750 765 self.assertNotIn("%_bar_cellm", matches)
751 766 self.assertIn("%%_bar_cellm", matches)
752 767
753 768 def test_magic_completion_order(self):
754 769 ip = get_ipython()
755 770 c = ip.Completer
756 771
757 772 # Test ordering of line and cell magics.
758 773 text, matches = c.complete("timeit")
759 774 self.assertEqual(matches, ["%timeit", "%%timeit"])
760 775
761 776 def test_magic_completion_shadowing(self):
762 777 ip = get_ipython()
763 778 c = ip.Completer
764 779 c.use_jedi = False
765 780
766 781 # Before importing matplotlib, %matplotlib magic should be the only option.
767 782 text, matches = c.complete("mat")
768 783 self.assertEqual(matches, ["%matplotlib"])
769 784
770 785 # The newly introduced name should shadow the magic.
771 786 ip.run_cell("matplotlib = 1")
772 787 text, matches = c.complete("mat")
773 788 self.assertEqual(matches, ["matplotlib"])
774 789
775 790 # After removing matplotlib from namespace, the magic should again be
776 791 # the only option.
777 792 del ip.user_ns["matplotlib"]
778 793 text, matches = c.complete("mat")
779 794 self.assertEqual(matches, ["%matplotlib"])
780 795
781 796 def test_magic_completion_shadowing_explicit(self):
782 797 """
783 798 If the user try to complete a shadowed magic, and explicit % start should
784 799 still return the completions.
785 800 """
786 801 ip = get_ipython()
787 802 c = ip.Completer
788 803
789 804 # Before importing matplotlib, %matplotlib magic should be the only option.
790 805 text, matches = c.complete("%mat")
791 806 self.assertEqual(matches, ["%matplotlib"])
792 807
793 808 ip.run_cell("matplotlib = 1")
794 809
795 810 # After removing matplotlib from namespace, the magic should still be
796 811 # the only option.
797 812 text, matches = c.complete("%mat")
798 813 self.assertEqual(matches, ["%matplotlib"])
799 814
800 815 def test_magic_config(self):
801 816 ip = get_ipython()
802 817 c = ip.Completer
803 818
804 819 s, matches = c.complete(None, "conf")
805 820 self.assertIn("%config", matches)
806 821 s, matches = c.complete(None, "conf")
807 822 self.assertNotIn("AliasManager", matches)
808 823 s, matches = c.complete(None, "config ")
809 824 self.assertIn("AliasManager", matches)
810 825 s, matches = c.complete(None, "%config ")
811 826 self.assertIn("AliasManager", matches)
812 827 s, matches = c.complete(None, "config Ali")
813 828 self.assertListEqual(["AliasManager"], matches)
814 829 s, matches = c.complete(None, "%config Ali")
815 830 self.assertListEqual(["AliasManager"], matches)
816 831 s, matches = c.complete(None, "config AliasManager")
817 832 self.assertListEqual(["AliasManager"], matches)
818 833 s, matches = c.complete(None, "%config AliasManager")
819 834 self.assertListEqual(["AliasManager"], matches)
820 835 s, matches = c.complete(None, "config AliasManager.")
821 836 self.assertIn("AliasManager.default_aliases", matches)
822 837 s, matches = c.complete(None, "%config AliasManager.")
823 838 self.assertIn("AliasManager.default_aliases", matches)
824 839 s, matches = c.complete(None, "config AliasManager.de")
825 840 self.assertListEqual(["AliasManager.default_aliases"], matches)
826 841 s, matches = c.complete(None, "config AliasManager.de")
827 842 self.assertListEqual(["AliasManager.default_aliases"], matches)
828 843
829 844 def test_magic_color(self):
830 845 ip = get_ipython()
831 846 c = ip.Completer
832 847
833 848 s, matches = c.complete(None, "colo")
834 849 self.assertIn("%colors", matches)
835 850 s, matches = c.complete(None, "colo")
836 851 self.assertNotIn("NoColor", matches)
837 852 s, matches = c.complete(None, "%colors") # No trailing space
838 853 self.assertNotIn("NoColor", matches)
839 854 s, matches = c.complete(None, "colors ")
840 855 self.assertIn("NoColor", matches)
841 856 s, matches = c.complete(None, "%colors ")
842 857 self.assertIn("NoColor", matches)
843 858 s, matches = c.complete(None, "colors NoCo")
844 859 self.assertListEqual(["NoColor"], matches)
845 860 s, matches = c.complete(None, "%colors NoCo")
846 861 self.assertListEqual(["NoColor"], matches)
847 862
848 863 def test_match_dict_keys(self):
849 864 """
850 865 Test that match_dict_keys works on a couple of use case does return what
851 866 expected, and does not crash
852 867 """
853 868 delims = " \t\n`!@#$^&*()=+[{]}\\|;:'\",<>?"
854 869
855 870 def match(*args, **kwargs):
856 871 quote, offset, matches = match_dict_keys(*args, delims=delims, **kwargs)
857 872 return quote, offset, list(matches)
858 873
859 874 keys = ["foo", b"far"]
860 875 assert match(keys, "b'") == ("'", 2, ["far"])
861 876 assert match(keys, "b'f") == ("'", 2, ["far"])
862 877 assert match(keys, 'b"') == ('"', 2, ["far"])
863 878 assert match(keys, 'b"f') == ('"', 2, ["far"])
864 879
865 880 assert match(keys, "'") == ("'", 1, ["foo"])
866 881 assert match(keys, "'f") == ("'", 1, ["foo"])
867 882 assert match(keys, '"') == ('"', 1, ["foo"])
868 883 assert match(keys, '"f') == ('"', 1, ["foo"])
869 884
870 885 # Completion on first item of tuple
871 886 keys = [("foo", 1111), ("foo", 2222), (3333, "bar"), (3333, "test")]
872 887 assert match(keys, "'f") == ("'", 1, ["foo"])
873 888 assert match(keys, "33") == ("", 0, ["3333"])
874 889
875 890 # Completion on numbers
876 891 keys = [
877 892 0xDEADBEEF,
878 893 1111,
879 894 1234,
880 895 "1999",
881 896 0b10101,
882 897 22,
883 898 ] # 0xDEADBEEF = 3735928559; 0b10101 = 21
884 899 assert match(keys, "0xdead") == ("", 0, ["0xdeadbeef"])
885 900 assert match(keys, "1") == ("", 0, ["1111", "1234"])
886 901 assert match(keys, "2") == ("", 0, ["21", "22"])
887 902 assert match(keys, "0b101") == ("", 0, ["0b10101", "0b10110"])
888 903
889 904 # Should yield on variables
890 905 assert match(keys, "a_variable") == ("", 0, [])
891 906
892 907 # Should pass over invalid literals
893 908 assert match(keys, "'' ''") == ("", 0, [])
894 909
895 910 def test_match_dict_keys_tuple(self):
896 911 """
897 912 Test that match_dict_keys called with extra prefix works on a couple of use case,
898 913 does return what expected, and does not crash.
899 914 """
900 915 delims = " \t\n`!@#$^&*()=+[{]}\\|;:'\",<>?"
901 916
902 917 keys = [("foo", "bar"), ("foo", "oof"), ("foo", b"bar"), ('other', 'test')]
903 918
904 919 def match(*args, extra=None, **kwargs):
905 920 quote, offset, matches = match_dict_keys(
906 921 *args, delims=delims, extra_prefix=extra, **kwargs
907 922 )
908 923 return quote, offset, list(matches)
909 924
910 925 # Completion on first key == "foo"
911 926 assert match(keys, "'", extra=("foo",)) == ("'", 1, ["bar", "oof"])
912 927 assert match(keys, '"', extra=("foo",)) == ('"', 1, ["bar", "oof"])
913 928 assert match(keys, "'o", extra=("foo",)) == ("'", 1, ["oof"])
914 929 assert match(keys, '"o', extra=("foo",)) == ('"', 1, ["oof"])
915 930 assert match(keys, "b'", extra=("foo",)) == ("'", 2, ["bar"])
916 931 assert match(keys, 'b"', extra=("foo",)) == ('"', 2, ["bar"])
917 932 assert match(keys, "b'b", extra=("foo",)) == ("'", 2, ["bar"])
918 933 assert match(keys, 'b"b', extra=("foo",)) == ('"', 2, ["bar"])
919 934
920 935 # No Completion
921 936 assert match(keys, "'", extra=("no_foo",)) == ("'", 1, [])
922 937 assert match(keys, "'", extra=("fo",)) == ("'", 1, [])
923 938
924 939 keys = [("foo1", "foo2", "foo3", "foo4"), ("foo1", "foo2", "bar", "foo4")]
925 940 assert match(keys, "'foo", extra=("foo1",)) == ("'", 1, ["foo2"])
926 941 assert match(keys, "'foo", extra=("foo1", "foo2")) == ("'", 1, ["foo3"])
927 942 assert match(keys, "'foo", extra=("foo1", "foo2", "foo3")) == ("'", 1, ["foo4"])
928 943 assert match(keys, "'foo", extra=("foo1", "foo2", "foo3", "foo4")) == (
929 944 "'",
930 945 1,
931 946 [],
932 947 )
933 948
934 949 keys = [("foo", 1111), ("foo", "2222"), (3333, "bar"), (3333, 4444)]
935 950 assert match(keys, "'", extra=("foo",)) == ("'", 1, ["2222"])
936 951 assert match(keys, "", extra=("foo",)) == ("", 0, ["1111", "'2222'"])
937 952 assert match(keys, "'", extra=(3333,)) == ("'", 1, ["bar"])
938 953 assert match(keys, "", extra=(3333,)) == ("", 0, ["'bar'", "4444"])
939 954 assert match(keys, "'", extra=("3333",)) == ("'", 1, [])
940 955 assert match(keys, "33") == ("", 0, ["3333"])
941 956
942 957 def test_dict_key_completion_closures(self):
943 958 ip = get_ipython()
944 959 complete = ip.Completer.complete
945 960 ip.Completer.auto_close_dict_keys = True
946 961
947 962 ip.user_ns["d"] = {
948 963 # tuple only
949 964 ("aa", 11): None,
950 965 # tuple and non-tuple
951 966 ("bb", 22): None,
952 967 "bb": None,
953 968 # non-tuple only
954 969 "cc": None,
955 970 # numeric tuple only
956 971 (77, "x"): None,
957 972 # numeric tuple and non-tuple
958 973 (88, "y"): None,
959 974 88: None,
960 975 # numeric non-tuple only
961 976 99: None,
962 977 }
963 978
964 979 _, matches = complete(line_buffer="d[")
965 980 # should append `, ` if matches a tuple only
966 981 self.assertIn("'aa', ", matches)
967 982 # should not append anything if matches a tuple and an item
968 983 self.assertIn("'bb'", matches)
969 984 # should append `]` if matches and item only
970 985 self.assertIn("'cc']", matches)
971 986
972 987 # should append `, ` if matches a tuple only
973 988 self.assertIn("77, ", matches)
974 989 # should not append anything if matches a tuple and an item
975 990 self.assertIn("88", matches)
976 991 # should append `]` if matches and item only
977 992 self.assertIn("99]", matches)
978 993
979 994 _, matches = complete(line_buffer="d['aa', ")
980 995 # should restrict matches to those matching tuple prefix
981 996 self.assertIn("11]", matches)
982 997 self.assertNotIn("'bb'", matches)
983 998 self.assertNotIn("'bb', ", matches)
984 999 self.assertNotIn("'bb']", matches)
985 1000 self.assertNotIn("'cc'", matches)
986 1001 self.assertNotIn("'cc', ", matches)
987 1002 self.assertNotIn("'cc']", matches)
988 1003 ip.Completer.auto_close_dict_keys = False
989 1004
990 1005 def test_dict_key_completion_string(self):
991 1006 """Test dictionary key completion for string keys"""
992 1007 ip = get_ipython()
993 1008 complete = ip.Completer.complete
994 1009
995 1010 ip.user_ns["d"] = {"abc": None}
996 1011
997 1012 # check completion at different stages
998 1013 _, matches = complete(line_buffer="d[")
999 1014 self.assertIn("'abc'", matches)
1000 1015 self.assertNotIn("'abc']", matches)
1001 1016
1002 1017 _, matches = complete(line_buffer="d['")
1003 1018 self.assertIn("abc", matches)
1004 1019 self.assertNotIn("abc']", matches)
1005 1020
1006 1021 _, matches = complete(line_buffer="d['a")
1007 1022 self.assertIn("abc", matches)
1008 1023 self.assertNotIn("abc']", matches)
1009 1024
1010 1025 # check use of different quoting
1011 1026 _, matches = complete(line_buffer='d["')
1012 1027 self.assertIn("abc", matches)
1013 1028 self.assertNotIn('abc"]', matches)
1014 1029
1015 1030 _, matches = complete(line_buffer='d["a')
1016 1031 self.assertIn("abc", matches)
1017 1032 self.assertNotIn('abc"]', matches)
1018 1033
1019 1034 # check sensitivity to following context
1020 1035 _, matches = complete(line_buffer="d[]", cursor_pos=2)
1021 1036 self.assertIn("'abc'", matches)
1022 1037
1023 1038 _, matches = complete(line_buffer="d['']", cursor_pos=3)
1024 1039 self.assertIn("abc", matches)
1025 1040 self.assertNotIn("abc'", matches)
1026 1041 self.assertNotIn("abc']", matches)
1027 1042
1028 1043 # check multiple solutions are correctly returned and that noise is not
1029 1044 ip.user_ns["d"] = {
1030 1045 "abc": None,
1031 1046 "abd": None,
1032 1047 "bad": None,
1033 1048 object(): None,
1034 1049 5: None,
1035 1050 ("abe", None): None,
1036 1051 (None, "abf"): None
1037 1052 }
1038 1053
1039 1054 _, matches = complete(line_buffer="d['a")
1040 1055 self.assertIn("abc", matches)
1041 1056 self.assertIn("abd", matches)
1042 1057 self.assertNotIn("bad", matches)
1043 1058 self.assertNotIn("abe", matches)
1044 1059 self.assertNotIn("abf", matches)
1045 1060 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
1046 1061
1047 1062 # check escaping and whitespace
1048 1063 ip.user_ns["d"] = {"a\nb": None, "a'b": None, 'a"b': None, "a word": None}
1049 1064 _, matches = complete(line_buffer="d['a")
1050 1065 self.assertIn("a\\nb", matches)
1051 1066 self.assertIn("a\\'b", matches)
1052 1067 self.assertIn('a"b', matches)
1053 1068 self.assertIn("a word", matches)
1054 1069 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
1055 1070
1056 1071 # - can complete on non-initial word of the string
1057 1072 _, matches = complete(line_buffer="d['a w")
1058 1073 self.assertIn("word", matches)
1059 1074
1060 1075 # - understands quote escaping
1061 1076 _, matches = complete(line_buffer="d['a\\'")
1062 1077 self.assertIn("b", matches)
1063 1078
1064 1079 # - default quoting should work like repr
1065 1080 _, matches = complete(line_buffer="d[")
1066 1081 self.assertIn('"a\'b"', matches)
1067 1082
1068 1083 # - when opening quote with ", possible to match with unescaped apostrophe
1069 1084 _, matches = complete(line_buffer="d[\"a'")
1070 1085 self.assertIn("b", matches)
1071 1086
1072 1087 # need to not split at delims that readline won't split at
1073 1088 if "-" not in ip.Completer.splitter.delims:
1074 1089 ip.user_ns["d"] = {"before-after": None}
1075 1090 _, matches = complete(line_buffer="d['before-af")
1076 1091 self.assertIn("before-after", matches)
1077 1092
1078 1093 # check completion on tuple-of-string keys at different stage - on first key
1079 1094 ip.user_ns["d"] = {('foo', 'bar'): None}
1080 1095 _, matches = complete(line_buffer="d[")
1081 1096 self.assertIn("'foo'", matches)
1082 1097 self.assertNotIn("'foo']", matches)
1083 1098 self.assertNotIn("'bar'", matches)
1084 1099 self.assertNotIn("foo", matches)
1085 1100 self.assertNotIn("bar", matches)
1086 1101
1087 1102 # - match the prefix
1088 1103 _, matches = complete(line_buffer="d['f")
1089 1104 self.assertIn("foo", matches)
1090 1105 self.assertNotIn("foo']", matches)
1091 1106 self.assertNotIn('foo"]', matches)
1092 1107 _, matches = complete(line_buffer="d['foo")
1093 1108 self.assertIn("foo", matches)
1094 1109
1095 1110 # - can complete on second key
1096 1111 _, matches = complete(line_buffer="d['foo', ")
1097 1112 self.assertIn("'bar'", matches)
1098 1113 _, matches = complete(line_buffer="d['foo', 'b")
1099 1114 self.assertIn("bar", matches)
1100 1115 self.assertNotIn("foo", matches)
1101 1116
1102 1117 # - does not propose missing keys
1103 1118 _, matches = complete(line_buffer="d['foo', 'f")
1104 1119 self.assertNotIn("bar", matches)
1105 1120 self.assertNotIn("foo", matches)
1106 1121
1107 1122 # check sensitivity to following context
1108 1123 _, matches = complete(line_buffer="d['foo',]", cursor_pos=8)
1109 1124 self.assertIn("'bar'", matches)
1110 1125 self.assertNotIn("bar", matches)
1111 1126 self.assertNotIn("'foo'", matches)
1112 1127 self.assertNotIn("foo", matches)
1113 1128
1114 1129 _, matches = complete(line_buffer="d['']", cursor_pos=3)
1115 1130 self.assertIn("foo", matches)
1116 1131 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
1117 1132
1118 1133 _, matches = complete(line_buffer='d[""]', cursor_pos=3)
1119 1134 self.assertIn("foo", matches)
1120 1135 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
1121 1136
1122 1137 _, matches = complete(line_buffer='d["foo","]', cursor_pos=9)
1123 1138 self.assertIn("bar", matches)
1124 1139 assert not any(m.endswith(("]", '"', "'")) for m in matches), matches
1125 1140
1126 1141 _, matches = complete(line_buffer='d["foo",]', cursor_pos=8)
1127 1142 self.assertIn("'bar'", matches)
1128 1143 self.assertNotIn("bar", matches)
1129 1144
1130 1145 # Can complete with longer tuple keys
1131 1146 ip.user_ns["d"] = {('foo', 'bar', 'foobar'): None}
1132 1147
1133 1148 # - can complete second key
1134 1149 _, matches = complete(line_buffer="d['foo', 'b")
1135 1150 self.assertIn("bar", matches)
1136 1151 self.assertNotIn("foo", matches)
1137 1152 self.assertNotIn("foobar", matches)
1138 1153
1139 1154 # - can complete third key
1140 1155 _, matches = complete(line_buffer="d['foo', 'bar', 'fo")
1141 1156 self.assertIn("foobar", matches)
1142 1157 self.assertNotIn("foo", matches)
1143 1158 self.assertNotIn("bar", matches)
1144 1159
1145 1160 def test_dict_key_completion_numbers(self):
1146 1161 ip = get_ipython()
1147 1162 complete = ip.Completer.complete
1148 1163
1149 1164 ip.user_ns["d"] = {
1150 1165 0xDEADBEEF: None, # 3735928559
1151 1166 1111: None,
1152 1167 1234: None,
1153 1168 "1999": None,
1154 1169 0b10101: None, # 21
1155 1170 22: None,
1156 1171 }
1157 1172 _, matches = complete(line_buffer="d[1")
1158 1173 self.assertIn("1111", matches)
1159 1174 self.assertIn("1234", matches)
1160 1175 self.assertNotIn("1999", matches)
1161 1176 self.assertNotIn("'1999'", matches)
1162 1177
1163 1178 _, matches = complete(line_buffer="d[0xdead")
1164 1179 self.assertIn("0xdeadbeef", matches)
1165 1180
1166 1181 _, matches = complete(line_buffer="d[2")
1167 1182 self.assertIn("21", matches)
1168 1183 self.assertIn("22", matches)
1169 1184
1170 1185 _, matches = complete(line_buffer="d[0b101")
1171 1186 self.assertIn("0b10101", matches)
1172 1187 self.assertIn("0b10110", matches)
1173 1188
1174 1189 def test_dict_key_completion_contexts(self):
1175 1190 """Test expression contexts in which dict key completion occurs"""
1176 1191 ip = get_ipython()
1177 1192 complete = ip.Completer.complete
1178 1193 d = {"abc": None}
1179 1194 ip.user_ns["d"] = d
1180 1195
1181 1196 class C:
1182 1197 data = d
1183 1198
1184 1199 ip.user_ns["C"] = C
1185 1200 ip.user_ns["get"] = lambda: d
1186 1201 ip.user_ns["nested"] = {"x": d}
1187 1202
1188 1203 def assert_no_completion(**kwargs):
1189 1204 _, matches = complete(**kwargs)
1190 1205 self.assertNotIn("abc", matches)
1191 1206 self.assertNotIn("abc'", matches)
1192 1207 self.assertNotIn("abc']", matches)
1193 1208 self.assertNotIn("'abc'", matches)
1194 1209 self.assertNotIn("'abc']", matches)
1195 1210
1196 1211 def assert_completion(**kwargs):
1197 1212 _, matches = complete(**kwargs)
1198 1213 self.assertIn("'abc'", matches)
1199 1214 self.assertNotIn("'abc']", matches)
1200 1215
1201 1216 # no completion after string closed, even if reopened
1202 1217 assert_no_completion(line_buffer="d['a'")
1203 1218 assert_no_completion(line_buffer='d["a"')
1204 1219 assert_no_completion(line_buffer="d['a' + ")
1205 1220 assert_no_completion(line_buffer="d['a' + '")
1206 1221
1207 1222 # completion in non-trivial expressions
1208 1223 assert_completion(line_buffer="+ d[")
1209 1224 assert_completion(line_buffer="(d[")
1210 1225 assert_completion(line_buffer="C.data[")
1211 1226
1212 1227 # nested dict completion
1213 1228 assert_completion(line_buffer="nested['x'][")
1214 1229
1215 1230 with evaluation_policy("minimal"):
1216 1231 with pytest.raises(AssertionError):
1217 1232 assert_completion(line_buffer="nested['x'][")
1218 1233
1219 1234 # greedy flag
1220 1235 def assert_completion(**kwargs):
1221 1236 _, matches = complete(**kwargs)
1222 1237 self.assertIn("get()['abc']", matches)
1223 1238
1224 1239 assert_no_completion(line_buffer="get()[")
1225 1240 with greedy_completion():
1226 1241 assert_completion(line_buffer="get()[")
1227 1242 assert_completion(line_buffer="get()['")
1228 1243 assert_completion(line_buffer="get()['a")
1229 1244 assert_completion(line_buffer="get()['ab")
1230 1245 assert_completion(line_buffer="get()['abc")
1231 1246
1232 1247 def test_dict_key_completion_bytes(self):
1233 1248 """Test handling of bytes in dict key completion"""
1234 1249 ip = get_ipython()
1235 1250 complete = ip.Completer.complete
1236 1251
1237 1252 ip.user_ns["d"] = {"abc": None, b"abd": None}
1238 1253
1239 1254 _, matches = complete(line_buffer="d[")
1240 1255 self.assertIn("'abc'", matches)
1241 1256 self.assertIn("b'abd'", matches)
1242 1257
1243 1258 if False: # not currently implemented
1244 1259 _, matches = complete(line_buffer="d[b")
1245 1260 self.assertIn("b'abd'", matches)
1246 1261 self.assertNotIn("b'abc'", matches)
1247 1262
1248 1263 _, matches = complete(line_buffer="d[b'")
1249 1264 self.assertIn("abd", matches)
1250 1265 self.assertNotIn("abc", matches)
1251 1266
1252 1267 _, matches = complete(line_buffer="d[B'")
1253 1268 self.assertIn("abd", matches)
1254 1269 self.assertNotIn("abc", matches)
1255 1270
1256 1271 _, matches = complete(line_buffer="d['")
1257 1272 self.assertIn("abc", matches)
1258 1273 self.assertNotIn("abd", matches)
1259 1274
1260 1275 def test_dict_key_completion_unicode_py3(self):
1261 1276 """Test handling of unicode in dict key completion"""
1262 1277 ip = get_ipython()
1263 1278 complete = ip.Completer.complete
1264 1279
1265 1280 ip.user_ns["d"] = {"a\u05d0": None}
1266 1281
1267 1282 # query using escape
1268 1283 if sys.platform != "win32":
1269 1284 # Known failure on Windows
1270 1285 _, matches = complete(line_buffer="d['a\\u05d0")
1271 1286 self.assertIn("u05d0", matches) # tokenized after \\
1272 1287
1273 1288 # query using character
1274 1289 _, matches = complete(line_buffer="d['a\u05d0")
1275 1290 self.assertIn("a\u05d0", matches)
1276 1291
1277 1292 with greedy_completion():
1278 1293 # query using escape
1279 1294 _, matches = complete(line_buffer="d['a\\u05d0")
1280 1295 self.assertIn("d['a\\u05d0']", matches) # tokenized after \\
1281 1296
1282 1297 # query using character
1283 1298 _, matches = complete(line_buffer="d['a\u05d0")
1284 1299 self.assertIn("d['a\u05d0']", matches)
1285 1300
1286 1301 @dec.skip_without("numpy")
1287 1302 def test_struct_array_key_completion(self):
1288 1303 """Test dict key completion applies to numpy struct arrays"""
1289 1304 import numpy
1290 1305
1291 1306 ip = get_ipython()
1292 1307 complete = ip.Completer.complete
1293 1308 ip.user_ns["d"] = numpy.array([], dtype=[("hello", "f"), ("world", "f")])
1294 1309 _, matches = complete(line_buffer="d['")
1295 1310 self.assertIn("hello", matches)
1296 1311 self.assertIn("world", matches)
1297 1312 # complete on the numpy struct itself
1298 1313 dt = numpy.dtype(
1299 1314 [("my_head", [("my_dt", ">u4"), ("my_df", ">u4")]), ("my_data", ">f4", 5)]
1300 1315 )
1301 1316 x = numpy.zeros(2, dtype=dt)
1302 1317 ip.user_ns["d"] = x[1]
1303 1318 _, matches = complete(line_buffer="d['")
1304 1319 self.assertIn("my_head", matches)
1305 1320 self.assertIn("my_data", matches)
1306 1321
1307 1322 def completes_on_nested():
1308 1323 ip.user_ns["d"] = numpy.zeros(2, dtype=dt)
1309 1324 _, matches = complete(line_buffer="d[1]['my_head']['")
1310 1325 self.assertTrue(any(["my_dt" in m for m in matches]))
1311 1326 self.assertTrue(any(["my_df" in m for m in matches]))
1312 1327 # complete on a nested level
1313 1328 with greedy_completion():
1314 1329 completes_on_nested()
1315 1330
1316 1331 with evaluation_policy("limited"):
1317 1332 completes_on_nested()
1318 1333
1319 1334 with evaluation_policy("minimal"):
1320 1335 with pytest.raises(AssertionError):
1321 1336 completes_on_nested()
1322 1337
1323 1338 @dec.skip_without("pandas")
1324 1339 def test_dataframe_key_completion(self):
1325 1340 """Test dict key completion applies to pandas DataFrames"""
1326 1341 import pandas
1327 1342
1328 1343 ip = get_ipython()
1329 1344 complete = ip.Completer.complete
1330 1345 ip.user_ns["d"] = pandas.DataFrame({"hello": [1], "world": [2]})
1331 1346 _, matches = complete(line_buffer="d['")
1332 1347 self.assertIn("hello", matches)
1333 1348 self.assertIn("world", matches)
1334 1349 _, matches = complete(line_buffer="d.loc[:, '")
1335 1350 self.assertIn("hello", matches)
1336 1351 self.assertIn("world", matches)
1337 1352 _, matches = complete(line_buffer="d.loc[1:, '")
1338 1353 self.assertIn("hello", matches)
1339 1354 _, matches = complete(line_buffer="d.loc[1:1, '")
1340 1355 self.assertIn("hello", matches)
1341 1356 _, matches = complete(line_buffer="d.loc[1:1:-1, '")
1342 1357 self.assertIn("hello", matches)
1343 1358 _, matches = complete(line_buffer="d.loc[::, '")
1344 1359 self.assertIn("hello", matches)
1345 1360
1346 1361 def test_dict_key_completion_invalids(self):
1347 1362 """Smoke test cases dict key completion can't handle"""
1348 1363 ip = get_ipython()
1349 1364 complete = ip.Completer.complete
1350 1365
1351 1366 ip.user_ns["no_getitem"] = None
1352 1367 ip.user_ns["no_keys"] = []
1353 1368 ip.user_ns["cant_call_keys"] = dict
1354 1369 ip.user_ns["empty"] = {}
1355 1370 ip.user_ns["d"] = {"abc": 5}
1356 1371
1357 1372 _, matches = complete(line_buffer="no_getitem['")
1358 1373 _, matches = complete(line_buffer="no_keys['")
1359 1374 _, matches = complete(line_buffer="cant_call_keys['")
1360 1375 _, matches = complete(line_buffer="empty['")
1361 1376 _, matches = complete(line_buffer="name_error['")
1362 1377 _, matches = complete(line_buffer="d['\\") # incomplete escape
1363 1378
1364 1379 def test_object_key_completion(self):
1365 1380 ip = get_ipython()
1366 1381 ip.user_ns["key_completable"] = KeyCompletable(["qwerty", "qwick"])
1367 1382
1368 1383 _, matches = ip.Completer.complete(line_buffer="key_completable['qw")
1369 1384 self.assertIn("qwerty", matches)
1370 1385 self.assertIn("qwick", matches)
1371 1386
1372 1387 def test_class_key_completion(self):
1373 1388 ip = get_ipython()
1374 1389 NamedInstanceClass("qwerty")
1375 1390 NamedInstanceClass("qwick")
1376 1391 ip.user_ns["named_instance_class"] = NamedInstanceClass
1377 1392
1378 1393 _, matches = ip.Completer.complete(line_buffer="named_instance_class['qw")
1379 1394 self.assertIn("qwerty", matches)
1380 1395 self.assertIn("qwick", matches)
1381 1396
1382 1397 def test_tryimport(self):
1383 1398 """
1384 1399 Test that try-import don't crash on trailing dot, and import modules before
1385 1400 """
1386 1401 from IPython.core.completerlib import try_import
1387 1402
1388 1403 assert try_import("IPython.")
1389 1404
1390 1405 def test_aimport_module_completer(self):
1391 1406 ip = get_ipython()
1392 1407 _, matches = ip.complete("i", "%aimport i")
1393 1408 self.assertIn("io", matches)
1394 1409 self.assertNotIn("int", matches)
1395 1410
1396 1411 def test_nested_import_module_completer(self):
1397 1412 ip = get_ipython()
1398 1413 _, matches = ip.complete(None, "import IPython.co", 17)
1399 1414 self.assertIn("IPython.core", matches)
1400 1415 self.assertNotIn("import IPython.core", matches)
1401 1416 self.assertNotIn("IPython.display", matches)
1402 1417
1403 1418 def test_import_module_completer(self):
1404 1419 ip = get_ipython()
1405 1420 _, matches = ip.complete("i", "import i")
1406 1421 self.assertIn("io", matches)
1407 1422 self.assertNotIn("int", matches)
1408 1423
1409 1424 def test_from_module_completer(self):
1410 1425 ip = get_ipython()
1411 1426 _, matches = ip.complete("B", "from io import B", 16)
1412 1427 self.assertIn("BytesIO", matches)
1413 1428 self.assertNotIn("BaseException", matches)
1414 1429
1415 1430 def test_snake_case_completion(self):
1416 1431 ip = get_ipython()
1417 1432 ip.Completer.use_jedi = False
1418 1433 ip.user_ns["some_three"] = 3
1419 1434 ip.user_ns["some_four"] = 4
1420 1435 _, matches = ip.complete("s_", "print(s_f")
1421 1436 self.assertIn("some_three", matches)
1422 1437 self.assertIn("some_four", matches)
1423 1438
1424 1439 def test_mix_terms(self):
1425 1440 ip = get_ipython()
1426 1441 from textwrap import dedent
1427 1442
1428 1443 ip.Completer.use_jedi = False
1429 1444 ip.ex(
1430 1445 dedent(
1431 1446 """
1432 1447 class Test:
1433 1448 def meth(self, meth_arg1):
1434 1449 print("meth")
1435 1450
1436 1451 def meth_1(self, meth1_arg1, meth1_arg2):
1437 1452 print("meth1")
1438 1453
1439 1454 def meth_2(self, meth2_arg1, meth2_arg2):
1440 1455 print("meth2")
1441 1456 test = Test()
1442 1457 """
1443 1458 )
1444 1459 )
1445 1460 _, matches = ip.complete(None, "test.meth(")
1446 1461 self.assertIn("meth_arg1=", matches)
1447 1462 self.assertNotIn("meth2_arg1=", matches)
1448 1463
1449 1464 def test_percent_symbol_restrict_to_magic_completions(self):
1450 1465 ip = get_ipython()
1451 1466 completer = ip.Completer
1452 1467 text = "%a"
1453 1468
1454 1469 with provisionalcompleter():
1455 1470 completer.use_jedi = True
1456 1471 completions = completer.completions(text, len(text))
1457 1472 for c in completions:
1458 1473 self.assertEqual(c.text[0], "%")
1459 1474
1460 1475 def test_fwd_unicode_restricts(self):
1461 1476 ip = get_ipython()
1462 1477 completer = ip.Completer
1463 1478 text = "\\ROMAN NUMERAL FIVE"
1464 1479
1465 1480 with provisionalcompleter():
1466 1481 completer.use_jedi = True
1467 1482 completions = [
1468 1483 completion.text for completion in completer.completions(text, len(text))
1469 1484 ]
1470 1485 self.assertEqual(completions, ["\u2164"])
1471 1486
1472 1487 def test_dict_key_restrict_to_dicts(self):
1473 1488 """Test that dict key suppresses non-dict completion items"""
1474 1489 ip = get_ipython()
1475 1490 c = ip.Completer
1476 1491 d = {"abc": None}
1477 1492 ip.user_ns["d"] = d
1478 1493
1479 1494 text = 'd["a'
1480 1495
1481 1496 def _():
1482 1497 with provisionalcompleter():
1483 1498 c.use_jedi = True
1484 1499 return [
1485 1500 completion.text for completion in c.completions(text, len(text))
1486 1501 ]
1487 1502
1488 1503 completions = _()
1489 1504 self.assertEqual(completions, ["abc"])
1490 1505
1491 1506 # check that it can be disabled in granular manner:
1492 1507 cfg = Config()
1493 1508 cfg.IPCompleter.suppress_competing_matchers = {
1494 1509 "IPCompleter.dict_key_matcher": False
1495 1510 }
1496 1511 c.update_config(cfg)
1497 1512
1498 1513 completions = _()
1499 1514 self.assertIn("abc", completions)
1500 1515 self.assertGreater(len(completions), 1)
1501 1516
1502 1517 def test_matcher_suppression(self):
1503 1518 @completion_matcher(identifier="a_matcher")
1504 1519 def a_matcher(text):
1505 1520 return ["completion_a"]
1506 1521
1507 1522 @completion_matcher(identifier="b_matcher", api_version=2)
1508 1523 def b_matcher(context: CompletionContext):
1509 1524 text = context.token
1510 1525 result = {"completions": [SimpleCompletion("completion_b")]}
1511 1526
1512 1527 if text == "suppress c":
1513 1528 result["suppress"] = {"c_matcher"}
1514 1529
1515 1530 if text.startswith("suppress all"):
1516 1531 result["suppress"] = True
1517 1532 if text == "suppress all but c":
1518 1533 result["do_not_suppress"] = {"c_matcher"}
1519 1534 if text == "suppress all but a":
1520 1535 result["do_not_suppress"] = {"a_matcher"}
1521 1536
1522 1537 return result
1523 1538
1524 1539 @completion_matcher(identifier="c_matcher")
1525 1540 def c_matcher(text):
1526 1541 return ["completion_c"]
1527 1542
1528 1543 with custom_matchers([a_matcher, b_matcher, c_matcher]):
1529 1544 ip = get_ipython()
1530 1545 c = ip.Completer
1531 1546
1532 1547 def _(text, expected):
1533 1548 c.use_jedi = False
1534 1549 s, matches = c.complete(text)
1535 1550 self.assertEqual(expected, matches)
1536 1551
1537 1552 _("do not suppress", ["completion_a", "completion_b", "completion_c"])
1538 1553 _("suppress all", ["completion_b"])
1539 1554 _("suppress all but a", ["completion_a", "completion_b"])
1540 1555 _("suppress all but c", ["completion_b", "completion_c"])
1541 1556
1542 1557 def configure(suppression_config):
1543 1558 cfg = Config()
1544 1559 cfg.IPCompleter.suppress_competing_matchers = suppression_config
1545 1560 c.update_config(cfg)
1546 1561
1547 1562 # test that configuration takes priority over the run-time decisions
1548 1563
1549 1564 configure(False)
1550 1565 _("suppress all", ["completion_a", "completion_b", "completion_c"])
1551 1566
1552 1567 configure({"b_matcher": False})
1553 1568 _("suppress all", ["completion_a", "completion_b", "completion_c"])
1554 1569
1555 1570 configure({"a_matcher": False})
1556 1571 _("suppress all", ["completion_b"])
1557 1572
1558 1573 configure({"b_matcher": True})
1559 1574 _("do not suppress", ["completion_b"])
1560 1575
1561 1576 configure(True)
1562 1577 _("do not suppress", ["completion_a"])
1563 1578
1564 1579 def test_matcher_suppression_with_iterator(self):
1565 1580 @completion_matcher(identifier="matcher_returning_iterator")
1566 1581 def matcher_returning_iterator(text):
1567 1582 return iter(["completion_iter"])
1568 1583
1569 1584 @completion_matcher(identifier="matcher_returning_list")
1570 1585 def matcher_returning_list(text):
1571 1586 return ["completion_list"]
1572 1587
1573 1588 with custom_matchers([matcher_returning_iterator, matcher_returning_list]):
1574 1589 ip = get_ipython()
1575 1590 c = ip.Completer
1576 1591
1577 1592 def _(text, expected):
1578 1593 c.use_jedi = False
1579 1594 s, matches = c.complete(text)
1580 1595 self.assertEqual(expected, matches)
1581 1596
1582 1597 def configure(suppression_config):
1583 1598 cfg = Config()
1584 1599 cfg.IPCompleter.suppress_competing_matchers = suppression_config
1585 1600 c.update_config(cfg)
1586 1601
1587 1602 configure(False)
1588 1603 _("---", ["completion_iter", "completion_list"])
1589 1604
1590 1605 configure(True)
1591 1606 _("---", ["completion_iter"])
1592 1607
1593 1608 configure(None)
1594 1609 _("--", ["completion_iter", "completion_list"])
1595 1610
1596 1611 def test_matcher_suppression_with_jedi(self):
1597 1612 ip = get_ipython()
1598 1613 c = ip.Completer
1599 1614 c.use_jedi = True
1600 1615
1601 1616 def configure(suppression_config):
1602 1617 cfg = Config()
1603 1618 cfg.IPCompleter.suppress_competing_matchers = suppression_config
1604 1619 c.update_config(cfg)
1605 1620
1606 1621 def _():
1607 1622 with provisionalcompleter():
1608 1623 matches = [completion.text for completion in c.completions("dict.", 5)]
1609 1624 self.assertIn("keys", matches)
1610 1625
1611 1626 configure(False)
1612 1627 _()
1613 1628
1614 1629 configure(True)
1615 1630 _()
1616 1631
1617 1632 configure(None)
1618 1633 _()
1619 1634
1620 1635 def test_matcher_disabling(self):
1621 1636 @completion_matcher(identifier="a_matcher")
1622 1637 def a_matcher(text):
1623 1638 return ["completion_a"]
1624 1639
1625 1640 @completion_matcher(identifier="b_matcher")
1626 1641 def b_matcher(text):
1627 1642 return ["completion_b"]
1628 1643
1629 1644 def _(expected):
1630 1645 s, matches = c.complete("completion_")
1631 1646 self.assertEqual(expected, matches)
1632 1647
1633 1648 with custom_matchers([a_matcher, b_matcher]):
1634 1649 ip = get_ipython()
1635 1650 c = ip.Completer
1636 1651
1637 1652 _(["completion_a", "completion_b"])
1638 1653
1639 1654 cfg = Config()
1640 1655 cfg.IPCompleter.disable_matchers = ["b_matcher"]
1641 1656 c.update_config(cfg)
1642 1657
1643 1658 _(["completion_a"])
1644 1659
1645 1660 cfg.IPCompleter.disable_matchers = []
1646 1661 c.update_config(cfg)
1647 1662
1648 1663 def test_matcher_priority(self):
1649 1664 @completion_matcher(identifier="a_matcher", priority=0, api_version=2)
1650 1665 def a_matcher(text):
1651 1666 return {"completions": [SimpleCompletion("completion_a")], "suppress": True}
1652 1667
1653 1668 @completion_matcher(identifier="b_matcher", priority=2, api_version=2)
1654 1669 def b_matcher(text):
1655 1670 return {"completions": [SimpleCompletion("completion_b")], "suppress": True}
1656 1671
1657 1672 def _(expected):
1658 1673 s, matches = c.complete("completion_")
1659 1674 self.assertEqual(expected, matches)
1660 1675
1661 1676 with custom_matchers([a_matcher, b_matcher]):
1662 1677 ip = get_ipython()
1663 1678 c = ip.Completer
1664 1679
1665 1680 _(["completion_b"])
1666 1681 a_matcher.matcher_priority = 3
1667 1682 _(["completion_a"])
1668 1683
1669 1684
1670 1685 @pytest.mark.parametrize(
1671 1686 "input, expected",
1672 1687 [
1673 1688 ["1.234", "1.234"],
1674 1689 # should match signed numbers
1675 1690 ["+1", "+1"],
1676 1691 ["-1", "-1"],
1677 1692 ["-1.0", "-1.0"],
1678 1693 ["-1.", "-1."],
1679 1694 ["+1.", "+1."],
1680 1695 [".1", ".1"],
1681 1696 # should not match non-numbers
1682 1697 ["1..", None],
1683 1698 ["..", None],
1684 1699 [".1.", None],
1685 1700 # should match after comma
1686 1701 [",1", "1"],
1687 1702 [", 1", "1"],
1688 1703 [", .1", ".1"],
1689 1704 [", +.1", "+.1"],
1690 1705 # should not match after trailing spaces
1691 1706 [".1 ", None],
1692 1707 # some complex cases
1693 1708 ["0b_0011_1111_0100_1110", "0b_0011_1111_0100_1110"],
1694 1709 ["0xdeadbeef", "0xdeadbeef"],
1695 1710 ["0b_1110_0101", "0b_1110_0101"],
1696 1711 # should not match if in an operation
1697 1712 ["1 + 1", None],
1698 1713 [", 1 + 1", None],
1699 1714 ],
1700 1715 )
1701 1716 def test_match_numeric_literal_for_dict_key(input, expected):
1702 1717 assert _match_number_in_dict_key_prefix(input) == expected
General Comments 0
You need to be logged in to leave comments. Login now