##// END OF EJS Templates
change regex fullmatch to match for <3.3 compatibility
sagnak -
Show More
@@ -1,1888 +1,1888 b''
1 # encoding: utf-8
1 # encoding: utf-8
2 """Completion for IPython.
2 """Completion for IPython.
3
3
4 This module started as fork of the rlcompleter module in the Python standard
4 This module started as fork of the rlcompleter module in the Python standard
5 library. The original enhancements made to rlcompleter have been sent
5 library. The original enhancements made to rlcompleter have been sent
6 upstream and were accepted as of Python 2.3,
6 upstream and were accepted as of Python 2.3,
7
7
8 This module now support a wide variety of completion mechanism both available
8 This module now support a wide variety of completion mechanism both available
9 for normal classic Python code, as well as completer for IPython specific
9 for normal classic Python code, as well as completer for IPython specific
10 Syntax like magics.
10 Syntax like magics.
11
11
12 Latex and Unicode completion
12 Latex and Unicode completion
13 ============================
13 ============================
14
14
15 IPython and compatible frontends not only can complete your code, but can help
15 IPython and compatible frontends not only can complete your code, but can help
16 you to input a wide range of characters. In particular we allow you to insert
16 you to input a wide range of characters. In particular we allow you to insert
17 a unicode character using the tab completion mechanism.
17 a unicode character using the tab completion mechanism.
18
18
19 Forward latex/unicode completion
19 Forward latex/unicode completion
20 --------------------------------
20 --------------------------------
21
21
22 Forward completion allows you to easily type a unicode character using its latex
22 Forward completion allows you to easily type a unicode character using its latex
23 name, or unicode long description. To do so type a backslash follow by the
23 name, or unicode long description. To do so type a backslash follow by the
24 relevant name and press tab:
24 relevant name and press tab:
25
25
26
26
27 Using latex completion:
27 Using latex completion:
28
28
29 .. code::
29 .. code::
30
30
31 \\alpha<tab>
31 \\alpha<tab>
32 Ξ±
32 Ξ±
33
33
34 or using unicode completion:
34 or using unicode completion:
35
35
36
36
37 .. code::
37 .. code::
38
38
39 \\greek small letter alpha<tab>
39 \\greek small letter alpha<tab>
40 Ξ±
40 Ξ±
41
41
42
42
43 Only valid Python identifiers will complete. Combining characters (like arrow or
43 Only valid Python identifiers will complete. Combining characters (like arrow or
44 dots) are also available, unlike latex they need to be put after the their
44 dots) are also available, unlike latex they need to be put after the their
45 counterpart that is to say, `F\\\\vec<tab>` is correct, not `\\\\vec<tab>F`.
45 counterpart that is to say, `F\\\\vec<tab>` is correct, not `\\\\vec<tab>F`.
46
46
47 Some browsers are known to display combining characters incorrectly.
47 Some browsers are known to display combining characters incorrectly.
48
48
49 Backward latex completion
49 Backward latex completion
50 -------------------------
50 -------------------------
51
51
52 It is sometime challenging to know how to type a character, if you are using
52 It is sometime challenging to know how to type a character, if you are using
53 IPython, or any compatible frontend you can prepend backslash to the character
53 IPython, or any compatible frontend you can prepend backslash to the character
54 and press `<tab>` to expand it to its latex form.
54 and press `<tab>` to expand it to its latex form.
55
55
56 .. code::
56 .. code::
57
57
58 \\Ξ±<tab>
58 \\Ξ±<tab>
59 \\alpha
59 \\alpha
60
60
61
61
62 Both forward and backward completions can be deactivated by setting the
62 Both forward and backward completions can be deactivated by setting the
63 ``Completer.backslash_combining_completions`` option to ``False``.
63 ``Completer.backslash_combining_completions`` option to ``False``.
64
64
65
65
66 Experimental
66 Experimental
67 ============
67 ============
68
68
69 Starting with IPython 6.0, this module can make use of the Jedi library to
69 Starting with IPython 6.0, this module can make use of the Jedi library to
70 generate completions both using static analysis of the code, and dynamically
70 generate completions both using static analysis of the code, and dynamically
71 inspecting multiple namespaces. The APIs attached to this new mechanism is
71 inspecting multiple namespaces. The APIs attached to this new mechanism is
72 unstable and will raise unless use in an :any:`provisionalcompleter` context
72 unstable and will raise unless use in an :any:`provisionalcompleter` context
73 manager.
73 manager.
74
74
75 You will find that the following are experimental:
75 You will find that the following are experimental:
76
76
77 - :any:`provisionalcompleter`
77 - :any:`provisionalcompleter`
78 - :any:`IPCompleter.completions`
78 - :any:`IPCompleter.completions`
79 - :any:`Completion`
79 - :any:`Completion`
80 - :any:`rectify_completions`
80 - :any:`rectify_completions`
81
81
82 .. note::
82 .. note::
83
83
84 better name for :any:`rectify_completions` ?
84 better name for :any:`rectify_completions` ?
85
85
86 We welcome any feedback on these new API, and we also encourage you to try this
86 We welcome any feedback on these new API, and we also encourage you to try this
87 module in debug mode (start IPython with ``--Completer.debug=True``) in order
87 module in debug mode (start IPython with ``--Completer.debug=True``) in order
88 to have extra logging information is :any:`jedi` is crashing, or if current
88 to have extra logging information is :any:`jedi` is crashing, or if current
89 IPython completer pending deprecations are returning results not yet handled
89 IPython completer pending deprecations are returning results not yet handled
90 by :any:`jedi`.
90 by :any:`jedi`.
91
91
92 Using Jedi for tab completion allow snippets like the following to work without
92 Using Jedi for tab completion allow snippets like the following to work without
93 having to execute any code:
93 having to execute any code:
94
94
95 >>> myvar = ['hello', 42]
95 >>> myvar = ['hello', 42]
96 ... myvar[1].bi<tab>
96 ... myvar[1].bi<tab>
97
97
98 Tab completion will be able to infer that ``myvar[1]`` is a real number without
98 Tab completion will be able to infer that ``myvar[1]`` is a real number without
99 executing any code unlike the previously available ``IPCompleter.greedy``
99 executing any code unlike the previously available ``IPCompleter.greedy``
100 option.
100 option.
101
101
102 Be sure to update :any:`jedi` to the latest stable version or to try the
102 Be sure to update :any:`jedi` to the latest stable version or to try the
103 current development version to get better completions.
103 current development version to get better completions.
104 """
104 """
105
105
106 # skip module docstests
106 # skip module docstests
107 skip_doctest = True
107 skip_doctest = True
108
108
109 # Copyright (c) IPython Development Team.
109 # Copyright (c) IPython Development Team.
110 # Distributed under the terms of the Modified BSD License.
110 # Distributed under the terms of the Modified BSD License.
111 #
111 #
112 # Some of this code originated from rlcompleter in the Python standard library
112 # Some of this code originated from rlcompleter in the Python standard library
113 # Copyright (C) 2001 Python Software Foundation, www.python.org
113 # Copyright (C) 2001 Python Software Foundation, www.python.org
114
114
115
115
116 import __main__
116 import __main__
117 import builtins as builtin_mod
117 import builtins as builtin_mod
118 import glob
118 import glob
119 import time
119 import time
120 import inspect
120 import inspect
121 import itertools
121 import itertools
122 import keyword
122 import keyword
123 import os
123 import os
124 import re
124 import re
125 import sys
125 import sys
126 import unicodedata
126 import unicodedata
127 import string
127 import string
128 import warnings
128 import warnings
129
129
130 from contextlib import contextmanager
130 from contextlib import contextmanager
131 from importlib import import_module
131 from importlib import import_module
132 from typing import Iterator, List
132 from typing import Iterator, List
133 from types import SimpleNamespace
133 from types import SimpleNamespace
134
134
135 from traitlets.config.configurable import Configurable
135 from traitlets.config.configurable import Configurable
136 from IPython.core.error import TryNext
136 from IPython.core.error import TryNext
137 from IPython.core.inputsplitter import ESC_MAGIC
137 from IPython.core.inputsplitter import ESC_MAGIC
138 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
138 from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
139 from IPython.utils import generics
139 from IPython.utils import generics
140 from IPython.utils.dir2 import dir2, get_real_method
140 from IPython.utils.dir2 import dir2, get_real_method
141 from IPython.utils.process import arg_split
141 from IPython.utils.process import arg_split
142 from IPython.utils.py3compat import cast_unicode_py2
142 from IPython.utils.py3compat import cast_unicode_py2
143 from traitlets import Bool, Enum, observe, Int
143 from traitlets import Bool, Enum, observe, Int
144
144
145 try:
145 try:
146 import jedi
146 import jedi
147 import jedi.api.helpers
147 import jedi.api.helpers
148 JEDI_INSTALLED = True
148 JEDI_INSTALLED = True
149 except ImportError:
149 except ImportError:
150 JEDI_INSTALLED = False
150 JEDI_INSTALLED = False
151 #-----------------------------------------------------------------------------
151 #-----------------------------------------------------------------------------
152 # Globals
152 # Globals
153 #-----------------------------------------------------------------------------
153 #-----------------------------------------------------------------------------
154
154
155 # Public API
155 # Public API
156 __all__ = ['Completer','IPCompleter']
156 __all__ = ['Completer','IPCompleter']
157
157
158 if sys.platform == 'win32':
158 if sys.platform == 'win32':
159 PROTECTABLES = ' '
159 PROTECTABLES = ' '
160 else:
160 else:
161 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
161 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
162
162
163
163
164 _deprecation_readline_sentinel = object()
164 _deprecation_readline_sentinel = object()
165
165
166
166
167 class ProvisionalCompleterWarning(FutureWarning):
167 class ProvisionalCompleterWarning(FutureWarning):
168 """
168 """
169 Exception raise by an experimental feature in this module.
169 Exception raise by an experimental feature in this module.
170
170
171 Wrap code in :any:`provisionalcompleter` context manager if you
171 Wrap code in :any:`provisionalcompleter` context manager if you
172 are certain you want to use an unstable feature.
172 are certain you want to use an unstable feature.
173 """
173 """
174 pass
174 pass
175
175
176 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
176 warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
177
177
178 @contextmanager
178 @contextmanager
179 def provisionalcompleter(action='ignore'):
179 def provisionalcompleter(action='ignore'):
180 """
180 """
181
181
182
182
183 This contest manager has to be used in any place where unstable completer
183 This contest manager has to be used in any place where unstable completer
184 behavior and API may be called.
184 behavior and API may be called.
185
185
186 >>> with provisionalcompleter():
186 >>> with provisionalcompleter():
187 ... completer.do_experimetal_things() # works
187 ... completer.do_experimetal_things() # works
188
188
189 >>> completer.do_experimental_things() # raises.
189 >>> completer.do_experimental_things() # raises.
190
190
191 .. note:: Unstable
191 .. note:: Unstable
192
192
193 By using this context manager you agree that the API in use may change
193 By using this context manager you agree that the API in use may change
194 without warning, and that you won't complain if they do so.
194 without warning, and that you won't complain if they do so.
195
195
196 You also understand that if the API is not to you liking you should report
196 You also understand that if the API is not to you liking you should report
197 a bug to explain your use case upstream and improve the API and will loose
197 a bug to explain your use case upstream and improve the API and will loose
198 credibility if you complain after the API is make stable.
198 credibility if you complain after the API is make stable.
199
199
200 We'll be happy to get your feedback , feature request and improvement on
200 We'll be happy to get your feedback , feature request and improvement on
201 any of the unstable APIs !
201 any of the unstable APIs !
202 """
202 """
203 with warnings.catch_warnings():
203 with warnings.catch_warnings():
204 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
204 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
205 yield
205 yield
206
206
207
207
208 def has_open_quotes(s):
208 def has_open_quotes(s):
209 """Return whether a string has open quotes.
209 """Return whether a string has open quotes.
210
210
211 This simply counts whether the number of quote characters of either type in
211 This simply counts whether the number of quote characters of either type in
212 the string is odd.
212 the string is odd.
213
213
214 Returns
214 Returns
215 -------
215 -------
216 If there is an open quote, the quote character is returned. Else, return
216 If there is an open quote, the quote character is returned. Else, return
217 False.
217 False.
218 """
218 """
219 # We check " first, then ', so complex cases with nested quotes will get
219 # We check " first, then ', so complex cases with nested quotes will get
220 # the " to take precedence.
220 # the " to take precedence.
221 if s.count('"') % 2:
221 if s.count('"') % 2:
222 return '"'
222 return '"'
223 elif s.count("'") % 2:
223 elif s.count("'") % 2:
224 return "'"
224 return "'"
225 else:
225 else:
226 return False
226 return False
227
227
228
228
229 def protect_filename(s):
229 def protect_filename(s):
230 """Escape a string to protect certain characters."""
230 """Escape a string to protect certain characters."""
231 if set(s) & set(PROTECTABLES):
231 if set(s) & set(PROTECTABLES):
232 if sys.platform == "win32":
232 if sys.platform == "win32":
233 return '"' + s + '"'
233 return '"' + s + '"'
234 else:
234 else:
235 return "".join(("\\" + c if c in PROTECTABLES else c) for c in s)
235 return "".join(("\\" + c if c in PROTECTABLES else c) for c in s)
236 else:
236 else:
237 return s
237 return s
238
238
239
239
240 def expand_user(path):
240 def expand_user(path):
241 """Expand ``~``-style usernames in strings.
241 """Expand ``~``-style usernames in strings.
242
242
243 This is similar to :func:`os.path.expanduser`, but it computes and returns
243 This is similar to :func:`os.path.expanduser`, but it computes and returns
244 extra information that will be useful if the input was being used in
244 extra information that will be useful if the input was being used in
245 computing completions, and you wish to return the completions with the
245 computing completions, and you wish to return the completions with the
246 original '~' instead of its expanded value.
246 original '~' instead of its expanded value.
247
247
248 Parameters
248 Parameters
249 ----------
249 ----------
250 path : str
250 path : str
251 String to be expanded. If no ~ is present, the output is the same as the
251 String to be expanded. If no ~ is present, the output is the same as the
252 input.
252 input.
253
253
254 Returns
254 Returns
255 -------
255 -------
256 newpath : str
256 newpath : str
257 Result of ~ expansion in the input path.
257 Result of ~ expansion in the input path.
258 tilde_expand : bool
258 tilde_expand : bool
259 Whether any expansion was performed or not.
259 Whether any expansion was performed or not.
260 tilde_val : str
260 tilde_val : str
261 The value that ~ was replaced with.
261 The value that ~ was replaced with.
262 """
262 """
263 # Default values
263 # Default values
264 tilde_expand = False
264 tilde_expand = False
265 tilde_val = ''
265 tilde_val = ''
266 newpath = path
266 newpath = path
267
267
268 if path.startswith('~'):
268 if path.startswith('~'):
269 tilde_expand = True
269 tilde_expand = True
270 rest = len(path)-1
270 rest = len(path)-1
271 newpath = os.path.expanduser(path)
271 newpath = os.path.expanduser(path)
272 if rest:
272 if rest:
273 tilde_val = newpath[:-rest]
273 tilde_val = newpath[:-rest]
274 else:
274 else:
275 tilde_val = newpath
275 tilde_val = newpath
276
276
277 return newpath, tilde_expand, tilde_val
277 return newpath, tilde_expand, tilde_val
278
278
279
279
280 def compress_user(path, tilde_expand, tilde_val):
280 def compress_user(path, tilde_expand, tilde_val):
281 """Does the opposite of expand_user, with its outputs.
281 """Does the opposite of expand_user, with its outputs.
282 """
282 """
283 if tilde_expand:
283 if tilde_expand:
284 return path.replace(tilde_val, '~')
284 return path.replace(tilde_val, '~')
285 else:
285 else:
286 return path
286 return path
287
287
288
288
289 def completions_sorting_key(word):
289 def completions_sorting_key(word):
290 """key for sorting completions
290 """key for sorting completions
291
291
292 This does several things:
292 This does several things:
293
293
294 - Lowercase all completions, so they are sorted alphabetically with
294 - Lowercase all completions, so they are sorted alphabetically with
295 upper and lower case words mingled
295 upper and lower case words mingled
296 - Demote any completions starting with underscores to the end
296 - Demote any completions starting with underscores to the end
297 - Insert any %magic and %%cellmagic completions in the alphabetical order
297 - Insert any %magic and %%cellmagic completions in the alphabetical order
298 by their name
298 by their name
299 """
299 """
300 # Case insensitive sort
300 # Case insensitive sort
301 word = word.lower()
301 word = word.lower()
302
302
303 prio1, prio2 = 0, 0
303 prio1, prio2 = 0, 0
304
304
305 if word.startswith('__'):
305 if word.startswith('__'):
306 prio1 = 2
306 prio1 = 2
307 elif word.startswith('_'):
307 elif word.startswith('_'):
308 prio1 = 1
308 prio1 = 1
309
309
310 if word.endswith('='):
310 if word.endswith('='):
311 prio1 = -1
311 prio1 = -1
312
312
313 if word.startswith('%%'):
313 if word.startswith('%%'):
314 # If there's another % in there, this is something else, so leave it alone
314 # If there's another % in there, this is something else, so leave it alone
315 if not "%" in word[2:]:
315 if not "%" in word[2:]:
316 word = word[2:]
316 word = word[2:]
317 prio2 = 2
317 prio2 = 2
318 elif word.startswith('%'):
318 elif word.startswith('%'):
319 if not "%" in word[1:]:
319 if not "%" in word[1:]:
320 word = word[1:]
320 word = word[1:]
321 prio2 = 1
321 prio2 = 1
322
322
323 return prio1, word, prio2
323 return prio1, word, prio2
324
324
325
325
326 class _FakeJediCompletion:
326 class _FakeJediCompletion:
327 """
327 """
328 This is a workaround to communicate to the UI that Jedi has crashed and to
328 This is a workaround to communicate to the UI that Jedi has crashed and to
329 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
329 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
330
330
331 Added in IPython 6.0 so should likely be removed for 7.0
331 Added in IPython 6.0 so should likely be removed for 7.0
332
332
333 """
333 """
334
334
335 def __init__(self, name):
335 def __init__(self, name):
336
336
337 self.name = name
337 self.name = name
338 self.complete = name
338 self.complete = name
339 self.type = 'crashed'
339 self.type = 'crashed'
340 self.name_with_symbols = name
340 self.name_with_symbols = name
341
341
342 def __repr__(self):
342 def __repr__(self):
343 return '<Fake completion object jedi has crashed>'
343 return '<Fake completion object jedi has crashed>'
344
344
345
345
346 class Completion:
346 class Completion:
347 """
347 """
348 Completion object used and return by IPython completers.
348 Completion object used and return by IPython completers.
349
349
350 .. warning:: Unstable
350 .. warning:: Unstable
351
351
352 This function is unstable, API may change without warning.
352 This function is unstable, API may change without warning.
353 It will also raise unless use in proper context manager.
353 It will also raise unless use in proper context manager.
354
354
355 This act as a middle ground :any:`Completion` object between the
355 This act as a middle ground :any:`Completion` object between the
356 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
356 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
357 object. While Jedi need a lot of information about evaluator and how the
357 object. While Jedi need a lot of information about evaluator and how the
358 code should be ran/inspected, PromptToolkit (and other frontend) mostly
358 code should be ran/inspected, PromptToolkit (and other frontend) mostly
359 need user facing information.
359 need user facing information.
360
360
361 - Which range should be replaced replaced by what.
361 - Which range should be replaced replaced by what.
362 - Some metadata (like completion type), or meta informations to displayed to
362 - Some metadata (like completion type), or meta informations to displayed to
363 the use user.
363 the use user.
364
364
365 For debugging purpose we can also store the origin of the completion (``jedi``,
365 For debugging purpose we can also store the origin of the completion (``jedi``,
366 ``IPython.python_matches``, ``IPython.magics_matches``...).
366 ``IPython.python_matches``, ``IPython.magics_matches``...).
367 """
367 """
368
368
369 def __init__(self, start: int, end: int, text: str, *, type: str=None, _origin=''):
369 def __init__(self, start: int, end: int, text: str, *, type: str=None, _origin=''):
370 warnings.warn("``Completion`` is a provisional API (as of IPython 6.0). "
370 warnings.warn("``Completion`` is a provisional API (as of IPython 6.0). "
371 "It may change without warnings. "
371 "It may change without warnings. "
372 "Use in corresponding context manager.",
372 "Use in corresponding context manager.",
373 category=ProvisionalCompleterWarning, stacklevel=2)
373 category=ProvisionalCompleterWarning, stacklevel=2)
374
374
375 self.start = start
375 self.start = start
376 self.end = end
376 self.end = end
377 self.text = text
377 self.text = text
378 self.type = type
378 self.type = type
379 self._origin = _origin
379 self._origin = _origin
380
380
381 def __repr__(self):
381 def __repr__(self):
382 return '<Completion start=%s end=%s text=%r type=%r>' % (self.start, self.end, self.text, self.type or '?')
382 return '<Completion start=%s end=%s text=%r type=%r>' % (self.start, self.end, self.text, self.type or '?')
383
383
384 def __eq__(self, other)->Bool:
384 def __eq__(self, other)->Bool:
385 """
385 """
386 Equality and hash do not hash the type (as some completer may not be
386 Equality and hash do not hash the type (as some completer may not be
387 able to infer the type), but are use to (partially) de-duplicate
387 able to infer the type), but are use to (partially) de-duplicate
388 completion.
388 completion.
389
389
390 Completely de-duplicating completion is a bit tricker that just
390 Completely de-duplicating completion is a bit tricker that just
391 comparing as it depends on surrounding text, which Completions are not
391 comparing as it depends on surrounding text, which Completions are not
392 aware of.
392 aware of.
393 """
393 """
394 return self.start == other.start and \
394 return self.start == other.start and \
395 self.end == other.end and \
395 self.end == other.end and \
396 self.text == other.text
396 self.text == other.text
397
397
398 def __hash__(self):
398 def __hash__(self):
399 return hash((self.start, self.end, self.text))
399 return hash((self.start, self.end, self.text))
400
400
401
401
402 _IC = Iterator[Completion]
402 _IC = Iterator[Completion]
403
403
404
404
405 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
405 def _deduplicate_completions(text: str, completions: _IC)-> _IC:
406 """
406 """
407 Deduplicate a set of completions.
407 Deduplicate a set of completions.
408
408
409 .. warning:: Unstable
409 .. warning:: Unstable
410
410
411 This function is unstable, API may change without warning.
411 This function is unstable, API may change without warning.
412
412
413 Parameters
413 Parameters
414 ----------
414 ----------
415 text: str
415 text: str
416 text that should be completed.
416 text that should be completed.
417 completions: Iterator[Completion]
417 completions: Iterator[Completion]
418 iterator over the completions to deduplicate
418 iterator over the completions to deduplicate
419
419
420
420
421 Completions coming from multiple sources, may be different but end up having
421 Completions coming from multiple sources, may be different but end up having
422 the same effect when applied to ``text``. If this is the case, this will
422 the same effect when applied to ``text``. If this is the case, this will
423 consider completions as equal and only emit the first encountered.
423 consider completions as equal and only emit the first encountered.
424
424
425 Not folded in `completions()` yet for debugging purpose, and to detect when
425 Not folded in `completions()` yet for debugging purpose, and to detect when
426 the IPython completer does return things that Jedi does not, but should be
426 the IPython completer does return things that Jedi does not, but should be
427 at some point.
427 at some point.
428 """
428 """
429 completions = list(completions)
429 completions = list(completions)
430 if not completions:
430 if not completions:
431 return
431 return
432
432
433 new_start = min(c.start for c in completions)
433 new_start = min(c.start for c in completions)
434 new_end = max(c.end for c in completions)
434 new_end = max(c.end for c in completions)
435
435
436 seen = set()
436 seen = set()
437 for c in completions:
437 for c in completions:
438 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
438 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
439 if new_text not in seen:
439 if new_text not in seen:
440 yield c
440 yield c
441 seen.add(new_text)
441 seen.add(new_text)
442
442
443
443
444 def rectify_completions(text: str, completions: _IC, *, _debug=False)->_IC:
444 def rectify_completions(text: str, completions: _IC, *, _debug=False)->_IC:
445 """
445 """
446 Rectify a set of completions to all have the same ``start`` and ``end``
446 Rectify a set of completions to all have the same ``start`` and ``end``
447
447
448 .. warning:: Unstable
448 .. warning:: Unstable
449
449
450 This function is unstable, API may change without warning.
450 This function is unstable, API may change without warning.
451 It will also raise unless use in proper context manager.
451 It will also raise unless use in proper context manager.
452
452
453 Parameters
453 Parameters
454 ----------
454 ----------
455 text: str
455 text: str
456 text that should be completed.
456 text that should be completed.
457 completions: Iterator[Completion]
457 completions: Iterator[Completion]
458 iterator over the completions to rectify
458 iterator over the completions to rectify
459
459
460
460
461 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
461 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
462 the Jupyter Protocol requires them to behave like so. This will readjust
462 the Jupyter Protocol requires them to behave like so. This will readjust
463 the completion to have the same ``start`` and ``end`` by padding both
463 the completion to have the same ``start`` and ``end`` by padding both
464 extremities with surrounding text.
464 extremities with surrounding text.
465
465
466 During stabilisation should support a ``_debug`` option to log which
466 During stabilisation should support a ``_debug`` option to log which
467 completion are return by the IPython completer and not found in Jedi in
467 completion are return by the IPython completer and not found in Jedi in
468 order to make upstream bug report.
468 order to make upstream bug report.
469 """
469 """
470 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
470 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
471 "It may change without warnings. "
471 "It may change without warnings. "
472 "Use in corresponding context manager.",
472 "Use in corresponding context manager.",
473 category=ProvisionalCompleterWarning, stacklevel=2)
473 category=ProvisionalCompleterWarning, stacklevel=2)
474
474
475 completions = list(completions)
475 completions = list(completions)
476 if not completions:
476 if not completions:
477 return
477 return
478 starts = (c.start for c in completions)
478 starts = (c.start for c in completions)
479 ends = (c.end for c in completions)
479 ends = (c.end for c in completions)
480
480
481 new_start = min(starts)
481 new_start = min(starts)
482 new_end = max(ends)
482 new_end = max(ends)
483
483
484 seen_jedi = set()
484 seen_jedi = set()
485 seen_python_matches = set()
485 seen_python_matches = set()
486 for c in completions:
486 for c in completions:
487 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
487 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
488 if c._origin == 'jedi':
488 if c._origin == 'jedi':
489 seen_jedi.add(new_text)
489 seen_jedi.add(new_text)
490 elif c._origin == 'IPCompleter.python_matches':
490 elif c._origin == 'IPCompleter.python_matches':
491 seen_python_matches.add(new_text)
491 seen_python_matches.add(new_text)
492 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin)
492 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin)
493 diff = seen_python_matches.difference(seen_jedi)
493 diff = seen_python_matches.difference(seen_jedi)
494 if diff and _debug:
494 if diff and _debug:
495 print('IPython.python matches have extras:', diff)
495 print('IPython.python matches have extras:', diff)
496
496
497
497
498 if sys.platform == 'win32':
498 if sys.platform == 'win32':
499 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
499 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
500 else:
500 else:
501 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
501 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
502
502
503 GREEDY_DELIMS = ' =\r\n'
503 GREEDY_DELIMS = ' =\r\n'
504
504
505
505
506 class CompletionSplitter(object):
506 class CompletionSplitter(object):
507 """An object to split an input line in a manner similar to readline.
507 """An object to split an input line in a manner similar to readline.
508
508
509 By having our own implementation, we can expose readline-like completion in
509 By having our own implementation, we can expose readline-like completion in
510 a uniform manner to all frontends. This object only needs to be given the
510 a uniform manner to all frontends. This object only needs to be given the
511 line of text to be split and the cursor position on said line, and it
511 line of text to be split and the cursor position on said line, and it
512 returns the 'word' to be completed on at the cursor after splitting the
512 returns the 'word' to be completed on at the cursor after splitting the
513 entire line.
513 entire line.
514
514
515 What characters are used as splitting delimiters can be controlled by
515 What characters are used as splitting delimiters can be controlled by
516 setting the ``delims`` attribute (this is a property that internally
516 setting the ``delims`` attribute (this is a property that internally
517 automatically builds the necessary regular expression)"""
517 automatically builds the necessary regular expression)"""
518
518
519 # Private interface
519 # Private interface
520
520
521 # A string of delimiter characters. The default value makes sense for
521 # A string of delimiter characters. The default value makes sense for
522 # IPython's most typical usage patterns.
522 # IPython's most typical usage patterns.
523 _delims = DELIMS
523 _delims = DELIMS
524
524
525 # The expression (a normal string) to be compiled into a regular expression
525 # The expression (a normal string) to be compiled into a regular expression
526 # for actual splitting. We store it as an attribute mostly for ease of
526 # for actual splitting. We store it as an attribute mostly for ease of
527 # debugging, since this type of code can be so tricky to debug.
527 # debugging, since this type of code can be so tricky to debug.
528 _delim_expr = None
528 _delim_expr = None
529
529
530 # The regular expression that does the actual splitting
530 # The regular expression that does the actual splitting
531 _delim_re = None
531 _delim_re = None
532
532
533 def __init__(self, delims=None):
533 def __init__(self, delims=None):
534 delims = CompletionSplitter._delims if delims is None else delims
534 delims = CompletionSplitter._delims if delims is None else delims
535 self.delims = delims
535 self.delims = delims
536
536
537 @property
537 @property
538 def delims(self):
538 def delims(self):
539 """Return the string of delimiter characters."""
539 """Return the string of delimiter characters."""
540 return self._delims
540 return self._delims
541
541
542 @delims.setter
542 @delims.setter
543 def delims(self, delims):
543 def delims(self, delims):
544 """Set the delimiters for line splitting."""
544 """Set the delimiters for line splitting."""
545 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
545 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
546 self._delim_re = re.compile(expr)
546 self._delim_re = re.compile(expr)
547 self._delims = delims
547 self._delims = delims
548 self._delim_expr = expr
548 self._delim_expr = expr
549
549
550 def split_line(self, line, cursor_pos=None):
550 def split_line(self, line, cursor_pos=None):
551 """Split a line of text with a cursor at the given position.
551 """Split a line of text with a cursor at the given position.
552 """
552 """
553 l = line if cursor_pos is None else line[:cursor_pos]
553 l = line if cursor_pos is None else line[:cursor_pos]
554 return self._delim_re.split(l)[-1]
554 return self._delim_re.split(l)[-1]
555
555
556
556
557
557
558 class Completer(Configurable):
558 class Completer(Configurable):
559
559
560 greedy = Bool(False,
560 greedy = Bool(False,
561 help="""Activate greedy completion
561 help="""Activate greedy completion
562 PENDING DEPRECTION. this is now mostly taken care of with Jedi.
562 PENDING DEPRECTION. this is now mostly taken care of with Jedi.
563
563
564 This will enable completion on elements of lists, results of function calls, etc.,
564 This will enable completion on elements of lists, results of function calls, etc.,
565 but can be unsafe because the code is actually evaluated on TAB.
565 but can be unsafe because the code is actually evaluated on TAB.
566 """
566 """
567 ).tag(config=True)
567 ).tag(config=True)
568
568
569 use_jedi = Bool(default_value=JEDI_INSTALLED,
569 use_jedi = Bool(default_value=JEDI_INSTALLED,
570 help="Experimental: Use Jedi to generate autocompletions. "
570 help="Experimental: Use Jedi to generate autocompletions. "
571 "Default to True if jedi is installed").tag(config=True)
571 "Default to True if jedi is installed").tag(config=True)
572
572
573 jedi_compute_type_timeout = Int(default_value=400,
573 jedi_compute_type_timeout = Int(default_value=400,
574 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
574 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
575 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
575 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
576 performance by preventing jedi to build its cache.
576 performance by preventing jedi to build its cache.
577 """).tag(config=True)
577 """).tag(config=True)
578
578
579 debug = Bool(default_value=False,
579 debug = Bool(default_value=False,
580 help='Enable debug for the Completer. Mostly print extra '
580 help='Enable debug for the Completer. Mostly print extra '
581 'information for experimental jedi integration.')\
581 'information for experimental jedi integration.')\
582 .tag(config=True)
582 .tag(config=True)
583
583
584 backslash_combining_completions = Bool(True,
584 backslash_combining_completions = Bool(True,
585 help="Enable unicode completions, e.g. \\alpha<tab> . "
585 help="Enable unicode completions, e.g. \\alpha<tab> . "
586 "Includes completion of latex commands, unicode names, and expanding "
586 "Includes completion of latex commands, unicode names, and expanding "
587 "unicode characters back to latex commands.").tag(config=True)
587 "unicode characters back to latex commands.").tag(config=True)
588
588
589
589
590
590
591 def __init__(self, namespace=None, global_namespace=None, **kwargs):
591 def __init__(self, namespace=None, global_namespace=None, **kwargs):
592 """Create a new completer for the command line.
592 """Create a new completer for the command line.
593
593
594 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
594 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
595
595
596 If unspecified, the default namespace where completions are performed
596 If unspecified, the default namespace where completions are performed
597 is __main__ (technically, __main__.__dict__). Namespaces should be
597 is __main__ (technically, __main__.__dict__). Namespaces should be
598 given as dictionaries.
598 given as dictionaries.
599
599
600 An optional second namespace can be given. This allows the completer
600 An optional second namespace can be given. This allows the completer
601 to handle cases where both the local and global scopes need to be
601 to handle cases where both the local and global scopes need to be
602 distinguished.
602 distinguished.
603 """
603 """
604
604
605 # Don't bind to namespace quite yet, but flag whether the user wants a
605 # Don't bind to namespace quite yet, but flag whether the user wants a
606 # specific namespace or to use __main__.__dict__. This will allow us
606 # specific namespace or to use __main__.__dict__. This will allow us
607 # to bind to __main__.__dict__ at completion time, not now.
607 # to bind to __main__.__dict__ at completion time, not now.
608 if namespace is None:
608 if namespace is None:
609 self.use_main_ns = True
609 self.use_main_ns = True
610 else:
610 else:
611 self.use_main_ns = False
611 self.use_main_ns = False
612 self.namespace = namespace
612 self.namespace = namespace
613
613
614 # The global namespace, if given, can be bound directly
614 # The global namespace, if given, can be bound directly
615 if global_namespace is None:
615 if global_namespace is None:
616 self.global_namespace = {}
616 self.global_namespace = {}
617 else:
617 else:
618 self.global_namespace = global_namespace
618 self.global_namespace = global_namespace
619
619
620 super(Completer, self).__init__(**kwargs)
620 super(Completer, self).__init__(**kwargs)
621
621
622 def complete(self, text, state):
622 def complete(self, text, state):
623 """Return the next possible completion for 'text'.
623 """Return the next possible completion for 'text'.
624
624
625 This is called successively with state == 0, 1, 2, ... until it
625 This is called successively with state == 0, 1, 2, ... until it
626 returns None. The completion should begin with 'text'.
626 returns None. The completion should begin with 'text'.
627
627
628 """
628 """
629 if self.use_main_ns:
629 if self.use_main_ns:
630 self.namespace = __main__.__dict__
630 self.namespace = __main__.__dict__
631
631
632 if state == 0:
632 if state == 0:
633 if "." in text:
633 if "." in text:
634 self.matches = self.attr_matches(text)
634 self.matches = self.attr_matches(text)
635 else:
635 else:
636 self.matches = self.global_matches(text)
636 self.matches = self.global_matches(text)
637 try:
637 try:
638 return self.matches[state]
638 return self.matches[state]
639 except IndexError:
639 except IndexError:
640 return None
640 return None
641
641
642 def global_matches(self, text):
642 def global_matches(self, text):
643 """Compute matches when text is a simple name.
643 """Compute matches when text is a simple name.
644
644
645 Return a list of all keywords, built-in functions and names currently
645 Return a list of all keywords, built-in functions and names currently
646 defined in self.namespace or self.global_namespace that match.
646 defined in self.namespace or self.global_namespace that match.
647
647
648 """
648 """
649 matches = []
649 matches = []
650 match_append = matches.append
650 match_append = matches.append
651 n = len(text)
651 n = len(text)
652 for lst in [keyword.kwlist,
652 for lst in [keyword.kwlist,
653 builtin_mod.__dict__.keys(),
653 builtin_mod.__dict__.keys(),
654 self.namespace.keys(),
654 self.namespace.keys(),
655 self.global_namespace.keys()]:
655 self.global_namespace.keys()]:
656 for word in lst:
656 for word in lst:
657 if word[:n] == text and word != "__builtins__":
657 if word[:n] == text and word != "__builtins__":
658 match_append(word)
658 match_append(word)
659
659
660 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?")
660 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
661 for lst in [self.namespace.keys(),
661 for lst in [self.namespace.keys(),
662 self.global_namespace.keys()]:
662 self.global_namespace.keys()]:
663 shortened = {"_".join([sub[0] for sub in word.split('_')]) : word
663 shortened = {"_".join([sub[0] for sub in word.split('_')]) : word
664 for word in lst if snake_case_re.fullmatch(word)}
664 for word in lst if snake_case_re.match(word)}
665 for word in shortened.keys():
665 for word in shortened.keys():
666 if word[:n] == text and word != "__builtins__":
666 if word[:n] == text and word != "__builtins__":
667 match_append(shortened[word])
667 match_append(shortened[word])
668 return [cast_unicode_py2(m) for m in matches]
668 return [cast_unicode_py2(m) for m in matches]
669
669
670 def attr_matches(self, text):
670 def attr_matches(self, text):
671 """Compute matches when text contains a dot.
671 """Compute matches when text contains a dot.
672
672
673 Assuming the text is of the form NAME.NAME....[NAME], and is
673 Assuming the text is of the form NAME.NAME....[NAME], and is
674 evaluatable in self.namespace or self.global_namespace, it will be
674 evaluatable in self.namespace or self.global_namespace, it will be
675 evaluated and its attributes (as revealed by dir()) are used as
675 evaluated and its attributes (as revealed by dir()) are used as
676 possible completions. (For class instances, class members are are
676 possible completions. (For class instances, class members are are
677 also considered.)
677 also considered.)
678
678
679 WARNING: this can still invoke arbitrary C code, if an object
679 WARNING: this can still invoke arbitrary C code, if an object
680 with a __getattr__ hook is evaluated.
680 with a __getattr__ hook is evaluated.
681
681
682 """
682 """
683
683
684 # Another option, seems to work great. Catches things like ''.<tab>
684 # Another option, seems to work great. Catches things like ''.<tab>
685 m = re.match(r"(\S+(\.\w+)*)\.(\w*)$", text)
685 m = re.match(r"(\S+(\.\w+)*)\.(\w*)$", text)
686
686
687 if m:
687 if m:
688 expr, attr = m.group(1, 3)
688 expr, attr = m.group(1, 3)
689 elif self.greedy:
689 elif self.greedy:
690 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
690 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
691 if not m2:
691 if not m2:
692 return []
692 return []
693 expr, attr = m2.group(1,2)
693 expr, attr = m2.group(1,2)
694 else:
694 else:
695 return []
695 return []
696
696
697 try:
697 try:
698 obj = eval(expr, self.namespace)
698 obj = eval(expr, self.namespace)
699 except:
699 except:
700 try:
700 try:
701 obj = eval(expr, self.global_namespace)
701 obj = eval(expr, self.global_namespace)
702 except:
702 except:
703 return []
703 return []
704
704
705 if self.limit_to__all__ and hasattr(obj, '__all__'):
705 if self.limit_to__all__ and hasattr(obj, '__all__'):
706 words = get__all__entries(obj)
706 words = get__all__entries(obj)
707 else:
707 else:
708 words = dir2(obj)
708 words = dir2(obj)
709
709
710 try:
710 try:
711 words = generics.complete_object(obj, words)
711 words = generics.complete_object(obj, words)
712 except TryNext:
712 except TryNext:
713 pass
713 pass
714 except AssertionError:
714 except AssertionError:
715 raise
715 raise
716 except Exception:
716 except Exception:
717 # Silence errors from completion function
717 # Silence errors from completion function
718 #raise # dbg
718 #raise # dbg
719 pass
719 pass
720 # Build match list to return
720 # Build match list to return
721 n = len(attr)
721 n = len(attr)
722 return [u"%s.%s" % (expr, w) for w in words if w[:n] == attr ]
722 return [u"%s.%s" % (expr, w) for w in words if w[:n] == attr ]
723
723
724
724
725 def get__all__entries(obj):
725 def get__all__entries(obj):
726 """returns the strings in the __all__ attribute"""
726 """returns the strings in the __all__ attribute"""
727 try:
727 try:
728 words = getattr(obj, '__all__')
728 words = getattr(obj, '__all__')
729 except:
729 except:
730 return []
730 return []
731
731
732 return [cast_unicode_py2(w) for w in words if isinstance(w, str)]
732 return [cast_unicode_py2(w) for w in words if isinstance(w, str)]
733
733
734
734
735 def match_dict_keys(keys: List[str], prefix: str, delims: str):
735 def match_dict_keys(keys: List[str], prefix: str, delims: str):
736 """Used by dict_key_matches, matching the prefix to a list of keys
736 """Used by dict_key_matches, matching the prefix to a list of keys
737
737
738 Parameters
738 Parameters
739 ==========
739 ==========
740 keys:
740 keys:
741 list of keys in dictionary currently being completed.
741 list of keys in dictionary currently being completed.
742 prefix:
742 prefix:
743 Part of the text already typed by the user. e.g. `mydict[b'fo`
743 Part of the text already typed by the user. e.g. `mydict[b'fo`
744 delims:
744 delims:
745 String of delimiters to consider when finding the current key.
745 String of delimiters to consider when finding the current key.
746
746
747 Returns
747 Returns
748 =======
748 =======
749
749
750 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
750 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
751 ``quote`` being the quote that need to be used to close current string.
751 ``quote`` being the quote that need to be used to close current string.
752 ``token_start`` the position where the replacement should start occurring,
752 ``token_start`` the position where the replacement should start occurring,
753 ``matches`` a list of replacement/completion
753 ``matches`` a list of replacement/completion
754
754
755 """
755 """
756 if not prefix:
756 if not prefix:
757 return None, 0, [repr(k) for k in keys
757 return None, 0, [repr(k) for k in keys
758 if isinstance(k, (str, bytes))]
758 if isinstance(k, (str, bytes))]
759 quote_match = re.search('["\']', prefix)
759 quote_match = re.search('["\']', prefix)
760 quote = quote_match.group()
760 quote = quote_match.group()
761 try:
761 try:
762 prefix_str = eval(prefix + quote, {})
762 prefix_str = eval(prefix + quote, {})
763 except Exception:
763 except Exception:
764 return None, 0, []
764 return None, 0, []
765
765
766 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
766 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
767 token_match = re.search(pattern, prefix, re.UNICODE)
767 token_match = re.search(pattern, prefix, re.UNICODE)
768 token_start = token_match.start()
768 token_start = token_match.start()
769 token_prefix = token_match.group()
769 token_prefix = token_match.group()
770
770
771 matched = []
771 matched = []
772 for key in keys:
772 for key in keys:
773 try:
773 try:
774 if not key.startswith(prefix_str):
774 if not key.startswith(prefix_str):
775 continue
775 continue
776 except (AttributeError, TypeError, UnicodeError):
776 except (AttributeError, TypeError, UnicodeError):
777 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
777 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
778 continue
778 continue
779
779
780 # reformat remainder of key to begin with prefix
780 # reformat remainder of key to begin with prefix
781 rem = key[len(prefix_str):]
781 rem = key[len(prefix_str):]
782 # force repr wrapped in '
782 # force repr wrapped in '
783 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
783 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
784 if rem_repr.startswith('u') and prefix[0] not in 'uU':
784 if rem_repr.startswith('u') and prefix[0] not in 'uU':
785 # Found key is unicode, but prefix is Py2 string.
785 # Found key is unicode, but prefix is Py2 string.
786 # Therefore attempt to interpret key as string.
786 # Therefore attempt to interpret key as string.
787 try:
787 try:
788 rem_repr = repr(rem.encode('ascii') + '"')
788 rem_repr = repr(rem.encode('ascii') + '"')
789 except UnicodeEncodeError:
789 except UnicodeEncodeError:
790 continue
790 continue
791
791
792 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
792 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
793 if quote == '"':
793 if quote == '"':
794 # The entered prefix is quoted with ",
794 # The entered prefix is quoted with ",
795 # but the match is quoted with '.
795 # but the match is quoted with '.
796 # A contained " hence needs escaping for comparison:
796 # A contained " hence needs escaping for comparison:
797 rem_repr = rem_repr.replace('"', '\\"')
797 rem_repr = rem_repr.replace('"', '\\"')
798
798
799 # then reinsert prefix from start of token
799 # then reinsert prefix from start of token
800 matched.append('%s%s' % (token_prefix, rem_repr))
800 matched.append('%s%s' % (token_prefix, rem_repr))
801 return quote, token_start, matched
801 return quote, token_start, matched
802
802
803
803
804 def cursor_to_position(text:int, line:int, column:int)->int:
804 def cursor_to_position(text:int, line:int, column:int)->int:
805 """
805 """
806
806
807 Convert the (line,column) position of the cursor in text to an offset in a
807 Convert the (line,column) position of the cursor in text to an offset in a
808 string.
808 string.
809
809
810 Parameters
810 Parameters
811 ----------
811 ----------
812
812
813 text : str
813 text : str
814 The text in which to calculate the cursor offset
814 The text in which to calculate the cursor offset
815 line : int
815 line : int
816 Line of the cursor; 0-indexed
816 Line of the cursor; 0-indexed
817 column : int
817 column : int
818 Column of the cursor 0-indexed
818 Column of the cursor 0-indexed
819
819
820 Return
820 Return
821 ------
821 ------
822 Position of the cursor in ``text``, 0-indexed.
822 Position of the cursor in ``text``, 0-indexed.
823
823
824 See Also
824 See Also
825 --------
825 --------
826 position_to_cursor: reciprocal of this function
826 position_to_cursor: reciprocal of this function
827
827
828 """
828 """
829 lines = text.split('\n')
829 lines = text.split('\n')
830 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
830 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
831
831
832 return sum(len(l) + 1 for l in lines[:line]) + column
832 return sum(len(l) + 1 for l in lines[:line]) + column
833
833
834 def position_to_cursor(text:str, offset:int)->(int, int):
834 def position_to_cursor(text:str, offset:int)->(int, int):
835 """
835 """
836 Convert the position of the cursor in text (0 indexed) to a line
836 Convert the position of the cursor in text (0 indexed) to a line
837 number(0-indexed) and a column number (0-indexed) pair
837 number(0-indexed) and a column number (0-indexed) pair
838
838
839 Position should be a valid position in ``text``.
839 Position should be a valid position in ``text``.
840
840
841 Parameters
841 Parameters
842 ----------
842 ----------
843
843
844 text : str
844 text : str
845 The text in which to calculate the cursor offset
845 The text in which to calculate the cursor offset
846 offset : int
846 offset : int
847 Position of the cursor in ``text``, 0-indexed.
847 Position of the cursor in ``text``, 0-indexed.
848
848
849 Return
849 Return
850 ------
850 ------
851 (line, column) : (int, int)
851 (line, column) : (int, int)
852 Line of the cursor; 0-indexed, column of the cursor 0-indexed
852 Line of the cursor; 0-indexed, column of the cursor 0-indexed
853
853
854
854
855 See Also
855 See Also
856 --------
856 --------
857 cursor_to_position : reciprocal of this function
857 cursor_to_position : reciprocal of this function
858
858
859
859
860 """
860 """
861
861
862 assert 0 < offset <= len(text) , "0 < %s <= %s" % (offset , len(text))
862 assert 0 < offset <= len(text) , "0 < %s <= %s" % (offset , len(text))
863
863
864 before = text[:offset]
864 before = text[:offset]
865 blines = before.split('\n') # ! splitnes trim trailing \n
865 blines = before.split('\n') # ! splitnes trim trailing \n
866 line = before.count('\n')
866 line = before.count('\n')
867 col = len(blines[-1])
867 col = len(blines[-1])
868 return line, col
868 return line, col
869
869
870
870
871 def _safe_isinstance(obj, module, class_name):
871 def _safe_isinstance(obj, module, class_name):
872 """Checks if obj is an instance of module.class_name if loaded
872 """Checks if obj is an instance of module.class_name if loaded
873 """
873 """
874 return (module in sys.modules and
874 return (module in sys.modules and
875 isinstance(obj, getattr(import_module(module), class_name)))
875 isinstance(obj, getattr(import_module(module), class_name)))
876
876
877
877
878 def back_unicode_name_matches(text):
878 def back_unicode_name_matches(text):
879 u"""Match unicode characters back to unicode name
879 u"""Match unicode characters back to unicode name
880
880
881 This does ``β˜ƒ`` -> ``\\snowman``
881 This does ``β˜ƒ`` -> ``\\snowman``
882
882
883 Note that snowman is not a valid python3 combining character but will be expanded.
883 Note that snowman is not a valid python3 combining character but will be expanded.
884 Though it will not recombine back to the snowman character by the completion machinery.
884 Though it will not recombine back to the snowman character by the completion machinery.
885
885
886 This will not either back-complete standard sequences like \\n, \\b ...
886 This will not either back-complete standard sequences like \\n, \\b ...
887
887
888 Used on Python 3 only.
888 Used on Python 3 only.
889 """
889 """
890 if len(text)<2:
890 if len(text)<2:
891 return u'', ()
891 return u'', ()
892 maybe_slash = text[-2]
892 maybe_slash = text[-2]
893 if maybe_slash != '\\':
893 if maybe_slash != '\\':
894 return u'', ()
894 return u'', ()
895
895
896 char = text[-1]
896 char = text[-1]
897 # no expand on quote for completion in strings.
897 # no expand on quote for completion in strings.
898 # nor backcomplete standard ascii keys
898 # nor backcomplete standard ascii keys
899 if char in string.ascii_letters or char in ['"',"'"]:
899 if char in string.ascii_letters or char in ['"',"'"]:
900 return u'', ()
900 return u'', ()
901 try :
901 try :
902 unic = unicodedata.name(char)
902 unic = unicodedata.name(char)
903 return '\\'+char,['\\'+unic]
903 return '\\'+char,['\\'+unic]
904 except KeyError:
904 except KeyError:
905 pass
905 pass
906 return u'', ()
906 return u'', ()
907
907
908 def back_latex_name_matches(text:str):
908 def back_latex_name_matches(text:str):
909 """Match latex characters back to unicode name
909 """Match latex characters back to unicode name
910
910
911 This does ``\\β„΅`` -> ``\\aleph``
911 This does ``\\β„΅`` -> ``\\aleph``
912
912
913 Used on Python 3 only.
913 Used on Python 3 only.
914 """
914 """
915 if len(text)<2:
915 if len(text)<2:
916 return u'', ()
916 return u'', ()
917 maybe_slash = text[-2]
917 maybe_slash = text[-2]
918 if maybe_slash != '\\':
918 if maybe_slash != '\\':
919 return u'', ()
919 return u'', ()
920
920
921
921
922 char = text[-1]
922 char = text[-1]
923 # no expand on quote for completion in strings.
923 # no expand on quote for completion in strings.
924 # nor backcomplete standard ascii keys
924 # nor backcomplete standard ascii keys
925 if char in string.ascii_letters or char in ['"',"'"]:
925 if char in string.ascii_letters or char in ['"',"'"]:
926 return u'', ()
926 return u'', ()
927 try :
927 try :
928 latex = reverse_latex_symbol[char]
928 latex = reverse_latex_symbol[char]
929 # '\\' replace the \ as well
929 # '\\' replace the \ as well
930 return '\\'+char,[latex]
930 return '\\'+char,[latex]
931 except KeyError:
931 except KeyError:
932 pass
932 pass
933 return u'', ()
933 return u'', ()
934
934
935
935
936 class IPCompleter(Completer):
936 class IPCompleter(Completer):
937 """Extension of the completer class with IPython-specific features"""
937 """Extension of the completer class with IPython-specific features"""
938
938
939 @observe('greedy')
939 @observe('greedy')
940 def _greedy_changed(self, change):
940 def _greedy_changed(self, change):
941 """update the splitter and readline delims when greedy is changed"""
941 """update the splitter and readline delims when greedy is changed"""
942 if change['new']:
942 if change['new']:
943 self.splitter.delims = GREEDY_DELIMS
943 self.splitter.delims = GREEDY_DELIMS
944 else:
944 else:
945 self.splitter.delims = DELIMS
945 self.splitter.delims = DELIMS
946
946
947 merge_completions = Bool(True,
947 merge_completions = Bool(True,
948 help="""Whether to merge completion results into a single list
948 help="""Whether to merge completion results into a single list
949
949
950 If False, only the completion results from the first non-empty
950 If False, only the completion results from the first non-empty
951 completer will be returned.
951 completer will be returned.
952 """
952 """
953 ).tag(config=True)
953 ).tag(config=True)
954 omit__names = Enum((0,1,2), default_value=2,
954 omit__names = Enum((0,1,2), default_value=2,
955 help="""Instruct the completer to omit private method names
955 help="""Instruct the completer to omit private method names
956
956
957 Specifically, when completing on ``object.<tab>``.
957 Specifically, when completing on ``object.<tab>``.
958
958
959 When 2 [default]: all names that start with '_' will be excluded.
959 When 2 [default]: all names that start with '_' will be excluded.
960
960
961 When 1: all 'magic' names (``__foo__``) will be excluded.
961 When 1: all 'magic' names (``__foo__``) will be excluded.
962
962
963 When 0: nothing will be excluded.
963 When 0: nothing will be excluded.
964 """
964 """
965 ).tag(config=True)
965 ).tag(config=True)
966 limit_to__all__ = Bool(False,
966 limit_to__all__ = Bool(False,
967 help="""
967 help="""
968 DEPRECATED as of version 5.0.
968 DEPRECATED as of version 5.0.
969
969
970 Instruct the completer to use __all__ for the completion
970 Instruct the completer to use __all__ for the completion
971
971
972 Specifically, when completing on ``object.<tab>``.
972 Specifically, when completing on ``object.<tab>``.
973
973
974 When True: only those names in obj.__all__ will be included.
974 When True: only those names in obj.__all__ will be included.
975
975
976 When False [default]: the __all__ attribute is ignored
976 When False [default]: the __all__ attribute is ignored
977 """,
977 """,
978 ).tag(config=True)
978 ).tag(config=True)
979
979
980 @observe('limit_to__all__')
980 @observe('limit_to__all__')
981 def _limit_to_all_changed(self, change):
981 def _limit_to_all_changed(self, change):
982 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
982 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
983 'value has been deprecated since IPython 5.0, will be made to have '
983 'value has been deprecated since IPython 5.0, will be made to have '
984 'no effects and then removed in future version of IPython.',
984 'no effects and then removed in future version of IPython.',
985 UserWarning)
985 UserWarning)
986
986
987 def __init__(self, shell=None, namespace=None, global_namespace=None,
987 def __init__(self, shell=None, namespace=None, global_namespace=None,
988 use_readline=_deprecation_readline_sentinel, config=None, **kwargs):
988 use_readline=_deprecation_readline_sentinel, config=None, **kwargs):
989 """IPCompleter() -> completer
989 """IPCompleter() -> completer
990
990
991 Return a completer object.
991 Return a completer object.
992
992
993 Parameters
993 Parameters
994 ----------
994 ----------
995
995
996 shell
996 shell
997 a pointer to the ipython shell itself. This is needed
997 a pointer to the ipython shell itself. This is needed
998 because this completer knows about magic functions, and those can
998 because this completer knows about magic functions, and those can
999 only be accessed via the ipython instance.
999 only be accessed via the ipython instance.
1000
1000
1001 namespace : dict, optional
1001 namespace : dict, optional
1002 an optional dict where completions are performed.
1002 an optional dict where completions are performed.
1003
1003
1004 global_namespace : dict, optional
1004 global_namespace : dict, optional
1005 secondary optional dict for completions, to
1005 secondary optional dict for completions, to
1006 handle cases (such as IPython embedded inside functions) where
1006 handle cases (such as IPython embedded inside functions) where
1007 both Python scopes are visible.
1007 both Python scopes are visible.
1008
1008
1009 use_readline : bool, optional
1009 use_readline : bool, optional
1010 DEPRECATED, ignored since IPython 6.0, will have no effects
1010 DEPRECATED, ignored since IPython 6.0, will have no effects
1011 """
1011 """
1012
1012
1013 self.magic_escape = ESC_MAGIC
1013 self.magic_escape = ESC_MAGIC
1014 self.splitter = CompletionSplitter()
1014 self.splitter = CompletionSplitter()
1015
1015
1016 if use_readline is not _deprecation_readline_sentinel:
1016 if use_readline is not _deprecation_readline_sentinel:
1017 warnings.warn('The `use_readline` parameter is deprecated and ignored since IPython 6.0.',
1017 warnings.warn('The `use_readline` parameter is deprecated and ignored since IPython 6.0.',
1018 DeprecationWarning, stacklevel=2)
1018 DeprecationWarning, stacklevel=2)
1019
1019
1020 # _greedy_changed() depends on splitter and readline being defined:
1020 # _greedy_changed() depends on splitter and readline being defined:
1021 Completer.__init__(self, namespace=namespace, global_namespace=global_namespace,
1021 Completer.__init__(self, namespace=namespace, global_namespace=global_namespace,
1022 config=config, **kwargs)
1022 config=config, **kwargs)
1023
1023
1024 # List where completion matches will be stored
1024 # List where completion matches will be stored
1025 self.matches = []
1025 self.matches = []
1026 self.shell = shell
1026 self.shell = shell
1027 # Regexp to split filenames with spaces in them
1027 # Regexp to split filenames with spaces in them
1028 self.space_name_re = re.compile(r'([^\\] )')
1028 self.space_name_re = re.compile(r'([^\\] )')
1029 # Hold a local ref. to glob.glob for speed
1029 # Hold a local ref. to glob.glob for speed
1030 self.glob = glob.glob
1030 self.glob = glob.glob
1031
1031
1032 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1032 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1033 # buffers, to avoid completion problems.
1033 # buffers, to avoid completion problems.
1034 term = os.environ.get('TERM','xterm')
1034 term = os.environ.get('TERM','xterm')
1035 self.dumb_terminal = term in ['dumb','emacs']
1035 self.dumb_terminal = term in ['dumb','emacs']
1036
1036
1037 # Special handling of backslashes needed in win32 platforms
1037 # Special handling of backslashes needed in win32 platforms
1038 if sys.platform == "win32":
1038 if sys.platform == "win32":
1039 self.clean_glob = self._clean_glob_win32
1039 self.clean_glob = self._clean_glob_win32
1040 else:
1040 else:
1041 self.clean_glob = self._clean_glob
1041 self.clean_glob = self._clean_glob
1042
1042
1043 #regexp to parse docstring for function signature
1043 #regexp to parse docstring for function signature
1044 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1044 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1045 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1045 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1046 #use this if positional argument name is also needed
1046 #use this if positional argument name is also needed
1047 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1047 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1048
1048
1049 # All active matcher routines for completion
1049 # All active matcher routines for completion
1050 self.matchers = [
1050 self.matchers = [
1051 self.python_matches,
1051 self.python_matches,
1052 self.file_matches,
1052 self.file_matches,
1053 self.magic_matches,
1053 self.magic_matches,
1054 self.python_func_kw_matches,
1054 self.python_func_kw_matches,
1055 self.dict_key_matches,
1055 self.dict_key_matches,
1056 ]
1056 ]
1057
1057
1058 # This is set externally by InteractiveShell
1058 # This is set externally by InteractiveShell
1059 self.custom_completers = None
1059 self.custom_completers = None
1060
1060
1061 def all_completions(self, text):
1061 def all_completions(self, text):
1062 """
1062 """
1063 Wrapper around the complete method for the benefit of emacs.
1063 Wrapper around the complete method for the benefit of emacs.
1064 """
1064 """
1065 return self.complete(text)[1]
1065 return self.complete(text)[1]
1066
1066
1067 def _clean_glob(self, text):
1067 def _clean_glob(self, text):
1068 return self.glob("%s*" % text)
1068 return self.glob("%s*" % text)
1069
1069
1070 def _clean_glob_win32(self,text):
1070 def _clean_glob_win32(self,text):
1071 return [f.replace("\\","/")
1071 return [f.replace("\\","/")
1072 for f in self.glob("%s*" % text)]
1072 for f in self.glob("%s*" % text)]
1073
1073
1074 def file_matches(self, text):
1074 def file_matches(self, text):
1075 """Match filenames, expanding ~USER type strings.
1075 """Match filenames, expanding ~USER type strings.
1076
1076
1077 Most of the seemingly convoluted logic in this completer is an
1077 Most of the seemingly convoluted logic in this completer is an
1078 attempt to handle filenames with spaces in them. And yet it's not
1078 attempt to handle filenames with spaces in them. And yet it's not
1079 quite perfect, because Python's readline doesn't expose all of the
1079 quite perfect, because Python's readline doesn't expose all of the
1080 GNU readline details needed for this to be done correctly.
1080 GNU readline details needed for this to be done correctly.
1081
1081
1082 For a filename with a space in it, the printed completions will be
1082 For a filename with a space in it, the printed completions will be
1083 only the parts after what's already been typed (instead of the
1083 only the parts after what's already been typed (instead of the
1084 full completions, as is normally done). I don't think with the
1084 full completions, as is normally done). I don't think with the
1085 current (as of Python 2.3) Python readline it's possible to do
1085 current (as of Python 2.3) Python readline it's possible to do
1086 better."""
1086 better."""
1087
1087
1088 # chars that require escaping with backslash - i.e. chars
1088 # chars that require escaping with backslash - i.e. chars
1089 # that readline treats incorrectly as delimiters, but we
1089 # that readline treats incorrectly as delimiters, but we
1090 # don't want to treat as delimiters in filename matching
1090 # don't want to treat as delimiters in filename matching
1091 # when escaped with backslash
1091 # when escaped with backslash
1092 if text.startswith('!'):
1092 if text.startswith('!'):
1093 text = text[1:]
1093 text = text[1:]
1094 text_prefix = u'!'
1094 text_prefix = u'!'
1095 else:
1095 else:
1096 text_prefix = u''
1096 text_prefix = u''
1097
1097
1098 text_until_cursor = self.text_until_cursor
1098 text_until_cursor = self.text_until_cursor
1099 # track strings with open quotes
1099 # track strings with open quotes
1100 open_quotes = has_open_quotes(text_until_cursor)
1100 open_quotes = has_open_quotes(text_until_cursor)
1101
1101
1102 if '(' in text_until_cursor or '[' in text_until_cursor:
1102 if '(' in text_until_cursor or '[' in text_until_cursor:
1103 lsplit = text
1103 lsplit = text
1104 else:
1104 else:
1105 try:
1105 try:
1106 # arg_split ~ shlex.split, but with unicode bugs fixed by us
1106 # arg_split ~ shlex.split, but with unicode bugs fixed by us
1107 lsplit = arg_split(text_until_cursor)[-1]
1107 lsplit = arg_split(text_until_cursor)[-1]
1108 except ValueError:
1108 except ValueError:
1109 # typically an unmatched ", or backslash without escaped char.
1109 # typically an unmatched ", or backslash without escaped char.
1110 if open_quotes:
1110 if open_quotes:
1111 lsplit = text_until_cursor.split(open_quotes)[-1]
1111 lsplit = text_until_cursor.split(open_quotes)[-1]
1112 else:
1112 else:
1113 return []
1113 return []
1114 except IndexError:
1114 except IndexError:
1115 # tab pressed on empty line
1115 # tab pressed on empty line
1116 lsplit = ""
1116 lsplit = ""
1117
1117
1118 if not open_quotes and lsplit != protect_filename(lsplit):
1118 if not open_quotes and lsplit != protect_filename(lsplit):
1119 # if protectables are found, do matching on the whole escaped name
1119 # if protectables are found, do matching on the whole escaped name
1120 has_protectables = True
1120 has_protectables = True
1121 text0,text = text,lsplit
1121 text0,text = text,lsplit
1122 else:
1122 else:
1123 has_protectables = False
1123 has_protectables = False
1124 text = os.path.expanduser(text)
1124 text = os.path.expanduser(text)
1125
1125
1126 if text == "":
1126 if text == "":
1127 return [text_prefix + cast_unicode_py2(protect_filename(f)) for f in self.glob("*")]
1127 return [text_prefix + cast_unicode_py2(protect_filename(f)) for f in self.glob("*")]
1128
1128
1129 # Compute the matches from the filesystem
1129 # Compute the matches from the filesystem
1130 if sys.platform == 'win32':
1130 if sys.platform == 'win32':
1131 m0 = self.clean_glob(text)
1131 m0 = self.clean_glob(text)
1132 else:
1132 else:
1133 m0 = self.clean_glob(text.replace('\\', ''))
1133 m0 = self.clean_glob(text.replace('\\', ''))
1134
1134
1135 if has_protectables:
1135 if has_protectables:
1136 # If we had protectables, we need to revert our changes to the
1136 # If we had protectables, we need to revert our changes to the
1137 # beginning of filename so that we don't double-write the part
1137 # beginning of filename so that we don't double-write the part
1138 # of the filename we have so far
1138 # of the filename we have so far
1139 len_lsplit = len(lsplit)
1139 len_lsplit = len(lsplit)
1140 matches = [text_prefix + text0 +
1140 matches = [text_prefix + text0 +
1141 protect_filename(f[len_lsplit:]) for f in m0]
1141 protect_filename(f[len_lsplit:]) for f in m0]
1142 else:
1142 else:
1143 if open_quotes:
1143 if open_quotes:
1144 # if we have a string with an open quote, we don't need to
1144 # if we have a string with an open quote, we don't need to
1145 # protect the names at all (and we _shouldn't_, as it
1145 # protect the names at all (and we _shouldn't_, as it
1146 # would cause bugs when the filesystem call is made).
1146 # would cause bugs when the filesystem call is made).
1147 matches = m0
1147 matches = m0
1148 else:
1148 else:
1149 matches = [text_prefix +
1149 matches = [text_prefix +
1150 protect_filename(f) for f in m0]
1150 protect_filename(f) for f in m0]
1151
1151
1152 # Mark directories in input list by appending '/' to their names.
1152 # Mark directories in input list by appending '/' to their names.
1153 return [cast_unicode_py2(x+'/') if os.path.isdir(x) else x for x in matches]
1153 return [cast_unicode_py2(x+'/') if os.path.isdir(x) else x for x in matches]
1154
1154
1155 def magic_matches(self, text):
1155 def magic_matches(self, text):
1156 """Match magics"""
1156 """Match magics"""
1157 # Get all shell magics now rather than statically, so magics loaded at
1157 # Get all shell magics now rather than statically, so magics loaded at
1158 # runtime show up too.
1158 # runtime show up too.
1159 lsm = self.shell.magics_manager.lsmagic()
1159 lsm = self.shell.magics_manager.lsmagic()
1160 line_magics = lsm['line']
1160 line_magics = lsm['line']
1161 cell_magics = lsm['cell']
1161 cell_magics = lsm['cell']
1162 pre = self.magic_escape
1162 pre = self.magic_escape
1163 pre2 = pre+pre
1163 pre2 = pre+pre
1164
1164
1165 # Completion logic:
1165 # Completion logic:
1166 # - user gives %%: only do cell magics
1166 # - user gives %%: only do cell magics
1167 # - user gives %: do both line and cell magics
1167 # - user gives %: do both line and cell magics
1168 # - no prefix: do both
1168 # - no prefix: do both
1169 # In other words, line magics are skipped if the user gives %% explicitly
1169 # In other words, line magics are skipped if the user gives %% explicitly
1170 bare_text = text.lstrip(pre)
1170 bare_text = text.lstrip(pre)
1171 comp = [ pre2+m for m in cell_magics if m.startswith(bare_text)]
1171 comp = [ pre2+m for m in cell_magics if m.startswith(bare_text)]
1172 if not text.startswith(pre2):
1172 if not text.startswith(pre2):
1173 comp += [ pre+m for m in line_magics if m.startswith(bare_text)]
1173 comp += [ pre+m for m in line_magics if m.startswith(bare_text)]
1174 return [cast_unicode_py2(c) for c in comp]
1174 return [cast_unicode_py2(c) for c in comp]
1175
1175
1176 def _jedi_matches(self, cursor_column:int, cursor_line:int, text:str):
1176 def _jedi_matches(self, cursor_column:int, cursor_line:int, text:str):
1177 """
1177 """
1178
1178
1179 Return a list of :any:`jedi.api.Completions` object from a ``text`` and
1179 Return a list of :any:`jedi.api.Completions` object from a ``text`` and
1180 cursor position.
1180 cursor position.
1181
1181
1182 Parameters
1182 Parameters
1183 ----------
1183 ----------
1184 cursor_column : int
1184 cursor_column : int
1185 column position of the cursor in ``text``, 0-indexed.
1185 column position of the cursor in ``text``, 0-indexed.
1186 cursor_line : int
1186 cursor_line : int
1187 line position of the cursor in ``text``, 0-indexed
1187 line position of the cursor in ``text``, 0-indexed
1188 text : str
1188 text : str
1189 text to complete
1189 text to complete
1190
1190
1191 Debugging
1191 Debugging
1192 ---------
1192 ---------
1193
1193
1194 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
1194 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
1195 object containing a string with the Jedi debug information attached.
1195 object containing a string with the Jedi debug information attached.
1196 """
1196 """
1197 namespaces = [self.namespace]
1197 namespaces = [self.namespace]
1198 if self.global_namespace is not None:
1198 if self.global_namespace is not None:
1199 namespaces.append(self.global_namespace)
1199 namespaces.append(self.global_namespace)
1200
1200
1201 completion_filter = lambda x:x
1201 completion_filter = lambda x:x
1202 # cursor_pos is an it, jedi wants line and column
1202 # cursor_pos is an it, jedi wants line and column
1203 offset = cursor_to_position(text, cursor_line, cursor_column)
1203 offset = cursor_to_position(text, cursor_line, cursor_column)
1204 # filter output if we are completing for object members
1204 # filter output if we are completing for object members
1205 if offset:
1205 if offset:
1206 pre = text[offset-1]
1206 pre = text[offset-1]
1207 if pre == '.':
1207 if pre == '.':
1208 if self.omit__names == 2:
1208 if self.omit__names == 2:
1209 completion_filter = lambda c:not c.name.startswith('_')
1209 completion_filter = lambda c:not c.name.startswith('_')
1210 elif self.omit__names == 1:
1210 elif self.omit__names == 1:
1211 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
1211 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
1212 elif self.omit__names == 0:
1212 elif self.omit__names == 0:
1213 completion_filter = lambda x:x
1213 completion_filter = lambda x:x
1214 else:
1214 else:
1215 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
1215 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
1216
1216
1217 interpreter = jedi.Interpreter(
1217 interpreter = jedi.Interpreter(
1218 text, namespaces, column=cursor_column, line=cursor_line + 1)
1218 text, namespaces, column=cursor_column, line=cursor_line + 1)
1219
1219
1220 try_jedi = False
1220 try_jedi = False
1221
1221
1222 try:
1222 try:
1223 # should we check the type of the node is Error ?
1223 # should we check the type of the node is Error ?
1224 from jedi.parser.tree import ErrorLeaf
1224 from jedi.parser.tree import ErrorLeaf
1225 next_to_last_tree = interpreter._get_module().tree_node.children[-2]
1225 next_to_last_tree = interpreter._get_module().tree_node.children[-2]
1226 completing_string = False
1226 completing_string = False
1227 if isinstance(next_to_last_tree, ErrorLeaf):
1227 if isinstance(next_to_last_tree, ErrorLeaf):
1228 completing_string = interpreter._get_module().tree_node.children[-2].value[0] in {'"', "'"}
1228 completing_string = interpreter._get_module().tree_node.children[-2].value[0] in {'"', "'"}
1229 # if we are in a string jedi is likely not the right candidate for
1229 # if we are in a string jedi is likely not the right candidate for
1230 # now. Skip it.
1230 # now. Skip it.
1231 try_jedi = not completing_string
1231 try_jedi = not completing_string
1232 except Exception as e:
1232 except Exception as e:
1233 # many of things can go wrong, we are using private API just don't crash.
1233 # many of things can go wrong, we are using private API just don't crash.
1234 if self.debug:
1234 if self.debug:
1235 print("Error detecting if completing a non-finished string :", e, '|')
1235 print("Error detecting if completing a non-finished string :", e, '|')
1236
1236
1237 if not try_jedi:
1237 if not try_jedi:
1238 return []
1238 return []
1239 try:
1239 try:
1240 return filter(completion_filter, interpreter.completions())
1240 return filter(completion_filter, interpreter.completions())
1241 except Exception as e:
1241 except Exception as e:
1242 if self.debug:
1242 if self.debug:
1243 return [_FakeJediCompletion('Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""' % (e))]
1243 return [_FakeJediCompletion('Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""' % (e))]
1244 else:
1244 else:
1245 return []
1245 return []
1246
1246
1247 def python_matches(self, text):
1247 def python_matches(self, text):
1248 """Match attributes or global python names"""
1248 """Match attributes or global python names"""
1249 if "." in text:
1249 if "." in text:
1250 try:
1250 try:
1251 matches = self.attr_matches(text)
1251 matches = self.attr_matches(text)
1252 if text.endswith('.') and self.omit__names:
1252 if text.endswith('.') and self.omit__names:
1253 if self.omit__names == 1:
1253 if self.omit__names == 1:
1254 # true if txt is _not_ a __ name, false otherwise:
1254 # true if txt is _not_ a __ name, false otherwise:
1255 no__name = (lambda txt:
1255 no__name = (lambda txt:
1256 re.match(r'.*\.__.*?__',txt) is None)
1256 re.match(r'.*\.__.*?__',txt) is None)
1257 else:
1257 else:
1258 # true if txt is _not_ a _ name, false otherwise:
1258 # true if txt is _not_ a _ name, false otherwise:
1259 no__name = (lambda txt:
1259 no__name = (lambda txt:
1260 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
1260 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
1261 matches = filter(no__name, matches)
1261 matches = filter(no__name, matches)
1262 except NameError:
1262 except NameError:
1263 # catches <undefined attributes>.<tab>
1263 # catches <undefined attributes>.<tab>
1264 matches = []
1264 matches = []
1265 else:
1265 else:
1266 matches = self.global_matches(text)
1266 matches = self.global_matches(text)
1267 return matches
1267 return matches
1268
1268
1269 def _default_arguments_from_docstring(self, doc):
1269 def _default_arguments_from_docstring(self, doc):
1270 """Parse the first line of docstring for call signature.
1270 """Parse the first line of docstring for call signature.
1271
1271
1272 Docstring should be of the form 'min(iterable[, key=func])\n'.
1272 Docstring should be of the form 'min(iterable[, key=func])\n'.
1273 It can also parse cython docstring of the form
1273 It can also parse cython docstring of the form
1274 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
1274 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
1275 """
1275 """
1276 if doc is None:
1276 if doc is None:
1277 return []
1277 return []
1278
1278
1279 #care only the firstline
1279 #care only the firstline
1280 line = doc.lstrip().splitlines()[0]
1280 line = doc.lstrip().splitlines()[0]
1281
1281
1282 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1282 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1283 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
1283 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
1284 sig = self.docstring_sig_re.search(line)
1284 sig = self.docstring_sig_re.search(line)
1285 if sig is None:
1285 if sig is None:
1286 return []
1286 return []
1287 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
1287 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
1288 sig = sig.groups()[0].split(',')
1288 sig = sig.groups()[0].split(',')
1289 ret = []
1289 ret = []
1290 for s in sig:
1290 for s in sig:
1291 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1291 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1292 ret += self.docstring_kwd_re.findall(s)
1292 ret += self.docstring_kwd_re.findall(s)
1293 return ret
1293 return ret
1294
1294
1295 def _default_arguments(self, obj):
1295 def _default_arguments(self, obj):
1296 """Return the list of default arguments of obj if it is callable,
1296 """Return the list of default arguments of obj if it is callable,
1297 or empty list otherwise."""
1297 or empty list otherwise."""
1298 call_obj = obj
1298 call_obj = obj
1299 ret = []
1299 ret = []
1300 if inspect.isbuiltin(obj):
1300 if inspect.isbuiltin(obj):
1301 pass
1301 pass
1302 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
1302 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
1303 if inspect.isclass(obj):
1303 if inspect.isclass(obj):
1304 #for cython embededsignature=True the constructor docstring
1304 #for cython embededsignature=True the constructor docstring
1305 #belongs to the object itself not __init__
1305 #belongs to the object itself not __init__
1306 ret += self._default_arguments_from_docstring(
1306 ret += self._default_arguments_from_docstring(
1307 getattr(obj, '__doc__', ''))
1307 getattr(obj, '__doc__', ''))
1308 # for classes, check for __init__,__new__
1308 # for classes, check for __init__,__new__
1309 call_obj = (getattr(obj, '__init__', None) or
1309 call_obj = (getattr(obj, '__init__', None) or
1310 getattr(obj, '__new__', None))
1310 getattr(obj, '__new__', None))
1311 # for all others, check if they are __call__able
1311 # for all others, check if they are __call__able
1312 elif hasattr(obj, '__call__'):
1312 elif hasattr(obj, '__call__'):
1313 call_obj = obj.__call__
1313 call_obj = obj.__call__
1314 ret += self._default_arguments_from_docstring(
1314 ret += self._default_arguments_from_docstring(
1315 getattr(call_obj, '__doc__', ''))
1315 getattr(call_obj, '__doc__', ''))
1316
1316
1317 _keeps = (inspect.Parameter.KEYWORD_ONLY,
1317 _keeps = (inspect.Parameter.KEYWORD_ONLY,
1318 inspect.Parameter.POSITIONAL_OR_KEYWORD)
1318 inspect.Parameter.POSITIONAL_OR_KEYWORD)
1319
1319
1320 try:
1320 try:
1321 sig = inspect.signature(call_obj)
1321 sig = inspect.signature(call_obj)
1322 ret.extend(k for k, v in sig.parameters.items() if
1322 ret.extend(k for k, v in sig.parameters.items() if
1323 v.kind in _keeps)
1323 v.kind in _keeps)
1324 except ValueError:
1324 except ValueError:
1325 pass
1325 pass
1326
1326
1327 return list(set(ret))
1327 return list(set(ret))
1328
1328
1329 def python_func_kw_matches(self,text):
1329 def python_func_kw_matches(self,text):
1330 """Match named parameters (kwargs) of the last open function"""
1330 """Match named parameters (kwargs) of the last open function"""
1331
1331
1332 if "." in text: # a parameter cannot be dotted
1332 if "." in text: # a parameter cannot be dotted
1333 return []
1333 return []
1334 try: regexp = self.__funcParamsRegex
1334 try: regexp = self.__funcParamsRegex
1335 except AttributeError:
1335 except AttributeError:
1336 regexp = self.__funcParamsRegex = re.compile(r'''
1336 regexp = self.__funcParamsRegex = re.compile(r'''
1337 '.*?(?<!\\)' | # single quoted strings or
1337 '.*?(?<!\\)' | # single quoted strings or
1338 ".*?(?<!\\)" | # double quoted strings or
1338 ".*?(?<!\\)" | # double quoted strings or
1339 \w+ | # identifier
1339 \w+ | # identifier
1340 \S # other characters
1340 \S # other characters
1341 ''', re.VERBOSE | re.DOTALL)
1341 ''', re.VERBOSE | re.DOTALL)
1342 # 1. find the nearest identifier that comes before an unclosed
1342 # 1. find the nearest identifier that comes before an unclosed
1343 # parenthesis before the cursor
1343 # parenthesis before the cursor
1344 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
1344 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
1345 tokens = regexp.findall(self.text_until_cursor)
1345 tokens = regexp.findall(self.text_until_cursor)
1346 iterTokens = reversed(tokens); openPar = 0
1346 iterTokens = reversed(tokens); openPar = 0
1347
1347
1348 for token in iterTokens:
1348 for token in iterTokens:
1349 if token == ')':
1349 if token == ')':
1350 openPar -= 1
1350 openPar -= 1
1351 elif token == '(':
1351 elif token == '(':
1352 openPar += 1
1352 openPar += 1
1353 if openPar > 0:
1353 if openPar > 0:
1354 # found the last unclosed parenthesis
1354 # found the last unclosed parenthesis
1355 break
1355 break
1356 else:
1356 else:
1357 return []
1357 return []
1358 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
1358 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
1359 ids = []
1359 ids = []
1360 isId = re.compile(r'\w+$').match
1360 isId = re.compile(r'\w+$').match
1361
1361
1362 while True:
1362 while True:
1363 try:
1363 try:
1364 ids.append(next(iterTokens))
1364 ids.append(next(iterTokens))
1365 if not isId(ids[-1]):
1365 if not isId(ids[-1]):
1366 ids.pop(); break
1366 ids.pop(); break
1367 if not next(iterTokens) == '.':
1367 if not next(iterTokens) == '.':
1368 break
1368 break
1369 except StopIteration:
1369 except StopIteration:
1370 break
1370 break
1371
1371
1372 # Find all named arguments already assigned to, as to avoid suggesting
1372 # Find all named arguments already assigned to, as to avoid suggesting
1373 # them again
1373 # them again
1374 usedNamedArgs = set()
1374 usedNamedArgs = set()
1375 par_level = -1
1375 par_level = -1
1376 for token, next_token in zip(tokens, tokens[1:]):
1376 for token, next_token in zip(tokens, tokens[1:]):
1377 if token == '(':
1377 if token == '(':
1378 par_level += 1
1378 par_level += 1
1379 elif token == ')':
1379 elif token == ')':
1380 par_level -= 1
1380 par_level -= 1
1381
1381
1382 if par_level != 0:
1382 if par_level != 0:
1383 continue
1383 continue
1384
1384
1385 if next_token != '=':
1385 if next_token != '=':
1386 continue
1386 continue
1387
1387
1388 usedNamedArgs.add(token)
1388 usedNamedArgs.add(token)
1389
1389
1390 # lookup the candidate callable matches either using global_matches
1390 # lookup the candidate callable matches either using global_matches
1391 # or attr_matches for dotted names
1391 # or attr_matches for dotted names
1392 if len(ids) == 1:
1392 if len(ids) == 1:
1393 callableMatches = self.global_matches(ids[0])
1393 callableMatches = self.global_matches(ids[0])
1394 else:
1394 else:
1395 callableMatches = self.attr_matches('.'.join(ids[::-1]))
1395 callableMatches = self.attr_matches('.'.join(ids[::-1]))
1396 argMatches = []
1396 argMatches = []
1397 for callableMatch in callableMatches:
1397 for callableMatch in callableMatches:
1398 try:
1398 try:
1399 namedArgs = self._default_arguments(eval(callableMatch,
1399 namedArgs = self._default_arguments(eval(callableMatch,
1400 self.namespace))
1400 self.namespace))
1401 except:
1401 except:
1402 continue
1402 continue
1403
1403
1404 # Remove used named arguments from the list, no need to show twice
1404 # Remove used named arguments from the list, no need to show twice
1405 for namedArg in set(namedArgs) - usedNamedArgs:
1405 for namedArg in set(namedArgs) - usedNamedArgs:
1406 if namedArg.startswith(text):
1406 if namedArg.startswith(text):
1407 argMatches.append(u"%s=" %namedArg)
1407 argMatches.append(u"%s=" %namedArg)
1408 return argMatches
1408 return argMatches
1409
1409
1410 def dict_key_matches(self, text):
1410 def dict_key_matches(self, text):
1411 "Match string keys in a dictionary, after e.g. 'foo[' "
1411 "Match string keys in a dictionary, after e.g. 'foo[' "
1412 def get_keys(obj):
1412 def get_keys(obj):
1413 # Objects can define their own completions by defining an
1413 # Objects can define their own completions by defining an
1414 # _ipy_key_completions_() method.
1414 # _ipy_key_completions_() method.
1415 method = get_real_method(obj, '_ipython_key_completions_')
1415 method = get_real_method(obj, '_ipython_key_completions_')
1416 if method is not None:
1416 if method is not None:
1417 return method()
1417 return method()
1418
1418
1419 # Special case some common in-memory dict-like types
1419 # Special case some common in-memory dict-like types
1420 if isinstance(obj, dict) or\
1420 if isinstance(obj, dict) or\
1421 _safe_isinstance(obj, 'pandas', 'DataFrame'):
1421 _safe_isinstance(obj, 'pandas', 'DataFrame'):
1422 try:
1422 try:
1423 return list(obj.keys())
1423 return list(obj.keys())
1424 except Exception:
1424 except Exception:
1425 return []
1425 return []
1426 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
1426 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
1427 _safe_isinstance(obj, 'numpy', 'void'):
1427 _safe_isinstance(obj, 'numpy', 'void'):
1428 return obj.dtype.names or []
1428 return obj.dtype.names or []
1429 return []
1429 return []
1430
1430
1431 try:
1431 try:
1432 regexps = self.__dict_key_regexps
1432 regexps = self.__dict_key_regexps
1433 except AttributeError:
1433 except AttributeError:
1434 dict_key_re_fmt = r'''(?x)
1434 dict_key_re_fmt = r'''(?x)
1435 ( # match dict-referring expression wrt greedy setting
1435 ( # match dict-referring expression wrt greedy setting
1436 %s
1436 %s
1437 )
1437 )
1438 \[ # open bracket
1438 \[ # open bracket
1439 \s* # and optional whitespace
1439 \s* # and optional whitespace
1440 ([uUbB]? # string prefix (r not handled)
1440 ([uUbB]? # string prefix (r not handled)
1441 (?: # unclosed string
1441 (?: # unclosed string
1442 '(?:[^']|(?<!\\)\\')*
1442 '(?:[^']|(?<!\\)\\')*
1443 |
1443 |
1444 "(?:[^"]|(?<!\\)\\")*
1444 "(?:[^"]|(?<!\\)\\")*
1445 )
1445 )
1446 )?
1446 )?
1447 $
1447 $
1448 '''
1448 '''
1449 regexps = self.__dict_key_regexps = {
1449 regexps = self.__dict_key_regexps = {
1450 False: re.compile(dict_key_re_fmt % '''
1450 False: re.compile(dict_key_re_fmt % '''
1451 # identifiers separated by .
1451 # identifiers separated by .
1452 (?!\d)\w+
1452 (?!\d)\w+
1453 (?:\.(?!\d)\w+)*
1453 (?:\.(?!\d)\w+)*
1454 '''),
1454 '''),
1455 True: re.compile(dict_key_re_fmt % '''
1455 True: re.compile(dict_key_re_fmt % '''
1456 .+
1456 .+
1457 ''')
1457 ''')
1458 }
1458 }
1459
1459
1460 match = regexps[self.greedy].search(self.text_until_cursor)
1460 match = regexps[self.greedy].search(self.text_until_cursor)
1461 if match is None:
1461 if match is None:
1462 return []
1462 return []
1463
1463
1464 expr, prefix = match.groups()
1464 expr, prefix = match.groups()
1465 try:
1465 try:
1466 obj = eval(expr, self.namespace)
1466 obj = eval(expr, self.namespace)
1467 except Exception:
1467 except Exception:
1468 try:
1468 try:
1469 obj = eval(expr, self.global_namespace)
1469 obj = eval(expr, self.global_namespace)
1470 except Exception:
1470 except Exception:
1471 return []
1471 return []
1472
1472
1473 keys = get_keys(obj)
1473 keys = get_keys(obj)
1474 if not keys:
1474 if not keys:
1475 return keys
1475 return keys
1476 closing_quote, token_offset, matches = match_dict_keys(keys, prefix, self.splitter.delims)
1476 closing_quote, token_offset, matches = match_dict_keys(keys, prefix, self.splitter.delims)
1477 if not matches:
1477 if not matches:
1478 return matches
1478 return matches
1479
1479
1480 # get the cursor position of
1480 # get the cursor position of
1481 # - the text being completed
1481 # - the text being completed
1482 # - the start of the key text
1482 # - the start of the key text
1483 # - the start of the completion
1483 # - the start of the completion
1484 text_start = len(self.text_until_cursor) - len(text)
1484 text_start = len(self.text_until_cursor) - len(text)
1485 if prefix:
1485 if prefix:
1486 key_start = match.start(2)
1486 key_start = match.start(2)
1487 completion_start = key_start + token_offset
1487 completion_start = key_start + token_offset
1488 else:
1488 else:
1489 key_start = completion_start = match.end()
1489 key_start = completion_start = match.end()
1490
1490
1491 # grab the leading prefix, to make sure all completions start with `text`
1491 # grab the leading prefix, to make sure all completions start with `text`
1492 if text_start > key_start:
1492 if text_start > key_start:
1493 leading = ''
1493 leading = ''
1494 else:
1494 else:
1495 leading = text[text_start:completion_start]
1495 leading = text[text_start:completion_start]
1496
1496
1497 # the index of the `[` character
1497 # the index of the `[` character
1498 bracket_idx = match.end(1)
1498 bracket_idx = match.end(1)
1499
1499
1500 # append closing quote and bracket as appropriate
1500 # append closing quote and bracket as appropriate
1501 # this is *not* appropriate if the opening quote or bracket is outside
1501 # this is *not* appropriate if the opening quote or bracket is outside
1502 # the text given to this method
1502 # the text given to this method
1503 suf = ''
1503 suf = ''
1504 continuation = self.line_buffer[len(self.text_until_cursor):]
1504 continuation = self.line_buffer[len(self.text_until_cursor):]
1505 if key_start > text_start and closing_quote:
1505 if key_start > text_start and closing_quote:
1506 # quotes were opened inside text, maybe close them
1506 # quotes were opened inside text, maybe close them
1507 if continuation.startswith(closing_quote):
1507 if continuation.startswith(closing_quote):
1508 continuation = continuation[len(closing_quote):]
1508 continuation = continuation[len(closing_quote):]
1509 else:
1509 else:
1510 suf += closing_quote
1510 suf += closing_quote
1511 if bracket_idx > text_start:
1511 if bracket_idx > text_start:
1512 # brackets were opened inside text, maybe close them
1512 # brackets were opened inside text, maybe close them
1513 if not continuation.startswith(']'):
1513 if not continuation.startswith(']'):
1514 suf += ']'
1514 suf += ']'
1515
1515
1516 return [leading + k + suf for k in matches]
1516 return [leading + k + suf for k in matches]
1517
1517
1518 def unicode_name_matches(self, text):
1518 def unicode_name_matches(self, text):
1519 u"""Match Latex-like syntax for unicode characters base
1519 u"""Match Latex-like syntax for unicode characters base
1520 on the name of the character.
1520 on the name of the character.
1521
1521
1522 This does ``\\GREEK SMALL LETTER ETA`` -> ``Ξ·``
1522 This does ``\\GREEK SMALL LETTER ETA`` -> ``Ξ·``
1523
1523
1524 Works only on valid python 3 identifier, or on combining characters that
1524 Works only on valid python 3 identifier, or on combining characters that
1525 will combine to form a valid identifier.
1525 will combine to form a valid identifier.
1526
1526
1527 Used on Python 3 only.
1527 Used on Python 3 only.
1528 """
1528 """
1529 slashpos = text.rfind('\\')
1529 slashpos = text.rfind('\\')
1530 if slashpos > -1:
1530 if slashpos > -1:
1531 s = text[slashpos+1:]
1531 s = text[slashpos+1:]
1532 try :
1532 try :
1533 unic = unicodedata.lookup(s)
1533 unic = unicodedata.lookup(s)
1534 # allow combining chars
1534 # allow combining chars
1535 if ('a'+unic).isidentifier():
1535 if ('a'+unic).isidentifier():
1536 return '\\'+s,[unic]
1536 return '\\'+s,[unic]
1537 except KeyError:
1537 except KeyError:
1538 pass
1538 pass
1539 return u'', []
1539 return u'', []
1540
1540
1541
1541
1542 def latex_matches(self, text):
1542 def latex_matches(self, text):
1543 u"""Match Latex syntax for unicode characters.
1543 u"""Match Latex syntax for unicode characters.
1544
1544
1545 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
1545 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``Ξ±``
1546
1546
1547 Used on Python 3 only.
1547 Used on Python 3 only.
1548 """
1548 """
1549 slashpos = text.rfind('\\')
1549 slashpos = text.rfind('\\')
1550 if slashpos > -1:
1550 if slashpos > -1:
1551 s = text[slashpos:]
1551 s = text[slashpos:]
1552 if s in latex_symbols:
1552 if s in latex_symbols:
1553 # Try to complete a full latex symbol to unicode
1553 # Try to complete a full latex symbol to unicode
1554 # \\alpha -> Ξ±
1554 # \\alpha -> Ξ±
1555 return s, [latex_symbols[s]]
1555 return s, [latex_symbols[s]]
1556 else:
1556 else:
1557 # If a user has partially typed a latex symbol, give them
1557 # If a user has partially typed a latex symbol, give them
1558 # a full list of options \al -> [\aleph, \alpha]
1558 # a full list of options \al -> [\aleph, \alpha]
1559 matches = [k for k in latex_symbols if k.startswith(s)]
1559 matches = [k for k in latex_symbols if k.startswith(s)]
1560 return s, matches
1560 return s, matches
1561 return u'', []
1561 return u'', []
1562
1562
1563 def dispatch_custom_completer(self, text):
1563 def dispatch_custom_completer(self, text):
1564 if not self.custom_completers:
1564 if not self.custom_completers:
1565 return
1565 return
1566
1566
1567 line = self.line_buffer
1567 line = self.line_buffer
1568 if not line.strip():
1568 if not line.strip():
1569 return None
1569 return None
1570
1570
1571 # Create a little structure to pass all the relevant information about
1571 # Create a little structure to pass all the relevant information about
1572 # the current completion to any custom completer.
1572 # the current completion to any custom completer.
1573 event = SimpleNamespace()
1573 event = SimpleNamespace()
1574 event.line = line
1574 event.line = line
1575 event.symbol = text
1575 event.symbol = text
1576 cmd = line.split(None,1)[0]
1576 cmd = line.split(None,1)[0]
1577 event.command = cmd
1577 event.command = cmd
1578 event.text_until_cursor = self.text_until_cursor
1578 event.text_until_cursor = self.text_until_cursor
1579
1579
1580 # for foo etc, try also to find completer for %foo
1580 # for foo etc, try also to find completer for %foo
1581 if not cmd.startswith(self.magic_escape):
1581 if not cmd.startswith(self.magic_escape):
1582 try_magic = self.custom_completers.s_matches(
1582 try_magic = self.custom_completers.s_matches(
1583 self.magic_escape + cmd)
1583 self.magic_escape + cmd)
1584 else:
1584 else:
1585 try_magic = []
1585 try_magic = []
1586
1586
1587 for c in itertools.chain(self.custom_completers.s_matches(cmd),
1587 for c in itertools.chain(self.custom_completers.s_matches(cmd),
1588 try_magic,
1588 try_magic,
1589 self.custom_completers.flat_matches(self.text_until_cursor)):
1589 self.custom_completers.flat_matches(self.text_until_cursor)):
1590 try:
1590 try:
1591 res = c(event)
1591 res = c(event)
1592 if res:
1592 if res:
1593 # first, try case sensitive match
1593 # first, try case sensitive match
1594 withcase = [cast_unicode_py2(r) for r in res if r.startswith(text)]
1594 withcase = [cast_unicode_py2(r) for r in res if r.startswith(text)]
1595 if withcase:
1595 if withcase:
1596 return withcase
1596 return withcase
1597 # if none, then case insensitive ones are ok too
1597 # if none, then case insensitive ones are ok too
1598 text_low = text.lower()
1598 text_low = text.lower()
1599 return [cast_unicode_py2(r) for r in res if r.lower().startswith(text_low)]
1599 return [cast_unicode_py2(r) for r in res if r.lower().startswith(text_low)]
1600 except TryNext:
1600 except TryNext:
1601 pass
1601 pass
1602
1602
1603 return None
1603 return None
1604
1604
1605 def completions(self, text: str, offset: int)->Iterator[Completion]:
1605 def completions(self, text: str, offset: int)->Iterator[Completion]:
1606 """
1606 """
1607 Returns an iterator over the possible completions
1607 Returns an iterator over the possible completions
1608
1608
1609 .. warning:: Unstable
1609 .. warning:: Unstable
1610
1610
1611 This function is unstable, API may change without warning.
1611 This function is unstable, API may change without warning.
1612 It will also raise unless use in proper context manager.
1612 It will also raise unless use in proper context manager.
1613
1613
1614 Parameters
1614 Parameters
1615 ----------
1615 ----------
1616
1616
1617 text:str
1617 text:str
1618 Full text of the current input, multi line string.
1618 Full text of the current input, multi line string.
1619 offset:int
1619 offset:int
1620 Integer representing the position of the cursor in ``text``. Offset
1620 Integer representing the position of the cursor in ``text``. Offset
1621 is 0-based indexed.
1621 is 0-based indexed.
1622
1622
1623 Yields
1623 Yields
1624 ------
1624 ------
1625 :any:`Completion` object
1625 :any:`Completion` object
1626
1626
1627
1627
1628 The cursor on a text can either be seen as being "in between"
1628 The cursor on a text can either be seen as being "in between"
1629 characters or "On" a character depending on the interface visible to
1629 characters or "On" a character depending on the interface visible to
1630 the user. For consistency the cursor being on "in between" characters X
1630 the user. For consistency the cursor being on "in between" characters X
1631 and Y is equivalent to the cursor being "on" character Y, that is to say
1631 and Y is equivalent to the cursor being "on" character Y, that is to say
1632 the character the cursor is on is considered as being after the cursor.
1632 the character the cursor is on is considered as being after the cursor.
1633
1633
1634 Combining characters may span more that one position in the
1634 Combining characters may span more that one position in the
1635 text.
1635 text.
1636
1636
1637
1637
1638 .. note::
1638 .. note::
1639
1639
1640 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
1640 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
1641 fake Completion token to distinguish completion returned by Jedi
1641 fake Completion token to distinguish completion returned by Jedi
1642 and usual IPython completion.
1642 and usual IPython completion.
1643
1643
1644 .. note::
1644 .. note::
1645
1645
1646 Completions are not completely deduplicated yet. If identical
1646 Completions are not completely deduplicated yet. If identical
1647 completions are coming from different sources this function does not
1647 completions are coming from different sources this function does not
1648 ensure that each completion object will only be present once.
1648 ensure that each completion object will only be present once.
1649 """
1649 """
1650 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
1650 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
1651 "It may change without warnings. "
1651 "It may change without warnings. "
1652 "Use in corresponding context manager.",
1652 "Use in corresponding context manager.",
1653 category=ProvisionalCompleterWarning, stacklevel=2)
1653 category=ProvisionalCompleterWarning, stacklevel=2)
1654
1654
1655 seen = set()
1655 seen = set()
1656 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
1656 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
1657 if c and (c in seen):
1657 if c and (c in seen):
1658 continue
1658 continue
1659 yield c
1659 yield c
1660 seen.add(c)
1660 seen.add(c)
1661
1661
1662 def _completions(self, full_text: str, offset: int, *, _timeout)->Iterator[Completion]:
1662 def _completions(self, full_text: str, offset: int, *, _timeout)->Iterator[Completion]:
1663 """
1663 """
1664 Core completion module.Same signature as :any:`completions`, with the
1664 Core completion module.Same signature as :any:`completions`, with the
1665 extra `timeout` parameter (in seconds).
1665 extra `timeout` parameter (in seconds).
1666
1666
1667
1667
1668 Computing jedi's completion ``.type`` can be quite expensive (it is a
1668 Computing jedi's completion ``.type`` can be quite expensive (it is a
1669 lazy property) and can require some warm-up, more warm up than just
1669 lazy property) and can require some warm-up, more warm up than just
1670 computing the ``name`` of a completion. The warm-up can be :
1670 computing the ``name`` of a completion. The warm-up can be :
1671
1671
1672 - Long warm-up the fisrt time a module is encountered after
1672 - Long warm-up the fisrt time a module is encountered after
1673 install/update: actually build parse/inference tree.
1673 install/update: actually build parse/inference tree.
1674
1674
1675 - first time the module is encountered in a session: load tree from
1675 - first time the module is encountered in a session: load tree from
1676 disk.
1676 disk.
1677
1677
1678 We don't want to block completions for tens of seconds so we give the
1678 We don't want to block completions for tens of seconds so we give the
1679 completer a "budget" of ``_timeout`` seconds per invocation to compute
1679 completer a "budget" of ``_timeout`` seconds per invocation to compute
1680 completions types, the completions that have not yet been computed will
1680 completions types, the completions that have not yet been computed will
1681 be marked as "unknown" an will have a chance to be computed next round
1681 be marked as "unknown" an will have a chance to be computed next round
1682 are things get cached.
1682 are things get cached.
1683
1683
1684 Keep in mind that Jedi is not the only thing treating the completion so
1684 Keep in mind that Jedi is not the only thing treating the completion so
1685 keep the timeout short-ish as if we take more than 0.3 second we still
1685 keep the timeout short-ish as if we take more than 0.3 second we still
1686 have lots of processing to do.
1686 have lots of processing to do.
1687
1687
1688 """
1688 """
1689 deadline = time.monotonic() + _timeout
1689 deadline = time.monotonic() + _timeout
1690
1690
1691
1691
1692 before = full_text[:offset]
1692 before = full_text[:offset]
1693 cursor_line, cursor_column = position_to_cursor(full_text, offset)
1693 cursor_line, cursor_column = position_to_cursor(full_text, offset)
1694
1694
1695 matched_text, matches, matches_origin, jedi_matches = self._complete(
1695 matched_text, matches, matches_origin, jedi_matches = self._complete(
1696 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column)
1696 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column)
1697
1697
1698 iter_jm = iter(jedi_matches)
1698 iter_jm = iter(jedi_matches)
1699 if _timeout:
1699 if _timeout:
1700 for jm in iter_jm:
1700 for jm in iter_jm:
1701 try:
1701 try:
1702 type_ = jm.type
1702 type_ = jm.type
1703 except Exception:
1703 except Exception:
1704 if self.debug:
1704 if self.debug:
1705 print("Error in Jedi getting type of ", jm)
1705 print("Error in Jedi getting type of ", jm)
1706 type_ = None
1706 type_ = None
1707 delta = len(jm.name_with_symbols) - len(jm.complete)
1707 delta = len(jm.name_with_symbols) - len(jm.complete)
1708 yield Completion(start=offset - delta,
1708 yield Completion(start=offset - delta,
1709 end=offset,
1709 end=offset,
1710 text=jm.name_with_symbols,
1710 text=jm.name_with_symbols,
1711 type=type_,
1711 type=type_,
1712 _origin='jedi')
1712 _origin='jedi')
1713
1713
1714 if time.monotonic() > deadline:
1714 if time.monotonic() > deadline:
1715 break
1715 break
1716
1716
1717 for jm in iter_jm:
1717 for jm in iter_jm:
1718 delta = len(jm.name_with_symbols) - len(jm.complete)
1718 delta = len(jm.name_with_symbols) - len(jm.complete)
1719 yield Completion(start=offset - delta,
1719 yield Completion(start=offset - delta,
1720 end=offset,
1720 end=offset,
1721 text=jm.name_with_symbols,
1721 text=jm.name_with_symbols,
1722 type='<unknown>', # don't compute type for speed
1722 type='<unknown>', # don't compute type for speed
1723 _origin='jedi')
1723 _origin='jedi')
1724
1724
1725
1725
1726 start_offset = before.rfind(matched_text)
1726 start_offset = before.rfind(matched_text)
1727
1727
1728 # TODO:
1728 # TODO:
1729 # Supress this, right now just for debug.
1729 # Supress this, right now just for debug.
1730 if jedi_matches and matches and self.debug:
1730 if jedi_matches and matches and self.debug:
1731 yield Completion(start=start_offset, end=offset, text='--jedi/ipython--', _origin='debug')
1731 yield Completion(start=start_offset, end=offset, text='--jedi/ipython--', _origin='debug')
1732
1732
1733 # I'm unsure if this is always true, so let's assert and see if it
1733 # I'm unsure if this is always true, so let's assert and see if it
1734 # crash
1734 # crash
1735 assert before.endswith(matched_text)
1735 assert before.endswith(matched_text)
1736 for m, t in zip(matches, matches_origin):
1736 for m, t in zip(matches, matches_origin):
1737 yield Completion(start=start_offset, end=offset, text=m, _origin=t)
1737 yield Completion(start=start_offset, end=offset, text=m, _origin=t)
1738
1738
1739
1739
1740 def complete(self, text=None, line_buffer=None, cursor_pos=None):
1740 def complete(self, text=None, line_buffer=None, cursor_pos=None):
1741 """Find completions for the given text and line context.
1741 """Find completions for the given text and line context.
1742
1742
1743 Note that both the text and the line_buffer are optional, but at least
1743 Note that both the text and the line_buffer are optional, but at least
1744 one of them must be given.
1744 one of them must be given.
1745
1745
1746 Parameters
1746 Parameters
1747 ----------
1747 ----------
1748 text : string, optional
1748 text : string, optional
1749 Text to perform the completion on. If not given, the line buffer
1749 Text to perform the completion on. If not given, the line buffer
1750 is split using the instance's CompletionSplitter object.
1750 is split using the instance's CompletionSplitter object.
1751
1751
1752 line_buffer : string, optional
1752 line_buffer : string, optional
1753 If not given, the completer attempts to obtain the current line
1753 If not given, the completer attempts to obtain the current line
1754 buffer via readline. This keyword allows clients which are
1754 buffer via readline. This keyword allows clients which are
1755 requesting for text completions in non-readline contexts to inform
1755 requesting for text completions in non-readline contexts to inform
1756 the completer of the entire text.
1756 the completer of the entire text.
1757
1757
1758 cursor_pos : int, optional
1758 cursor_pos : int, optional
1759 Index of the cursor in the full line buffer. Should be provided by
1759 Index of the cursor in the full line buffer. Should be provided by
1760 remote frontends where kernel has no access to frontend state.
1760 remote frontends where kernel has no access to frontend state.
1761
1761
1762 Returns
1762 Returns
1763 -------
1763 -------
1764 text : str
1764 text : str
1765 Text that was actually used in the completion.
1765 Text that was actually used in the completion.
1766
1766
1767 matches : list
1767 matches : list
1768 A list of completion matches.
1768 A list of completion matches.
1769
1769
1770
1770
1771 .. note::
1771 .. note::
1772
1772
1773 This API is likely to be deprecated and replaced by
1773 This API is likely to be deprecated and replaced by
1774 :any:`IPCompleter.completions` in the future.
1774 :any:`IPCompleter.completions` in the future.
1775
1775
1776
1776
1777 """
1777 """
1778 warnings.warn('`Completer.complete` is pending deprecation since '
1778 warnings.warn('`Completer.complete` is pending deprecation since '
1779 'IPython 6.0 and will be replaced by `Completer.completions`.',
1779 'IPython 6.0 and will be replaced by `Completer.completions`.',
1780 PendingDeprecationWarning)
1780 PendingDeprecationWarning)
1781 # potential todo, FOLD the 3rd throw away argument of _complete
1781 # potential todo, FOLD the 3rd throw away argument of _complete
1782 # into the first 2 one.
1782 # into the first 2 one.
1783 return self._complete(line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0)[:2]
1783 return self._complete(line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0)[:2]
1784
1784
1785 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
1785 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
1786 full_text=None, return_jedi_results=True) -> (str, List[str], List[object]):
1786 full_text=None, return_jedi_results=True) -> (str, List[str], List[object]):
1787 """
1787 """
1788
1788
1789 Like complete but can also returns raw jedi completions as well as the
1789 Like complete but can also returns raw jedi completions as well as the
1790 origin of the completion text. This could (and should) be made much
1790 origin of the completion text. This could (and should) be made much
1791 cleaner but that will be simpler once we drop the old (and stateful)
1791 cleaner but that will be simpler once we drop the old (and stateful)
1792 :any:`complete` API.
1792 :any:`complete` API.
1793
1793
1794
1794
1795 With current provisional API, cursor_pos act both (depending on the
1795 With current provisional API, cursor_pos act both (depending on the
1796 caller) as the offset in the ``text`` or ``line_buffer``, or as the
1796 caller) as the offset in the ``text`` or ``line_buffer``, or as the
1797 ``column`` when passing multiline strings this could/should be renamed
1797 ``column`` when passing multiline strings this could/should be renamed
1798 but would add extra noise.
1798 but would add extra noise.
1799 """
1799 """
1800
1800
1801 # if the cursor position isn't given, the only sane assumption we can
1801 # if the cursor position isn't given, the only sane assumption we can
1802 # make is that it's at the end of the line (the common case)
1802 # make is that it's at the end of the line (the common case)
1803 if cursor_pos is None:
1803 if cursor_pos is None:
1804 cursor_pos = len(line_buffer) if text is None else len(text)
1804 cursor_pos = len(line_buffer) if text is None else len(text)
1805
1805
1806 if self.use_main_ns:
1806 if self.use_main_ns:
1807 self.namespace = __main__.__dict__
1807 self.namespace = __main__.__dict__
1808
1808
1809 # if text is either None or an empty string, rely on the line buffer
1809 # if text is either None or an empty string, rely on the line buffer
1810 if (not line_buffer) and full_text:
1810 if (not line_buffer) and full_text:
1811 line_buffer = full_text.split('\n')[cursor_line]
1811 line_buffer = full_text.split('\n')[cursor_line]
1812 if not text:
1812 if not text:
1813 text = self.splitter.split_line(line_buffer, cursor_pos)
1813 text = self.splitter.split_line(line_buffer, cursor_pos)
1814
1814
1815 if self.backslash_combining_completions:
1815 if self.backslash_combining_completions:
1816 # allow deactivation of these on windows.
1816 # allow deactivation of these on windows.
1817 base_text = text if not line_buffer else line_buffer[:cursor_pos]
1817 base_text = text if not line_buffer else line_buffer[:cursor_pos]
1818 latex_text, latex_matches = self.latex_matches(base_text)
1818 latex_text, latex_matches = self.latex_matches(base_text)
1819 if latex_matches:
1819 if latex_matches:
1820 return latex_text, latex_matches, ['latex_matches']*len(latex_matches), ()
1820 return latex_text, latex_matches, ['latex_matches']*len(latex_matches), ()
1821 name_text = ''
1821 name_text = ''
1822 name_matches = []
1822 name_matches = []
1823 for meth in (self.unicode_name_matches, back_latex_name_matches, back_unicode_name_matches):
1823 for meth in (self.unicode_name_matches, back_latex_name_matches, back_unicode_name_matches):
1824 name_text, name_matches = meth(base_text)
1824 name_text, name_matches = meth(base_text)
1825 if name_text:
1825 if name_text:
1826 return name_text, name_matches, [meth.__qualname__]*len(name_matches), {}
1826 return name_text, name_matches, [meth.__qualname__]*len(name_matches), {}
1827
1827
1828
1828
1829 # If no line buffer is given, assume the input text is all there was
1829 # If no line buffer is given, assume the input text is all there was
1830 if line_buffer is None:
1830 if line_buffer is None:
1831 line_buffer = text
1831 line_buffer = text
1832
1832
1833 self.line_buffer = line_buffer
1833 self.line_buffer = line_buffer
1834 self.text_until_cursor = self.line_buffer[:cursor_pos]
1834 self.text_until_cursor = self.line_buffer[:cursor_pos]
1835
1835
1836 # Start with a clean slate of completions
1836 # Start with a clean slate of completions
1837 matches = []
1837 matches = []
1838 custom_res = self.dispatch_custom_completer(text)
1838 custom_res = self.dispatch_custom_completer(text)
1839 # FIXME: we should extend our api to return a dict with completions for
1839 # FIXME: we should extend our api to return a dict with completions for
1840 # different types of objects. The rlcomplete() method could then
1840 # different types of objects. The rlcomplete() method could then
1841 # simply collapse the dict into a list for readline, but we'd have
1841 # simply collapse the dict into a list for readline, but we'd have
1842 # richer completion semantics in other evironments.
1842 # richer completion semantics in other evironments.
1843 completions = ()
1843 completions = ()
1844 if self.use_jedi and return_jedi_results:
1844 if self.use_jedi and return_jedi_results:
1845 if not full_text:
1845 if not full_text:
1846 full_text = line_buffer
1846 full_text = line_buffer
1847 completions = self._jedi_matches(
1847 completions = self._jedi_matches(
1848 cursor_pos, cursor_line, full_text)
1848 cursor_pos, cursor_line, full_text)
1849 if custom_res is not None:
1849 if custom_res is not None:
1850 # did custom completers produce something?
1850 # did custom completers produce something?
1851 matches = [(m, 'custom') for m in custom_res]
1851 matches = [(m, 'custom') for m in custom_res]
1852 else:
1852 else:
1853 # Extend the list of completions with the results of each
1853 # Extend the list of completions with the results of each
1854 # matcher, so we return results to the user from all
1854 # matcher, so we return results to the user from all
1855 # namespaces.
1855 # namespaces.
1856 if self.merge_completions:
1856 if self.merge_completions:
1857 matches = []
1857 matches = []
1858 for matcher in self.matchers:
1858 for matcher in self.matchers:
1859 try:
1859 try:
1860 matches.extend([(m, matcher.__qualname__)
1860 matches.extend([(m, matcher.__qualname__)
1861 for m in matcher(text)])
1861 for m in matcher(text)])
1862 except:
1862 except:
1863 # Show the ugly traceback if the matcher causes an
1863 # Show the ugly traceback if the matcher causes an
1864 # exception, but do NOT crash the kernel!
1864 # exception, but do NOT crash the kernel!
1865 sys.excepthook(*sys.exc_info())
1865 sys.excepthook(*sys.exc_info())
1866 else:
1866 else:
1867 for matcher in self.matchers:
1867 for matcher in self.matchers:
1868 matches = [(m, matcher.__qualname__)
1868 matches = [(m, matcher.__qualname__)
1869 for m in matcher(text)]
1869 for m in matcher(text)]
1870 if matches:
1870 if matches:
1871 break
1871 break
1872 seen = set()
1872 seen = set()
1873 filtered_matches = set()
1873 filtered_matches = set()
1874 for m in matches:
1874 for m in matches:
1875 t, c = m
1875 t, c = m
1876 if t not in seen:
1876 if t not in seen:
1877 filtered_matches.add(m)
1877 filtered_matches.add(m)
1878 seen.add(t)
1878 seen.add(t)
1879
1879
1880 filtered_matches = sorted(
1880 filtered_matches = sorted(
1881 set(filtered_matches), key=lambda x: completions_sorting_key(x[0]))
1881 set(filtered_matches), key=lambda x: completions_sorting_key(x[0]))
1882
1882
1883 matches = [m[0] for m in filtered_matches]
1883 matches = [m[0] for m in filtered_matches]
1884 origins = [m[1] for m in filtered_matches]
1884 origins = [m[1] for m in filtered_matches]
1885
1885
1886 self.matches = matches
1886 self.matches = matches
1887
1887
1888 return text, matches, origins, completions
1888 return text, matches, origins, completions
General Comments 0
You need to be logged in to leave comments. Login now