IPCompleter doesn't work inside tuples/arrays #14585

divyansshhh · 2024-11-21T05:48:45Z

Consider the following snippet -

y = "abc"
x = (1, y.<tab>

This doesn't return any autocomplete suggestions. This is fixed by enabling jedi but can we look into fixing this for the IPCompleter?

This is reproducible on both ipython v8.22.0 and v8.28.0

The text was updated successfully, but these errors were encountered:

krassowski · 2024-11-21T14:50:22Z

The case of unclosed parenthesis is already handled, but this case is not. Briefly the relevant code is:

ipython/IPython/core/completer.py

Lines 1144 to 1150 in d7eebae

    
           def _attr_matches(self, text, include_prefix=True) -> Tuple[Sequence[str], str]: 
        
               m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer) 
        
               if not m2: 
        
                   return [], "" 
        
               expr, attr = m2.group(1, 2) 
        
               obj = self._evaluate_expr(expr)

and then in evaluation we use naive approach of trimming on the left side if there is a syntax error - unless there is not.

ipython/IPython/core/completer.py

Lines 1221 to 1228 in d7eebae

    
           except Exception as e: 
        
               if self.debug: 
        
                   print("Evaluation exception", e) 
        
               # trim the expression to remove any invalid prefix 
        
               # e.g. user starts `(d[`, so we get `expr = '(d'`, 
        
               # where parenthesis is not closed. 
        
               # TODO: make this faster by reusing parts of the computation? 
        
               expr = expr[1:]

I think that the problem might be that 1, y is a valid Python expression which evaluates to a tuple (assuming that y is defined).

Partial parsing of Python is not easy in the general case. We could possibly consider some heuristics for closing parenthesis for tuples and lists and after guarded evaluation extract the relevant part.

To spell it out we cannot just split on space as in general case it could be x = (1, a[" "].y.

krassowski · 2024-11-21T14:54:29Z

Indeed, <tab> returns results corresponding to methods available on a tuple:

krassowski · 2024-11-21T14:58:52Z

Some more related cases:

x = (1, a[" "].y.
x = (")", y.
x = [1, y.
x = {1, y.

x = (
   1, y.

Carreau · 2024-11-25T13:56:57Z

I'm wondering, finding completions using jedi is a problem, (if you set use_jedi = False, but maybe we an use parso (or even jedi itself but not as a completer), to get just the expression we are trying to complete.

mlucool · 2024-11-27T01:41:53Z

FWIW we turned off jedi because it had historically been too slow, so using Jedi as a solution is unlikely to be ideal (unless its much faster than it used to be)

mlucool · 2024-11-27T15:33:25Z

The following is also a fix

ip = get_ipython()
ip.Completer.python_matcher = ip.Completer.python_matches

Carreau · 2024-11-28T07:44:36Z

FWIW we turned off jedi because it had historically been too slow, so using Jedi as a solution is unlikely to be ideal (unless its much faster than it used to be)

I think it's computing the completions that is slow with jedi – because of all the inference and need to access libraries code; Jedi (and its dependency, parso), also offer the ability to simply parse incomplete Python code – which should be much faster – I'm suggesting using only the parsing to extract the current expression the cursor is in.

@krassowski pointed out that a disadvantage is jedi/parso need a new release on every new Python version because of Python syntax changes so that may be a better reason to not use it.

krassowski · 2024-11-29T10:28:32Z

I think restoring the previous behaviour by adding back the attempt to evaluate:

# Another option, seems to work great. Catches things like ''.<tab>
m = re.match(r"(\S+(\.\w+)*)\.(\w*)$", text)

prior to:

m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)

would be a sensible solution improving the current situation. In the edge cases like x = (1, a[" "].y.<tab> it does not matter as it does not work right now either. However, we should be careful not to brake a[" "].y.<tab>, so maybe it needs two checks? Definitely it needs more tests.

In progress work toward #14585 guarded eval strip leading characters until it find soemthing, this is problematic as `(1, x`, becomes valid after 1 char strip: `1, x` is a tuple; So now we trim until it is valid an not a tuple. This is still imperfect as things like `(1, a[" "].y` will be trimmed to `y`, while it should stop with `a[" "].y` ? I think maybe we should back-propagate; build back up from `y`, to `a[" "].y`, greedily until we get the last valid expression – skipping any unbalanced parentheses/quotes if we encounter imblanced.

krassowski added the tab-completion label Nov 21, 2024

Carreau self-assigned this Nov 25, 2024

Carreau mentioned this issue Dec 2, 2024

Fix completion tuple #14594

Merged

Carreau closed this as completed Dec 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IPCompleter doesn't work inside tuples/arrays #14585

IPCompleter doesn't work inside tuples/arrays #14585

IPCompleter doesn't work inside tuples/arrays #14585

IPCompleter doesn't work inside tuples/arrays #14585

Comments