Skip to content

Unexpected conditional expression type inference change #7986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JukkaL opened this issue Nov 20, 2019 · 6 comments · Fixed by #7991
Closed

Unexpected conditional expression type inference change #7986

JukkaL opened this issue Nov 20, 2019 · 6 comments · Fixed by #7991

Comments

@JukkaL
Copy link
Collaborator

JukkaL commented Nov 20, 2019

#7955 caused this example to start generating type errors:

from typing import List, Union

def g(a: List[str], b: bool) -> bool:
    x = a and b
    y: bool
    return x or y  # Incompatible return value type (got "Union[List[str], bool]", expected "bool")

Previously, this generated no errors. It's unclear whether this fixed a false negative or whether this is a regression.

@Michael0x2a Do you have an idea of what might be going on?

@ilevkivskyi
Copy link
Member

I would say this is rather a bug (I am however not sure whether it previously worked on purpose or by coincidence).

I think the inferred type of x should be like UnionType(Instance('list', can_be_true=False), Instance('bool')), and then the or in return statement should eliminate the false-only items from union.

@JukkaL
Copy link
Collaborator Author

JukkaL commented Nov 20, 2019

Just to be clear: the new behavior is a bug, or the old behavior is a bug?

@Michael0x2a
Copy link
Collaborator

The only thing I can think of is the changes to LastKnownValueEraser.visit_instance: now that we try erasing types if there are any args, that visitor will end up calling Instance.copy_modified more frequently -- and it doesn't seem that copy method actually preserves the can_be_true or can_be_false fields.

I'm not 100% sure, but I feel the new behavior is probably a bug? I tried playing around with the example function, and I could never get it to return a list, so a inferred return type of bool is probably want we want.

But maybe the deeper bug is that there's a discrepancy between how we handle last_known_value and can_be_true and can_be_false? It seems we want to erase the latter less aggressively -- so this would probably be yet another binder/"it would be nice if we had a more sophisticated abstract interpreter" issue?

@ilevkivskyi
Copy link
Member

Just to be clear: the new behavior is a bug, or the old behavior is a bug?

IMO the new behaviour is a false positive.

@ilevkivskyi
Copy link
Member

@Michael0x2a

so this would probably be yet another binder/"it would be nice if we had a more sophisticated abstract interpreter" issue?

I think there is a chance a pretty simple fix exists. Will you have time to look into preserving can_be_true and can_be_false in Instance.copy_modified()? If not, I can look into this tomorrow.

@Michael0x2a
Copy link
Collaborator

Sure, I can take a look.

Michael0x2a added a commit to Michael0x2a/mypy that referenced this issue Nov 21, 2019
This pull request modifies all of our `copy_modified` type methods
so they always copy over the previously inferred values of the
`can_be_true` and `can_be_false` fields.

I'm not entirely convinced copying over these fields is fully
correct: we can sometimes end up accidentally inferring the wrong
result when we try reassigning variables or calling mutating
functions. For example, mypy incorrectly infers 'bool' in the
following two variants of the example in python#7986
instead of `Union[List[str], bool]`:

    def bad1(a: List[str], b: bool) -> bool:
        x = a and b
        a = ["foo"]
        y: bool
        return x or y   # Should be an error

    def bad2(a: List[str], b: bool) -> bool:
        x = a and b
        a.append("foo")
        y: bool
        return x or y   # Should be an error

But we always copy over these two fields for non-generic Instances,
I think it would be consistent to just always do so, for all types.

Fixes python#7986, probably.
Michael0x2a added a commit to Michael0x2a/mypy that referenced this issue Nov 21, 2019
This pull request modifies the `last_known_value` eraser so it
always copies over the previously inferred value of the
`can_be_true` and `can_be_false` fields.

This isn't entirely typesafe -- for example, we can sometimes
accidentally infer the wrong result if we try reassigning variables
or call a mutating function :

    def bad1(a: List[str], b: bool) -> bool:
        x = a and b
        a = ["foo"]
        y: bool
        return x or y   # Should be an error

    def bad2(a: List[str], b: bool) -> bool:
        x = a and b
        a.append("foo")
        y: bool
        return x or y   # Should be an error

But this was apparently the old behavior/something mypy wasn't
previously able to detect, so I guess this is fine for now.

I also decided against modifying every `copy_modified` method in
all the type classes because of this reason: I wanted to limit
the spread of this potentially misleading additional inferred info.

Fixes python#7986, probably.
JukkaL pushed a commit that referenced this issue Nov 21, 2019
This pull request modifies the `last_known_value` eraser so it
always copies over the previously inferred value of the
`can_be_true` and `can_be_false` fields.

This isn't entirely typesafe -- for example, we can sometimes
accidentally infer the wrong result if we try reassigning variables
or call a mutating function :

```
    def bad1(a: List[str], b: bool) -> bool:
        x = a and b
        a = ["foo"]
        y: bool
        return x or y   # Should be an error

    def bad2(a: List[str], b: bool) -> bool:
        x = a and b
        a.append("foo")
        y: bool
        return x or y   # Should be an error
```

But this was apparently the old behavior/something mypy wasn't
previously able to detect, so I guess this is fine for now.

I also decided against modifying every `copy_modified` method in
all the type classes because of this reason: I wanted to limit
the spread of this potentially misleading additional inferred info.

Fixes #7986, probably.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants