Better error message when eq has unexpected signature #6106

David-Wobrock · 2018-12-26T19:42:00Z

Hey there,

This PR tackles an old issue #2055 opened in 2016 - to get started in the mypy source code.

This PR will add a message and an example when the arguments of the dunder method __eq__ do not match the expected signature. With this PR, the added message looks like this:

It is recommended for "__eq__" to work with arbitrary objects, for example:
    def __eq__(self, other: object) -> bool:
        if not isinstance(other, {class_name}):
            raise NotImplementedError
        return <logic to compare two {class_name} instances>

The 3 commits can and should be reviewed individually since they represent independent bulks of work:

Fix a docstring of errors.render_messages
Add an option so that reporting an error/warning/note eventually doesn't strip the message. This was necessary because the new message needs to preserve indentation.
The core of the work: add the message when one of these methods is used and add a test

Looking forward for some reviews
Cheers,
David

Fixes #2055

The result tuple type was missing a field

and __ne__ dunder methods signatures. When the 'other' argument is not an 'object' type, we show a better error message with an example code. * And add a test

ilevkivskyi

Thank you for the PR! Here I have some comments and suggestions.

ilevkivskyi · 2018-12-27T13:23:38Z

mypy/messages.py

                               origin_line=origin.get_line() if origin else None)

    def fail(self, msg: str, context: Optional[Context], file: Optional[str] = None,
-             origin: Optional[Context] = None) -> None:
+             origin: Optional[Context] = None, strip_msg: bool = True) -> None:


I would add the new strip_msg argument only where it is actually used, i.e. in notes. It is unlikely that we will have multiline errors or warnings.

ilevkivskyi · 2018-12-27T13:24:00Z

mypy/messages.py


    def warn(self, msg: str, context: Context, file: Optional[str] = None,
-             origin: Optional[Context] = None) -> None:
+             origin: Optional[Context] = None, strip_msg: bool = True) -> None:


ilevkivskyi · 2018-12-27T13:26:47Z

mypy/messages.py

+
+    def note_multiline(self, messages: str, context: Context, file: Optional[str] = None,
+             origin: Optional[Context] = None, offset: int = 0,
+             strip_msg: bool = True) -> None:


I think strip_msg is not needed for note_multiline. It is quite clear that multiline notes will be often indented.

ilevkivskyi · 2018-12-27T13:28:09Z

mypy/messages.py

            self.errors.report(context.get_line() if context else -1,
                               context.get_column() if context else -1,
-                               msg.strip(), severity=severity, file=file, offset=offset,


After some thinking I am not sure why do we actually need to strip the message here. How many tests fail if you remove the .strip() here? Maybe we can instead update our test framework?

ilevkivskyi · 2018-12-27T13:29:20Z

mypy/messages.py

@@ -860,6 +871,21 @@ def argument_incompatible_with_supertype(
        self.fail('Argument {} of "{}" incompatible with {}'
                  .format(arg_num, name, target), context)

+        if name in ("__eq__", "__ne__"):


I think this note should be applied only for __eq__.

ilevkivskyi · 2018-12-27T13:31:21Z

mypy/messages.py

+        return '''It is recommended for "{method_name}" to work with arbitrary objects.
+The snippet below shows an example of how you can implement "{method_name}":
+
+class Foo(...):


Can we use the actual class name instead here?

ilevkivskyi · 2018-12-27T13:31:40Z

mypy/messages.py

+class Foo(...):
+    ...
+    def {method_name}(self, other: object) -> bool:
+        if not isinstance(other, Foo):


...and here

ilevkivskyi · 2018-12-27T13:33:36Z

test-data/unit/check-classes.test

+main:2: error: Argument 1 of "__eq__" incompatible with supertype "object"
+main:2: note: It is recommended for "__eq__" to work with arbitrary objects.
+main:2: note: The snippet below shows an example of how you can implement "__eq__":
+main:2: note:


This "preamble" is too wordy. I would use shorter It is recommended for "__eq__" to work with arbitrary objects, for example: and skip the empty line.

ilevkivskyi · 2018-12-27T13:34:23Z

test-data/unit/check-classes.test

+main:2: note: The snippet below shows an example of how you can implement "__eq__":
+main:2: note:
+main:2: note: class Foo(...):
+main:2: note:     ...


I would actually skip these two lines (however I would keep the indentation of the next lines).

ilevkivskyi · 2018-12-27T13:36:44Z

mypy/messages.py

+    def {method_name}(self, other: object) -> bool:
+        if not isinstance(other, Foo):
+            raise NotImplementedError
+        return <logic to compare two Foo instances>'''.format(method_name=method_name)


And here I would also use the actual class name instead of Foo.

ilevkivskyi · 2018-12-27T13:39:25Z

(I also edited the PR description to add the "magic" text for GitHub.)

* Delete outdated comment

* Use textwrap.dedent to have a nicer multiline string * Only show the note for __eq__ * Use the actual class name in the shown code snippet * Adapt the test

David-Wobrock · 2018-12-28T13:30:09Z

Hey @ilevkivskyi, thanks for the comments.

The PR is open for review again and I took your changes into account in the last 2 commits:

Completely removing the strip() call on the message reporting works like charm. No test broke. So my best guess is to go without it.
Changing the displayed messaged

ilevkivskyi

Thanks for updates! This is now almost ready, I just have three small comments.

ilevkivskyi · 2018-12-30T15:40:55Z

mypy/messages.py

+                    offset=offset)
+
+    def note_multiline(self, messages: str, context: Context, file: Optional[str] = None,
+             origin: Optional[Context] = None, offset: int = 0) -> None:


Please use the same style as everywhere else: indent for argument coincides with the start of first argument.

ilevkivskyi · 2018-12-30T15:57:49Z

test-data/unit/check-classes.test

+main:2: note: It is recommended for "__eq__" to work with arbitrary objects, for example:
+main:2: note:     def __eq__(self, other: object) -> bool:
+main:2: note:         if not isinstance(other, A):
+main:2: note:             raise NotImplementedError


I think return NotImplemented is more idiomatic, but I don't think this should delay this PR if there are other opinions.

Yes, NotImplemented is right here. It signals to the runtime to try the other object's __eq__.

ilevkivskyi · 2018-12-30T16:02:44Z

mypy/messages.py

@@ -860,6 +869,20 @@ def argument_incompatible_with_supertype(
        self.fail('Argument {} of "{}" incompatible with {}'
                  .format(arg_num, name, target), context)

+        if name == "__eq__":
+            assert isinstance(context, FuncDef)
+            multiline_msg = self.comparison_method_example_msg(class_name=context.info.name())


Getting anything except line/column from context is a bad idea. First, this kind of defeats the purpose of the type annotation. Second, what if this is an OverloadedFuncDef, or a Decorator?

I think it is better to add an optional argument current_type_name to argument_incompatible_with_supertype, and if name == '__eq__': assert current_type_name is not None etc. (Or maybe even make it a required argument if there are only couple of call sites to update).

Instead of inspecting the context, we directly pass the type name to the messages methods as argument.

David-Wobrock · 2018-12-31T17:37:56Z

Hi @ilevkivskyi, thanks again for the feedback, and thank you @JelleZijlstra.

Again, the last 3 commits contain the suggested changes.
The messages method argument_incompatible_with_supertype is only called in one place, so that is easy to update. But to get the class/type name, I fall back on a hard-coded Foo string when I don't have access to the function definition.
I don't know the codebase enough to know in what cases this can happen - and if it makes sense in mypy to fallback on such a class name for the note. Do you have a cleaner suggestion for this?

Cheers and happy new year!
David

ilevkivskyi · 2019-01-03T12:41:08Z

mypy/checker.py

+                if isinstance(override.definition, FuncDef):
+                    type_name = override.definition.info.name()
+                else:
+                    type_name = "Foo"


I think a better alternative is to not show the note at all if __eq__ is not a function definition. So you can just set type_name to None here, and add a check in argument_incompatible_with_supertype() that will not show the note if the name is None.

ilevkivskyi

Thanks, looks ready now!

Type added in a way recommended at python/mypy#6106

David-Wobrock added 3 commits December 26, 2018 12:57

Fix docstring of errors.render_messages

cf81474

The result tuple type was missing a field

Make it optional to strip reported message

80bc5eb

Improve error message for unexpected __eq__

d524a45

and __ne__ dunder methods signatures. When the 'other' argument is not an 'object' type, we show a better error message with an example code. * And add a test

ilevkivskyi reviewed Dec 27, 2018

View reviewed changes

David-Wobrock added 2 commits December 28, 2018 13:22

Completely remove striping the message before reporting it

16178b2

* Delete outdated comment

Adapt __eq__ note:

a630c67

* Use textwrap.dedent to have a nicer multiline string * Only show the note for __eq__ * Use the actual class name in the shown code snippet * Adapt the test

David-Wobrock force-pushed the master branch from 7ee376e to a630c67 Compare December 28, 2018 13:14

ilevkivskyi reviewed Dec 30, 2018

View reviewed changes

David-Wobrock added 3 commits December 30, 2018 22:33

Use return NotImplemented instead of raise NotImplError

a09fdc4

Start argument on same indent as first arg

15f1d06

Pass the type name to the messages method

9854898

Instead of inspecting the context, we directly pass the type name to the messages methods as argument.

David-Wobrock force-pushed the master branch from d247fde to 9854898 Compare December 31, 2018 17:29

ilevkivskyi reviewed Jan 3, 2019

View reviewed changes

David-Wobrock force-pushed the master branch from 4ba86f6 to b087cdb Compare January 3, 2019 19:07

Do not display __eq__ note when not function definition

6fc602e

David-Wobrock force-pushed the master branch from b087cdb to 6fc602e Compare January 3, 2019 19:10

ilevkivskyi approved these changes Jan 3, 2019

View reviewed changes

ilevkivskyi merged commit c33da74 into python:master Jan 3, 2019

HiromuHota pushed a commit to HiromuHota/fonduer that referenced this pull request Sep 10, 2019

Add types to __eq__, __ne__, __gt__ of classes that inherit object

1169b27

Type added in a way recommended at python/mypy#6106

lukehsiao pushed a commit to HazyResearch/fonduer that referenced this pull request Sep 10, 2019

Add types to __eq__, __ne__, __gt__ of classes that inherit object

996b2bc

Type added in a way recommended at python/mypy#6106

Uh oh!

Better error message when __eq__ has unexpected signature #6106

Better error message when __eq__ has unexpected signature #6106

Uh oh!

Conversation

David-Wobrock commented Dec 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi commented Dec 27, 2018

Uh oh!

David-Wobrock commented Dec 28, 2018

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

David-Wobrock commented Dec 31, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Better error message when eq has unexpected signature #6106

Better error message when eq has unexpected signature #6106

David-Wobrock commented Dec 26, 2018 •

edited

Loading