Skip to content

Restrict dynamic attribute creation with slots #7388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jun 3, 2020
Merged

Restrict dynamic attribute creation with slots #7388

merged 14 commits into from
Jun 3, 2020

Conversation

xmunoz
Copy link
Contributor

@xmunoz xmunoz commented Nov 21, 2019

Fixes #7313.

@chrahunt
Copy link
Member

Hi @xmunoz! Thanks so much for looking at this.

I think there's probably a way to avoid manually managing the slots, instance variables, and parameters to __init__ with a library like attrs or dataclasses. I just left a comment in #7313 to that effect. If you're interested please take a look! One important factor for us will be how straightforward it is for new people to the code base to be able to understand it so if you've never used it before that's even better.

@xmunoz
Copy link
Contributor Author

xmunoz commented Nov 21, 2019

Great, thanks! Yeah, I think attrs makes a lot more sense than what I did here. I'll close this and open another PR with that approach instead. I am new to both the attrs module and pip development, but it seems pretty straight-forward. Thanks for the quick follow up :)

@xmunoz
Copy link
Contributor Author

xmunoz commented Nov 21, 2019

After some more discussion on #7313 about attrs, it seems like direct usage of __slots__ may be acceptable.

@xmunoz xmunoz reopened this Nov 21, 2019
@xmunoz
Copy link
Contributor Author

xmunoz commented Nov 21, 2019

Once the existential question about this PR is resolved, I have two follow up questions for the reviewers:

  • Would unit tests for this slots feature contribute meaningfully to test coverage?
  • Is this a "trivial" change?

@xavfernandez xavfernandez added the skip news Does not need a NEWS file entry (eg: trivial changes) label Nov 21, 2019
@BrownTruck
Copy link
Contributor

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

@BrownTruck BrownTruck added the needs rebase or merge PR has conflicts with current master label May 20, 2020
@pradyunsg
Copy link
Member

Would unit tests for this slots feature contribute meaningfully to test coverage?

Nope. :)

@xmunoz Would it be possible for you to update this PR?

@pypa-bot pypa-bot removed the needs rebase or merge PR has conflicts with current master label May 22, 2020
@xmunoz
Copy link
Contributor Author

xmunoz commented May 26, 2020

We're green!

@xmunoz
Copy link
Contributor Author

xmunoz commented May 26, 2020

I opened this pull request months ago, honestly am not too sure why I did this the way that I did. That said, I rolled back the Base class stuff locally and ran tests locally, which failed. I won't have too much time to dedicate to this in the near term, but given the concern I'll go ahead and push my local changes and leave the failures for someone else to debug.

@xmunoz
Copy link
Contributor Author

xmunoz commented May 29, 2020

Scrap my last comment, looks like everything is green :)

attr_getters = [operator.attrgetter(attr)
for attr in self.__slots__]
return all(getter(self) == getter(other)
for getter in attr_getters)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Equality on list could have ordering issues; we probably need to cast the result; something like

self_attrs = {k: getattr(self, k) for k in self.__slots__}
other_attrs = {k: getattr(other, k) for k in other.__slots__}
return self_attrs == other_attrs

Copy link
Member

@pradyunsg pradyunsg May 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're protected by a self.__slots__ == other.__slots__, which is reasonable with self.__class__ IMO.

Your suggestion is more robust tho.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're protected by a self.__slots__ == other.__slots__, which is reasonable with self.__class__ IMO.

Subclassing may result in self.__class__ != other.__class__ (therefore not guarenteeing the __slots__ declarations match), which is the main issue I have with the current implementation.

Copy link
Member

@pradyunsg pradyunsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me! I'm fairly sure we can iterate on this in a follow up, and I'm biased toward merge everything-we-agree-on-and-iterate-on-the-rest. :)

return False

if self.__slots__ != other.__slots__:
return False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, this is suspect to false positives when __slots__ is redefined.

Example:

class Foo:
    __slots__ = ("a", "b")

    def __init__(self, a, b):
        self.a = a
        self.b = b

    def __eq__(self, other):
        if self.__slots__ != other.__slots__:
            return False
        return all(
            getattr(self, k) == getattr(other, k)
            for k in self.__slots__
        )

class Bar(Foo):
    __slots__ = ("b", "a")

# False!
print(Foo(1, 2) == Bar(1, 2))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we want to be a little more explicit rather than pulling out a bunch of hacks here 😄 I mean there are only 2 attributes, which can be compared manually. Subclasses extending this may define their own comparison methods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to leave this as-is for now.

Copy link
Contributor

@deveshks deveshks Jun 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if I understand correctly, the __eq__ should be updated to

def __eq__(self, other):

    self_attrs = {k: getattr(self, k) for k in self.__slots__}
    other_attrs = {k: getattr(other, k) for k in other.__slots__}
    return self_attrs == other_attrs

as per this comment and #7388 (comment)

@pradyunsg
Copy link
Member

@uranusjr are you OK to iterate on the equality function in a follow up PR? I'd like to get this PR merged since it's good enough IMO.

@uranusjr
Copy link
Member

uranusjr commented Jun 3, 2020

I am fine with merging this to master as long as the tests are able to pass.

@pradyunsg pradyunsg merged commit cdeb377 into pypa:master Jun 3, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
skip news Does not need a NEWS file entry (eg: trivial changes)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add "no-new-attributes" behavior to our models
9 participants