Skip to content

Is it possible to create Protocols for Django Models #651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
GlennS opened this issue Jun 23, 2021 · 11 comments
Open

Is it possible to create Protocols for Django Models #651

GlennS opened this issue Jun 23, 2021 · 11 comments
Labels
bug Something isn't working upstream mypy Bugs in mypy

Comments

@GlennS
Copy link

GlennS commented Jun 23, 2021

My aim is to create a Protocol which could match either a dataclass (ideally a frozen dataclass) or a Django model (I only really care about the getter).

Does this seem like a plausible thing to expect to work? If so could you point me in the right direction?

The main issue is that the various types django-stubs defines to match Django fields like IntegerField and CharField aren't compatible with int, str, and so on when you use those in a protocol.

Obviously this does work when you use get and set from Django model fields in normal code, so I'm wondering where the magic lives which makes that happen (makes CharField compatible with str)?

Sample error 1 (caused by str not being the same as CharField):

expression has type "CharField[, ]", base class "P" defined the type as "str"

Sample error 2 (caused by Django model inheriting from a protocol which defines its field as a basic type):

Cannot determine type of 'x'

Approaches I've tried (these are probably a little bit mad):

  1. Define the protocol field as an int field. Django model won't accept it as a protocol.
  2. Defining a protocol for the field with some __get__ function overrides. If I fairly carefully copy parts of the Field definition https://github.com/typeddjango/django-stubs/blob/master/django-stubs/db/models/fields/__init__.pyi#L45 then I can make it partially work, but then I can't use my newly defined field protocol as if it were a str elsewhere in the code.
  3. Defining the protocol field as a Union[CharField, str]. I then have to tell the Django model which one it is, and I can't use it the field elsewhere in code.

Here's some code for just approach (1):

import dataclasses
from typing_extensions import Protocol

from django.db import models


# Naive attempt at defining protocol.
class P(Protocol):
    x: str


# Django doesn't like our protocol.
class M(models.Model, P):
    x = models.CharField(null=False)


# Dataclass is happy with our protocol.
@dataclasses.dataclass(frozen=True)
class D(P):
    x: str


# Defining some functions that act on everything
def f_p(p: P) -> str:
    return p.x


def f_m(m: M) -> str:
    # This function is now confused about its return type.
    # (The model inheriting from the protocol causes this error.)
    return m.x


def f_d(d: D) -> str:
    return d.x


# Instantiating everything doesn't add any extra errors.
m = M(x="a")
d = D(x="a")


# Call the functions
f_p(m)
f_p(d)
f_d(d)
f_m(m)
@sobolevn
Copy link
Member

sobolevn commented Jun 23, 2021

You don't have to inherit from a protocol.
I am using protocols in several projects together with django-stubs and it works fine.

Just define them and use where you expect a matching structure.

@GlennS
Copy link
Author

GlennS commented Jun 23, 2021

@sobolevn that sounds like a perfect solution, but I haven't been able to make it work.

Do you have an example you could show me?

@sobolevn
Copy link
Member

@GlennS I hope, that this would be helpful to understand how mypy and descriptors work: 159a0e4 This is what we use to infer model attribute types.

But, while working with this docs, I have noticed that mypy actually has a bug in find_member: https://github.com/python/mypy/blob/master/mypy/subtypes.py#L617-L618 It does not infer __get__ method type on descriptors. I will fix this right now.

@GlennS
Copy link
Author

GlennS commented Jun 23, 2021

@sobolevn thank you very much! I'll read your new docs tomorrow and let you know how I get on.

@GlennS
Copy link
Author

GlennS commented Jun 23, 2021

I think python/mypy#5481 is a much clearer description of the problem, so I'll close this one as a duplicate.

Thanks again for your help @sobolevn , I think I understand this much better now.

@GlennS GlennS closed this as completed Jun 23, 2021
@GlennS
Copy link
Author

GlennS commented Jun 23, 2021

Changed my mind when I realised that other issue is on mypy itself. It's probably useful for people to be able to search django-stub's issues for "Protocol" and see this one.

@GlennS GlennS reopened this Jun 23, 2021
@flaeppe
Copy link
Member

flaeppe commented Sep 5, 2022

I just want to complement this issue with a (kind of) minimal example:

from typing import Protocol

from django.db import models


class Person(Protcol):
    @property
    def given_name(self) -> str:
        ...
    
    @property
    def family_name(self) -> str:
        ...


class User(models.Model):
    given_name = models.CharField(max_length=255)
    family_name = models.CharField(max_length=255)


def x(person: Person) -> None:
    return None


def y() -> None:
    user = User.objects.get()
    x(user)
    return None

Which gives me the following errors:

 error: Argument 1 to "x" has incompatible type "User"; expected "Person"  [arg-type]
 note: Following member(s) of "User" have conflicts:
 note:     family_name: expected "str", got "CharField[Union[str, int, Combinable], str]"
 note:     given_name: expected "str", got "CharField[Union[str, int, Combinable], str]"

@sobolevn
Copy link
Member

sobolevn commented Sep 5, 2022

This is a mypy bug, because mypy uses find_member for protocol subtyping, while using checkmember for regular subtyping.

I don't think we can solve this quickly.

@intgr intgr added bug Something isn't working upstream mypy Bugs in mypy labels Nov 11, 2022
@wolfskaempf
Copy link

I'm running into the same issue. Does someone have insight if it has become easier to fix since 2022?

@alan-andrade
Copy link

It's 2025 and I landed here lol. Should I give up trying to capture django models properties in a protocol?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working upstream mypy Bugs in mypy
Development

No branches or pull requests

6 participants