Removing behaviour with OrderedIntersection and Protocols

[alicederyn]

Is there a way to remove behaviour from a type with OrderedIntersection and a Protocol, while having the resulting type still compatible with a type that supports that behaviour?

More concretely, suppose I was trying to replicate PEP 705 for list and create a type like list[ReadOnly[T]], let's call it ReadOnlyList[T]. This should have the following properties:

def foo(param: ReadOnlyList[Sequence[int]]) -> None:
  foo.append([4])  # Type check error: append is not accessible via ReadOnlyList
  foo.clear()  # Don't worry about making this illegal for now

bar: list[tuple[int]] = [(), (3,), (4,5)]
foo(bar)  # OK: tuple[int] is a subtype of Sequence[int]

Could I do that? Maybe with something like this?

class NoListMutatorMethods:
  def append(x: typing.Never) -> None: ...
  def extend(iterable: typing.Never) -> None: ...
  # etc
  
type ReadOnlyList[T] = OrderedIntersection[list[T], NoListMutatorMethods]

I feel like the answer is probably "no, there's no way to spell this using OrderedIntersection and Protocol", since you can't remove methods in a subclass/subtype. However, some of the conversation in #42 led me to wonder if I misunderstood the construct.

So I guess a prerequisite question is: Should both (or either) of these always be valid for any A and B where OrderedIntersection[A, B] is valid:

def as_first_type(x: OrderedIntersection[A, B]) -> A:
  return x

def as_second_type(x: OrderedIntersection[A, B]) -> B:
  return x

(I have not thoroughly read all the conversations up to this point, my apologies if this is already in there or is obvious to someone who understands MRO more than me :( )

[mikeshardmind]

Is there a way to remove behaviour from a type with OrderedIntersection and a Protocol, while having the resulting type still compatible with a type that supports that behaviour?

It depends on what you mean by being compatible. You can specify such an intersection, and then in the scope where that intersection is what is known, limit the behavior of things done to a value consistent with it, but it's still a requirement that the actual type is consistent with all operands, so this is really only something that helps detect issues statically. I suppose that's actually okay for things like replicating ReadOnly but it wasn't a goal to create that capability, just a side effect of using consistent, easy to implement rules that interact well with all of the goals we had and helped resolve an issue of ambiguity with gradual types.

class PreventsListMutationViaMethods(Protocol):
    def append(x: typing.Never, /) -> Never: ...
    # etc

def example(x: OrderedIntersection[PreventsListMutationViaMethods, list[int]]):
    x.append(1)  # This would be a type error, 1 isn't consistent with Never (from PreventsListMutationViaMethods.append) in this context

x: OrderedIntersection[PreventsListMutationViaMethods, list[int]] = [1, 2, 3]  # this is fine
example(x)  # this is fine
example([1,2,3])  # this is fine too

# however

def why_not_actually_read_only(x: list[int]):
    x.append(1)

why_not_actually_read_only(x)  # this is also fine
y: list[int] = x  # This is also fine

you somewhat got to that being the case with:

def as_first_type(x: OrderedIntersection[A, B]) -> A:
  return x

def as_second_type(x: OrderedIntersection[A, B]) -> B:
  return x

So it's not as strict as ReadOnly, not a full replacement for it, and such a protocol hiding the mutable capabilities only applies to the specific scope where that is what you are treating it as the intersection. Overall consistency and subtyping rules still require you being able to treat it as either operand individually if you want to.

If you just want to know if it's possible, and not the why, you can skip the below, but I've included it based on the rest of the post

This particular comment and the two after it may help clarify a bit: #41 (comment)

Importantly, OrderedIntersection in an annotation context does not ever directly equate to a runtime type, but to a specification of specific qualities of one or more runtime types that values need to conform to, this distinction prevents a few issues with the interaction with gradual types, but that's a very long discussion that was had over the course of months, and it would take a while to rehash it. A summary of why this was necessary will be included in the PEP draft.

(Without rehashing all of it, you can explore the difference between what you are allowed to do with class MyList(Any, list[int]): pass and class MyList(list[int], Any): pass to get a sense for why an order wasn't just expedient but necessary to differing desired behavior)

Additional definitions will be proposed as part of this to help discuss this and other cases where the distinction between typing special forms in an annotation context and concrete types in an annotation context carry additional meaning due to them only being "type-like" at runtime due to implementation, while being intended to describe things at a level of description above types in the type system.

It wasn't strictly a goal of adding ordering to create the possibility to place a more restrictive bound on usage, but the possibility was observed to follow from the rules, and seeing as it was easy to find real use cases which could benefit from leaving this capability in place, I cannot see myself attempting to find yet another construction that fits all of our needs, but removes this capability, nor one that makes the capability more powerful and a true replacement for something like ReadOnly

If you have any other questions on this, I'd love to hear them as anything unclear about this will absolutely need to be clearer by the time of proposal.

[alicederyn]

Interesting. So you end up with a type that superficially seems like it has removed some operations, but does not prevent you accidentally "reenabling" them by calling a function that uses the unrestricted parent type.

I'm curious what use cases were found for this.

[mikeshardmind]

I'm curious what use cases were found for this.

copied from the other thread:

__all__ = ["X",]

class _X:
    def foo(self, _private_cache: dict[str, int] = {}) -> int:
        """
        The private cache being implemented this way prevents the issues of lru_cache on methods...
        """
        ...

class ProtoFoo:
    def foo(self) -> int:
        ...

X: type[OrderedIntersection[ProtoFoo, _X]] = _X

This is better than a library lying about the interface in a typestub instead, as subclassing of the provided type will be detected at an issue if it can no longer be used as _X, but the public type exports the supported interface by users. This one won't have a situation where you accidentally go to having _X, as _X wasn't exported to users, they have to intentionally use something private, and it should be meaningfully obvious to them that this isn't supported by the library.

It's not the most useful extra behavior, but it's a neat side effect of what we otherwise need.