Match.group can return None

[rmorshea]

All usages of Match.group appear to return strings however, for optional named groups it can return None:

import re
assert re.match("(?P<numbers>[1-9]+)?", "abc").group("numbers") is None

[srittau]

There is a note in the stubs:

typeshed/stdlib/3/typing.pyi

Line 549 in 23e380a

# TODO: The return for a group may be None, except if __group is 0 or not given.

This needs to be amended about it being potentially None in case of strings. Overloads and literals should be able to narrow this down. PR welcome.

[djb4ai]

@rmorshea Are you working on it? I can submit a patch if you want

[djb4ai]

@srittau When I am trying to replicate it since return type of re.match is Optional[Match[AnyStr]], mypy complains about calling .group on None is there any way to handle this scenario?

[srittau]

@lladhibhutall This should use overloads, e.g. the return type should depend on whether an argument is given, it is 0, another int or a str.

[hauntsaninja]

Note it can return None for (non-zero) numbered groups as well: assert re.match("(x)?y", "y").group(1) is None. This change, while correct, might hurt ergonomics in the happy case, pretty much all uses of numbered groups I've seen assume if there's a match their groups will be not None.

[rmorshea]

IMO having a false positive (i.e. the return type is Optional[str] for required match groups) is better than a false negative, but if the maintainers want to keep as-is to avoid inconvenience I'd understand that as well.

[hauntsaninja]

typeshed generally prefers false negatives over false positives (https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#what-to-include), but maybe Sebastian has an opinion on whether to make Optional the return type of zero, one or both of named and (non-zero) numbered groups.

[hauntsaninja]

It looks like the change was made, and then reverted in #3190

[rmorshea]

Ok, seems like false-positive caused problems. Thanks @hauntsaninja.

[hauntsaninja]

There's also some hope a plugin could improve results here, e.g.: python/mypy#7803

[mr-c]

This is quite needed for mypyc users. Can this issue be re-opened?