False positive unsubscriptable-object

[DarthLegiON]

Bug description

Hi! I have weird behavior in my code.

Here is a code example:

from sqlalchemy import ForeignKey
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.sql.sqltypes import String, Integer


class RoleReaction(Base):
    __tablename__ = 'user_roles_reactions'

    id: Mapped[int] = mapped_column(Integer, primary_key=True)
    role_name: Mapped[str] = mapped_column(String)
    reaction_name: Mapped[str] = mapped_column(String)
    group_id: Mapped[int] = mapped_column(ForeignKey('user_roles_groups.id'))
    group: Mapped[RoleGroup] = relationship(RoleGroup)

It causes several unsubscriptable-object errors in pylint (look into the output section).

But when I remove the last line (group definition), all of them disappear. When I move it upper, they don't. There is only one file in the whole project that causes such problem.

Configuration

[MASTER]
disable=
    C0114, # missing-module-docstring
    C0115, # missing-class-docstring
    C0116, # missing-function-docstring
    R0903, # too-few-public-methods
    R0801, # similar lines

extension-pkg-allow-list=
    dependency_injector,
    discord

max-line-length=120

Command used

PYTHONPATH=./src pylint --recursive=y --rcfile=.pylintrc ./src/roles_reactions/db.py

Pylint output

************* Module src.roles_reactions.db
src/roles_reactions/db.py:19:8: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/roles_reactions/db.py:20:11: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/roles_reactions/db.py:21:15: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/roles_reactions/db.py:22:19: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/roles_reactions/db.py:23:14: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 9.58/10, -9.58)

Expected behavior

No errors, 10/10

Pylint version

pylint 3.1.0
astroid 3.1.0
Python 3.10.5 (main, Aug  2 2022, 10:31:34) [GCC 10.2.1 20210110]

OS / Environment

Linux, docker

Additional dependencies

No response

[DanielNoord]

Hi, without the relevant import statements we won't be able to diagnose the issue. Please make sure to include all relevant code and dependencies :)

[jenstroeger]

@DarthLegiON thank you for the report, I’ve seen similar problems in my own project. Unfortunately, we didn’t investigate more but in our case deleting the virtual env and rebuilding it worked. Not a fix, I know, but it may unblock you?

@DanielNoord the code shown above is a SQLAlchemy v2 declarative table mapping (docs).

[behnazh-w]

We have faced a similar problem too.

@jenstroeger deleting the virtual env and rebuilding it doesn't fix our issue. We call pylint using a pre-commit hook. So, I ran pre-commit clean to clean up its cach, but still I face the same issue.

This is the error we get:

src/macaron/slsa_analyzer/checks/infer_artifact_pipeline_check.py:36:8: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/macaron/slsa_analyzer/checks/infer_artifact_pipeline_check.py:39:16: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/macaron/slsa_analyzer/checks/infer_artifact_pipeline_check.py:42:17: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/macaron/slsa_analyzer/checks/infer_artifact_pipeline_check.py:45:13: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)

You can find the code here: https://github.com/oracle/macaron/blob/f9be5387ae5ed4e04ceabb9b86579929d2ddcb8f/src/macaron/slsa_analyzer/checks/infer_artifact_pipeline_check.py#L36-L45

[Pierre-Sassoulas]

Could you provide the output of pylint --version @behnazh-w, please ? @DarthLegiON I upgraded your example but we still miss the content of Base and RoleGroup, would you mind providing us the detail or create a minimal reproducer, please ?

[behnazh-w]

Could you provide the output of pylint --version @behnazh-w, please ?

pylint 3.1.0
astroid 3.1.0
Python 3.11.3 (main, Apr  4 2023, 22:36:41) [GCC 11.3.0]

sqlalchemy version:
"SQLAlchemy >=2.0.0,<3.0.0", so 2.0.29 atm.

[jenstroeger]

I upgraded your example but we still miss the content of Base and RoleGroup, would you mind providing us the detail or create a minimal reproducer, please ?

Base is probably defined as described here and RoleGroup seems to be another table mapping just like the RoleReaction in the example. (It may even be enough to declare an __abstract__ table mapping, for the sake of a reproducible example?)

[DarthLegiON]

Sorry for the delay and thanks for your response!

Base definition:

Base = declarative_base()

Rolegroup definition:

class RoleGroup(Base):
    __tablename__ = 'user_roles_groups'

    id: Mapped[int] = mapped_column(Integer, primary_key=True)
    give_role_channel_id: Mapped[int] = mapped_column(BigInteger)
    give_role_message_id: Mapped[int] = mapped_column(BigInteger)

@jenstroeger about deleting a virtual env. Maybe it will surprise you but I don't use one even in development mode. I run the project in Docker. And it fails both in local environment and in CI pipeline, but docker and pip was rebuilt every time. Problem disappears only when I delete the last line with group definition.

[leemurus]

I have the same error for declared_attr decorated function. In pylint 2.* it was ok. sqlalchemy=2.0.30

class NetworkAwareModel(ApostroBaseModel):
    __abstract__ = True

    network_id: Mapped[int] = mapped_column(ForeignKey("network.id"))

    @declared_attr
    def network(self) -> Mapped[Network]:
        """Ensures that a new instance is created for each subclass"""
        return relationship("Network")

def network(self) -> Mapped[Network]:E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)

[DarthLegiON]

If you're still intersted, here is my requirements.txt:

aiohttp==3.9.3
aiosignal==1.3.1
alembic==1.13.1
astroid==3.1.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.2.2
dependency-injector==4.41.0
dill==0.3.8
discord==2.3.2
discord-py-slash-command==4.2.1
discord.py==2.3.2
frozenlist==1.4.1
greenlet==3.0.3
idna==3.6
isort==5.13.2
Mako==1.3.2
MarkupSafe==2.1.5
mccabe==0.7.0
multidict==6.0.5
platformdirs==4.2.0
psycopg2-binary==2.9.9
sentry-sdk==1.44.1
six==1.16.0
SQLAlchemy==2.0.29
tomli==2.0.1
tomlkit==0.12.4
typing_extensions==4.11.0
urllib3==2.2.1
yarl==1.9.4

[diegoortizmatajira]

I had the same issue, by looking at the source code from the mapped_column and relationship functions, I saw their return types, so I modified my hints.

You can change the hinted types to match the return type of sqlalchemy functions (MappedColumn, Relationship)

from sqlalchemy import ForeignKey
from sqlalchemy.orm import MappedColumn, Relationship, mapped_column, relationship

from sqlalchemy.sql.sqltypes import String, Integer


class RoleReaction(Base):
    __tablename__ = 'user_roles_reactions'

    id: MappedColumn[int] = mapped_column(Integer, primary_key=True)
    role_name: MappedColumn[str] = mapped_column(String)
    reaction_name: MappedColumn[str] = mapped_column(String)
    group_id: MappedColumn[int] = mapped_column(ForeignKey('user_roles_groups.id'))
    group: Relationship[RoleGroup] = relationship(RoleGroup)

It made my pylint error messages disappear.

[benmss]

We have experienced this problem again under new circumstances that may be of interest to the pylint team. See above comment for our previous report.

After some refactoring of our codebase, pylint when ran as part of the pre-commit hooks, reports this error for (only) some of our scripts that make use of the Mapped class. We investigated the cause of this error suddenly appearing, and discovered that it is the deletion of two unrelated files that triggers it.

In the original refactor, two files were moved and modified slightly:
src/macaron/repo_finder/provenance_extractor.py -> src/macaron/provenance/provenance_extractor.py
src/macaron/repo_finder/provenance_finder.py -> src/macaron/provenance/provenance_extractor.py

As part of the investigation this refactor was re-done in a piecemeal fashion to help clarify what was happening. The two files above were created in the new location, in their final form, instead of simply moved. Then everything using the old files was modified to use the new ones. Finally, the old files were deleted, and it is only after this deletion that the unsubscriptable error appears:

************* Module macaron.slsa_analyzer.checks.build_script_check
src/macaron/slsa_analyzer/checks/build_script_check.py:31:8: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
...
src/macaron/slsa_analyzer/checks/build_script_check.py:61:24: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
************* Module macaron.slsa_analyzer.checks.provenance_commit_check
src/macaron/slsa_analyzer/checks/provenance_commit_check.py:26:8: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)
src/macaron/slsa_analyzer/checks/provenance_commit_check.py:29:17: E1136: Value 'Mapped' is unsubscriptable (unsubscriptable-object)

This investigation is available for inspection on this branch: pylint-bug-unsubscriptable-3
The final commit of this branch deletes the two files: b1f49f7

Also worth noting:

• The unsubscriptable error only happens if both files are deleted
• The contents of the files do not seem to matter. See commit d8e61ba, where everything except comment headers are removed
• To run pylint on our codebase outside of the pre-commit hooks, we use make check-lint in the base directory

[torotil]

I have observed this issue as well in one of our projects.

At first I tried using git bisect to find the commit that triggered this error. This pointed to a commit not even touching the model classes.

Since it was clearly not a code change triggering this I suspected code size might be an issue. That’s why I tried deleting the tests/ folder of the project containing all the unit tests and the error disappeared. I then checked out the folder again (git checkout -- .) and the error was still gone.

My theory is now: The result depends on the order in which pylint evaluates the files in the project. Deleting and recreating files changes that order.

[torotil]

I was able to create a very minimal example that is reproducible for me on Debian 12 (with Python 3.11.2):

mkdir pylint-test; cd pylint-test
python -m venv .venv
. .venv/bin/activate
pip install sqlalchemy pylint
pylint models.py   # see below

Output:

************* Module models
models.py:23:8: E1136: Value 'orm.Mapped' is unsubscriptable (unsubscriptable-object)
models.py:24:15: E1136: Value 'orm.Mapped' is unsubscriptable (unsubscriptable-object)
models.py:25:12: E1136: Value 'orm.Mapped' is unsubscriptable (unsubscriptable-object)

pip freeze

astroid==3.3.8
dill==0.3.9
greenlet==3.1.1
isort==6.0.1
mccabe==0.7.0
platformdirs==4.3.6
pylint==3.3.4
SQLAlchemy==2.0.38
tomlkit==0.13.2
typing_extensions==4.12.2

models.py

"""DB models."""

# Model classes are useful even without methods.
# pylint: disable=too-few-public-methods

import typing as t

import sqlalchemy as sa
from sqlalchemy import orm

# Create database connection object
db = sa.declarative_base()

class Organization(db.Model):
    """Organization model."""

    id: orm.Mapped[int] = orm.mapped_column(primary_key=True)


class Component(db.Model):
    """Model for a component."""

    id: orm.Mapped[int] = orm.mapped_column(primary_key=True)
    parent_id: orm.Mapped[t.Optional[int]]
    parent: orm.Mapped["Component"] = orm.relationship(
        remote_side=[id],
        back_populates="components",
    )

Update: I was also able to reproduce this on Gentoo with Python 3.13.1.
Update2: Changing the order of classes in the file (putting Organization after Component) makes the problem disappear.