[firestore-emulator: datastore-mode] Kind query return empty list

[artoale]

General description

dastore-mode does not support Kind Queries

This is absolutely essential to support using the emulator in a testing environment, since you need to be able to list all the kinds in the db to be able to (potentially selectively) delete all of them.

[REQUIRED] Environment info

firebase-tools: Using the gcloud cli version 468.0.0, firestore emulator: 1.19.3

Platform: macOS (intel), python client library

[REQUIRED] Test case

import os

from google.cloud import datastore
from google.cloud.datastore.query import PropertyFilter


def main():
    PORT = 10901
    os.environ["DATASTORE_EMULATOR_HOST"] = "127.0.0.1:%s" % PORT
    os.environ["DATASTORE_PROJECT_ID"] = "test"

    def get_new_client():
        return datastore.Client(
            project="test",
            namespace=None,
            _http=None,
        )

    client = get_new_client()

    kind = "a_kind"
    key = client.key(kind)
    entity = datastore.Entity(key=key)
    entity.update({"foo": "bar"})

    client.put(entity)

    all_kinds = list(client.query(kind="__kind__").fetch())
    print(all_kinds)
    assert kind in [x.key.name for x in all_kinds]


if __name__ == "__main__":
    main()

[REQUIRED] Steps to reproduce

1. Start the emulator with gcloud emulators firestore start --host-port=127.0.0.1:10901 --database-mode=datastore-mode
2. Install the python deps: pip install google-cloud-datastore==2.19.0
3. Run the above python script

[REQUIRED] Expected behavior

Completes, printing the following output:

[<Entity('__kind__', 'a_kind') {}>]

[REQUIRED] Actual behavior

Output is

[]

and the assertion fails.

Additional information

Note: the same test works as expected with both a real datastore instance or the old legacy datastore emulator

[harshyyy21]

Thanks for bringing this up Alessandro! This is unsupported in the current implementation of the new emulator. Will try to get it out as soon as possible and will update this thread when done. Thanks again!

[jacobg]

Any update on this? This capability is crucial in order to use the emulator in unit tests, and match both real datastore and legacy datastore emulator behavior.

[JoolsHattey]

Yeah, we really need this, along with namespace support. Right now, we're completely stuck using the legacy bundled Datastore emulator with App Engine (java) local just to get the same behavior as deployed Cloud Datastore. There's no real workaround for this, so it's a major blocker.

[jacobg]

Or alternatively, please open source the Java emulator so your customers can implement these features themselves. There's a discussion opened about it here:
#8282