Performance issues with referencing

[robherring]

I'm working on moving dtschema over to use referencing Registry instead of RefResolver. With the change, the validation time has gone from ~12 sec to 6 minutes.

The schemas I have are ~3k schema files which are mostly defining a single object. They are all loaded into a Registry. Each validation run iterates over 100s of instances with all the schemas. I'm

Profiling puts almost all the time spent in Registry.combine():

 4324    0.016    0.000   10.630    0.002 _core.py:457(combine)                                        
 8648    8.496    0.001    8.514    0.001 {method 'update' of 'rpds.HashTrieMap' objects}              
 4323    2.014    0.000    2.014    0.000 {method 'update' of 'rpds.HashTrieSet' objects}              

(This was Ctrl-C out of after 12 secs)

The root problem seems to be every new Validator instance created in 'descend' is creating a new Registry to add the json-schema.org schemas into my Registry. Commenting out this gets the performance back:

diff --git a/jsonschema/validators.py b/jsonschema/validators.py
index 23ea17c878d4..783a931f9dc2 100644
--- a/jsonschema/validators.py
+++ b/jsonschema/validators.py
@@ -202,7 +202,7 @@ def create(
# TODO: include new meta-schemas added at runtime
_registry: referencing.jsonschema.SchemaRegistry = field(
default=SPECIFICATIONS,

•        converter=SPECIFICATIONS.combine,  # type: ignore[misc]
  

•        #converter=SPECIFICATIONS.combine,  # type: ignore[misc]
         kw_only=True,
         repr=False,
     )
  

Obviously not the right fix. It needs to do the combine once, rather than every 'evolve'.

[Julian]

I can't look at this without some information. Please include something which reproduces.

[robherring]

Now that I understand what 'converter' does, it's really quite simple. If you pass in a registry, then every subsequent (internal) instantiation of Validator is going to create a new Registry. That is because on each Validator instantiation SPECIFICATIONS.combine() creates a new Registry unless the Registry is SPECIFICATIONS (as 'self' will always be equal to SPECIFICATIONS). What's needed is a check if the registry doesn't contain SPECIFICATIONS already, then only then add it.

[robherring]

Something like this gets me down to ~40 secs:

diff --git a/jsonschema/validators.py b/jsonschema/validators.py
index 23ea17c878d4..91f8eac3fe4a 100644
--- a/jsonschema/validators.py
+++ b/jsonschema/validators.py
@@ -202,7 +202,6 @@ def create(
         # TODO: include new meta-schemas added at runtime
         _registry: referencing.jsonschema.SchemaRegistry = field(
             default=SPECIFICATIONS,
-            converter=SPECIFICATIONS.combine,  # type: ignore[misc]
             kw_only=True,
             repr=False,
         )
@@ -247,6 +246,7 @@ def create(
 
         def __attrs_post_init__(self):
             if self._resolver is None:
+                self._registry = self._registry.combine(SPECIFICATIONS)
                 self._resolver = self._registry.resolver_with_root(
                     resource=specification.create_resource(self.schema),
                 )

That's still too much of a slow down for me though (take the times here and multiply by 1000-2000 for my total run times).

[robherring]

Here's a test case to demonstrate. Run the tests with duration timing. Looking at the execution time for the 2 test cases, validation is 5x slower when a registry is used.

diff --git a/jsonschema/tests/test_validators.py b/jsonschema/tests/test_validators.py
index 12280781a9d7..9bb4f65e430c 100644
--- a/jsonschema/tests/test_validators.py
+++ b/jsonschema/tests/test_validators.py
@@ -2145,6 +2145,8 @@ class TestThreading(TestCase):
         thread.join()
         self.assertEqual((thread.is_alive(), failed), (False, []))
 
+from referencing import Registry
+from referencing.jsonschema import DRAFT201909
 
 class TestRefResolver(TestCase):
 
@@ -2344,6 +2346,30 @@ class TestRefResolver(TestCase):
         validator = validators.Draft7Validator(another, resolver=two)
         self.assertFalse(validator.is_valid({"maxLength": "foo"}))
 
+    def test_registry(self):
+        sch = {
+            "$id": "test",
+            "$ref": "https://json-schema.org/draft/2019-09/schema",
+        }
+        rsrc = ("test", DRAFT201909.create_resource(sch))
+        registry = Registry().with_resources([rsrc])
+
+        validator = validators.Draft201909Validator(sch, registry=registry)
+        for i in range(10000):
+            validator.is_valid({"maxLength": 4})
+
+    def test_no_registry(self):
+        sch = {
+            "$id": "test",
+            "$ref": "https://json-schema.org/draft/2019-09/schema",
+        }
+        rsrc = ("test", DRAFT201909.create_resource(sch))
+        registry = Registry().with_resources([rsrc])
+
+        validator = validators.Draft201909Validator(sch)
+        for i in range(10000):
+            validator.is_valid({"maxLength": 4})
+
 
 def sorted_errors(errors):
     def key(error):

[Julian]

Great, thanks, will have a look in a bit.

[Julian]

Took a bit longer than I'd have liked to get to but had a few other things on the plate --

This should now be addressed (and in a moment released in 4.18.0a9).

Thanks again for the example!

Let me know if you notice any other issues, appreciated.