From c8017f80e89ba5dfa94943caa6fd03b23f141a92 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Sat, 24 Aug 2019 15:14:11 -0700 Subject: [PATCH 1/6] Move core examples/best practices to appendixes These sections are not normative (RFC 2119 terms will be removed in a subsequent commit), and moving them to an appendix reduces the length of the sizable normative portion of the spec. And also improves flow, I think. The examples within core tend to rely on understanding more of the core keywords than have necessarily been introduced, so they work better after the entire vocabulary has been introduced. --- jsonschema-core.xml | 587 ++++++++++++++++++++++---------------------- 1 file changed, 296 insertions(+), 291 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 5e8bba20..4082cd5c 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -1308,177 +1308,6 @@ interoperable across implementations. -
- - Vocabulary authors SHOULD - take care to avoid keyword name collisions if the vocabulary is intended - for broad use, and potentially combined with other vocabularies. JSON - Schema does not provide any formal namespacing system, but also does - not constrain keyword names, allowing for any number of namespacing - approaches. - - - Vocabularies may build on each other, such as by defining the behavior - of their keywords with respect to the behavior of keywords from another - vocabulary, or by using a keyword from another vocabulary with - a restricted or expanded set of acceptable values. Not all such - vocabulary re-use will result in a new vocabulary that is compatible - with the vocabulary on which it is built. Vocabulary authors SHOULD - clearly document what level of compatibility, if any, is expected. - - - Meta-schema authors SHOULD NOT use "$vocabulary" to combine multiple - vocabularies that define conflicting syntax or semantics for the same - keyword. As semantic conflicts are not generally detectable through - schema validation, implementations are not expected to detect such - conflicts. If conflicting vocabularies are declared, the resulting - behavior is undefined. - - - Vocabulary authors SHOULD provide a meta-schema that validates the - expected usage of the vocabulary's keywords on their own. Such meta-schemas - SHOULD NOT forbid additional keywords, and MUST NOT forbid any - keywords from the Core vocabulary. - - - It is RECOMMENDED that meta-schema authors reference each vocabulary's - meta-schema using the "allOf" keyword, - although other mechanisms for constructing the meta-schema may be - appropriate for certain use cases. - - - The recursive nature of meta-schemas makes the "$recursiveAnchor" - and "$recursiveRef" keywords particularly useful for extending - existing meta-schemas, as can be seen in the JSON Hyper-Schema meta-schema - which extends the Validation meta-schema. - - - Meta-schemas MAY impose additional constraints, including describing - keywords not present in any vocabulary, beyond what the meta-schemas - associated with the declared vocabularies describe. This allows for - restricting usage to a subset of a vocabulary, and for validating - locally defined keywords not intended for re-use. - - - However, meta-schemas SHOULD NOT contradict any vocabularies that - they declare, such as by requiring a different JSON type than - the vocabulary expects. The resulting behavior is undefined. - - - Meta-schemas intended for local use, with no need to test for - vocabulary support in arbitrary implementations, can safely omit - "$vocabulary" entirely. - -
-
- - This meta-schema explicitly declares both the Core and Applicator vocabularies, - together with an extension vocabulary, and combines their meta-schemas with - an "allOf". The extension vocabulary's meta-schema, which describes only the - keywords in that vocabulary, is shown after the main example meta-schema. - - - The main example meta-schema also restricts the usage of the Applicator - vocabulary by forbidding the keywords prefixed with "unevaluated", which - are particularly complex to implement. This does not change the semantics - or set of keywords defined by the Applicator vocabulary. It just ensures - that schemas using this meta-schema that attempt to use the keywords - prefixed with "unevaluated" will fail validation against this meta-schema. - - - Finally, this meta-schema describes the syntax of a keyword, "localKeyword", - that is not part of any vocabulary. Presumably, the implementors and users - of this meta-schema will understand the semantics of "localKeyword". - JSON Schema does not define any mechanism for expressing keyword semantics - outside of vocabularies, making them unsuitable for use except in a - specific environment in which they are understood. - -
- - This meta-schema combines several vocabularies for general use. - - - - -
-
- - This meta-schema describes only a single extension vocabulary. - - - - -
- - As shown above, even though each of the single-vocabulary meta-schemas - referenced in the general-use meta-schema's "allOf" declares its - corresponding vocabulary, this new meta-schema must re-declare them. - - - The standard meta-schemas that combine all vocabularies defined by - the Core and Validation specification, and that combine all vocabularies - defined by those specifications as well as the Hyper-Schema specification, - demonstrate additional complex combinations. These URIs for these - meta-schemas may be found in the Validation and Hyper-Schema specifications, - respectively. - - - While the general-use meta-schema can validate the syntax of "minDate", - it is the vocabulary that defines the logic behind the semantic meaning - of "minDate". Without an understanding of the semantics (in this example, - that the instance value must be a date equal to or after the date - provided as the keyword's value in the schema), an implementation can - only validate the syntactic usage. In this case, that means validating - that it is a date-formatted string (using "pattern" to ensure that it is - validated even when "format" functions purely as an annotation, as explained - in the Validation specification. - -
@@ -1710,126 +1539,6 @@ raise an error if such usage is detected.
-
-
- - Consider the following schema, which shows "$id" being used to identify - both the root schema and various subschemas, and "$anchor" being used - to define plain name fragment identifiers. - - - - -
- - The schemas at the following URI-encoded JSON - Pointers (relative to the root schema) have the following - base URIs, and are identifiable by any listed URI in accordance with - sections and - above. As previously - noted, support for non-canonical URIs is OPTIONAL. - - - - - - - https://example.com/root.json - - - https://example.com/root.json# - - - - - - https://example.com/root.json - - https://example.com/root.json#foo - - - https://example.com/root.json#/$defs/A - - - - - - https://example.com/other.json - - https://example.com/other.json# - - - https://example.com/root.json#/$defs/B - - - - - - https://example.com/other.json - - https://example.com/other.json#bar - - - https://example.com/other.json#/$defs/X - - - https://example.com/root.json#/$defs/B/$defs/X - - - - - - https://example.com/t/inner.json - - https://example.com/t/inner.json#bar - - - https://example.com/t/inner.json# - - - https://example.com/other.json#/$defs/Y - - - https://example.com/root.json#/$defs/B/$defs/Y - - - - - - - urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f - - - urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f# - - - https://example.com/root.json#/$defs/C - - - - - -
@@ -3606,6 +3315,302 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 +
+
+ + Consider the following schema, which shows "$id" being used to identify + both the root schema and various subschemas, and "$anchor" being used + to define plain name fragment identifiers. + + + + +
+ + The schemas at the following URI-encoded JSON + Pointers (relative to the root schema) have the following + base URIs, and are identifiable by any listed URI in accordance with + sections and + above. As previously + noted, support for non-canonical URIs is OPTIONAL. + + + + + + + https://example.com/root.json + + + https://example.com/root.json# + + + + + + https://example.com/root.json + + https://example.com/root.json#foo + + + https://example.com/root.json#/$defs/A + + + + + + https://example.com/other.json + + https://example.com/other.json# + + + https://example.com/root.json#/$defs/B + + + + + + https://example.com/other.json + + https://example.com/other.json#bar + + + https://example.com/other.json#/$defs/X + + + https://example.com/root.json#/$defs/B/$defs/X + + + + + + https://example.com/t/inner.json + + https://example.com/t/inner.json#bar + + + https://example.com/t/inner.json# + + + https://example.com/other.json#/$defs/Y + + + https://example.com/root.json#/$defs/B/$defs/Y + + + + + + + urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f + + + urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f# + + + https://example.com/root.json#/$defs/C + + + + + +
+ +
+
+ + Vocabulary authors SHOULD + take care to avoid keyword name collisions if the vocabulary is intended + for broad use, and potentially combined with other vocabularies. JSON + Schema does not provide any formal namespacing system, but also does + not constrain keyword names, allowing for any number of namespacing + approaches. + + + Vocabularies may build on each other, such as by defining the behavior + of their keywords with respect to the behavior of keywords from another + vocabulary, or by using a keyword from another vocabulary with + a restricted or expanded set of acceptable values. Not all such + vocabulary re-use will result in a new vocabulary that is compatible + with the vocabulary on which it is built. Vocabulary authors SHOULD + clearly document what level of compatibility, if any, is expected. + + + Meta-schema authors SHOULD NOT use "$vocabulary" to combine multiple + vocabularies that define conflicting syntax or semantics for the same + keyword. As semantic conflicts are not generally detectable through + schema validation, implementations are not expected to detect such + conflicts. If conflicting vocabularies are declared, the resulting + behavior is undefined. + + + Vocabulary authors SHOULD provide a meta-schema that validates the + expected usage of the vocabulary's keywords on their own. Such meta-schemas + SHOULD NOT forbid additional keywords, and MUST NOT forbid any + keywords from the Core vocabulary. + + + It is RECOMMENDED that meta-schema authors reference each vocabulary's + meta-schema using the "allOf" keyword, + although other mechanisms for constructing the meta-schema may be + appropriate for certain use cases. + + + The recursive nature of meta-schemas makes the "$recursiveAnchor" + and "$recursiveRef" keywords particularly useful for extending + existing meta-schemas, as can be seen in the JSON Hyper-Schema meta-schema + which extends the Validation meta-schema. + + + Meta-schemas MAY impose additional constraints, including describing + keywords not present in any vocabulary, beyond what the meta-schemas + associated with the declared vocabularies describe. This allows for + restricting usage to a subset of a vocabulary, and for validating + locally defined keywords not intended for re-use. + + + However, meta-schemas SHOULD NOT contradict any vocabularies that + they declare, such as by requiring a different JSON type than + the vocabulary expects. The resulting behavior is undefined. + + + Meta-schemas intended for local use, with no need to test for + vocabulary support in arbitrary implementations, can safely omit + "$vocabulary" entirely. + +
+ +
+ + This meta-schema explicitly declares both the Core and Applicator vocabularies, + together with an extension vocabulary, and combines their meta-schemas with + an "allOf". The extension vocabulary's meta-schema, which describes only the + keywords in that vocabulary, is shown after the main example meta-schema. + + + The main example meta-schema also restricts the usage of the Applicator + vocabulary by forbidding the keywords prefixed with "unevaluated", which + are particularly complex to implement. This does not change the semantics + or set of keywords defined by the Applicator vocabulary. It just ensures + that schemas using this meta-schema that attempt to use the keywords + prefixed with "unevaluted" will fail validation against this meta-schema. + + + Finally, this meta-schema describes the syntax of a keyword, "localKeyword", + that is not part of any vocabulary. Presumably, the implementors and users + of this meta-schema will understand the semantics of "localKeyword". + JSON Schema does not define any mechanism for expressing keyword semantics + outside of vocabularies, making them unsuitable for use except in a + specific environment in which they are understood. + +
+ + This meta-schema combines several vocabularies for general use. + + + + +
+
+ + This meta-schema describes only a single extension vocabulary. + + + + +
+ + As shown above, even though each of the single-vocabulary meta-schemas + referenced in the general-use meta-schema's "allOf" declares its + corresponding vocabulary, this new meta-schema must re-declare them. + + + The standard meta-schemas that combine all vocabularies defined by + the Core and Validation specification, and that combine all vocabularies + defined by those specifications as well as the Hyper-Schema specification, + demonstrate additional complex combinations. These URIs for these + meta-schemas may be found in the Validation and Hyper-Schema specifications, + respectively. + + + While the general-use meta-schema can validate the syntax of "minDate", + it is the vocabulary that defines the logic behind the semantic meaning + of "minDate". Without an understanding of the semantics (in this example, + that the instance value must be a date equal to or after the date + provided as the keyword's value in the schema), an implementation can + only validate the syntactic usage. In this case, that means validating + that it is a date-formatted string (using "pattern" to ensure that it is + validated even when "format" functions purely as an annotation, as explained + in the Validation specification. + +
+
+
Thanks to From 332abe6d5a3be30f24c9df0dcc98fb68393188c0 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Sat, 24 Aug 2019 22:05:00 -0700 Subject: [PATCH 2/6] Remove RFC 2119 language from examples/guidance Vocabulary authorship is not part of the JSON Schema specification, so this guidance should not use RFC 2119 terms. --- jsonschema-core.xml | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 4082cd5c..c7f69143 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -1502,7 +1502,7 @@ Further examples of such non-canonical URIs, as well as the appropriate - canonical URIs to use instead, are provided in section + canonical URIs to use instead, are provided in appendix .
@@ -1839,7 +1839,7 @@ on the trust that the validator has in the schema. Such URIs and schemas can be supplied to an implementation prior to processing instances, or may be noted within a schema document as it is processed, producing associations - as shown in section . + as shown in appendix . A schema MAY (and likely will) have multiple URIs, but there is no way for a @@ -3351,8 +3351,7 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 Pointers (relative to the root schema) have the following base URIs, and are identifiable by any listed URI in accordance with sections and - above. As previously - noted, support for non-canonical URIs is OPTIONAL. + above. @@ -3439,7 +3438,7 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0
- Vocabulary authors SHOULD + Vocabulary authors should take care to avoid keyword name collisions if the vocabulary is intended for broad use, and potentially combined with other vocabularies. JSON Schema does not provide any formal namespacing system, but also does @@ -3452,11 +3451,11 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 vocabulary, or by using a keyword from another vocabulary with a restricted or expanded set of acceptable values. Not all such vocabulary re-use will result in a new vocabulary that is compatible - with the vocabulary on which it is built. Vocabulary authors SHOULD + with the vocabulary on which it is built. Vocabulary authors should clearly document what level of compatibility, if any, is expected. - Meta-schema authors SHOULD NOT use "$vocabulary" to combine multiple + Meta-schema authors should not use "$vocabulary" to combine multiple vocabularies that define conflicting syntax or semantics for the same keyword. As semantic conflicts are not generally detectable through schema validation, implementations are not expected to detect such @@ -3464,13 +3463,13 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 behavior is undefined. - Vocabulary authors SHOULD provide a meta-schema that validates the + Vocabulary authors should provide a meta-schema that validates the expected usage of the vocabulary's keywords on their own. Such meta-schemas - SHOULD NOT forbid additional keywords, and MUST NOT forbid any + should not forbid additional keywords, and must not forbid any keywords from the Core vocabulary. - It is RECOMMENDED that meta-schema authors reference each vocabulary's + It is recommended that meta-schema authors reference each vocabulary's meta-schema using the "allOf" keyword, although other mechanisms for constructing the meta-schema may be appropriate for certain use cases. @@ -3482,14 +3481,14 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 which extends the Validation meta-schema. - Meta-schemas MAY impose additional constraints, including describing + Meta-schemas may impose additional constraints, including describing keywords not present in any vocabulary, beyond what the meta-schemas associated with the declared vocabularies describe. This allows for restricting usage to a subset of a vocabulary, and for validating locally defined keywords not intended for re-use. - However, meta-schemas SHOULD NOT contradict any vocabularies that + However, meta-schemas should not contradict any vocabularies that they declare, such as by requiring a different JSON type than the vocabulary expects. The resulting behavior is undefined. From 61c6bf168195bf5f4afabb0c6e9d27661717ab12 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Sat, 24 Aug 2019 22:54:41 -0700 Subject: [PATCH 3/6] Generative use case guidance This is going to be a powerful technique for generative use cases, which have been observed either requiring references (as opposed to subschemas) in certain places, or implementing various sorts of inferencing around references on their own or in conjunction with other keywords. For example, one OpenAPI documentation tool assumes that in an "allOf" of several references, the first is a base class of some sort. While this is not correct behavior by either JSON Schema or OpenAPI, it demonstrates a feature built by making implicit assumptions about certain references. This section provides a recommendation for making such behavior explicit, and also guides writers of such tools to the concept of extension vocabularies and annotations. --- jsonschema-core.xml | 62 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index c7f69143..8a93f0b9 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -3610,6 +3610,68 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0
+
+ + While the presence of references is expected to be transparent + to validation results, generative use cases such as code generators + and UI renderers often consider references to be semantically significant. + + + To make such use case-specific semantics explicit, the best practice + is to create an annotation keyword can be created for use in the same + schema object alongside of a reference keyword such as "$ref". + +
+ + For example, here is a hypothetical keyword for determining + whether a code generator should consider the reference + target to be a distinct class, and how those classes are related. + Note that this example is solely for illustrative purposes, and is + not intended to propose a functional code generation keyword. + + + + +
+ + Here, this schema represents some sort of object-oriented class. + The first reference in the "allOf" is noted as the base class. + The second is not assigned a class relationship, meaning that the + code generator should combine the target's definition with this + one as if no reference were involved. + + + Looking at the properties, "foo" is flagged as object composition, + while the "date" property is not. It is simply a field with + sub-fields, rather than an instance of a distinct class. + + + This style of usage requires the annotation to be in the same object + as the reference, which must be recognizable as a reference. + +
+
Thanks to From d7ed9b14eeded20f7e653d1da032688b18e3e97c Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Wed, 4 Sep 2019 20:04:09 -0700 Subject: [PATCH 4/6] Fix munged sentence based on PR feedback --- jsonschema-core.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 8a93f0b9..dd5d931e 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -3618,7 +3618,7 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 To make such use case-specific semantics explicit, the best practice - is to create an annotation keyword can be created for use in the same + is to create an annotation keyword for use in the same schema object alongside of a reference keyword such as "$ref".
From 440cf2e5b29c788832602f8d429c2de3243bdbce Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Sat, 24 Aug 2019 22:32:20 -0700 Subject: [PATCH 5/6] Add guidance on bundling and references This should make clear what use cases are considered valid and safe, vs when someone is on their own in terms of figuring that out. --- jsonschema-core.xml | 51 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index dd5d931e..3a0f5658 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -3435,6 +3435,57 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0
+
+ + Various tools have been created to rearrange schema documents + based on how and where references ("$ref") appear. This appendix discusses + which use cases and actions are compliant with this specification. + +
+ + A set of schema resources intended for use together can be organized + with each in its own schema document, all in the same schema document, + or any granularity of document grouping in between. + + + Numerous tools exist to perform various sorts of reference removal. + A common case of this is producing a single file where all references + can be resolved within that file. This is typically done to simplify + distribution, or to simplify coding so that various invocations + of JSON Schema libraries do not have to keep track of and load + a large number of resources. + + + This transformation can be safely and reversibly done as long as + all static references (e.g. "$ref") use URI-references that resolve + to canonical URIs, and all schema resources have an absolute-URI + as the "$id" in their root schema. + + + With these conditions met, each external resource can be copied + under "$defs", without breaking any references among the resources' + schema objects, and without changing any aspect of validation or + annotation results. The names of the schemas under "$defs" do + not affect behavior, assuming they are each unique, as they + do not appear in canonical URIs for the embedded resources. + +
+
+ + Attempting to remove all references and produce a single schema document does not, + in all cases, produce a schema with identical behavior to the original form. + + + Since "$ref" is now treated like any other keyword, with other keywords allowed + in the same schema objects, fully supporting non-recursive "$ref" removal in + all cases can require relatively complex schema manipulations. It is beyond + the scope of this specification to determine or provide a set of safe "$ref" + removal transformations, as they depend not only on the schema structure + but also on the intende usage. + +
+
+
From 7e982af8a40586b440fcc70e80812168cd416ab0 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Mon, 26 Aug 2019 10:39:18 -0700 Subject: [PATCH 6/6] Fix type from review feedback Co-Authored-By: Phil Sturgeon --- jsonschema-core.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 3a0f5658..f4d662c2 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -3481,7 +3481,7 @@ User-Agent: product-name/5.4.1 so-cool-json-schema/1.0.2 curl/7.43.0 all cases can require relatively complex schema manipulations. It is beyond the scope of this specification to determine or provide a set of safe "$ref" removal transformations, as they depend not only on the schema structure - but also on the intende usage. + but also on the intended usage.