Merge pull request #80 from handrews/refs

Shorter, more focused referencing guide

Author: handrews

referencing/dynamic.md

@@ -0,0 +1,16 @@
+---
+layout: default
+title: Dynamic References
+nav_order: 4
+parent: Using References
+---
+
+## Dynamic References
+
+Dynamic referencing is a feature of JSON Schema draft 2020-12, and therefore included in OAS 3.1.  It allows setting a placeholder reference target that can be overridden during validation depending on how the dynamic reference was reached.  If that explanation didn't help you at all, don't worry, it boils down to two use cases:
+
+* [Implementing generic types](https://json-schema.org/blog/posts/dynamicref-and-generics#using-dynamic-references-to-support-generic-types) (a.k.a. templatized types), such as container data structures or payload envelopes
+* Extending recursive schemas, as the OAS schema needed to do to [add OAS-specific keywords to the standard JSON Schema dialect](https://json-schema.org/blog/posts/validating-openapi-and-json-schema)
+
+While there may be other creative ways to use dynamic references, these are the situations most likely to show up in API descriptions.
+

referencing/index.md

@@ -0,0 +1,25 @@
+---
+layout: default
+title: Using References
+nav_order: 5
+has_children: true
+has_toc: false
+---
+
+# Using References in OpenAPI Descriptions
+
+OpenAPI Referencing is a powerful tool.  It allows managing document size and complexity, and allows re-using shared components and avoiding copy-paste or change management errors.
+
+However, the history of referencing and the `"$ref"` keyword is complex, leading to slightly different behavior depending on the version of the OpenAPI Specification (OAS) that you are using, and on where in your OpenAPI Description (OAD) the reference occurs.  There are also other `"$ref"`-like keywords (`"operationRef"`, `"mapping"`) and behaviors (referencing by component name or operation ID) that are related but somewhat different.  Referencing can also be challenging to use due to incomplete and inconsistent support across different tools, some of which require references to be pre-processed before they can read the OAD.
+
+The resources in this section explain how to use referencing, and what to look for when assessing the referencing support in your OpenAPI tools.
+
+## The Future of References
+
+There are plans to reduce the complexity around references in future OpenAPI Specifications.
+
+The [Moonwalk project](https://github.com/OAI/sig-moonwalk/discussions) is considering a different approach that imports complete documents, associates them with namespaces, and only supports referencing by component name (not `"$ref"`).  A small example can be seen in the [Moonwalk deployments proposal](https://github.com/OAI/sig-moonwalk/blob/main/Deployments.md), and there are discussions around an [initial draft proposal for imports](https://github.com/OAI/sig-moonwalk/discussions/72) and a few ideas on [how to manage interactions with JSON Schema referencing](https://github.com/OAI/sig-moonwalk/discussions/73).
+
+The [proposed Workflows Specification](https://github.com/OAI/sig-workflows/blob/main/versions/1.0.0.md) is already using a `"sourceDescription"` field that is not unlike the Moonwalk proposal.
+
+

referencing/overview.md

@@ -0,0 +1,27 @@
+---
+layout: default
+title: References Overview
+nav_order: 1
+parent: Using References
+---
+
+# What are references?
+
+A *reference* is a keyword and value that identifies a *reference target* with a URI.  In some cases, this URI can be treated as a URL and de-referenced directly.  In other cases, as we will see in the (forthcoming) guide to resolving references, it is helpful to separate the target's *identity* from its *location*.
+
+External references are how multiple documents are linked into a single OpenAPI Description (OAD).  This means that referencing impacts how other linkages, such as those that use Components Object names, or values such as `operationId` in the Path Item Object, work.  These other linkages can only work if the document (or with many tools, the specific JSON object) containing the name or other identifier has been referenced.
+
+## A taxonomy of references
+
+References exist in several variations in the OpenAPI Specification (OAS) versions 3.0 and 3.1, as shown in the following table.  Note that an *adjacent keyword* is a keyword in the same JSON Object (whether it is written in JSON or YAML) as the reference keyword.
+
+| OAS Version | Object | Reference Keyword | Adjacent Keywords | Behavior |
+| ----------- | ------ | ----------------- | ----------------- | -------- |
+| 3.0         | Reference Object | `"$ref"`       | ignored           | logically replace the Reference Object with the reference target |
+| 3.1         | Reference Object | `"$ref"`       | `"summary"` and `"description"` allowed; others ignored | logically replace the Reference Object with a copy of the target, overwriting the target's `"summary"` and/or `"description"` fields with those of the Reference Object if they are present |
+| 3.x         | Path Item Object | `"$ref"`       | allowed under some circumstances | logically replace the Path Item Object containing the `"$ref"` with a Path Item Object that combines the fields of the target Path Item Object with the non-`"$ref"` fields of the Path Item Object containing the `"$ref"`, as long as none of those fields conflict |
+| 3.1         | Schema Object | `"$ref"`   | allowed | Apply the target Schema Object to the same instance location as the Schema Object containing the `"$ref"`, and combine the results with the results of other keywords in the Schema Object containing the `"$ref"` just as you would any other keyword results; this is more-or-less equivalent to using a one-element `"allOf"` |
+| 3.x         | Link Object   | `"operationRef"` | allowed, except `"operationId"` | treat the reference target as the target of the link described by the Link Object |
+| 3.x         | Discriminator Object | `"mapping"` | n/a | for each name under `"mapping"`, if the value is not the name of a Schema Object under the Components Object, treat it as a reference to the schema to use when the discriminator field matches the mapping name |
+
+Note that not all references are handled by replacing the source with the target.  Some are defined in terms of behavior, and **cannot** be replaced with an inline value.

referencing/remove.md

@@ -0,0 +1,38 @@
+---
+layout: default
+title: Remove References
+nav_order: 3
+parent: Using References
+---
+
+# Remove References
+
+Sometimes it is useful to remove references in an OpenAPI Description (OAD).  A common use case is working with tools that don't understand references well.  This isn't always straightforward, however, and this page covers the main points to look out for.
+
+## Convert an OAD to a single-document form without any references
+
+Not all OADs can be converted but there are many cases where it is possible to do such a thing without problems.
+
+The difficult cases are:
+
+* Cyclic references in JSON Schema ***cannot*** be removed, although if the references cross documents, it is possible to reduce them to same-document references in a single document
+* While [dynamic references](dynamic) in OAS 3.1 Schema Objects technically can be converted to static references, it can [grow the schema size exponentially](https://dl.acm.org/doi/10.1145/3632891) ([pdf](https://dl.acm.org/doi/pdf/10.1145/3632891))
+* OAS 3.1 Schema Objects, OAS 3.0 Reference Objects, and both OAS 3.0 and 3.1 Path Item Objects all allow keywords alongside a`"$ref"`, each with slightly different semantics; generic reference removal tools, particularly ones that predate OAS 3.1, may fail to preserve the behavior while removing references
+* In OAS 3.1 Schema Objects that use `"$id"`, reference removal tools that are not aware of `"$id"` will resolve references impacted by the `"$id"` incorrectly
+* The Link Object's `"operationRef"` cannot be replaced with an inline Operation Object (although if the target Operation Object has a unique-within-the-OAD `"operationId"`, then the `"operationRef"` can be replaced by the `"operationId"`)
+* The Discriminator Object's `"mapping"` field forbids inline Schema Objects; URI-reference values can be replaced with names from the `"schemas"` section of the Components Object, but when the Discriminator Object is in a referenced document, there are ambiguities involved in how to resolve those names
+* When using components published by 3rd parties such as standards organizations, tools might rely on reference URIs to identify standard components; while this usage is currently uncommon, it may become more common as OpenAPI is more broadly adopted by standards bodies
+
+If your OAD does not run afoul of any of the above difficulties, you _can_ produce a reference-less single-document OAD, and there are [tools](https://tools.openapis.org) that will do so (but see the caveats in the next section).
+
+## Convert a multi-document OAD into a single document
+
+It is possible to convert a multi-document OAD into a single document event if not all references can be removed.  However, depending on exactly how your multi-document OAD is structured, this may be more complex than just removing references.  Sadly, the reasons have to do with [painfully obscure and technical ambiguities](https://github.com/OAI/oascomply/blob/main/reports/processing-model-summary.md) regarding how multi-document OADs are intended to be parsed.  The OpenAPI Initiative hopes to clarify this in future versions of the specification.
+
+* It is possible for a Link Object's `"operationId"` to point to an Operation Object in a Path Item Object that is never referenced; it is unclear how this case should handle it, which is one reason the OAS itself [advises against](https://spec.openapis.org/oas/v3.1.0#fixed-fields-16) using `"operationId"` in  multi-document OADs
+* Security Scheme Objects are never referenced, only named in Security Requirement Objects, and must be taken into account
+* There is no standard place to put referenced Path Item Objects in OAS 3.0
+* The resolution of server configuration and security settings can be unclear in multi-document OADs
+
+Again, in many cases none of this is a problem, and there are [tools](https://tools.openapis.org) that will do what you want.
+

referencing/resolve.md

@@ -0,0 +1,23 @@
+---
+layout: default
+title: Resolve References
+nav_order: 2
+parent: Using References
+---
+
+## Resolve References
+
+A reference is said to be _resolved_ within a tool if:
+
+* Its target has been identified
+* Any modifications to the target required by the OAS (e.g. because of fields adjacent to `"$ref"`) have been performed on a copy of the target
+* The resulting target value has been associated with the reference source in some way that the tool can easily use when needed
+
+A reference is said to be _removed_ if it has been replaced by its (possibly modified) target.
+
+Reference resolution usually preserves the referencing information such as the URI used to identify the target, while reference removal usually discards it.  In many cases this is not significant, except that not knowing how the parsed OAD relates to the references in your JSON or YAML document may make debugging more difficult.
+
+While plain JSON documents form a [tree](https://en.wikipedia.org/wiki/Tree_%28data_structure%29) structure, an OpenAPI Description with resolved references is not _necessarily_ a tree, but a [graph](https://en.wikipedia.org/wiki/Graph_%28abstract_data_type%29).
+
+Tools that resolve references in-memory and work with the graph structure can process all OADs. Tools that rely on reference removal, either as part of the tool or by a separate pre-processing tool, can only support OADs that form trees.
+