Skip to content

Refined guidelines for external file reuse #535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 26, 2016
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
186 changes: 127 additions & 59 deletions guidelines/REUSE.md
Original file line number Diff line number Diff line change
@@ -1,85 +1,140 @@
# Reuse Philosophy

We encourage reuse and patterns through references.

## What is reusable

The following types are reusable, as defined by the spec:

* Operations
* Parameters
* Models (_or Schema Objects in general_)
* Responses
* Models (or Schema Objects in general)
* Operations (_Operations can only be remote references_)

## Reuse strategy
When reusing components in an API design, a pointer is created from the definition to target design. The references are maintained in the structure, and can be updated by modifying the source definitions. This is different from a "copy on design" approach where references are injected into the design itself.

The reuse technique is transparent between JSON or YAML and is lossless when converting between the two.
When authoring API design documents, common object definitions can be utilized to avoid duplication. For example, imagine multiple path definitions that each share a common path parameter, or a common response structure. The OpenAPI specification allows reuse of common object definitions through the use of "references".

A reference is a construct in your API design document that indicates "the content for this portion of the document is defined elsewhere". To create a reference, at the location in your document where you want to reuse some other definition, create an object that has a `$ref` property whose value is a URI pointing to where the definition is (more on this in later sections).

OpenAPI's provides reference capabilities using the [JSON Reference](https://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03) specification.

### JSON Example

``` js
{
// ...
definitions: {
Person: {
type: 'object',
properties: {
friends: {
type: 'array',
items: {
$ref: '#/definitions/Person'
}
}
}
}
}
}
```

YAML anchors are technically allowed but break the general reuse strategy in Swagger, since anchors are "injected" into a single document. They are not recommended.
### YAML Example

``` yaml
# ...
definitions:
Person:
type: object
properties:
friends:
type: array
items:
$ref: '#/definitions/Person'
```

Referenes can be made either inside the Swagger definition file or to external files. References are done using [JSON Reference](http://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03).
Note: YAML has a very similar feature, [YAML anchors](http://yaml.org/spec/1.2/spec.html#id2765878). Examples from this point will only be in JSON, using JSON References.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well played. :)


## Techniques

### Guidelines for Referencing

When referencing internally, the target references have designated locations:
All references should follow the [JSON Reference](https://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03) specification.

* Parameters -> `parameters`
* Responses -> `responses`
* Models (and general Schema Objects) -> `definitions`
JSON Reference provides guidance on the resolution of references, notably:

Operations can only be referenced externally.
> If the URI contained in the JSON Reference value is a relative URI,
then the base URI resolution MUST be calculated according to
[RFC3986], section 5.2. Resolution is performed relative to the
referring document.

An example for an internal reference - `#/definitions/MyModel`. All references are canonical and must be a qualified [JSON Pointer](http://tools.ietf.org/html/rfc6901). For example, simply referencing `MyModel` is not allowed, even if there are no other definitions of it in the file.
Whether you reference definitions locally or remote, you can never override or change their definitions from the referring location. The definitions can only be used as-is.

When referencing externally, use a valid URI as the reference value. If your referenced file contains only one definition that should be used, you can refer to the file directly. For example:
#### Local references

_Assuming file https://my.company.com/definitions/Model.json_
```json
{
"description": "A simple model",
"type": "object",
"properties": {
"id": {
"type": "integer"
}
}
}
When referencing locally (within the current document), the target references should follow the conventions, as defined by the spec:

* Parameters -> `#/parameters`
* Responses -> `#/responses`
* Definitions (Models/Schema) -> `#/definitions`

An example of a local definition reference:

_Example from https://github.com/OAI/OpenAPI-Specification/blob/master/examples/v2.0/json/petstore.json_
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to use more examples from the repo, rather than contrived examples.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also isn't necessarily true. While swagger has created some top level locations to put reusable objects of a specific type, most JSON Reference processors will resolve any JSON Reference. There's no reason I can't have something like #/info/title as a JSON Reference within my Swagger document.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd agree, and I've seen this before. However, there is a long-standing assumed convention on the aforementioned elements, and changing it could break existing implementations (probably not yours ;) ).

If this isn't already an existing issue, maybe it should be, tagged for proposal on next version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @whitlockjc here. It has always seemed weird to me that the Swagger 2.0 spec defined its own Reference Object and then defined specific places in the spec where that object was allowed. Seems arbitrarily limiting, when JSON Reference is so much more powerful than that. Parsers like mine and @whitlockjc's allow users to use JSON References anywhere they want, and in my experience, this is something that almost everyone ends up doing in the real world.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure that's accurate. By using JSON References, which Swagger knowingly did in 2.0, they knew that this would allow URIs to any place locally or remote. The type-specific containers were just so you could do all of this in one document if you wanted to while still retaining the ability to validate the content of the containers. @webron can set me straight if I'm wrong.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, $ref values are not guaranteed to be JSON Pointers. JSON Pointers are only for local references, otherwise the $ref value is any URI. This is based on the JSON References spec: http://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03#section-4

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @webron, I stand corrected. I had no idea that creating these containers was to imply that you could only reference definitions within a container. I can see why you'd want containers so you could do validation on their contents but I would had never assume that implied they were the only things you could officially reference. Knowing that was the intent now, you should definitely document this. But...like you said, knowing how JSON References works means many people will just use them as they were intended and reference whatever they need. ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't really go into the details of the containers. There are two reasons for the containers (that I recall). 1 - allow for easier validation. Since you know that anything under definitions is a Schema Object you can easily validate them. To me, that's a major downside of external references as it makes validating a spec more difficult (not getting into the why now). 2 - All spec objects have no additionalProperties allowed except for vendor extensions. That again is intentional, and that's another reason to provide the internal containers to hold referenced objects, because those can't just be added to the spec arbitrarily.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If json reference must be pointer in the swagger spec, maybe adding different jsonReference with a pattern would help like the following :

"jsonReferenceSchema" : {
    "type": "object",
    "properties": {
        "$ref": {
            "type": "string",
            "required": true,
            "pattern": "^#/definitions/[^/]+?$"
        }
    }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't have to be a pointer, can also be a URL. I don't think the schema should deal with that level of validation.

``` json
// ...
"200": {
"description": "pet response",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/Pet"
}
}
```

The reference would be `https://my.company.com/definitions/Model.json`.
#### Remote references

_Assuming file https://my.company.com/definitions/models.json_
```json
{
"models": {
"Model": {
"description": "A simple model",
"type": "object",
"properties": {
"id": {
"type": "integer"
}
},
"Tag": {
"description": "A tag entity in the system",
"type": "object",
"properties": {
"name": {
"type": "string"
}
}
}
}
##### Relative path

Files can be referred to in relative paths to the current document.

_Example from https://github.com/OAI/OpenAPI-Specification/tree/master/examples/v2.0/json/petstore-separate/spec/swagger.json_

``` json
// ...
"responses": {
"default": {
"description": "unexpected error",
"schema": {
"$ref": "../common/Error.json"
}
}
}
```

The reference to the `Model` model would be `https://my.company.com/definitions/models.json#/models/Model`. Make sure you include the full path to the model itself, including its containers if needed.
Remote references may also reference properties within the relative remote file.

_Example from https://github.com/OAI/OpenAPI-Specification/tree/master/examples/v2.0/json/petstore-separate/spec/swagger.json_
``` json
// ...
"parameters": [
{
"$ref": "parameters.json#/tagsParam"
},
{
"$ref": "parameters.json#/limitsParam"
}
]
```

Whether you reference definitions internally or externally, you can never override or change their definitions from the referring location. The definitions can only be used as-is.

### Definitions
Reuse schema definitions by creating a repository of definitions. This is done by simply hosting a file or set of files for commonly used definitions across a company or organization. In the case of multiple files, the models can be referenced directly as such:
##### URL

Remote files can be hosted on an HTTP server (rather than the local file system).

One risk of this approach is that environment specific issues could arise if DNS is not taken into account (as the reference can only contain one hostname).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is worth mentioning. I think it's obvious that if you cannot access the referenced document for any reason that things will fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like a common request, "how do I handle multiple environments with ". This was my attempt at stating, "you don't".
In other words, it's clarifying that hostnames are not parameterizable, at least in the word of the spec.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why would someone expect it to be if there is no mention of this anywhere except for path parameters? I'm not against keeping it, I just thought by adding the details that it could be confusing. Your call.


_Assuming file https://my.company.com/definitions/Model.json_
```json
Expand All @@ -92,14 +147,14 @@ _Assuming file https://my.company.com/definitions/Model.json_
"format": "int64"
},
"tag": {
"description": "a complex, shared property. Note the absolute reference",
"description": "A complex, shared property. Note the absolute reference",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this was in the original example as well, but the description is obviously ignored and does not override the value inside the reference (as previously mentioned in the doc). I get this is used here for documentation to the readers, but it may also send the wrong message. I'll merge as-is, but we may consider removing it after all.

"$ref": "https://my.company.com/definitions/Tag.json"
}
}
}
```

For a single file, you can package the definitions in an object:
Remote references may also reference properties within the remote file.

_Assuming file https://my.company.com/definitions/models.json_
```json
Expand Down Expand Up @@ -133,8 +188,20 @@ _Assuming file https://my.company.com/definitions/models.json_
```


### Definitions

Reuse schema definitions by creating a repository of definitions. This is done by simply hosting a file or set of files for commonly used definitions across a company or organization.

Refer to [Guidelines for Referencing](#guidelines-for-referencing) for referencing strategies.


### Parameters
Similar to model schemas, you can create a repository of `parameters` to describe the common entities that appear throughout a set of systems. Using the same technique as above, you can host on either a single or multiple files. For simplicity, the example below assumes a single file.

Similar to model schemas, you can create a repository of `parameters` to describe the common entities that appear throughout a set of systems.

Refer to [Guidelines for Referencing](#guidelines-for-referencing) for referencing strategies.

Using the same technique as above, you can host on either a single or multiple files. For simplicity, the example below assumes a single file.

_Assuming file https://my.company.com/parameters/parameters.json_

Expand Down Expand Up @@ -205,7 +272,10 @@ To include these parameters, you would need to add them individually as such:
```

### Operations
Again, Operations can be shared across files. Although the reusability of operations will be less than with Parameters and models. For this example, we will share a common `health` resource so that all APIs can reference it:

Again, Operations can be shared across files. Although the reusability of operations will be less than with Parameters and Definitions. For this example, we will share a common `health` resource so that all APIs can reference it:

Refer to [Guidelines for Referencing](#guidelines-for-referencing) for referencing strategies.

```json
{
Expand Down Expand Up @@ -246,7 +316,8 @@ Which points to the reference in the `operations.json` file:
Remember, you cannot override the definitions, but in this case, you can add additional operations on the same path level.

### Responses
Just like the other objects, responses can be reused as well.

Refer to [Guidelines for Referencing](#guidelines-for-referencing) for referencing strategies.

Assume the file `responses.json`:

Expand Down Expand Up @@ -299,6 +370,3 @@ You can refer to it from a response definition:
}
}
```

### Constraints
* Referenced objects must be to JSON structures. YAML reuse structures may be supported in a future version.