v3.2: $self field (Alternative Approach)

[handrews]

Fixes:

• OAS3 Tag/template to define external-components URL #1979

This is an alternative to PR #4389. The only functional difference is that $self is required to be absolute (a change that @karenetheridge and @mikekistler are debating).

The main difference is that I drastically reduced the normative specification text and instead gave examples of all of the different ways to establish a base URI. [EDIT: and put the examples in an appendix] This is probably more understandable than trying to explain the options in prose.

@notEthan I also dumped a lot of the use cases as I think I figured out a more concise way to get the point across.

This adds $self as a way for a document to define its own URI for use in reference targets, and as the base URI for relative URI references in the document.

This does not impact the resolution of relative API URLs.

• schema changes are included in this pull request
• schema changes are needed for this pull request but not done yet
• no schema changes are needed for this pull request

[handrews]

After staring at the giant examples section a lot, I decided it works better as an appendix. (force push is just because I accidentally picked up a stray file in the commit and had to edit it to pull that out).

[lornajane]

This is the most readable of the lot, and I think I "get" the whole thing a lot more now. Thank you for doing this work and all the iterations!

Minor points/suggestions from me (I'm not ready to approve but I've only got small questions):

• in the proposal we called it self but it looks like now it is $self. We have other URI fields with plain-text names, why does this one need the $? Since it's being added at the top level, we can be confident of not having name clashes. (why isn't it in info with the other document information like license? Am I asking too many questions?)
• I'd like to propose we move the appendix to the learn site and link to it. Is the learn site really good/mature/stable enough to link to from a spec file? I think it is, and the information would be better there (but needs to be very well signposted since we haven't done this for other content).

[lornajane]

Good discussion in the meeting:

• content does need to go to the learn site, but we're not going to block this change for it
• $self makes sense because it's like $ref in being "special" and related to how references work

[karenetheridge]

I am in agreement with the shuffling of the examples to the learn site rather than in the main spec, but not with the change from relative to absolute uri.

[mikeschinkel]

@handrews — A big +1 from me for including examples in the spec. I would prefer them to be in-line so a reader does not have to jump around to follow them, but if those examples need to be in an appendix that is better than not having any or having a link to someplace that might eventually 404.

My biggest pet peeve about specs in general is that they do not include examples which make most specs all-but-inaccessible to those like me who need examples to understand the prose.

P.S. Moving to the learn site only works IMO if there is a guarantee that the learn site will never change those examples nor take them down which, I believe, is impossible to guarantee. FTW.

[ralfhandl]

Relative vs. absolute $self doesn't seem to make much of a difference.

• A relative $ref is now relative to the retrieval URL. Making it relative to a relative $self which is itself relative to the retrieval URL only adds the convenience of making the $ref value shorter. It does not add a feature.

• A relative $ref that is relative to an absolute $self also only adds the convenience of making the $ref value shorter (and relative instead of absolute). It does not add a feature.

Seems the only feature added by $self is an in-document identifier for an OpenAPI description, and that feature seems to benefit from an absolute URI value.

What am I missing?

[mikekistler]

I think there may be some errors in the examples of URI resolution in Appendix G. I pointed out two and I think the rest should probably be checked for accuracy before we merge this.

[mikekistler]

This too does not jive with the resolver that Copilot built. It says that the target URI is actually "https://example.com/api/schemas/foo".

[handrews]

@mikekistler should be fixed. Really, "these look wrong" would have been sufficient here. The errors were just because I reworked the various paths at some point to make the examles cover more cases, and some things didn't get updated post-rework. Nothing obscure was going on.

Although it sounds like you produced something useful beyond this which is cool!

[karenetheridge]

Another reason to allow $self to be relative is that it is consistent with JSON Schema, where $id can be relative, so it would be easier to reason through how $ref resolution works in a document and apply RFC3986 consistently if both keywords worked the same way.

I see no gain from forcing $self to be absolute; it just restricts what users and implementations can do with the keyword. We could just as easily say that $self is not needed at all, because the user can always supply the base URI via the retrieval URI. Our goal should be to open up usecases, not restrict them.

[ralfhandl]

$self is not needed at all, because the user can always supply the base URI via the retrieval URI

The main feature $self adds is an in-document identifier allowing to determine whether documents retrieved from different URIs or locally stored without their retrieval URI are actually the same and not just textually identical by chance.

Using $self with a relative URI means opting out its main feature and can be considered an abuse of $self. This abuse would be prevented by requiring an absolute URI value.

[karenetheridge]

Using $self with a relative URI means opting out its main feature and can be considered an abuse of $self

Is there an example (for the learn site, appendix or wherever) that you can construct that would demonstrate this issue, as a way of cautioning against it?

[lornajane]

After many discussions about relative vs absolute, we agreed in TDC this week that supporting both would make a more open feature and since the retrieval URL is already used in other places, the additional burden for tools maintainers is not huge.

[handrews]

I have updated this with the outcome of today's TDC call. I have also:

• @ralfhandl further clarified the consequences of using alternative URIs
• Added an example (a random UUID URN) for the application-specific default option
• Added an example of relative $self and $id

The new commits were added in one push, and then I force-pushed the rebased commits unchanged to pick up the latest syntax highlighting that supports this change building without warnings.

[handrews]

hmm... I thought I was only adding two new commits, looks like maybe I had not pushed some other local fixes? My apologies for any confusion, just be aware that I have never edited commits once pushed, so if a commit looks familiar it's the same as it was.

[handrews]

One more new bugfix commit plus an unmodified rebase force-push.

[ralfhandl]

+1 to the content, suggestions for its representation

[handrews]

@ralfhandl I have tried to address everything that I did not accept as a commit.

• Everything is now "retrieval URI"
• I reworked the requirement-then-caveat ordering
• I reworked the crossing-$id text and dropped the mention of the unclear "point too but not across $id" case; I think @karenetheridge had wanted that included? But it's tricky to word without just being more confusing, and idk how much we should try to compensate for JSON Schema's shortcomings here
• I added "only" to the weird JSON Pointer case that you asked if people actually do, because I agree it read strangely, and I think the new wording is much more clear

Regarding the schema media type parameter, while that usage is the most correct usage, it is a bit confusing because of the need to put YYYY-MM-DD in the URI, so I might be willing to drop it on those grounds. Otherwise, we'll need to update that to the correct date so people don't copy-paste the placeholder.

[handrews]

@ralfhandl went ahead and removed the parameter because of the YYYY-MM-DD problem. Which makes me sad because I would rather promote the long-established correct use of the media type.