Remove tests with urn:// URIs from required tests

[jdesrosiers]

It occurred to me that we have a bunch of tests that test urn scheme URIs, but nothing in the specification says that implementations need to support the urn URI scheme. Does anyone disagree with moving those tests to "optional"?

[handrews]

Yes. Implementations have to support URIs, and URNs are URIs.

While the urn: scheme specifically is not required, all of the major variations in syntax described in RFC 3986 should be supported and tested.

[handrews]

urn: being a scheme using the rootless syntax which implementations that are lazy about URI support might not handle properly.

[jdesrosiers]

It turned out I had a misunderstanding that led to creating this issue. However, I wasn't entirely wrong and I'm not sure how the piece I got right should effect the test suite and/or the specification.

URI schemes define certain semantics about a URI. My misunderstanding was that the scheme could affect how resolution is performed. For example, relative resolution doesn't make sense for urn and tag schemes. I thought that meant that URI implementations shouldn't try to resolve relative references for these schemes and therefore implementations would need to behave differently when asked to resolve for urn than it does for http. In which case, the specification would have to specifically reference the scheme specifications for the schemes we expect implementations to support. However, it turns out that what really happens is that the standard resolution algorithm can apply to urn and tag. Those scheme specifications say don't do that because it doesn't make sense. They don't say it's not allowed.

However, there is one bit of semantics that is defined by the scheme that could be problematic. The scheme determines how URI equivalence is defined. That means that when fetching a schema from local storage, it's necessary to code for specific schemes to correctly look up the schema. JSON Schema evaluation is a client and RFC 3986 is sufficient without needing to code for schemes. But, local storage is a server and would have to know about schemes and it would be necessary to define what schemes internal storage should support.

So, since you can't use urn or tag without internal storage and we don't define what schemes should be understood, it could be argued that those schemes don't need to to be supported. It could also be argued that http doesn't need to be supported by internal storage, although it should work for external fetching if that's implemented. I'm not sure I want to make that argument at this point, but unless I'm missing something else, I think internal storage is under-specified.

[handrews]

However, it turns out that what really happens is that the standard resolution algorithm can apply to urn and tag. Those scheme specifications say don't do that because it doesn't make sense. They don't say it's not allowed.

That's because it is allowed and in some cases it does make sense. Actually, it always makes sense, it's just not always interesting - when you can only have one path component, you can't have relative paths, and when your path component contains a :, you can't have a scheme-relative reference. But sometimes it is interesting:

• urn, tag, and about all allow fragments
• about and mailto allow query parameters

While some schemes note that relative resolution doesn't make sense, it is always allowed. For example, the latest urn RFC notes that the relative resolution algorithm applied independent of scheme-specific rules will work, to the extent that it is meaningful.

However, there is one bit of semantics that is defined by the scheme that could be problematic. The scheme determines how URI equivalence is defined. That means that when fetching a schema from local storage, it's necessary to code for specific schemes to correctly look up the schema. JSON Schema evaluation is a client and RFC 3986 is sufficient without needing to code for schemes. But, local storage is a server and would have to know about schemes and it would be necessary to define what schemes internal storage should support.

No, this is not true at all. RFC 3986 defines a comparison ladder and does not mandate how far any particular application has to go up the ladder. At most, we might want to note that JSON Schema implementations need not concern themselves with comparisons beyond RFC 3986 §6.2.1 (simple string comparison), but SHOULD document whatever additional normalizations (scheme-specific or otherwise) they apply, if any.

So, since you can't use urn or tag without internal storage and we don't define what schemes should be understood, it could be argued that those schemes don't need to to be supported.

That does not follow at all. I've been using tag in JSON Schemas for years and I've seen urn in the wild. No special support was required. The tag RFC explicitly states that only character-by-character comparisons are necessary to determine equivalence. The urn RFC explicitly states that §6.2.1 comparisons are valid, and that in accordance with RFC 3986's principles of allowing false negatives but not false positives, urn-specific comparison will find more equivalent URIs, never fewer.

It could also be argued that http doesn't need to be supported by internal storage, although it should work for external fetching if that's implemented.

I don't understand this argument at all. I'm not sure I've ever configured a JSON Schema implementation to fetch HTTP resources externally except for hypermedia clients. It's a security nightmare and wouldn't have passed code review in some places I worked.

[handrews]

Plus it's more complicated to carve out scheme-specific exceptions than to just say that URIs are supported, including relative resolution and comparison, to the extent defined fully within RFC 3986. And perhaps add a note that scheme/protocol/media type-specific support beyond that SHOULD be documented.

There is no reason whatsoever that I can think of the exclude any URI scheme from support by internal storage. The whole point of internal storage is to avoid having to deal with external resolution and retrieval. A string lookup table is sufficient for compliance, if not necessarily ideal.

[Julian]

@jdesrosiers I'll try to have a re-read of the spec portions at some point soon just to double check, but yeah the tl;dr on my read was something like the comments above -- "it's a URI, the spec says you're supposed to support URIs as IDs"

I'm not sure I want to make that argument at this point, but unless I'm missing something else, I think internal storage is under-specified.

Just very tactically, are you saying that you're OK with this being closed then?

[jdesrosiers]

@Julian Yes, this can be closed. I couldn't manage to read most of what Henry wrote, but this last line is essentially where my opinion ended up.

A string lookup table is sufficient for compliance, if not necessarily ideal.

It's sufficient based on what the spec currently says, so there is no need to change tests. I still think we should look into this further from the spec perspective at some point, but I'm not going to pursue that at this time.

[Julian]

Sounds good thanks!

[Julian]

(no idea how I clicked the reopen button)