Add tests for referencing / identifying schemas via URN URIs

[Relequestual]

Came across the folliwng issue...

$ref tags will work for the python validators if they refer to an internal path.
If they refer to a URI (urn:something:...) then it will attempt to resolve it as an URI and fail.

phenopackets/phenopacket-format#55

Is this something we should test for?
Something like "ids defined should be checked before trying to resolve externally"?

[Julian]

I'm not sure whether I follow whether that issue means to be a bug report or not.

Do you know what full schema we'd be talking about?

What would it mean to $ref an urn URI without specifying what that urn refers to?

[handrews]

What would it mean to $ref an urn URI without specifying what that urn refers to?

If you have pre-loaded schemas that declared is with urn:, or have a database to search by id, or some other arrangement than loading schemas over the network via URL semantics. Another URI scheme that would be used in this way is tag:, which is basically a human-readable alternative to URNs.

[handrews]

In my totally subjective opinion, implementations should not only attempt to retrieve a reference over the network if the URI is a URL, and then unless there's some other clear reason to do otherwise should use the protocol associated with the scheme. Even then technically it's just an identifier according to the spec.

[Julian]

Ah, OK, I figured that's what that meant, but the Python library (well, mine) does support that, so maybe the author of that just didn't find it.

It might be nice to test that sort of stuff here, but right now the test suite has no way of communicating that out of band info (of "here's some other stuff, coordinate being able to look them up"). Might need a bit of extending to our test vocabulary..

[handrews]

It might be nice to test that sort of stuff here, but right now the test suite has no way of communicating that out of band info (of "here's some other stuff, coordinate being able to look them up"). Might need a bit of extending to our test vocabulary..

That would also help build a hyper-schema conformance suite, which I hope becomes more relevant very soon :-)

[Julian]

Yup, probably just needs some schematizing (ha ha) and to be done in a not-backwards-incompatible way.

Would be nice to hit #53 at the same time too I think, and tbh I've been considering whether to propose we switch to using TOML for test cases.

[handrews]

I have to admit I'm not a fan of TOML for JSON Schema (its been proposed to me elsewhere). TOML's approach to nested hierarchy doesn't visually convey the structure well. It's a great alternative to INI by providing some actually standardized structure and semantics on that basic pattern, plus advanced syntax.

But even aside from the visual thing (which I prioritize highly, but is somewhat subjective), it can't express all JSON Schemas because of restrictions on key names and mixing types. You can't use a "." in a key name (but URI Templates allow a "." and those names are keys in "hrefSchema"). You can't mix types in an array (but "enum" allows exactly that).

[Relequestual]

Ach sorry. I didn't realise I handn't also included a link to the schema.

The urn in full is used as the id for fields. I forgot about the fact that the value of $ref need to start with a hash # to be relative to the current document. I confess I didn't look into this in great detail; it's the first I've heard of urn's. From my limited research, it looks like they are really depricated, and URLs in full should be used, but there you go.

It might be nice to test that sort of stuff here, but right now the test suite has no way of communicating that out of band info (of "here's some other stuff, coordinate being able to look them up"). Might need a bit of extending to our test vocabulary..

That's fair enough, and it sounds like it would be a complex ask. Maybe we could add a section to the repos readme to clarify what the test suite DOES NOT test in terms of expected compliance. I think that somoene implementing JSON Schema and inetegrating the test suite should probably be able to work that out, but if it's simple to compile, then it sounds like it could be useful! (Not suggesting it be an exhaustive list.)

[handrews]

I forgot about the fact that the value of $ref need to start with a hash # to be relative to the current document.

How does that play into this case? URNs are URIs so a reference to them would not start with a # whether they are in the same file or not.

[Julian]

I have to admit I'm not a fan of TOML for JSON Schema (its been proposed to me elsewhere). TOML's approach to nested hierarchy doesn't visually convey the structure well. It's a great alternative to INI by providing some actually standardized structure and semantics on that basic pattern, plus advanced syntax.

I think for that you just do what you usually do in that case -- introduce indentation, which TOML allows for.

it can't express all JSON Schemas because of restrictions on key names

It supports "." in key names, you just need to quote them. But yup, the mixed type thing is certainly an issue for parsers that strictly follow the spec.

But to be clear, I was not suggesting we use TOML to represent the actual structures. Just the meta-language.

I.e., the test data still written in JSON, just the test cases themselves written in TOML, and then we still generate full JSON to preserve backwards compat.

Anyways, I don't mean to tie all these things together, I haven't even fully thought ^ through -- just collecting together a bunch of the related tickets on "I want to do write test cases that the current form doesn't allow me to write".

[awwright]

It'd probably be useful to have a few different tests just to show the diversity of possible names, for example have at least one specified as a UUID URN, and another as an ni: URI.

---

As for how tests are written, I'm a fan of alternative syntaxes for JSON, which is pretty horrible as far as human-writable data formats go (No comments, no trailing commas, or newlines in strings? Really?)... But since JSON Schema is only defined over JSON it'd be kind of awkward for the official test suite to use anything else (it feels like a conflict of interest sort of thing).

[Relequestual]

I forgot about the fact that the value of $ref need to start with a hash # to be relative to the current document.

How does that play into this case? URNs are URIs so a reference to them would not start with a # whether they are in the same file or not.

Right you are.

So the solution would be that pre loading the JSON shchema, they should define a urn registry of some kind? (I've never seen urn's before coming across this repo. Whole thing seems sort of odd to me)

[handrews]

@Relequestual I think the only thing that needs validating here is that if an implementation has seen a schema with an id/$id that is a URN-ish URI (e.g. urn:uuid: or tag:) then any subsequently evaluated $ref to that URN-ish URI should successfully resolve.

Basically, it doesn't matter what kind of URI or URI Reference is used in an ID, an implementation should be able to remember and use it. On the other hand, if a $ref points to a URN-ish URI that it has NOT already seen, then there is, by definition, no clear way to resolve it and the resolution is expected to fail.

These seem verifiable in the test suite. We don't need to cover all schemes, but basically there are two general URI syntaxes, as seen in Section 3 of RFC 3986:

• scheme + authority + path + query + fragment (I think of these as URL=ish)
• scheme + path (I think of these as URN-ish)

Since an authority starts with '//' these are easy to distinguish, and we should test that both work since many URL-oriented libraries fail to properly handle URN-ish URIs.