Future changes to JSON Schema

[HotelDon]

So there have been a few changes to roadmap for JSON Schema in the last couple months - for one, v5 of the spec has added no new features, and was instead done to clarify a bunch of language in the spec for future improvements. This means that the features originally meant for draft 5 have been pushed to draft 6 and beyond. Currently, work is being done on draft 6, but some features (including the $data keyword) are being tabled until draft 7 or later in order to allow more discussion of potentially controversial changes.

So, in the mean time, I want to work on some changes to JSL to clarify a few things and prepare for the future. Firstly, drafts 4 and 5 are now functionally the same, so those should probably be combined as "draft-04/05" or "draft-04/draft-05" on the drop down menu. Version 5.0.0 of AJV will support most of the v6 proposals, and will likely be done sometime in January - so adding draft-06 to the list should be easy.

However, that does leave the keywords destined for draft 7+ in a bit of a weird spot - most of them are being moved out of AJV proper and into ajv-keywords until those specs start getting worked on (it also includes a some custom keywords, and any keywords that end up rejected might end up in there as well). Conversely, the $data keyword is being broken out into it's own run time option, which will only work in v5+. So I propose adding an "experimental" entry on the drop down menu that enables these keywords for anyone who wants to use them, as a sort of "draft-n+1".

I was going to start working on it today, but since I'm waiting on the next version of AJV to be finished anyways, I figured I'd ask for your input first - any suggestions or concerns are appreciated.

[HotelDon]

So, it turns out that there aren't really any changes in draft-06 that break backwards compatibility - so the next version of AJV will support drafts 4,5, and 6 by default without needing any special options. However, this means if you wanted an option to validate to draft-04/05 only, you'd have to use a different validator (or keep the old version of AJV around, but that's a messier option). I'm not sure how important keeping that separation would be, but I'll leave that for you to decide.

[nickcmaynard]

Hey. Thanks for the detailed thoughts. I'm on holiday right now, but promise I'll have a think and get back to you as soon as I'm back.

[nickcmaynard]

Up until now, I haven't paid a lot of attention to the differences between draft-04 and draft-05. The hard isolation was added mostly because of the (hideous) breaking changes between IIRC draft-03 and draft-04. So I now, by default, isolate every version.

So... are they retrospectively changing the definition of draft-05 to, in essence, be draft-04 + $data? Or have I misunderstood that?

[nickcmaynard]

FYI, big refactoring going on right now in componentise, so please base any changes on that.

[HotelDon]

Draft-05 doesn't include the $data reference in the spec - nothing was added to draft-05 at all. In fact, $data isn't planned for draft-06 either, but it will probably get in eventually (it's definitely getting added as a hyperschema keyword at some point). I was only talking about $data support as far as AJV is concerned, and that comment was written before I learned that AJV isn't doing version flags anymore - instead, it just loads the draft-06 meta-schema by default, and you can load the draft-04 schema (which is also the draft-05 schema) instead if you need it. As a result, the $data option will work with either.

[nickcmaynard]

OK. So draft-04/05 are identical. Draft-06 is a moving target which AJV will track. But - although we can load draft-04/05 meta schema (for validating our schemas) the actual language support won't change.

So - if we validate a scheme against the appropriate meta schema, then if that fails (i.e. draft-04/05 target, but draft-06 keyword) then we should refuse to attempt document validation, as the schema is invalid.

Does that sound workable? If so, from what you describe we can switch AJV into draft-04/05 meta schema mode and effectively have a draft-04/05 compliant validator. Or have I misunderstood?

[HotelDon]

Yes, that should work. I've already done some work setting things up (although I need to move those changes over to the componentise tag), so I should be able to test that tonight to make sure there aren't any weird edgecases hiding in there.

[nickcmaynard]

Great, thanks. I'll try to close off the componentise work tomorrow so you don't have a moving target.

[nickcmaynard]

Looks like draft-04 and draft-05 have different meta-schema. So let's keep those isolated for now.

[HotelDon]

Do you have a link to the draft-05 meta-schema? Because json-schema.org still lists the draft-04 meta schema as the latest.

[nickcmaynard]

It's included in AJV apparently - https://github.com/epoberezkin/ajv/releases/tag/5.0.1-beta.0

[HotelDon]

So that meta-schema is the one that was created specifically for AJV to support the old v5 proposals - there never was an official v5 meta-schema. When you pass the v5: true option to AJV, it would just swap out the official v4 meta-schema for that one. AJV still includes it for backwards compatibility reasons. You could include that as an option, but calling it "draft-05" would be really confusing since there would be no continuity between it and draft-06.

[nickcmaynard]

Branch merged. Do you have any links where these draft versions are discussed?

[HotelDon]

There was a pull request (json-schema-org/json-schema-spec#132) that references the fact draft-05 didn't update the meta-schema, and that some of the updates originally meant for it are being pushed to draft-06. Unfortunately, I can't find the exact discussion where this decision was made (I'm pretty sure it's hidden somewhere in the issue tracker), but I know that @awwright or @handrews can explain the reasoning behind the decision themselves if you message them about it.

[handrews]

@HotelDon @nickcmaynard At this point we're just moving to Draft 06 because it is very close to completion (I am optimistic for February). @awwright did not want to publish a meta-schema with Draft 05 because it was a cleanup more than anything else- there was some discussion among various of us at json-schema-org/json-schema-spec over following up with one but by now Draft 06 is just a better focus.

The TL;DR here is that we're going to heavily encourage people to implement Draft 06 and give feedback on it, rather than spending time trying to make Draft 05 a major presence "in the wild."

Draft 06 validation is nearly complete- I think the biggest remaining validation PR is on how to spell the new formats ("uri-ref", "uri-template", "json-pointer" vs "uriref", "uritemplate", "jsonpointer", with the hyphen forms having more support due to the existing "date-time"). Everything else with Draft 06 is about wording clarifications or hyper-schema, I'm pretty sure.