Clarify the nature of schema placeholder keywords

[handrews]

Fixes #778 (paging @about-code).

Acknowledge that "$defs" and placeholders like it also indicate
that their values are schemas, which is relevant to whether
they can be detected as a proper "$ref" target or place where
"$id" may appear.

This ensures that similar extension keywords are understood
to have the same concerns as extension applicator keywords.

I did not add anything about $defs being recommended, because
now that it is covered in the same way as the applicator vocab,
it is clear that you can add more extension keywords like it.

And I think it is inherently clear that keywords in Core are recommended.

[See later comments from me about a few further changes]

[about-code]

Sorry for my late feedback. To be honest I am not sure if this would clarify things (for me) considerably compared to before. I weren't able to draw the line in the opposite direction (as a reader of section Schema Re-Use With "$defs" right to this section and the essence of its meaning). But that might be caused by my personal situation of "not being in the specs" in such depth, so isn't meant to disqualify your answer. Though, the newly introduced term placeholder keyword would require its own definition.

I would accept this PR as closing my issue but would also accept the issue being closed without any further action if majority doesn't see any necessity for clarification.

Thank you.

Just a little note: I can't comment on things in a timely manner ATM so don't make any progress with this depend on me.

[handrews]

Thanks, @about-code! That's helpful feedback, I'll tweak it a bit more (we don't want people to have to already be experts when reading the standard- it's not a tutorial but it should be accessible). If you have time to look at the update, great, but thanks for letting me know that you might not be available.

[Relequestual]

In the original issue #512 @handrews stated that $defs is actually more of a recommendation while definitions and basically any other arbitrary keys (see #512 (comment)) should continue to work, too.

• @about-code in the related issue

$defs is the recognised keyword, changed form definitions in previous draft-7.

Essentially, your question is, should definitions continue to work with draft-8?

The answer is, maybe, but you shouldn't rely on such, because the behaviour is now undefined.

Some implementations may choose to support referencing using arbitrary keys, but it's not required. There are lots of "super edge" cases that we would need to consider to define the behaviour, so rather than forbid it, leaving it undefined leaves implementations to document specific support for users who don't need their schemas to be portable. "behaviour is undefined" essentially translates to, "Probably not a good idea, as it's complex, but we won't stop you doing it if you really want, and understand it will likely mean your implementation isn't portable".

I hope that gives some additional clarification.

[Relequestual]

I feel this reads better.
In terms of the change itself. Fine by me!

[handrews]

@Relequestual @about-code OK I actually did some serious-ish reworking.

I had come up with the idea of $defs and $comment as "reserved location" keywords, and forgot that I'd never actually put that in. So I added a fourth keyword category, which I think makes it obviously you can add more keywords like that. I'm not going to outright say "you can make a keyword exactly like $defs because we don't really want people to do that. But it should be clear that it's valid now.

@Relequestual definitions is actually sort of still reserved by the default meta-schema. I clarified that in the validation spec. The idea of calling that particular meta-schema the "default meta-schema" is part of #780.

I updated both $defs and $comment to use wording about reserving locations, which also avoids the question of how forceful "standard" is. Reserving a location does not imply that all other locations are unusable. It just implies that this location is reliably usable.

I think this is a good improvement overall, so thanks for bringing it to our attention, @about-code !

[gregsdennis]

@Relequestual Some implementations may choose to support referencing using arbitrary keys, but it's not required.

For example, mine does this. It allows $ref to point anywhere, but if what it finds is not a schema, it throws an exception.

[handrews]

@gregsdennis also, presumably if you have a structure like:

{
  "$id": "https://this.example.com/schemas/foo",
  "$ref": "#/a/b/c",
  "a": {
    "b": {
      "$id": "https://that.example.com/schemas/bar",
      "c": {
        "$ref": "somewhere#/$defs/whatever"
      }
    }
  }
}

where a, b, and c are all unrecognized keywords and all of them take schemas as values, you will end up evaluating the $ref in c incorrectly because you did not take into account the $id in b.

Or if you're implementation does look at that $id, and if b in fact instead does not take a schema, but allows arbitrary keys which take schemas (like properties), then that would also produce the wrong $ref resolution behavior.

@gregsdennis @Relequestual , the point here isn't about implementations choosing to "support" arbitrary keys. The point is that it is absolutely not possible to guarantee the correct behavior of nested arbitrary unknown keys. There is no way to do it. You have to decide whether you look for $id in intervening levels, but since there are two possibilities and you can't implement both at once, you are guaranteed to produce the wrong behavior for one use case or the other.

This is a strange corner case, but it is critically important not to over-promise spec behavior.

You cannot do this correctly, and shouldn't claim to do so.

However, you can get away with (for lack of a better term) schema duck typing for a single level of unknown keyword, because it's not possible to insert an unexpected $id anywhere. But that's it. Claiming to match whatever assumptions people make beyond that is in violation of the spec. You can document a consistent behavior, but understand that there are cases where it will be wrong.

[gregsdennis]

Yeah, hopefully people don't do that because it's really weird.

[handrews]

@gregsdennis

Yeah, hopefully people don't do that because it's really weird.

LOL have you met people? 😆

Are you OK with these changes as I've updated things?

[about-code]

Here I'd suggest to use the term placeholder keyword at least once, maybe even emphasized. For example:

These placeholder keywords do not affect validation[...]

Then when the term placeholder keyword is used further down below such as in section References to Possible Non-Schemas it is terminologically clear that placeholder keywords is the phrase used to refer to keywords falling into the category of reserved locations.

Overall the recent additions make things much more consistent and comprehensible, I think. 👍

[handrews]

Oh, I'd actually meant to get rid of "placeholder" and use the "location" terminology. I'll sort that out and fix the out-of-order wording, too- thanks!

[about-code]

is reserved a location for

I guess it should be

is a reserved location

[handrews]

OK, @about-code I made both of those places consistent by using "location-reserving keywords" and "reserves a location", respectively.

Since those changes are pretty trivial, and @Relequestual 's only specific change request was a comma that I fixed yesterday, I'm going to go ahead and merge. @Relequestual seemed to approve the changes aside from that grammar thing. And Greg approved yesterday.

I fixed the comma, I think that was the only actual change requested.