Should `unevaluatedProperties` count annotations from `if`?

[jdesrosiers]

The current interpretation is that if does count annotations if it passes. We have tests that check for this and every current implementation behaves that way. However, @Relequestual brought up on slack that annotations should never count for if because it's just a conditional whose purpose is to select between then and else.

Here's an example that illustrates the consequences of making such a change.

{
  "type": "object",
  "if": {
    "properties": {
      "foo": { "const": "then" }
    },
    "required": ["foo"]
  },
  "then": {
    "properties": {
      "bar": { "type": "string" }
    },
    "required": ["bar"]
  },
  "unevaluatedProperties": false
}

If we don't count if annotations, then there is no value that matches the if and passes validation over the whole schema. { "foo": "then", "bar": "" } would pass the if/then, but fail the unevaluatedProperties. The "foo" property declaration would have to be added outside of the if. If we can determine any reason why that would be a bad thing, then we shouldn't change anything. I can't think of any good reason for it.

Another way to think about it is, what is the behavior of unevaluatedProperties using the Implication Pattern. if/then is supposed to be sugar for the Implication Pattern, so I would expect them to work the same with unevaluatedProperties. Here's the same example as above except with implication,

{
  "type": "object",
  "anyOf": [
    {
      "not": {
        "properties": {
          "foo": { "const": "then" }
        },
        "required": ["foo"]
      }
    },
    {
      "properties": {
        "bar": { "type": "string" }
      },
      "required": ["bar"]
    }
  ],
  "unevaluatedProperties": false
}

Now the annotations from the "if" don't apply because of the not schema. So, that indicates that it might be desirable for if to not count annotations to be compatible with implication. To me, that's a pretty convincing argument that we go this wrong and annotations from if schemas should never be counted.

[karenetheridge]

I find this a compelling argument. +1 from me.

edit: I am changing my vote to 0, in the face of compelling arguments from the other side. :)

[notEthan]

I also wondered why this is the specified behavior, and had intended to open an issue about it. it is counterintuitive to me that annotations from if - either validation annotations, or informational, would apply. +1 from me for changing to not collect these annotations.

[handrews]

I absolutely intended for annotations to be collected from if whenever it passes. Sometimes I don't remember what I meant, but this time I do 😄

The validation result from if is essentially discarded after determining whether to evaluate then or else, but annotations are explicitly said to be collected:

If annotations (Section 7.7) are being collected, they are collected
from this keyword's subschema in the usual way, including when the
keyword is present without either "then" or "else".

When I wrote "in the usual way" I meant collecting them and propagating them up as normal if the if schema passes, and dropping all annotations from the if schema if it fails. That could perhaps be spelled out more clearly, but that is definitely what I meant. The unevaluatedProperties case is one reason for that, as otherwise you have to duplicate some part of the if schema in the then schema in order to make it work, which is not good design.

including when the keyword is present without either "then" or "else" means that you can use a standalone if to conditionally collect some annotations without ever causing validation to fail. I don't think I had a use case in mind, but it felt more consistent, and earlier drafts failed to explain what a standalone if meant (other than that someone forgot a then or else, most likely).

[gregsdennis]

So are you saying that then runs when if produces annotations, and else runs when if does not produce annotations but is still present?

[Relequestual]

So are you saying that then runs when if produces annotations, and else runs when if does not produce annotations but is still present?

That seems to be the gist of it. Why do you ask?

[Relequestual]

I absolutely intended for annotations to be collected from if whenever it passes. Sometimes I don't remember what I meant, but this time I do 😄

Do you remember what the use case you had in mind for this was @handrews ?

The validation result from if is essentially discarded after determining whether to evaluate then or else, but annotations are explicitly said to be collected:

If annotations (Section 7.7) are being collected, they are collected
from this keyword's subschema in the usual way, including when the
keyword is present without either "then" or "else".

When I wrote "in the usual way" I meant collecting them and propagating them up as normal if the if schema passes, and dropping all annotations from the if schema if it fails. That could perhaps be spelled out more clearly, but that is definitely what I meant.

Yeah, that's how I read the spec, and why I asked for the issue to be raised. I certinly didn't expect this to be the case, and I'd have raised a concern if I'd have understood the implications at the time.

The unevaluatedProperties case is one reason for that, as otherwise you have to duplicate some part of the if schema in the then schema in order to make it work, which is not good design.

I actually think this is preferable design, which I often suggest, but putting minimal content in then.
I always suggest people break up their validation into the conditional logic and remaining subschema application logic. I find it's cleaner, and allows for better decoupling of concerns. Plus, useful if you want to later break up your schema into multiple files or refactor.

including when the keyword is present without either "then" or "else" means that you can use a standalone if to conditionally collect some annotations without ever causing validation to fail. I don't think I had a use case in mind, but it felt more consistent, and earlier drafts failed to explain what a standalone if meant (other than that someone forgot a then or else, most likely).

I truly expected if on its own to have zero implications, and that it could be ignored.

---

However, having said all that, I can't think of an example of where this (as is currently) would be a problem, and I can think of potential use cases in terms of annotation collections beyond just for unevaluated* keywords.

This leaves me feeling a little conflicted.

[Relequestual]

The unevaluatedProperties case is one reason for that, as otherwise you have to duplicate some part of the if schema in the then schema in order to make it work, which is not good design.

I actually think this is preferable design, which I often suggest, but putting minimal content in then.
I always suggest people break up their validation into the conditional logic and remaining subschema application logic. I find it's cleaner, and allows for better decoupling of concerns. Plus, useful if you want to later break up your schema into multiple files or refactor.

Taking @jdesrosiers implication pattern example from the first post. If you remove the unevaluatedProperties, using anyOf doesn't make sense to me here. I would probably suggest something like this was written as...

[some time passes]

It appears I can't build schemas today...
I do normally find the implication pattern unintiutive, but I my gut tells me I'd have arrived at a different solution where the annotations would have come out as expected. (Given I have no evidence, it's not a pragmatic argument at all 😩)

[gregsdennis]

That seems to be the gist of it. Why do you ask?

It seems fine to me that then should respond to an annotation - the annotation implies the presence of if. This is how "keyword communication" seems to work.

Annotations are the mechanism by which keywords can know that other keywords are present. If no annotation is present, it should be equivalent to the keyword that generates it being absent.

But that's not the case for else. else shouldn't run if if isn't present (so no annotations), but it should run if if is present but still doesn't produce annotations.

In other words, else seems to be an oddball keyword that actually scans its own schema for the presence of another keyword. All of the other keyword interdependencies can be managed in the second keyword by whether annotations are produced by the first.

[jdesrosiers]

when the keyword is present without either "then" or "else" means that you can use a standalone if to conditionally collect some annotations without ever causing validation to fail.

That feels like an abuse of the keyword to me, but I could be convinced otherwise with a good example.

I actually think this is preferable design, which I often suggest, but putting minimal content in then. I always suggest people break up their validation into the conditional logic and remaining subschema application logic. I find it's cleaner, and allows for better decoupling of concerns. Plus, useful if you want to later break up your schema into multiple files or refactor.

💯 Completely agree

[karenetheridge]

In other words, else seems to be an oddball keyword that actually scans its own schema for the presence of another keyword.

That's not unique. maxContains and minContains are also dependent on the presence of another keyword (contains), and if it is not present, these keywords do nothing

[karenetheridge]

That feels like an abuse of the keyword to me, but I could be convinced otherwise with a good example.

Abuse is a bit negative.. how about "(sometimes) convenient side effect". I view the "not": { "not": { ... } } pattern similarly - the facct that it serves to hide annotations from the inner schema is useful, but the keyword wasn't designed for this purpose specifically. I'm sure we can all think of other similar side effects in our primary languages that are (ab)used as a pattern. There was a famous stackoverflow post on "the --> operator) (as in while (i --> 0) { // while i decreases to zero..) a while back which is another one of these cases.

[handrews]

Ooof. I am shocked to discover that section 7.2 reads:

Keyword behavior MAY be defined in terms of the annotation results of
subschemas (Section 4.3.5) and/or adjacent keywords (keywords within
the same schema object) and their subschemas. Such keywords MUST NOT
result in a circular dependency. Keywords MAY modify their behavior
based on the presence or absence of another keyword in the same
schema object (Section 4.3).

instead of (note the boldface addition):

Keyword behavior MAY be defined in terms of the annotation or assertion results of
subschemas (Section 4.3.5) and/or adjacent keywords (keywords within
the same schema object) and their subschemas. Such keywords MUST NOT
result in a circular dependency. Keywords MAY modify their behavior
based on the presence or absence of another keyword in the same
schema object (Section 4.3).

The whole point of this section is that you can define an execution order for certain keywords with respect to other keywords or other keyword classes (e.g. in-place applicators), and affect the later-executing keywords with the results of the earlier-executing keywords. While this is most notably done with annotation results, it was supposed to allow using either or both types of results.

• if runs before then or else
• prefixItems runs before items, which runs before unevaluatedItems
• properties and patternProperties run before additionalProperties, which runs before unevaluatedProperties
• in-place applicators run before unevaluated*

I might be missing something but you get the point.

I thought I had a very clear memory of writing that paragraph with both kinds of results, but I guess I only thought about it? It was something I figured out after first writing the part about annotations, so it's not too shocking to find that I never actually fixed it, I suppose.

However, the specification for if is very clear that the selection of then vs else is done based on the assertion result and not an annotation:

This validation outcome of this keyword's subschema has no direct
effect on the overall validation result. Rather, it controls which
of the "then" or "else" keywords are evaluated.

"validation result" here is synonymous with "assertion result," I'd always meant to go back and replace all of the "validation result"s with "assertion result"s as it is more generic, but apparently I never did.

I consider this a simple error in section 7.2. The phrase or assertion was intended to be in that sentence and got left out by error. I'm tempted to file it as errata on the draft, that's the degree to which I consider this to just be a bug/typo.

To sum up:

• Applying then vs else is done based on the assertion result of if
• If if is not present, then neither then nor else are ever applied
• if is always applied, and in terms of annotation results it is a totally normal subschema application

All of this is necessary for it (including interaction with unevaluated* to work at all. If you take any of it out the whole thing falls apart. Forbidding if on its own is more trouble than it's worth. It's weird to use it on its own and I don't know that I'd recommend it, but there are a lot of weird not-recommended things you can do in JSON Schema. This one isn't a problem, it's just a bit odd.

---

@Relequestual

Do you remember what the use case you had in mind for this was @handrews ?

Yes, the unevaluted* keywords. They only work if annotations are collected. I really thought the annotation collection behavior was quite clear. I made a point of calling it out specifically, just as I specifically called out that the assertion result was to be discarded after determining whether to apply then or else.

[handrews]

(and yes, I missed the contains keywords but @karenetheridge already noted those- thanks!)

[handrews]

@Relequestual

The unevaluatedProperties case is one reason for that, as otherwise you have to duplicate some part of the if schema in the then schema in order to make it work, which is not good design.

I actually think this is preferable design, which I often suggest, but putting minimal content in then.
I always suggest people break up their validation into the conditional logic and remaining subschema application logic. I find it's cleaner, and allows for better decoupling of concerns. Plus, useful if you want to later break up your schema into multiple files or refactor.

This doesn't make sense to me. If your condition is a test against property "foo", and your remaining logic has to do with property "bar", then schema authors "break[ing] up their validation into the conditional logic and remaining subschema application logic" would exactly mean putting the subschema for "foo" under if, and the subschema for "bar" under then, and not mentioning "bar" in if or "foo" in then.

I can definitely see an abstract argument that the entire if schema should have no results, but I think the concrete implications of doing so- that successfully evaluating a property under if would fail to affect unevaluatedProperties- are too problematic to ignore.

Part of the whole point of unevaluatedProperties was to make it so that you no longer had to keep putting "someProperty": true in odd places to make additionalProperties work.

[handrews]

@jdesrosiers

when the keyword is present without either "then" or "else" means that you can use a standalone if to conditionally collect some annotations without ever causing validation to fail.

That feels like an abuse of the keyword to me, but I could be convinced otherwise with a good example.

I get where that is coming from, but if you think of a standalone if as one that implicitly has a "then": true, "else": true attached to it, it's weird, but not really abusive. I don't actually remember why then and else don't default to true, although I do remember that trying to set defaults for if/then/else was more complicated than expected. But it also took us a couple of tries to nail down how it really worked in general, so maybe defaults would make sense now?