Key-based item uniqueness

[gregsdennis]

It may be handy to be able to specify key-based uniqueness within an array of items.

Given the schema

{
  "type" : "array",
  "items" : {
    "type" : "object",
    "properties" : {
      "key" : { "type" : "string" },
      "value" : { "type" : "integer" }
    }
  },
  "uniqueItems" : true
}

the instance

[ { "key" : 1, "value" : "value1" }, { "key" : 1, "value" : "value2" } ]

would pass. However, the user may want this to fail as the value of the key property is repeated.

I propose a uniquenessKey (or similar) keyword that would allow the author to specify a pointer to an object property that should be unique among all items within the array. This would update the above schema to

{
  "type" : "array",
  "items" : {
    "type" : "object",
    "properties" : {
      "key" : { "type" : "string" },
      "value" : { "type" : "integer" }
    }
  },
  "uniquenessKey" : "#/key"
}

(The pointer would be resolved using the item in the array as the root.)

[handrews]

This has some of the same difficulties as #21 ordered which also discusses possibly ordering on a key. We should probably tackle them together. These are in what I consider the grey zone between "yeah let's add that to the standard vocab" and "hmm... definitely see the use, but can easily become either too complicated to implement or too limited to be useful, so maybe extension vocabularies should play with this first."

I'm hoping as we improve our notion of what a vocabulary is and how to organize them, it will become more clear where this and ordered best fit.

[TakingItCasual]

Perhaps a way to do this would be to add an objectArray type? For example converting the following:

{
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "Name": {"type" : "string"},
            "Desc": {"type" : "string"}
        },
        "required": ["Name", "Desc"]
    },
    "minItems": 1,
    "uniqueItems": true
}

To something along the lines of the the following:

{
    "type": "objectArray",
    "properties": {
        "Name": {"type" : "string"},
        "Desc": {"type" : "string"}
    },
    "required": ["Name", "Desc"]
    "minItems": 1,
    "uniqueItems": true
}

A keyword such as uniqueKeys (or uniqueKey with just one value, if that makes more sense) could be used with this type to achieve the desired effect.

[gregsdennis]

@TakingItCasual I'm not sure how your suggestion solves the problem of key-based uniqueness. You seem to be merely adding a new type keyword as a shorthand for functionality that already exists.

[lalloni]

We are right now needing almost exactly what @gregsdennis proposed except that we must have composed keys so the "uniquenessKey" should be an array.

Having said that, I find very odd that the pointer points "#" to an element in the array somwhere in the instance... which would be very confusing to users, having all other pointers refering to the root of the schema as "#", I believe. Maybe there's no other way, though.

[handrews]

@lalloni

Having said that, I find very odd that the pointer points "#" to an element in the array somwhere in the instance... which would be very confusing to users, having all other pointers refering to the root of the schema as "#", I believe. Maybe there's no other way, though.

If we do that, that should be done with a Relative JSON Pointer, not changing the meaning of #.

[gregsdennis]

We could do something like what JSON path does for array predicates: use @ instead of # to indicate a local root instead of the global root. It's not a proper pointer, but it makes the distinction well enough.

[handrews]

@gregsdennis Or we could just use Relative JSON Pointer which is designed specifically to solve this problem, is one of the specs we publish, and is used throughout JSON Hyper-Schema already.

[rrodini]

This would be very useful.

[ralfhandl]

This would be very useful, and uniquenessKey needs to accept an array of Relative JSON Pointers to also cover the multi-part key case.

[senior-pomidorka]

This would be very useful, and uniquenessKey needs to accept an array of Relative JSON Pointers to also cover the multi-part key case.

Totally agree. Would be really useful

[awwright]

I'm skeptical about the effectiveness of things like this for a few reasons:

• The primary role of "uniqueItems" is to signify that an array is functioning as a mathematical set, and so array items should be unique. It's also one of the most difficult keywords to validate.

• Verifying whether or not some data exists elsewhere (in the document, or another database) is somewhat outside the scope of JSON Schema. "uniqueItems" is an exception, and it has exceptionally poor performance. And I'm concerned that if we start adding keywords like this, it would encourage people to design schemas in a way that isn't ideal, by forcing too much data into the same document, when maybe it should be separated. See json-schema-org/json-schema-spec#549

[handrews]

@awwright if we were adding uniqueItems now I would lobby hard to make it an annotation and not an assertion, and leave it up to the application layer to decide whether and how to validate it.

[gregsdennis]

@awwright It's also one of the most difficult keywords to validate.

While uniqueItems is difficult to implement, uniquenessKey would make things easier because you only have to check that the values at the keys are unique rather than the whole item.

To your second point, I'm not proposing that this has any relation to data in schemas. I think that item uniqueness holds a valid place within a schema, and identifying that uniqueness based on a property within the item (for example an ID) is a worthy addition.

[speedplane]

I found this issue after asking a question on SO. The view count (currently 21 views after 7 days) may provide insight into prioritizing this issue.

[Relequestual]

@handrews @awwright In terms of

Verifying whether or not some data exists elsewhere (in the document, or another database) is somewhat outside the scope of JSON Schema.

This is often needed, and often if checks need to be pushed up a few levels to enable what's required.

@awwright if we were adding uniqueItems now I would lobby hard to make it an annotation and not an assertion, and leave it up to the application layer to decide whether and how to validate it.

@handrews

(Assuming you do not mean uniqueItems as that's already a keyword, and mean some new uniqueItemsByKey keyword) I would very much prefer NOT adding this as an annotation only. Doing so would create confusion, and I would find it preferable to take a stance which says "no, you must do this in creating a new 'database' sort of vocabulary. probably need to collaborate with mongodb". If it was only an annotation, I'm sure there are PLENTY more database type annotations that people might want, and we're not in the best place to see or collect them right now.

I personally don't feel this is unreasonable, however I don't think the number of people who've requested it is high enough to divert focus on remaining tasks for draft-8. (This issue is currently not in the draft-8 milestone. Let's also discuss if we should move it there or not.)

If the value was simply a string, which represented the value in the object to be checked, it sounds fairly simple to me. I don't think we need to allow for pointers. Values which are ids for objects should be at the top most layer, and doing so reduces the complexity of the task here.