Is there a way to define additional metadata in JSON-LD @context?

[jechterhoff]

Hello,
I'm currently analysing JSON-LD 1.1 (reviewed the Final Community Group Report) with the goal of semantically enabling JSON data. Is there a way to define additional metadata in a JSON-LD context?

(I hope it is ok to ask this question here. If I should ask somewhere else, for example the mailing list, let me know, please.)

Let me explain what I mean with an example:

{
 "@context": {
  "@base": "http://example.org/baumregister/",
  "@version": 1.1,
  "xsd": "http://www.w3.org/2001/XMLSchema#",
  "geosparql": "http://www.opengis.net/ont/geosparql#",
  "ex": "http://example.org/ontology/flora/",
  "ort": "geosparql:hasGeometry",
  "wkt": {
   "@id": "geosparql:asWKT",
   "@type": "geosparql:wktLiteral"
  },
  "eid": "@id",
  "art": "@type",
  "Eiche": "ex:oak",
  "Walnuss": "ex:walnut",
  "hoehe": {   
   "@id": "ex:height",
   "@type": "xsd:double",
   "@derivedBy": "A specific description ...",
   "uom": "m"
  },
  "alter": {
   "@id": "ex:age",
   "@type": "xsd:integer"
  }
 },
 "art": "Eiche",
 "hoehe": "16",
 "eid": "08218adf-7947-4f28-bcaf-e069ef43e012",
 "alter": 242,
 "ort": {"wkt": "POINT(8.191035,51.899666)"}
}

In this example, the key "hoehe" expands to the IRI http://example.org/ontology/flora/height. In addition, the value type is defined as http://www.w3.org/2001/XMLSchema#double.

In the example, I've added two keys as metadata to the definition of term "hoehe":

• "@derivedBy" - The value is supposed to contain a textual description. I used the leading "@" on purpose for testing.
• "uom" - An indication of the unit of measure for "hoehe".

When I test this on the JSON-LD dev playground (a great tool, by the way - so helpful!), I get the following syntax errors:

• For "@derivedBy": Invalid JSON-LD syntax; a term definition must not contain @derivedBy
• For "uom": Invalid JSON-LD syntax; a term definition must not contain uom

When the two keys are removed, the example "works".
In the JSON-LD 1.1 specification I find the following:

An expanded term definition MUST be a JSON object composed of zero or more keys from @id, @reverse, @type, @language, @context, @Prefix, or @container. An expanded term definition SHOULD NOT contain any other keys.

Apparently I should not use the two additional keys, but they do not appear to be explicitly forbidden either. Now I am wondering if the dev playground is too strict or not. Can someone please enlighten me as to where in a JSON-LD context I can add custom keys - if at all - and what restrictions, if any, I need to be aware of? Any additional insight would be highly appreciated.

I did come across the following two issues, which may be related to my question:

• Warn or error if non-keyword strings having "@" are encountered
• Reserved keywords

[gkellogg]

We have become more prescriptive about what can be in a context, but could consider issuing warnings for non-keyword-like keys (e.g., "uom" in your example) and issue a warning. It might be better to tackle this with a new keyword, such as @comment, which would be explicitly allowed and content ignored.

[azaroth42]

@comment only in contexts? Otherwise we should add it to the round tripping discussion.

[BigBlueHat]

The uom (Unit of Measure) thing is interesting, but I'm not sure the context is the best place for that--as it seems data related--i.e. you'd want this in the output graph somewhere.

Perhaps something like https://schema.org/QuantitativeValue (or similar) would be useful here?

{
  "hoehe": {
    "@type": "schema:QualitativeValue",
    "value": 6,
    "unitText": "m"
  }
}

[azaroth42]

Related to #31 in terms of context adding actual data, which is currently out of scope (and I feel should remain out of scope).

[jechterhoff]

Thank you all for your responses and comments so far.

Having @comment as new keyword would be very useful, I think. In my example, it could replace @derivedBy. In my opinion, it would be useful to allow @comment on any level of nesting within a JSON-LD context.

Regarding the "uom" from the example: My use case is to semantically enable JSON data, by adding an external JSON-LD context document as described in section "JSON data as JSON-LD" from the JSON-LD final community group report. I do not intend to add any actual data to the JSON data through the JSON-LD context. That might cause issues with digital signatures of the data.

The idea of having the "uom" within the definition of term "hoehe" in the example was to allow any applications that process the context to use the information of this application specific metadata to correctly interpret the height value - without requiring the actual data to contain the unit of measure. However, "uom" is just an example. I was thinking that a term definition could contain any non-keyword-like keys which might be useful for a specific application or community. An application that parses the context should simply ignore such keys when they are encountered in a term definition and are unknown to the application. The JSON-LD playground could issue a warning when it finds such keys, to emphasize what is currently defined by the standard: that an expanded term definition should not contain any keys other than the keywords defined by the standard. However, when application specific extensions of term definitions are not explicitly forbidden, then the note in the standard could actually be changed to:

An expanded term definition MUST be a JSON object composed of zero or more keys from @id, @reverse, @type, @language, @context, @Prefix, or @container. The object MAY contain non-keyword-like keys. Each non-keyword-like key that is unknown to an application that processes the term definition MUST be ignored. The application MAY issue a warning if such a key is encountered.

[azaroth42]

Part of the way towards consensus about this issue, is the resolution not to add comment type data into graph instance data.

The remaining discussion is whether or not to allow comments in contexts and frames. Allowing comments in context data would allow embedded micro-contexts within instance data that contained only a comment (c.f. @language rationale in #5).

WG members with opinions are to add their proposal and rationales to the issue before the call on 2018-08-03.

[msporny]

-1 to adding @comment.

I get the use case, but adding a new JSON-LD keyword such as this, while easy from an implementer perspective creates added cognitive burden for authors and potentially problematic outcomes (your comment moves to a place you didn't intend it to move to). There is yet another keyword that they have to know.

Comments should be in Vocabulary documents and other human-readable documents. JSON-LD Context files and documents are not intended to be human-readable... especially since we can't place @comment by the item that was originally intended due to not being able to control where it ends up when the JSON object is reserialized.

[azaroth42]

I am +1 to adding @comment as a reserved keyword, valid in the scope of contexts only. The value of @comment could be any valid JSON construction, and JSON-LD processors MUST ignore the value.

Rationale:

• All (?) other RDF serializations have syntax level commenting. JSON does not have syntax level commenting, requiring any data not intended for machine processing to be embedded within the same representation in "magic" fields such as @comment. We control the schema for contexts and frames, and thus we are the correct authority to determine where such commentary could be included. This would be bringing JSON-LD up to parity with the functionality of other syntaxes.
• It gives a consistent place to put comments intended for humans looking at the data, such that they are better able to understand the intent or usage.
• It gives a consistent place to put metadata about the document, such as creator and date of last modification.
• It gives a consistent place to put out of band information that is guaranteed not to cause an error or collide with other context-defined mappings. For example, when exporting data from a system, it is useful to include internal metadata not intended for JSON-LD processors but instead for other components in that pipeline. In a generic pipeline, there would be no guarantee that the key chosen did not collide with context-defined mappings. That pipeline might span across systems.

I disagree with @msporny that this adds a significant burden for developers:

• It is intended to make it easier for developers to understand the data
• It is (I think) self-evident as to its intent ... more so than some existing keyword usage (e.g. the confusion caused by overloading of @type for both datatype and class experienced by many members of the WG, per the previous call)
• It would be optional. If the audience of a particular context is solely machines, and no human ever needs to develop using the serialization, then don't put in comments. This is not true for many communities, however, and it would be better to have consistency than all of them using different hacks to work around the syntax level lack of functionality.
• Unless the comment is intended to be associated with a literal, it can always be within the JSON object, and thus be serialized with it.

I also try to address some arguments against:

• It doesn't survive round-tripping through other RDF syntaxes, when put into the @context of a node within an instance -- Yes, that's intentional not a bug :) If you need something like that, then put it in the data.
• JSON shouldn't have comments -- Yet everyone asks for them, and uses them in other machine readable syntaxes.
• Just put it in the body of the context document -- this works for context documents, but there is the convention of putting the ontology within the body, meaning that the data might collide. Also, it would mean mapping into RDF in order to put arbitrary structure into the data, and defining those terms in the context.
• Just put it in external documentation -- this is a good computer science solution, but not a good practical one. In this case there is no link to the next level of abstraction from the context document found on the web to the documentation for that context. That problem could be solved separately, but would not meet all of the use cases described above.

[ajs6f]

Is there any serious effort currently afoot of which anyone knows to introduce a comment facility for JSON?

[msporny]

All (?) other RDF serializations have syntax level commenting. JSON does not have syntax level commenting, requiring any data not intended for machine processing to be embedded within the same representation in "magic" fields such as @comment. We control the schema for contexts and frames, and thus we are the correct authority to determine where such commentary could be included. This would be bringing JSON-LD up to parity with the functionality of other syntaxes.

The point is that the base syntax doesn't have comments. I'll note that JSON still doesn't have comments and developers that use it are just fine without them.

It gives a consistent place to put comments intended for humans looking at the data, such that they are better able to understand the intent or usage.

No, it's not consistent... @comment can find itself anywhere in the object it's included in and can only be used once per object. It doesn't work like a comment that most developers are familiar with (that is, something that sticks next to the code for which it was intended). This @comment can move around depending on the implementation that's reading it. So, calling it @comment is bad because it doesn't actually act like a comment that most developers are used to.

It gives a consistent place to put metadata about the document, such as creator and date of last modification.

How do you know it's metadata about the document instead of metadata about the field it is next to? Won't this just force people to create microsyntaxes for the comment like "Author: J. Doe, Date: 2018-10-19"?

It gives a consistent place to put out of band information that is guaranteed not to cause an error or collide with other context-defined mappings. For example, when exporting data from a system, it is useful to include internal metadata not intended for JSON-LD processors but instead for other components in that pipeline. In a generic pipeline, there would be no guarantee that the key chosen did not collide with context-defined mappings. That pipeline might span across systems.

If the metadata and pipeline spans across systems, it is no longer internal. JSON-LD is the mechanism that can be used for annotating system data. That said, if this is a use case, we should write it up and then see if this new mechanism is the best way to solve it.

Comments are supposed to be metadata that is out of band of the data structure. This suggests that we put something in band of the data structure that is metadata about the data structure. Feels like the wrong level of abstraction.

[msporny]

I think a better feature here would be explicit drop support for JSON-LD Processors... as in, you see these terms, drop them... This may be useful when we go to making JSON-LD Signatures throw errors for any unknown value... in some cases, people may not want to map the value, but rather drop the value.

That said, I hesitate to suggest that as a feature at this point in time. The only reason I'm doing it is that it would prevent the addition of "@comment" with a more useful feature that can be used for things that are more than just comments.

[azaroth42]

I agree that developers get by without comments in JSON ... clearly. However that the stackoverflow question was asked 10 years ago, and continues to be active sends the signal to me that it's still of keen interest: https://stackoverflow.com/questions/244777/can-comments-be-used-in-json
Viewed 1.5 million times, with thousands of up votes, suggests that it's desirable.
The top ranked workaround is, of course, to create _comment ... which is what we would be standardizing for JSON-LD contexts.

By consistent, I meant (consistently) that it would be the same across contexts. If we don't define it, then thousands of implementers will pick something different to use. I agree it's not consistent with other syntaxes commenting functionality, and that unordered keys makes the comment node unreliable in position. Happy to call it @ignored or @outofband or anything else.

The pipeline use case came up here where the system needs additional information when processing the JSON-LD input to correctly manage it, due to internal-to-the-implementation limitations. I agree that it generates micro-syntaxes, but I would rather these meta- level microsyntaxes live somewhere that is trivial to ignore rather than having to determine if they're real syntax or serialization level annotations.

I could also see the functionality being implemented with @documentation that takes a URI as value. Then humans can go and read the docs, and machines can do conneg to get something useful to them.

[hsolbrig]

Comments can also serve as extension points -- either explicit such as appinfo in XML Schema or by agreed-upon inner syntax, as exemplified by decorators in Java or quoted strings in python. I wonder whether we couldn't kill both birds (keeping the namespace clean versions and allowing extensibility) by allowing any key starting with '@@'?

One might also note that YAML supports comments.