Is there a need for both an authored and a canonical manifests?

[iherman]

At the moment, there is an authored and a canonical manifest, with a separate canonicalization step to transform the authored manifest into the canonical one. The goal is to allow the author to express data more succinctly (eg, use only simple file names instead of complete LinkedResource instances or person names instead of Person structures).

It was raised, in #11, that the price being paid for having this is too high:

For the few people typing these by hand, [...] but for the vast majority of implementers (i.e. CMS's generating these manifests), I think they'd find the consistency of using the canonical representation (and the lack of "overhead" of needing it to be canonicalized every time...) to be a win. (#11 (comment))

Question: do we want to simplify the manifest by removing this extra step and defining the manifest purely in terms of what is currently called the canonical manifest?

[iherman]

Two relevant comment from the other issue:

• Canonicalization algorithm should incorporate @base #11 (comment)
• Canonicalization algorithm should incorporate @base #11 (comment)

[iherman]

Cc: @dauwhe @mattgarrish @llemeurfr @wareid @RachelComerford @GarthConboy @laudrain

[iherman]

My own opinion: as a technical person, I would be happy to use a canonical manifest only, i.e., to remove the notion of an authoring manifest. It would undeniably simplify the spec and make the technical content cleaner.

This is, however, a usability issue, and the question really relies on what the community would accept or not; I do not have the necessary experience to have an informed opinion on this.

[laudrain]

IMO, I don't see "people typing these by hand" (@BigBlueHat).
Our ebooks are build using data transformation processes or export plugins, then I'm in favor of only one manifest expression, the canonical one.

[llemeurfr]

For reading systems, the mapping from the authored manifest to the canonical one is undoubtely some work. We're doing exactly that, from the "authored" Readium Webpub manifest to the in-memory object processed by reading software.

For instance
"author": "James Joyce"
will become the in-memory equivalent of
"author": [
{ "name": ["en": "James Joyce"],
"identifier": null,
"sortAs": null,
"links": null
}
]

This is not a huge work however. Those curious can look at the Readium Mobile / Kotlin code:
https://github.com/readium/r2-shared-kotlin/blob/develop/r2-shared/src/main/java/org/readium/r2/shared/Contributor.kt#L76

A JSON schema capable of dealing with flexible structures is also doable; this is done e.g. in https://github.com/readium/webpub-manifest/blob/master/schema/contributor-object.schema.json.

In conclusion yes, there is a price to this flexibility but developers can overcome it.
So the question is more: would the group feel better specifying
"author": [ { "name": ["en": "James Joyce"]}] ?

Re. other shortcuts in the authored manifest, there is also the question of JSON-LD types (e.g. "type": "LinkedResource"). The @type associated with the property definition in the context file is enough to define the type and I don't see why "type": "LinkedResource" should be added in each resource.
The type issue only happens for contributors, as we can't know for sure if it is a Person or Organization from the context. But that is another story.

[iherman]

@llemeurfr

In conclusion yes, there is a price to this flexibility but developers can overcome it.
So the question is more: would the group feel better specifying
"author": [ { "name": ["en": "James Joyce"]}] ?

Exactly. That is the core question.

[mattgarrish]

Wasn't the main motivator for the canonicalization steps (relating to expanding values, at least) that schema.org metadata values aren't rigidly enforced, so whether machines generate the manifest or not, there's the problem of whatever creates the file following general conventions, like: "author": "So and So"

I thought the idea was to be practical and have the steps to sanitize the data for the user agent rather than fail on it or not process it, since the compact forms don't seem to hinder seo processing?

But if we go this route, we should be thorough about it and also remove the steps about obtaining information from the html document that references/embeds the manifest.

We should figure out why we want/need full json-ld conformance, too, when we don't expect user agents to be json-ld processors.

[iherman]

Wasn't the main motivator for the canonicalization steps (relating to expanding values, at least) that schema.org metadata values aren't rigidly enforced, so whether machines generate the manifest or not, there's the problem of whatever creates the file following general conventions, like: "author": "So and So"

That was certainly part of it. I suspect schema.org has some similar procedure as our canonicalization.

That being said, if we decide we are stricter than schema.org this does not "harm" our data v.a.v. schema.org.

But if we go this route, we should be thorough about it and also remove the steps about obtaining information from the html document that references/embeds the manifest.

Oops, that is true. If we do not do any canonicalization at all, this is a consequence... The biggest "loss" would be the potential reuse of the <title> element.

We should figure out why we want/need full json-ld conformance, too, when we don't expect user agents to be json-ld processors.

By "full conformance" I presume you mean being able to use all JSON-LD feature, right? At this moment I do not see any reason for having it…

[dauwhe]

In case of conflict, consider users over authors over implementors over specifiers over theoretical purity.

[mattgarrish]

By "full conformance" I presume you mean being able to use all JSON-LD feature, right? At this moment I do not see any reason for having it…

Right, we've never said that a user agent must be a fully-conforming json-ld processor, only that it be able to process the json in the manifest into an internal structure. Maybe I'm wrong, but json-ld only seemed to enter the equation as a means of allowing search crawlers to get at the information without duplicating the metadata. (Is that (still) a primary use case?)

In terms of discrepancies between the graphs, sure, I don't think it's ideal, either, but I thought the point was finding a balance? If you don't care about the seo angle, or it's not relevant to your format, you aren't burdened with strict authoring of json-ld. If it does matter, you can choose to be stricter in authoring.

But maybe this is where layering comes in if we want to formalize this duality.

[iherman]

Maybe I'm wrong, but json-ld only seemed to enter the equation as a means of allowing search crawlers to get at the information without duplicating the metadata. (Is that (still) a primary use case?)

That is certainly the main motivation, yes.

One could imagine a full JSON-LD engine which would then also allow a simple and frictionless extensibility of the metadata by adding new terms, vocabularies, etc, using the JSON-LD facilities. This is what, e.g., Verifiable Claims do. But I do not see that as a use case for the publication manifest, at least not in this version.

[mattgarrish]

But I do not see that as a use case for the publication manifest, at least not in this version.

I'm admittedly not too keen on another major overhaul of the specification at this stage. We got where we are because there wasn't consensus on requiring strict authoring, so reversing course now seems a bit fraught. We'd be undoing a lot.