Secret Vault Schema #4
Description
Secret bundles (or whatever we shall call them), are just basically a basic filesystem. But to ensure we can have structured access, a schema must be applied to them.
Why would we want structured access? Well the fact that it is a filesystem and not just some binary blob already imposes some structure. The structure of files, directories, symlinks, hardlinks, permissions... etc. Actually we want to make this more specific. Properties like metadata and permissions of the files must be stripped when they are put into a secret bundle, they just don't make sense in the context of what we need. Even hardlinks might not make sense. However files, directories and symlinks should be fine. (Except on Windows). We can assume symlinks simply because we always work on our secret bundle in-memory via an in-memory filesystem (as provided by js-virtualfs). For other implementations they will need their own filesystem implementation.
However beyond just a filesystem, one has to consider what kind of files can be put into the filepaths. What file paths can exist, what are their names. If we put additional structure into it, we can offer more constraints over the "shape" of the data which gives any external consumers using Polykey more confidence/guarantees/type-safety over the key-values/info/secrets that they are putting into the secret bundles and taking out of the secret bundles. This means there is less of a need for discovery intelligence in the consumers (as in the consumers don't have to be too smart).
Here's an example. Say that within a secret bundle there is a single file:
/cards.json
The schema is saying that it must be a JSON file. Then we would not allow any consumer using the API to stick a PNG file and call it a JSON file. But even more than this, we can code up a plugin for these schemas. That is a JSON encoder/decoder, which means the API allows structured access the JSON file. It can pass a JSON dictionary, or even access a single attribute via an "attribute path" into the JSON file. Imagine using jq to directly acquire something within that file. Ideally we would not need to have plugins, since this means we are using magic strings like "JSON" to mean that there is something that knows about that format. Ideally the encoder/decoders should be derived from the schema itself. For example see: https://github.com/mozilla-services/react-jsonschema-form it uses the JSON schema and autogenerates HTML forms for React. There's also research in this from dependent types, OMeta, http://blog.rafaelferreira.net/2008/04/couple-of-interesting-dsls.html, https://categoricaldata.net/aql.html, boomerang, https://en.wikipedia.org/wiki/Bidirectional_transformation... etc. Consider this a low priority however.
Here are some examples of filesystem schemas:
There is a relationship between the schema language (and its associated tooling) and the ability to check, model, verify and generate marshalling interfaces (https://en.wikipedia.org/wiki/Marshalling_(computer_science)), also see how people write protobuf interfaces and autogenerate data marshalling libraries for given languages.
Consider all of this a low priority!