Generated `user-data` is failing cloud-config schema validation

[afbjorklund]

Description

There are some reports in the log, about the user-data failing validation.

[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.

We should probably run yaml and jsonschema validation, on the user-data?

But at least fix the syntax errors, and try to conform to the cloud-init schema...

[afbjorklund]

Note that changing to "ssh_authorized_keys" will actually fail some alternative cloud-init implementations...

Such as lima-init, or tinycloudinit. They have "ssh-authorized-keys" hardcoded, in their alternate parsers (awk)

Seems to be a (python) bug in the cloud-init documentation:

• https://bugs.launchpad.net/cloud-init/+bug/1420018

[jandubois]

Such as lima-init

We can easily fix lima-init, but don't know about tinycloudinit. I did the same already for ca-certs and ca_certs: https://github.com/lima-vm/alpine-lima/blob/5a09f68c35124ba6049ec15fd972991c445856f7/lima-init.sh#L129-L130

awk -f- "${LIMA_CIDATA_MNT}"/user-data <<'EOF' > ${LIMA_CA_CERTS}
# Lima currently uses "ca-certs", which is deprecated and should be "ca_certs"
/^ca.certs:/ {
…

[afbjorklund]

The tinycloudinit was just a hack that I used when playing with tinycorelinux, it is a similar awk script (with similar fix)

https://github.com/spinto/tinycloudinit

---

EDIT: fixed on master now (by checking for both dashes and underscores...), along with some other bugs and features

afbjorklund/tinycloudinit@e44d4e0

[afbjorklund]

https://raw.githubusercontent.com/canonical/cloud-init/main/cloudinit/config/schemas/schema-cloud-config-v1.json

Example validation with jv -annotations

$ jv https://raw.githubusercontent.com/canonical/cloud-init/main/cloudinit/config/schemas/schema-cloud-config-v1.json cloud-config.yaml 
[I#] [S#] doesn't validate with https://raw.githubusercontent.com/canonical/cloud-init/main/cloudinit/config/schemas/schema-cloud-config-v1.json#
  [I#] [S#/allOf/8] allOf failed
    [I#] [S#/allOf/8/$ref] doesn't validate with '/$defs/cc_ca_certs'
      [I#/ca_certs] [S#/$defs/cc_ca_certs/properties/ca_certs/$ref] doesn't validate with '/$defs/ca_certs.properties'
        [I#/ca_certs/trusted] [S#/$defs/ca_certs.properties/properties/trusted/type] expected array, but got null
  [I#] [S#/allOf/49] allOf failed
    [I#] [S#/allOf/49/$ref] doesn't validate with '/$defs/cc_users_groups'
      [I#/users/0] [S#/$defs/cc_users_groups/properties/users/items/oneOf] oneOf failed
        [I#/users/0] [S#/$defs/cc_users_groups/properties/users/items/oneOf/0/type] expected string, but got object
        [I#/users/0] [S#/$defs/cc_users_groups/properties/users/items/oneOf/1/type] expected array, but got object
        [I#/users/0] [S#/$defs/cc_users_groups/properties/users/items/oneOf/2/$ref] doesn't validate with '/$defs/users_groups.user'
          [I#/users/0] [S#/$defs/users_groups.user/additionalProperties] additionalProperties 'ssh-authorized-keys' not allowed

https://github.com/santhosh-tekuri/jsonschema

[afbjorklund]

Checking for deprecated and changed fields actually requires both enabling "annotations" and patching jsonschema...

        "uid": {
          "description": "The user's ID. Default value [system default]",
          "oneOf": [
            {
              "type": "integer"
            },
            {
              "type": "string",
              "changed": true,
              "changed_description": "The use of ``string`` type is deprecated. Use an ``integer`` instead.",
              "changed_version": "22.3"
            }
          ]
        }

The "changed" feature seems to have been invented by cloud-init, since I can't find it on https://json-schema.org/

There is a very similar "deprecated" feature that handles removal, but doesn't cover haphazard renaming of fields.

[afbjorklund]

As mentioned in the PRs, the revision v1 (or main) of this jsonschema is actually quite a lot of versions.

And there are even more variants of cloud-config, if you count the years before the jsonschema existed...

So we should not change the type of these, just because they are deprecated (or outright missing):

users.uid, users.ssh-authorized-keys - since it breaks stuff, and some lima distributions stop working

[afbjorklund]

Related issue:

• Add templating support for cloud-init in lima.yaml #2128

[afbjorklund]

The lima generated cloud-config.yaml now passes validation with cloud-init schema,
after the ssh-authorized keys was deprecated in schema (24.2~1g370e680c-0ubuntu1)

check-jsonschema --schemafile schema-cloud-config-v1.json cloud-config.yaml
https://github.com/canonical/cloud-init/blob/370e680c/cloudinit/config/schemas/schema-cloud-config-v1.json

• Generate separate cloud-config.yaml for when not using cidata.iso #2268