Revise public key metadata format

[lukpueh]

Description of issue or feature request:

#251 discusses various consistency concerns related to securesystemslib (TUF/in-toto) cryptographic keys.

This is an attempt to carve (and flesh) out only those concerns related to the format of public keys as they appear in TUF/in-toto metadata, with the goal of a clear specification likely as part (or appendix) of the new signing-spec and its reference implementation in securesystemslib.

Below in 'Current behavior' is an exhaustive list of available values for public keys as currently implemented in securesystemslib. This includes rsa, ecdsa, ed25519 and spx (WIP #169) keys generated with securesystemslib, plus keys exported from a GnuPG keyring using securesystemslib.gpg or from a cryptographic token (e.g. HSM) using securesystemslib.hsm (WIP #229).

Related concerns and calls for action are listed further below in 'Expected behavior'.

Current behavior:

Public key container (mandatory fields)

{
  'keyid' <...>, # Optional in the spec, but included in the in-toto reference implementation
  'keytype': <...>,
  'scheme': <...>,
  'keyval': {
    'private': <...>, # Must be omitted or empty for public keys
    'public': <...>
  }
}

'keyid'

• ascii hex of canonical JSON (OLPC) of parts of the key dictionary
• RFC4880 OpenPGP Issuer Fingerprint (gpg keys only)
• custom value (see WIP sphincs+ support, for post-quantum crypto #169 , TAP 12 and Implement TAP 12 theupdateframework/python-tuf#1084)

Note that 'keyid' is not part of the public key metadata format according to in-toto/TUF specifications and tuf reference implementation, but it is included in the in-toto reference implementation (see #251 PR description).

'keytype'

• 'rsa'
• 'ed25519'
• 'ecdsa'
• 'ecdsa-sha2-nistp(256|384)' (legacy, new keys use 'ecdsa', see Better distinguish between keytype and scheme for ECDSA keys #239 and Better distinguish between keytype and scheme for ECDSA keys #267)
• 'spx' (WIP sphincs+ support, for post-quantum crypto #169)

'scheme'

• 'rsassa-pss-(md5|sha1|sha224|sha256|sha384|sha512)'
• 'rsa-pkcs1v15-(md5|sha1|sha224|sha256|sha384|sha512)'
• 'ecdsa-sha2-nistp(256|384)'
• 'ed25519'
• 'spx' (WIP sphincs+ support, for post-quantum crypto #169)

'type'

(gpg keys only, instead of 'keytype')

• 'rsa'
• 'dsa'
• 'eddsa'

'method'

(gpg keys only, instead of 'scheme')

• 'pgp+rsa-pkcsv1.5'
• 'pgp+dsa-fips-180-2'
• 'pgp+eddsa-ed25519'

'hashes'

(gpg keys only, instead of 'scheme')

• 'pgp+SHA2'

'keyval'

'public'

• rsa, ecdsa, ecdsa from HSM (WIP Add PKCS11-based HSM interface #229)
  X.509 SubjectPublicKeyInfo PEM string (RFC5280)
• ed25519
  32 bytes ascii hex key data (RFC8032)
• spx (WIP sphincs+ support, for post-quantum crypto #169)
  64 bytes key data (SPHINCS+-SHAKE256)
• rsa gpg (RFC4880)

    {
      "e": <public encryption exponent (ascii hex)>,
      "n": <public modulus (ascii hex)>
    }
  

• dsa gpg (RFC4880)

  {
    "y": <public value (ascii hex)>,
    "p": <public modulus (ascii hex)>,
    "q": <group order (ascii hex)>,
    "g": <group generator (ascii hex)>
  }
  

• eddsa gpg (RFC4880)

  {
    "q": <MPI of an EC point representing a public key (ascii hex)>
  }
  

'private'

(NOTE: The 'private' portion is not directly relevant for this issue, as it must not be included in TUF/in-toto metadata. For the sake of completeness we still list the private key values that are currently kept internally in securesystemslib).

• rsa, ecdsa
  PKCS#1/TraditionalOpenSSL PEM string (RFC8017)
• ed25519
  32 bytes ascii hex key data (RFC8032)
• spx (WIP sphincs+ support, for post-quantum crypto #169)
  128 bytes key data (SPHINCS+-SHAKE256)**

Optional fields

• 'creation_time'
• 'validity_period'
• 'expires'
• 'keyid_hash_algorithms' (Implement TAP 12 theupdateframework/python-tuf#1084)

Expected behavior:

• 'keyid'

  • better decouple default keyids from keys, to allow the use of custom keyids, but with a canonical json computed keyid as fallback
  • clarify if keyid is optional in the key object

• Consolidate 'keytype', 'type', 'scheme', 'method', 'hashes' fields and values

  • 'scheme' usually includes key type, signing scheme and hash algorithm, could one field be enough?
  • revise correctness/appropriateness of values (see e.g. 'eddsa' vs. 'ed25519' as different '(key)type's for the same type of key).
  • is there a standard for 'scheme' strings that we could use?

• 'keyval'

  • consolidate values (see e.g. serialized rsa PEM vs. non-serialized rsa numbers)
  • clarify if 'private' field may be omitted or should be just empty for public keys

• optional fields

  • deprecate 'keyid_hash_algorithms'
  • maybe consolidate 'validity_period' and 'expires'?