Add os_purpose field. (#970)

Also add a link to OpenStack specs for os_distro and explain why we
need it.

This is patch 1 from several. It adds this os_purpose field as
*recommended*. As it does not break certification, this can be done in
the existing v1 of the scs-0102 standard. I suggest we discuss this one
first until we are aligned. In parallel, I'd like to prepare a blog
article that serves as user guide how to select images in a portable way
without relying on the name. We can reference it in tne implementation
notes. We should also enhance the image-manager input to carry this
field soon.

In a future patch, we create a v2 of scs-0102 which *requires* the
os_purpose field. The naming in scs-0102 is already a recommendation
only, so that can remain. scs-0104 will need a new version then,
as it does use the recommended names as mandatory for the mandatory
images.

Signed-off-by: Kurt Garloff <[email protected]>
Signed-off-by: Marvin Frommhold <[email protected]>
Signed-off-by: Matthias Büchse <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Co-authored-by: Marvin Frommhold <[email protected]>

Author: 3 people

Standards/scs-0102-v1-image-metadata.md

@@ -6,7 +6,8 @@ status: Stable
 track: IaaS
 replaces: Image-Metadata-Spec.md
 description: |
-  The SCS-0102 Image Metadata Standard outlines how to categorize and manage metadata for cloud-based operating
+  This is version 1.1 of the SCS-0102 Image Metadata Standard.
+  It outlines how to categorize and manage metadata for cloud-based operating
   system images to ensure usability and clarity. The standard encompasses naming conventions, technical requirements,
   image handling protocols including updating and origin, and licensing/support details. These guidelines ensure
   that users can understand, access, and utilize OS images effectively, with clear information on features, updates,
@@ -65,14 +66,51 @@ The following property is recommended:
 
 - `hypervisor_type`
 
+The values for `architecture` and `os_distro` and `hypervisor_type` (the latter only if specified) values
+must follow the [OpenStack specifications](https://docs.openstack.org/glance/2025.1/admin/useful-image-properties.html).
+The `os_version` string should be numeric if the distribution uses numbers, the pair `os_distro` `os_version` should
+for example be `ubuntu` `24.04` for Ubuntu Noble Numbat 24.04[.x] LTS.
+
+To allow the distinction between general purpose images (which should come from upstream with at most some
+targeted adjustments as required by the cloud such as e.g. drivers) and images that are purpose-built, we
+recommend an additional field:
+
+- `os_purpose`
+
+  The following values are allowed
+
+  | `os_purpose` value | Intention                                                 |
+  |--------------------|-----------------------------------------------------------|
+  | `generic`          | A general purpose image, (mostly) vanilla from upstream   |
+  | `minimal`          | A much more barebones general purpose image               |
+  | `k8snode`          | Node image built for k8s with CRI and kubelet             |
+  | `gpu`              | Image with GPU drivers e.g. for HPC or AI                 |
+  | `network`          | Network appliance (firewall, router, loadbalancer, ...)   |
+  | `custom`           | None of the above                                         |
+
+  Note that no other values are currently allowed and `custom` should be used in case
+  of doubt. Talk to the SCS standardization bodies if you'd like to see this list extended which is
+  likely the case if you fall back to `custom`.
+
+  The usage of standardized `os_distro`, `os_version`, `architecture`, and `os_purpose` help cloud users to create
+  automation that works across clouds without requiring image names to be standardized.
+
+  _Uniqueness requirement_: whenever there are two images that have `os_hidden=False`, `visibility=public`,
+  and that coincide in all three fields `os_distro`, `os_version`, and `architecture`, then only one of them may
+  have `os_purpose=generic`. In other words, users who search visible public images for a generic OS
+  of a certain distro, version, and architecture will not get more than one result.
+
+  Note: Expect this field to become mandatory in v2 of this standard sooner rather than later.
+
 The following further properties are recommended (if the features are supported):
 
 - `hw_rng_model`
 - `os_secure_boot`, `hw_firmware_type`
 - `hw_watchdog_action`, `hw_mem_encryption`, `hw_pmu`, `hw_video_ram`, `hw_vif_multiqueue_enabled`
 
 The `trait:XXX=required` property can be used to indicate that certain virtual hardware
-features `XXX` are required.
+features `XXX` are required which may be advertised in matching
+[flavor extra specs](https://docs.openstack.org/nova/latest/user/flavors.html#extra-specs).
 
 ## Image handling
 
@@ -239,3 +277,11 @@ A boolean property that is not present is considered to be `false`.
   contact for issues with this image. Note that this field must only be set if the
   service provider does provide support for this image included in the image/flavor
   pricing (but it might be provided by a contracted 3rd party, e.g. the OS vendor).
+
+### Version history
+
+- Version 1.0 has existed without notable changes since June 2021.
+- Version 1.1 was created in preparation for a new major version 2.0 and has the following additional recommendations:
+  - Reference OpenStack image spec for standard values of `os_distro`, `architecture` and `hypervisor_type`.
+  - Recommendation on `os_version` to be a version number (if such a value exists).
+  - Recommended field `os_purpose`.

Tests/iaas/openstack_test.py

@@ -24,6 +24,7 @@
 from scs_0102_image_metadata.image_metadata import \
     compute_scs_0102_prop_architecture, compute_scs_0102_prop_hash_algo, compute_scs_0102_prop_min_disk, \
     compute_scs_0102_prop_min_ram, compute_scs_0102_prop_os_version, compute_scs_0102_prop_os_distro, \
+    compute_scs_0102_prop_os_purpose, \
     compute_scs_0102_prop_hw_disk_bus, compute_scs_0102_prop_hypervisor_type, compute_scs_0102_prop_hw_rng_model, \
     compute_scs_0102_prop_image_build_date, compute_scs_0102_prop_image_original_user, \
     compute_scs_0102_prop_image_source, compute_scs_0102_prop_image_description, \
@@ -90,6 +91,7 @@ def make_container(cloud):
     c.add_function('scs_0102_prop_min_ram', lambda c: compute_scs_0102_prop_min_ram(c.images))
     c.add_function('scs_0102_prop_os_version', lambda c: compute_scs_0102_prop_os_version(c.images))
     c.add_function('scs_0102_prop_os_distro', lambda c: compute_scs_0102_prop_os_distro(c.images))
+    c.add_function('scs_0102_prop_os_purpose', lambda c: compute_scs_0102_prop_os_purpose(c.images))
     c.add_function('scs_0102_prop_hw_disk_bus', lambda c: compute_scs_0102_prop_hw_disk_bus(c.images))
     c.add_function('scs_0102_prop_hypervisor_type', lambda c: compute_scs_0102_prop_hypervisor_type(c.images))
     c.add_function('scs_0102_prop_hw_rng_model', lambda c: compute_scs_0102_prop_hw_rng_model(c.images))

Tests/iaas/scs_0102_image_metadata/image_metadata.py

@@ -12,8 +12,7 @@
 HW_DISK_BUSES = ("virtio", "scsi", None)  # FIXME why None?
 HYPERVISOR_TYPES = ("qemu", "kvm", "xen", "hyper-v", "esxi", None)
 HW_RNG_MODELS = ("virtio", None)
-# Just for nice formatting of image naming hints -- otherwise we capitalize the 1st letter
-OS_LIST = ("CentOS", "AlmaLinux", "Windows Server", "RHEL", "SLES", "openSUSE")
+OS_PURPOSES = ("generic", "minimal", "k8snode", "gpu", "network", "custom")
 # Auxiliary mapping for `freq2secs` (note that values are rounded up a bit on purpose)
 FREQ_TO_SEC = {
     "never": 0,
@@ -31,15 +30,6 @@
 KIB, MIB, GIB = (1024 ** n for n in (1, 2, 3))
 
 
-def recommended_name(nm, os_list=OS_LIST):
-    """Return capitalized name"""
-    for osnm in os_list:
-        osln = len(osnm)
-        if nm[:osln].casefold() == osnm.casefold():
-            return osnm + nm[osln:]
-    return nm[0].upper() + nm[1:]
-
-
 def is_url(stg):
     """Is string stg a URL?"""
     idx = stg.find("://")
@@ -156,6 +146,7 @@ def compute_scs_0102_prop_os_version(images):
     # NOTE currently we are content when the property is not empty, but we could be more strict,
     # because the standard was recently edited to refer to the OpenStack docs, which prescribe
     # certain values for common operating systems.
+    # - os_version not matching regexp r'[0-9\.]*' (should be a numeric version no)
     offenders = [img for img in images if not img.os_version]
     _log_error('property os_version not set', offenders)
     return not offenders
@@ -164,11 +155,20 @@ def compute_scs_0102_prop_os_version(images):
 def compute_scs_0102_prop_os_distro(images):
     """This test ensures that each image has a proper value for the property `os_distro`."""
     # NOTE see note in `compute_scs_0102_prop_os_version`
+    # - os_distro not being all-lowercase (they all should be acc. to
+    #   https://docs.openstack.org/glance/2025.1/admin/useful-image-properties.html
     offenders = [img for img in images if not img.os_distro]
     _log_error('property os_distro not set', offenders)
     return not offenders
 
 
+def compute_scs_0102_prop_os_purpose(images, os_purposes=OS_PURPOSES):
+    """This test ensures that each image has a proper value for the property `os_distro`."""
+    offenders = [img for img in images if img.properties.get('os_purpose') not in os_purposes]
+    _log_error('property os_purpose not set or not correct', offenders)
+    return not offenders
+
+
 def compute_scs_0102_prop_hw_disk_bus(images, hw_disk_buses=HW_DISK_BUSES):
     """This test ensures that each image has a proper value for the property `hw_disk_bus`."""
     offenders = [img for img in images if img.hw_disk_bus not in hw_disk_buses]
@@ -227,7 +227,7 @@ def compute_scs_0102_prop_image_source(images):
         img
         for img in images
         if img.properties.get('image_source') != 'private'
-        if not is_url(img.properties.get('image_source'))
+        if not is_url(img.properties.get('image_source', ''))
     ]
     _log_error('property image_source INVALID (url or "private")', offenders)
     return not offenders