Update prereq_python to use support matrix lookup

roles/prereq_python/README.md

@@ -1,23 +1,21 @@
 # prereq_python
 
-Install Python
+Install Python and pip.
 
-This role ensures that the correct versions of Python are installed on a host to meet the requirements of specified Cloudera Manager and Cloudera Runtime versions. It also handles the installation and update of the `pip` package manager for Python 3. For environments that still require Python 2, the role provides a flag to control whether the installation is done via the system package manager or from source.
+This role ensures that the correct versions of Python are installed on a host to meet the requirements of specified Cloudera Manager and Cloudera Runtime versions. It also handles the installation and update of the `pip` package manager. For environments that still require Python 2, the role provides a flag to control whether the installation is done via the system package manager or from source.
 
 To validate the required Python version for a given Cloudera Manager and Runtime, the [Cloudera on premise documentation](https://docs.cloudera.com/cdp-private-cloud-base/latest/installation/topics/cdpdc-cm-install-python-3.8.html) and the support matrix variables defined in the `cloudera.exe.prereq_supported` role are used.
 
 The role will:
 - Determine the required Python versions based on `cloudera_manager_version` and `cloudera_runtime_version`.
-- Install the necessary Python 3 packages if a supported version is not already present.
-- Install or update the `pip` package for Python 3.
-- If Python 2 is required, install it either via the system's package manager (`python2_package_install: true`) or from source (`python2_package_install: false`).
-- Ensure that the installed Python versions and `pip` are properly configured and accessible.
+- Install the necessary Python packages if a supported version is not already present and installations and upgrades are enabled.
+- Install or update the `pip` package for Python.
+- Ensure that the installed Python versions and `pip` are properly configured and updated.
 
 # Requirements
 
 - Root or `sudo` privileges are required on the target host to install system packages.
 - Network access to package repositories (for system packages) and PyPI (for pip).
-- If installing Python 2 from source, the host will require build tools (e.g., `gcc`, `make`, development headers).
 
 # Dependencies
 
@@ -27,36 +25,34 @@ None.
 
 | Variable | Type | Required | Default | Description |
 | --- | --- | --- | --- | --- |
-| `cloudera_manager_version` | `str` | `False` | `7.11.3` | The version of Cloudera Manager to use for determining Python version requirements. |
-| `cloudera_runtime_version` | `str` | `False` | `7.1.9` | The version of Cloudera Runtime to use for determining Python version requirements. |
-| `python3_package` | `str` | `False` | - | An optional name of the Python 3 package to install. This is only used when a supported version of Python 3 is not found. If not specified, the role will use OS-specific default package names. |
-| `python3_pip_package` | `str` | `False` | `python-pip` | The name of the Python 3 Pip package to be installed or updated. |
-| `python2_package_install` | `bool` | `False` | `true` | Flag to specify if Python 2 should be installed via the system package manager. If `false`, the role will attempt to install Python 2 from source. |
+| `cloudera_manager_version` | `str` | `True` | | The version of Cloudera Manager to use for determining Python version requirements. |
+| `cloudera_runtime_version` | `str` | `True` | | The version of Cloudera Runtime to use for determining Python version requirements. |
+| `python_packages` | `list[str]` | `False` | - | Optional names of the Python packages to install. This is only used when a supported version of Python is not found. If not specified, the role will use OS-specific default package names. |
+| `python_pip_packages` | `list[str]` | `False` | `[ python-pip ]` | The names of the Python pip packages to be installed or updated. |
+| `prereq_python_upgrade` | `bool` | `False` | `true` | Flag to enable Python installation or upgrade if a supported version is not present. If `true`, the latest supported version or the `python_packages` will be installed. |
 
 # Example Playbook
 
 ```yaml
 - hosts: all
   tasks:
     - name: Install Python for Cloudera Manager 7.11.3 / Cloudera Runtime 7.1.9
-      ansible.builtin.import_role:
-        name: cloudera.exe.prereq_python
-      # All variables will use their defaults, installing Python 3 with pip and Python 2 via package manager.
-
-    - name: Install Python for a different version of Cloudera Runtime
       ansible.builtin.import_role:
         name: cloudera.exe.prereq_python
       vars:
-        cloudera_runtime_version: "7.1.8"
-        # The role will adjust Python version requirements accordingly.
+        cloudera_manager_version: 7.11.3
+        cloudera_runtime_version: 7.1.9
 
-    - name: Install Python with custom package names and install Python 2 from source
+    - name: Install Python with custom package names
       ansible.builtin.import_role:
         name: cloudera.exe.prereq_python
       vars:
-        python3_package: "python39" # Example custom package name for Python 3.9
-        python3_pip_package: "python3-pip"
-        python2_package_install: false # Install Python 2 from source
+        cloudera_manager_version: 7.11.3
+        cloudera_runtime_version: 7.1.9
+        python_packages:
+          - python39 # Example custom package name for Python 3.9
+        python_pip_package:
+          - python3-pip
 ```
 
 # License

roles/prereq_python/defaults/main.yml

@@ -1,3 +1,4 @@
+---
 # Copyright 2024 Cloudera, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -12,11 +13,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
----
-cloudera_manager_version: 7.11.3
-cloudera_runtime_version: 7.1.9
+cloudera_manager_version: "{{ undef(hint='Please specify the Cloudera Manager version') }}"
+cloudera_runtime_version: "{{ undef(hint='Please specify the Cloudera Runtime version') }}"
 
-# python3_package:
-python3_pip_package: python-pip
+# python_packages:
+# python_pip_packages:
 
-python2_package_install: true
+prereq_python_upgrade: true

roles/prereq_python/meta/argument_specs.yml

@@ -18,34 +18,35 @@ argument_specs:
     short_description: Install Python for deployments
     description:
       - Ensures that the supported versions of Python for specified versions of Cloudera Manager and Runtime.
-      - Also ensures that Pip is installed and updated.
+      - Also ensures that pip is installed and updated.
     author:
       - "Jim Enright <[email protected]>"
+      - "Webster Mudge <[email protected]>"
     version_added: "3.0.0"
     options:
       cloudera_manager_version:
-        description: Version of Cloudera Manager
-        type: "str"
-        default: "7.11.3"
+        description: Version of Cloudera Manager to discover supported Python versions.
+        type: str
+        required: true
       cloudera_runtime_version:
-        description: Version of Cloudera Runtime
-        type: "str"
-        default: "7.1.9"
-      python3_package:
-        description:
-          - Optional Python3 package to be installed.
-          - This is used only when a supported version of Python3 is not already installed.
+        description: Version of Cloudera Runtime to discover supported Python versions
         type: str
-      python3_pip_package:
+        required: true
+      python_packages:
         description:
-          - Python3 Pip package to be installed.
-        type: str
+          - Override Python packages to be installed.
+          - This is used only when a supported version of Python is not already installed.
+        type: list
+        elements: str
+      python_pip_packages:
+        description:
+          - Override Python pip packages to be installed.
+        type: list
+        elements: str
         default: python-pip
-      python2_package_install:
+      prereq_python_upgrade:
         description:
-          - Flag to specify if Python2, should be installed via system package manager.
-          - This is used only when Python2 is required and not already installed.
-          - If I(True) Python2 will be installed using pacakage manager.
-          - If I(False) Python2 will be installed from source.
+          - Flag to enable Python installation or upgrade if a supported version is not present.
+          - If V(true), the latest supported version or the O(python_packages) will be installed
         type: bool
         default: true

roles/prereq_python/molecule/default/molecule.yml

@@ -30,6 +30,8 @@ platforms:
   - name: rhel9-4.molecule.internal
     image_owner: "309956199498"
     image_name: RHEL-9.4.0_HVM-*
+    image_filters:
+      architecture: x86_64
     instance_type: t3.medium
     boot_wait_seconds: 15
     vpc_subnet_id: ${TEST_VPC_SUBNET_ID}
@@ -48,6 +50,8 @@ platforms:
       Project: Molecule testing for prereq_python
 provisioner:
   name: ansible
-  # inventory:
-  #   group_vars:
-  #     all:
+  inventory:
+    group_vars:
+      all:
+        cloudera_manager_version: 7.13.1
+        cloudera_runtime_version: 7.1.9 SP1

roles/prereq_python/tasks/RedHat.yml

@@ -13,43 +13,43 @@
 # limitations under the License.
 
 ---
-# Python3 and Pip
-- name: Install supported version of Python3 if required
+
+- name: Install supported Python
   when:
-    - not __python3_version is version(__required_python_version, 'ge', version_type='pep440')
+    prereq_python_upgrade and
+    prereq_python_supported is not contains(
+      [
+        ansible_python["version"]["major"],
+        ansible_python["version"]["minor"]
+      ] |
+      join(".")
+    )
+  vars:
+    __python_version: "{{ prereq_python_supported | first | string }}"
   block:
-    - name: Install python3 package if defined
-      when: python3_package is defined
+    - name: Install supported version of Python3
+      when:
+        prereq_python_supported |
+        first |
+        ansible.builtin.split(".") |
+        first == "3"
       ansible.builtin.package:
-        name: "{{ python3_package }}"
+        name: "{{ python_packages | default('python' + __python_version) }}"
         state: present
 
-    - name: Download and install
-      when: python3_package is not defined
-      ansible.builtin.debug:
-        msg:
-          - "TODO - install Python version {{ __required_python_version }}"
+    # TODO Confirm Python2 package names
+    - name: Install supported version of Python2
+      when:
+        prereq_python_supported |
+        first |
+        ansible.builtin.split(".") |
+        first == "2"
+      ansible.builtin.package:
+        name: "{{ python_packages | default('python' + __python_version) }}"
+        state: present
 
-- name: Ensure pip is installed
+# TODO Confirm Python2-pip package names
+- name: Install pip
   ansible.builtin.package:
-    name: "{{ python3_pip_package }}"
+    name: "{{ python_pip_packages | default('python-pip') }}"
     state: present
-
-- name: Ensure pip is upgraded
-  ansible.builtin.pip:
-    name: pip
-    extra_args: --upgrade
-
-# Python2 and Pip
-- name: Ensure supported version of Python2
-  when:
-    - __required_python2_version is defined
-  block:
-    - name: Install Python2 if required
-      when:
-        - not __python2_version is version(__required_python2_version, 'ge', version_type='pep440')
-      block:
-        - name: Download and install
-          ansible.builtin.debug:
-            msg:
-              - "TODO - install Python version {{ __required_python2_version }}"