{Package} Upgrade setuptools and wheel to the latest version.

[wangzelin007]

Describe the bug

All users using azdev must pin wheel==0.30.0 and setuptools==70.0.0, otherwise the following issues may occur:

1. Generated index.json is incomplete, missing metadata. (Azdev relies on metadata generated by wheel 0.30.0 to update index.json. )
2. Error: ModuleNotFoundError: No module named 'wheel.wheelfile'
3. The wheel generated with 0.30.0 doesn't support metadata version 2.1 and cannot publish to pypi.

Example of a incomplete index.json:

{
    "downloadUrl": "https://azuremlsdktestpypi.blob.core.windows.net/wheels/sdk-cli-v2-public/ml-2.36.0-py3-none-any.whl",
    "filename": "ml-2.36.0-py3-none-any.whl",
    "metadata": {
        "azext.minCliCoreVersion": "2.15.0"
    },
    "sha256Digest": "xxx"
}

Root cause analysis:

The setuptools >=70.1.0 adds a vendored wheel 0.4.3 to generate wheel and it does not generate metadata.json.
Related PR:
pypa/setuptools#4369
Wheel version higher than 0.30.0 has removed metadata.json from the generated files in .dist-info
pypa/wheel#195

If we upgrade the setuptools to the latest version and decouple wheel will also face many issues:

1. The latest editable mode will change sys.path
2. Default build isolation behavior affecting extension installation
3. The latest editable mode will change location
4. If we generate metadata ourselves, it cannot be completely consistent with the existing metadata in idex.json.
5. Need to upgrade to pyproject.toml in the future

azure-cli-extensions:

azure-cli-extensions/.azure-pipelines/templates/azdev_setup.yml

Line 21 in ea6fff6

pip install setuptools==70.0.0

azure-cli-extensions/azure-pipelines.yml

Line 79 in ea6fff6

pip install wheel==0.30.0 requests packaging setuptools

azure-cli-extensions/azure-pipelines.yml

Lines 104 to 105 in ea6fff6

	- bash: pip install wheel==0.30.0
	displayName: 'Install wheel==0.30.0'

azure-cli-extensions/scripts/ci/util.py

Lines 55 to 78 in ea6fff6

	def get_ext_metadata(ext_dir, ext_file, ext_name):
	# Modification of https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension.py#L89
	WHL_METADATA_FILENAME = 'metadata.json'
	zip_ref = zipfile.ZipFile(ext_file, 'r')
	zip_ref.extractall(ext_dir)
	zip_ref.close()
	metadata = {}
	dist_info_dirs = [f for f in os.listdir(ext_dir) if f.endswith('.dist-info')]
	azext_metadata = _get_azext_metadata(ext_dir)
	if not azext_metadata:
	raise ValueError('azext_metadata.json for Extension "{}" Metadata is missing'.format(ext_name))
	metadata.update(azext_metadata)
	for dist_info_dirname in dist_info_dirs:
	parsed_dist_info_dir = WHEEL_INFO_RE(dist_info_dirname)
	if parsed_dist_info_dir and parsed_dist_info_dir.groupdict().get('name') == ext_name.replace('-', '_'):
	whl_metadata_filepath = os.path.join(ext_dir, dist_info_dirname, WHL_METADATA_FILENAME)
	if os.path.isfile(whl_metadata_filepath):
	with open(whl_metadata_filepath) as f:
	metadata.update(json.load(f))
	return metadata

azure-cli:
self.assertNotIn('metadata.json', os.listdir(os.path.join(self.ext_dir, ext_name + '-' + ext_version)))
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_dev_type_extension.py#L42
self.assertIn('metadata.json', os.listdir(os.path.join(self.ext_dir, dist_info)))
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_wheel_type_extension.py#L46
self.assertNotIn('metadata.json', os.listdir(os.path.join(self.ext_dir, dist_info)))
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_wheel_type_extension.py#L77
self.assertNotIn('metadata.json', os.listdir(os.path.join(self.ext_dir, ext_name + '-' + ext_version)))
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_wheel_type_extension.py#L105
get_cli_dependencies: whl_metadata_filepath = os.path.join(tmp_dir, dist_info_dir, 'metadata.json')
https://github.com/Azure/azure-cli/blob/50ccb24528eb36529d1432808f3ff385cc4c04aa/tools/automation/verify/default_modules.py#L33
pip install setuptools==70.0.0 wheel==0.30.0
https://github.com/Azure/azure-cli/blob/bf7352f1e9d20ff04c2f7cf5dd7c1b0988ce96c1/azure-pipelines.yml#L1176

azdev:
'wheel==0.30.0',
https://github.com/Azure/azure-cli-dev-tools/blob/8aa045a249297d063334c89f3d70f1e84c678007/setup.py#L89
get_ext_metadata: WHL_METADATA_FILENAME = 'metadata.json'
https://github.com/Azure/azure-cli-dev-tools/blob/dev/azdev/operations/extensions/util.py#L48

Related command

az extension add
az extension list
az extension list-available
az extension remove
az extension show
az extension update
azdev extension build
azdev extension publish
azdev extension update-index

Errors

metadata in the wheel file is: {
  "azext.minCliCoreVersion": "2.56.0",
  "azext.isPreview": true
}
Traceback (most recent call last):
  File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 290, in <module>
    main()
  File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 281, in main
    azdev_on_internal_extension(modified_files, azdev_type)
  File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 233, in azdev_on_internal_extension
    azdev_extension.check_extension_name()
  File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 116, in check_extension_name
    if metadata['name'] != extension_name:
       ~~~~~~~~^^^^^^^^
KeyError: 'name'

Issue script & Debug output

None

Expected behavior

None

Environment Summary

None

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[wangzelin007]

Azure/azure-cli#29467

[wangzelin007]

Azure/azure-cli#29421

[mainred]

@wangzelin007 can we push this work forward? setuptool has cve-2025-47273