HTTPError: 400 Client Error: Invalid distribution file. for url: https://upload.pypi.org/legacy/

[zqfan]

Using same procedure, I can upload other packages.
Using same procedure, I can upload my project's history tag sucessfully, but fail with the latest tag of my project
no change in setup.py & setup.cfg, only add and change some python files.
it is very odd, and no more information returned after twine failed with http 400

Your Environment

Thank you for taking the time to report an issue.

To more efficiently resolve this issue, we'd like to know some basic information about your system and setup.

1. Your operating system:
   cat /etc/redhat-release
   CentOS Linux release 7.7.1908 (Core)

2. Version of python you are running:

python --version

python3 --version
Python 3.6.8

3. How did you install twine? Did you use your operating system's package manager or pip or something else?
   pip show twine
   Name: twine
   Version: 1.11.0
   Summary: Collection of utilities for publishing packages on PyPI
   Home-page: https://twine.readthedocs.io/
   Author: Donald Stufft and individual contributors
   Author-email: donald@stufft.io
   License: Apache License, Version 2.0
   Location: /usr/local/lib/python3.6/site-packages
   Requires: requests-toolbelt, requests, pkginfo, tqdm, setuptools
   Required-by:
   python3 -m twine upload dist/*
4. Version of twine you have installed (include complete output of):

twine --version

twine --version
twine version 1.11.0 (pkginfo: 1.7.1, requests: 2.27.1, setuptools: 57.4.0,
requests-toolbelt: 0.9.1, tqdm: 4.62.2)

5. Which package repository are you targeting?
   tccli
   https://github.com/TencentCloud/tencentcloud-cli
   3.0.894.1 fail to upload, while 3.0.890.1 is ok

If you're having issues uploading a specific package, you must include a copy of the following:

• The package's PKG-INFO file
• A redacted version of your .pypirc file (REMOVE ALL USERNAMES & PASSWORDS BEFORE UPLOADING)

The Issue

Please describe the issue that you are experiencing.

python3 -m twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading tccli-3.0.894.1-py2.py3-none-any.whl
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6.58M/6.58M [00:04<00:00, 1.45MB/s]
HTTPError: 400 Client Error: Invalid distribution file. for url: https://upload.pypi.org/legacy/

Steps to Reproduce

If the issue is predictable and consistently reproducible, please list the steps here.

git clone https://github.com/TencentCloud/tencentcloud-cli
git checkout 3.0.894.1
python3 setup.py sdist bdist_wheel
python3 -m twine upload dist/*

[bhrutledge]

I'm sorry to heart that you're having trouble, but the error you're seeing comes from from PyPI, not Twine. I suspect there's an issue with the setup.py-based build process, which is now deprecated in favor of the pyproject.toml-based process. It also looks like you're using a very old version of Twine; I recommend upgrading to the latest version.

I'm going to transfer this issue to the general-purpose support project.

[zqfan]

➜  tencentcloud-cli git:(625157d) ✗ pip show twine
Name: twine
Version: 3.8.0
Summary: Collection of utilities for publishing packages on PyPI
Home-page: https://twine.readthedocs.io/
Author: Donald Stufft and individual contributors
Author-email: donald@stufft.io
License: UNKNOWN
Location: /usr/local/lib/python3.6/site-packages
Requires: keyring, tqdm, pkginfo, urllib3, requests, rfc3986, readme-renderer, colorama, requests-toolbelt, importlib-metadata
Required-by: 
➜  tencentcloud-cli git:(625157d) ✗ twine upload --verbose -r testpypi dist/*
Using configuration from /root/.pypirc
Uploading distributions to https://test.pypi.org/legacy/
  dist/tccli-3.0.890.3-py2.py3-none-any.whl (6.6 MB)
username set from config file
password set from config file
username: tencentcloudapi
password: <hidden>
Uploading tccli-3.0.890.3-py2.py3-none-any.whl
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6.56M/6.56M [00:03<00:00, 1.80MB/s]
Response from https://test.pypi.org/legacy/:
400 Invalid distribution file.
<html>
 <head>
  <title>400 Invalid distribution file.</title>
 </head>
 <body>
  <h1>400 Invalid distribution file.</h1>
  The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
Invalid distribution file.


 </body>
</html>
HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
Invalid distribution file.

It seems caused by file size limit? when the packaged file size over 5.4 MB, it will fail with this error, after I delete some file (which I shouldn't because it breaks functionality) and shink the file to less than 5.4MB, it will success

But IIRC, the default file size limit is 50 MB instead? So can my package file size limit restored to default 50 MB?

I will try to upgrade my project's package process from setup.py to the new favored toml to check if it can be resolved as well.

[zqfan]

change from setuptools to pyproject.toml and using default hatching build, but get same error:

➜  tencentcloud-cli git:(625157d) ✗ python3 -m build  
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatchling)
* Getting build dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatchling)
* Getting build dependencies for wheel...
* Building wheel...
Successfully built tccli-3.0.890.3.tar.gz and tccli-3.0.890.3-py3-none-any.whl
➜  tencentcloud-cli git:(625157d) ✗ twine upload --verbose -r testpypi dist/*
Using configuration from /root/.pypirc
Uploading distributions to https://test.pypi.org/legacy/
  dist/tccli-3.0.890.3-py2.py3-none-any.whl (6.6 MB)
  dist/tccli-3.0.890.3-py3-none-any.whl (12.2 MB)
  dist/tccli-3.0.890.3.tar.gz (14.8 MB)
username set from config file
password set from config file
username: tencentcloudapi
password: <hidden>
Uploading tccli-3.0.890.3-py2.py3-none-any.whl
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6.56M/6.56M [00:03<00:00, 1.79MB/s]
Response from https://test.pypi.org/legacy/:
400 Invalid distribution file.
<html>
 <head>
  <title>400 Invalid distribution file.</title>
 </head>
 <body>
  <h1>400 Invalid distribution file.</h1>
  The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
Invalid distribution file.


 </body>
</html>
HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
Invalid distribution file.

[henryiii]

Why do you have both a py2.py3 tagged wheel and a py3 tagged wheel? I believe it's invalid to upload two wheels with the same tags.

I'd recommend setting your requires-python setting (or python_requires in the old setuptools format). I think at least hatchling might use that to decide on adding a py2. tag. The old setuptools way used [bdist_wheel] universal=True in setup.cfg.

Did you try twine check on the produced files? There aren't all that many errors it can find, but it's worth a check.

[henryiii]

You should delete the old setup.py, setup.cfg, and MANIFEST.in if you are moving to a [project] table in pyproject.toml. Hatchling can produce py2.py3 wheels; I don't remember how, exactly, but it can. Try requires-python=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" first.

I think your problem is this:

https://github.com/TencentCloud/tencentcloud-cli/blob/7409bd05f76112a7a266ee58bda5dfa90d3abbf2/setup.py#L45

That's invalid - besides being broken on Windows PyPy since it doesn't close the file, it doesn't have an encoding, so the README processing will be broken on systems that are not unicode by default. You should instead do:

with open("README.rst", encoding="utf-8") as f:
    readme = f.read()
# Then use the "readme" text variable in setup() call

(Or use hatchling + pyproject.toml which does this for you)

[zqfan]

@henryiii Thank you very much, the python version is a mistake, I've already remove it from pyproject.toml.

I tested a lot and found that:

• rm setup.py setup.cfg and use hatchling build will upload successfully
• rm setup.py setup.cfg and use setuptools build will upload fail
• use pyproject.toml along with setup.cfg and setup.py , setup.py sdist will upload successfully
• use pyproject.toml along with setup.cfg and setup.py , setup.py bdist_wheel will fail
• but if I downgrade setuptools from latest to 61.0, bdist_wheel will success but with corrputed package (missing some files, maybe due to wrong config)
• but if I delete some source files to make sure the wheel file smaller than 5.4MB, bdist_wheel upload will success

[zqfan]

I think the setuptools still has advantage because it builds a wheel file which is just 50% size comparing to hatchling build.

[moriyoshi]

See this: pypi/warehouse@77c0432

[henryiii]

I'd unzip the wheels (unzip <filename>.whl or just rename it to .zip and use your file browser) and see what's inside. They are just zip files. Why is there a difference? If you do that, you can see what files are in one vs. the other.

[moriyoshi]

I'm not relevant to the issue itself, but I experienced the same problem and successfully tracked down the problem to be the validation logic that was just introduced last week in PyPI. This is not a packaging problem at all.

pypi/warehouse#13962

[zqfan]

I'm not relevant to the issue itself, but I experienced the same problem and successfully tracked down the problem to be the validation logic that was just introduced last week in PyPI. This is not a packaging problem at all.

pypi/warehouse#13962

that's it, thank you very much!