pip_import fails with under python 3.7

[aohren]

pip_import fails to find the dataclasses==0.7 dependency if the local python version is 3.7.0:

$ python2 --version
Python 2.7.18rc1
$ python3 --version
Python 3.7.0
$ bazel build //google/ads/googleads/v3:googleads_csharp_gapic
... [removed earlier status messages]
ERROR: An error occurred during the fetch of repository 'gapic_generator_python_pip_deps':
   pip_import failed: Collecting click==7.1.2 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
  Saved ./click-7.1.2-py2.py3-none-any.whl
Collecting google-api-core==1.20.1 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/00/c4/5ade7b68872f5ac7be2bec4b2e78800a7f4cd4cd587e9ccbcb67bb6722c7/google_api_core-1.20.1-py2.py3-none-any.whl
  Saved ./google_api_core-1.20.1-py2.py3-none-any.whl
Collecting googleapis-common-protos==1.52.0 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/03/74/3956721ea1eb4bcf7502a311fdaa60b85bd751de4e57d1943afe9b334141/googleapis_common_protos-1.52.0-py2.py3-none-any.whl
  Saved ./googleapis_common_protos-1.52.0-py2.py3-none-any.whl
Collecting jinja2==2.11.2 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
  Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/98/7b/ff284bd8c80654e471b769062a9b43cc5d03e7a615048d96f4619df8d420/MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl
  Saved ./MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl
Collecting protobuf==3.12.2 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/07/63/2c505711827446bfdb544e7bcc0d7694b115d22d56175902a2581fe1172a/protobuf-3.12.2-cp37-cp37m-manylinux1_x86_64.whl
  Saved ./protobuf-3.12.2-cp37-cp37m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
Collecting PyYAML==5.3.1 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 8))
  Using cached https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz
Collecting dataclasses==0.7 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 9))
 (  Could not find a version that satisfies the requirement dataclasses==0.7 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 9)) (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6)
No matching distribution found for dataclasses==0.7 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 9))
)
ERROR: no such package '@gapic_generator_python_pip_deps//': pip_import failed: Collecting click==7.1.2 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
  Saved ./click-7.1.2-py2.py3-none-any.whl
Collecting google-api-core==1.20.1 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/00/c4/5ade7b68872f5ac7be2bec4b2e78800a7f4cd4cd587e9ccbcb67bb6722c7/google_api_core-1.20.1-py2.py3-none-any.whl
  Saved ./google_api_core-1.20.1-py2.py3-none-any.whl
Collecting googleapis-common-protos==1.52.0 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/03/74/3956721ea1eb4bcf7502a311fdaa60b85bd751de4e57d1943afe9b334141/googleapis_common_protos-1.52.0-py2.py3-none-any.whl
  Saved ./googleapis_common_protos-1.52.0-py2.py3-none-any.whl
Collecting jinja2==2.11.2 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
  Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/98/7b/ff284bd8c80654e471b769062a9b43cc5d03e7a615048d96f4619df8d420/MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl
  Saved ./MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl
Collecting protobuf==3.12.2 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/07/63/2c505711827446bfdb544e7bcc0d7694b115d22d56175902a2581fe1172a/protobuf-3.12.2-cp37-cp37m-manylinux1_x86_64.whl
  Saved ./protobuf-3.12.2-cp37-cp37m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
Collecting PyYAML==5.3.1 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 8))
  Using cached https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz
Collecting dataclasses==0.7 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 9))
 (  Could not find a version that satisfies the requirement dataclasses==0.7 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 9)) (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6)
No matching distribution found for dataclasses==0.7 (from -r /usr/local/google/home/aohren/.cache/bazel/_bazel_aohren/0806572a3a8ec469df4a6a2174534dde/external/gapic_generator_python/requirements.txt (line 9))
)
INFO: Elapsed time: 2.208s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

Switching to python 3.6.10 fixes the issue:

$ pyenv global 3.6.10
$ python2 --version
Python 2.7.18rc1
$ python3 --version
Python 3.6.10
$ bazel build //google/ads/googleads/v4:googleads_csharp_gapic
...... [no errors] 

[bobhancock]

dataclasses, as a library, is a backport for 3.6. The py_binary checks for 3.6, but in our case we are running with various version of 3.7.x.

From BUILD.bazel

py_binary(
name = "gapic_plugin",
srcs = glob(["gapic//*.py"]),
data = [":pandoc_binary"] + glob(["gapic//*.j2"]),
main = "gapic/cli/generate_with_pandoc.py",
python_version = "PY3",
visibility = ["//visibility:public"],
deps = [
"@com_google_protobuf//:protobuf_python",
"@com_github_grpc_grpc//src/python/grpcio/grpc:grpcio",
requirement("click"),
requirement("google-api-core"),
requirement("googleapis-common-protos"),
requirement("jinja2"),
requirement("MarkupSafe"),
requirement("pypandoc"),
requirement("PyYAML"),
] + select({
":gapic_gen_python_3_6": [requirement("dataclasses")],
"//conditions:default": [],
}),
)

[vam-google]

#439 - this broke it. For some unknown reason dataclasses==0.7 causes problems while dataclasses==0.6 works fine.

Given this whole dataclasses thing is needed only to build on python3.6 it is not worth investigating why dataclasses==0.7 breaks it, sticking to 0.6 seems like good enough. We just need to tell the bot stop sending prs to update that dependency. Or at least not approve them ever.

[busunkim96]

The 0.7 release pinned to 3.6 specifically. (noted in their release history). I opened #475 to adjust the pin in requirements.txt

In 3.7+, dataclasses support comes with the core library.

[software-dov]

Closed by #475