FreeBSD support (fix ready) #17

LukaszMoskala · 2023-12-06T20:21:46Z

Hello,

I get NotImplementedError: Unsupported platform: freebsd13 when doing pip install --upgrade homeassistant. Turns out this package recently became dependency of another dependency of homeassistant.

Please replace in setup.py

SYSTEM_IS_UNIX = (sys.platform.startswith("linux") or
                  sys.platform.startswith("darwin"))

with:

SYSTEM_IS_UNIX = (sys.platform.startswith("linux") or
                  sys.platform.startswith("darwin") or
                  sys.platform.startswith("freebsd"))

Probably we also should handle openbsd, netbsd and dragonflybsd, but I didn't verify if these names are correct. Or maybe we should just do:

SYSTEM_IS_UNIX = (sys.platform.startswith("linux") or
                  sys.platform.startswith("darwin") or
                  'bsd' in sys.platform)

I can also confirm that after making my change, package successfully installs on freebsd.

Thank you in advance, let me know if you need anything else. I can open pull request if that's easier for you.

The text was updated successfully, but these errors were encountered:

rhpvorderman · 2023-12-08T08:10:31Z

Hi. Sure, if that works I can implement that. Thanks for the suggestion!

ryanwinter · 2023-12-12T00:18:26Z

I'm still having issues when installing wheel for this package:

 OverflowError: could not allocate a large enough CPU set

ryanwinter · 2023-12-12T00:31:15Z

Processing /opt/python-zlib-ng
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: zlib-ng
  Building wheel for zlib-ng (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for zlib-ng (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [74 lines of output]
      /tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/dist.py:314: InformationOnly: Normalizing '0.3.0-dev' to '0.3.0.dev0'
        self.metadata.version = self._normalize_version(self.metadata.version)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311
      creating build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng
      copying src/zlib_ng/__init__.py -> build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng
      copying src/zlib_ng/gzip_ng.py -> build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng
      copying src/zlib_ng/zlib_ng.pyi -> build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng
      copying src/zlib_ng/py.typed -> build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng
      creating build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng/zlib-ng
      copying src/zlib_ng/zlib-ng/LICENSE.md -> build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng/zlib-ng
      copying src/zlib_ng/zlib-ng/README.md -> build/lib.freebsd-13.1-RELEASE-p7-amd64-cpython-311/zlib_ng/zlib-ng
      running build_ext
      Traceback (most recent call last):
        File "/opt/homeassistant.2023.12/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/homeassistant.2023.12/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/homeassistant.2023.12/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 389, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 123, in <module>
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/tmp/pip-build-env-o8jrof8i/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "<string>", line 67, in build_extension
        File "<string>", line 99, in build_zlib_ng
      OverflowError: could not allocate a large enough CPU set
      [end of output]

rhpvorderman · 2023-12-12T07:54:50Z

This seems to be an issue in pip/setuptools that is triggered by this package. I don't have freebsd, so I can't help you.
Since python-zlib-ng is a drop-in replacement, it is technically redundant. If it is a dependency of a package you use, please request of them that they make it an optional requirement.

LukaszMoskala · 2023-12-12T08:37:18Z

@ryanwinter I'm unable to reproduce this with Python 3.11.5 and pip 23.2.1 on FreeBSD 13.2.

You can look at this similar issue on stackoverflow.com

One thing that comes to mind is that FreeBSD 13.1 is no longer supported, and maybe upgrading to 13.2 or 14.0 would help.

Please provide output of:

freebsd-version -k #(on host when running in jail)
freebsd-version -u
pip --version
python --version

precurse · 2023-12-12T15:42:19Z

Looks like when I try to compile the latest develop version, it's not adding the /usr/local/include/zlib-ng.h path when calling cc. If I have some time I'll try to fix, but wanted to mention it (FreeBSD 13.2-RELEASE in a Jail and Python311 venv):

 $ python3 setup.py install
/usr/home/hass/havenv_py11/lib/python3.11/site-packages/setuptools/dist.py:530: UserWarning: Normalizing '0.3.0-dev' to '0.3.0.dev0'
  warnings.warn(tmpl.format(**locals()))
running install
/usr/home/hass/havenv_py11/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/home/hass/havenv_py11/lib/python3.11/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing src/zlib_ng.egg-info/PKG-INFO
writing dependency_links to src/zlib_ng.egg-info/dependency_links.txt
writing top-level names to src/zlib_ng.egg-info/top_level.txt
reading manifest file 'src/zlib_ng.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no directories found matching 'src/zlib_ng/zlib-ng'
adding license file 'LICENSE'
writing manifest file 'src/zlib_ng.egg-info/SOURCES.txt'
installing library code to build/bdist.freebsd-13.2-RELEASE-amd64/egg
running install_lib
running build_py
running build_ext
CMake Warning:
  Ignoring extra path from command line:

   "/tmp/tmpk9qr9w95"


CMake Error: The source directory "/tmp/tmpk9qr9w95" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
Error: could not load cache
building 'zlib_ng.zlib_ng' extension
cc -pthread -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -fPIC -I/tmp/tmpk9qr9w95 -I/usr/home/hass/havenv_py11/include -I/usr/local/include/python3.11 -c src/zlib_ng/zlib_ngmodule.c -o build/temp.freebsd-13.2-RELEASE-amd64-cpython-311/src/zlib_ng/zlib_ngmodule.o
src/zlib_ng/zlib_ngmodule.c:8:10: fatal error: 'zlib-ng.h' file not found
#include "zlib-ng.h"
         ^~~~~~~~~~~
1 error generated.
error: command '/usr/bin/cc' failed with exit code 1

It does exist:

# ls /usr/local/include/zlib-ng.h
/usr/local/include/zlib-ng.h

I tried adding /usr/local to possible_prefixes, but that didn't seem to help.

LukaszMoskala · 2023-12-12T15:49:03Z

You probably need to do git submodule init and git submodule update as it expects it to be inside python-zlib-ng repo (which makes sense as you probably want to pin zlib-ng version to specific python-zlib-ng version)

precurse · 2023-12-12T15:50:12Z

That worked. Thanks!

ryanwinter · 2023-12-14T18:06:01Z

Thanks @LukaszMoskala,

I'm running 13.2 right now in the jail, however the host is still on 13.1.

I actually found the cause, the os.sched_getaffinity(0) in the setup.py is throwing an error (or maybe returning 0?). I just commented it out to use the os.cpu_count() instead. Not sure why this isn't working, didnt have time to investigate :(

@ryanwinter I'm unable to reproduce this with Python 3.11.5 and pip 23.2.1 on FreeBSD 13.2.

You can look at this similar issue on stackoverflow.com

One thing that comes to mind is that FreeBSD 13.1 is no longer supported, and maybe upgrading to 13.2 or 14.0 would help.

Please provide output of:
freebsd-version -k #(on host when running in jail)
freebsd-version -u
pip --version
python --version

LukaszMoskala · 2023-12-14T18:27:09Z

I'm running 13.2 right now in the jail, however the host is still on 13.1.

That's probably your problem, as userspace should never be on higher version than kernel. Looks like some kernel API has changed (or been added in 13.2), you have application that uses it (python compiled for 13.2) but your kernel does not support it.

I'm always updating my host before updating my jails to solve that problem. So far having newer kernel (13.2) with older userspace (13.1 and 13.2) does not seem to cause any problems like that.

Anyway, I think that's outside of scope of this issue

rhpvorderman · 2023-12-22T10:48:39Z

BSD installation enabled now on 0.3.0

pixel8383 · 2024-01-09T14:02:57Z

If I try to install the package (pip install zlib-ng --no-cache-dir), I get the same error as @ryanwinter (OverflowError: could not allocate a large enough CPU set).

I am running a jail (13.2-RELEASE-p6) on an host running FreeBSD 13.1-RELEASE-p9. The jail runs with pip 23.3.2 and Python 3.11.5.

Could you please ELI5 how can I fix the issue?

Thank you!

LukaszMoskala · 2024-01-09T14:05:00Z

Don't run newer version in jail than on host Dnia 9 stycznia 2024 15:03:08 CET, Emanuele ***@***.***> napisał/a:

…

If I try to install the package (`pip install zlib-ng --no-cache-dir`), I get the same error as @ryanwinter (_OverflowError: could not allocate a large enough CPU set_). I am running a jail (13.2-RELEASE-p6) on an host running FreeBSD 13.1-RELEASE-p9. The jail runs with pip 23.3.2 and Python 3.11.5. Could you please ELI5 how can I fix the issue? Thank you!

pixel8383 · 2024-01-09T14:10:25Z

Yep @LukaszMoskala, I got it and, for sure, I will keep it in mind for the future but I think I cannot rollback the jail version easily. As far as I understood, I can workaround this forcing a git update somewhere, am I wrong?

LukaszMoskala · 2024-01-09T14:30:57Z

Basically you can either reinstall jail to older version (or rollback snapshot/backup if you have) or upgrade kernel on host. I don't know about any other way unfortunately. Dnia 9 stycznia 2024 15:10:44 CET, Emanuele ***@***.***> napisał/a:

…

Yep @LukaszMoskala, I got it and, for sure, I will keep it in mind for the future but I think I cannot rollback the jail version easily. As far as I understood, I can workaround this forcing a git update somewhere, am I wrong?

pixel8383 · 2024-01-20T14:07:28Z

Basically you can either reinstall jail to older version (or rollback snapshot/backup if you have) or upgrade kernel on host. I don't know about any other way unfortunately.

Unfortunately I have no snapshot to roolback to (for different reasons I had to delete them). I guess I will have to wait until I can upgrade the kernel on the host.

LukaszMoskala mentioned this issue Dec 7, 2023

Add FreeBSD support #18

Merged

2 tasks

rhpvorderman added enhancement New feature or request Implement before next release labels Dec 8, 2023

rhpvorderman closed this as completed Dec 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FreeBSD support (fix ready) #17

FreeBSD support (fix ready) #17

LukaszMoskala commented Dec 6, 2023

rhpvorderman commented Dec 8, 2023

Uh oh!

ryanwinter commented Dec 12, 2023

Uh oh!

ryanwinter commented Dec 12, 2023

Uh oh!

rhpvorderman commented Dec 12, 2023

Uh oh!

LukaszMoskala commented Dec 12, 2023

Uh oh!

precurse commented Dec 12, 2023 •

edited

Loading

Uh oh!

LukaszMoskala commented Dec 12, 2023

Uh oh!

precurse commented Dec 12, 2023

Uh oh!

ryanwinter commented Dec 14, 2023 •

edited

Loading

Uh oh!

LukaszMoskala commented Dec 14, 2023

Uh oh!

rhpvorderman commented Dec 22, 2023

Uh oh!

pixel8383 commented Jan 9, 2024

Uh oh!

LukaszMoskala commented Jan 9, 2024 via email

Uh oh!

pixel8383 commented Jan 9, 2024

Uh oh!

LukaszMoskala commented Jan 9, 2024 via email

Uh oh!

pixel8383 commented Jan 20, 2024

Uh oh!

FreeBSD support (fix ready) #17

FreeBSD support (fix ready) #17

Comments

LukaszMoskala commented Dec 6, 2023

rhpvorderman commented Dec 8, 2023

Uh oh!

ryanwinter commented Dec 12, 2023

Uh oh!

ryanwinter commented Dec 12, 2023

Uh oh!

rhpvorderman commented Dec 12, 2023

Uh oh!

LukaszMoskala commented Dec 12, 2023

Uh oh!

precurse commented Dec 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LukaszMoskala commented Dec 12, 2023

Uh oh!

precurse commented Dec 12, 2023

Uh oh!

ryanwinter commented Dec 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LukaszMoskala commented Dec 14, 2023

Uh oh!

rhpvorderman commented Dec 22, 2023

Uh oh!

pixel8383 commented Jan 9, 2024

Uh oh!

LukaszMoskala commented Jan 9, 2024 via email

Uh oh!

pixel8383 commented Jan 9, 2024

Uh oh!

LukaszMoskala commented Jan 9, 2024 via email

Uh oh!

pixel8383 commented Jan 20, 2024

Uh oh!

precurse commented Dec 12, 2023 •

edited

Loading

ryanwinter commented Dec 14, 2023 •

edited

Loading