Build Fails on MacOS

[mhworth]

Building python-rocksdb fails on recent version of MacOS. The final error you get is when linking with libstdc++:

clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
ld: library not found for -lstdc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'g++' failed with exit status 1

Summary of how to work around right now:

brew install rocksdb
export MACOSX_DEPLOYMENT_TARGET=10.9
pip install python-rocksdb

The pandas folks ran into this too; here is a discussion on how to resolve the issue by passing additional args to Clang: pandas-dev/pandas#23424

[zzzhc]

export CFLAGS=-stdlib=libc++
pip install python-rocksdb

this works for me on macOS 10.14.3

[cvarma21]

Thanks. This worked perfectly

[deed02392]

I'm getting rocksdb/_rocksdb.cpp:622:10: fatal error: 'rocksdb/slice.h' file not found

Above did not help.

[vincentwyshan]

I'm getting rocksdb/_rocksdb.cpp:622:10: fatal error: 'rocksdb/slice.h' file not found

Above did not help.

I get same problem, guess we need install rocksdb in mac and make complier link to proper headers

[mrugacz95]

Installing rocksdb with brew helped:
brew install rocksdb

[alexreg]

None of the above solutions work on macOS 12.2, I'm afraid...

[faberchri]

The following procedure worked for me on:

• MacBook Pro M1
• macOS 12.2.1 Monterey
• rocksdb 6.29.3

Install rocksdb with brew:

brew install rocksdb

Use brew to look up the path on your system of the libs required for the install of python-rocksdb:

brew info rocksdb lz4 snappy

I had all libs already installed, maybe due to the install of rocksdb. If libs are missing, I guess you need to install them.

Export (for gcc?) the paths of the libs with CPPFLAGS and LDFLAGS and install python-rocksdb with pip.
For me that was (adapt paths according to your setup):

export CPPFLAGS=-I/opt/homebrew/Cellar/rocksdb/6.29.3/include
export LDFLAGS="-L/opt/homebrew/Cellar/rocksdb/6.29.3/lib -L/opt/homebrew/Cellar/snappy/1.1.9/lib -L/opt/homebrew/Cellar/lz4/1.9.3/lib"
pip install python-rocksdb

The exports fixed also the poetry install of a poetry project with faust-streaming-rocksdb (faust-streaming = {extras = ["rocksdb"], version = "^0.8.2"}) as dependency.

[alexreg]

@faberchri Thank you for sharing, although that didn't work for me either. The only thing that worked in the end was to use the more recent fork of this library, patch that for the latest rocksdb (just removing a few definitions), and then build under brew sh. I'll try to get a PR submitted to that project.

[Littlecowherd]

The following procedure worked for me on:

• MacBook Pro M1
• macOS 12.2.1 Monterey
• rocksdb 6.29.3

Install rocksdb with brew:

brew install rocksdb

Use brew to look up the path on your system of the libs required for the install of python-rocksdb:

brew info rocksdb lz4 snappy

I had all libs already installed, maybe due to the install of rocksdb. If libs are missing, I guess you need to install them.

Export (for gcc?) the paths of the libs with CPPFLAGS and LDFLAGS and install python-rocksdb with pip. For me that was (adapt paths according to your setup):

export CPPFLAGS=-I/opt/homebrew/Cellar/rocksdb/6.29.3/include
export LDFLAGS="-L/opt/homebrew/Cellar/rocksdb/6.29.3/lib -L/opt/homebrew/Cellar/snappy/1.1.9/lib -L/opt/homebrew/Cellar/lz4/1.9.3/lib"
pip install python-rocksdb

The exports fixed also the poetry install of a poetry project with faust-streaming-rocksdb (faust-streaming = {extras = ["rocksdb"], version = "^0.8.2"}) as dependency.

Thank you anyway, the installation succeeded but runs failed ……

[rollue]

@faberchri Thank you for sharing, although that didn't work for me either. The only thing that worked in the end was to use the more recent fork of this library, patch that for the latest rocksdb (just removing a few definitions), and then build under brew sh. I'll try to get a PR submitted to that project.

Could you share how this could be done in more detail?

I tried to install from your PR branch by adding the below line to poetry,

rocksdb = { git = "https://github.com/alexreg/python-rocksdb.git", branch = "update-api" }

but still getting this error

...
rocksdb/_rocksdb.cpp:796:10: fatal error: 'rocksdb/utilities/backupable_db.h' file not found
#include "rocksdb/utilities/backupable_db.h"
...

P.S: what do you mean by build under brew sh?

[alexreg]

@rollue I mean to start the Homebrew shell using brew sh and run the build from within that. But anyway, the way I recommend using my branch (and when it's merged, NightTsarina's fork), is to configure the rocksdb dependency exactly how you did above, but also, before running pip install (or the poetry equivalent), to export some environment variables:

export INCLUDE_PATH="$(brew --prefix)/include" LIBRARY_PATH="$(brew --prefix)/lib"

This presumes you installed rocksdb with Homebrew. Note that $(brew --prefix) is Bash syntax, but if you're using another shell, replace it with the equivalent (or just hard code it to /opt/homebrew or /usr/local, as appropriate).

[rollue]

@alexreg thanks for the explanation, but pardon my ignorance.
When I try to poetry install your branch after installing rocksdb with Homebrew (with brew install rocksdb).
I keep getting the following error. (I'm on M1 Mac pro)

.... 
    rocksdb/_rocksdb.cpp:819:10: fatal error: 'rocksdb/utilities/backupable_db.h' file not found
    #include "rocksdb/utilities/backupable_db.h"
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    3 warnings and 1 error generated.
    error: command '/usr/bin/clang' failed with exit code 1
    [end of output]
...

I've seen in your conversation with NightTsarina, especially in NightTsarina#13 (comment) that this may be deprecated.
But I can't find a workaround for this. Any ideas how to go about this?

[alexreg]

@rollue I don't use poetry so I can't say much about it, but if you set the environment variables like above and then pip install my branch, it seems to work fine on macOS. And yes, backupable_db is deprecated, though it still exists with rocksdb 6.

[jrj-d]

Hello @alexreg, I'm experiencing the same issue as @rollue.
We're blocked because there's no easy way to install rocksdb v6 on mac (with brew). I'll try to find a workaround

[alexreg]

Hello @alexreg, I'm experiencing the same issue as @rollue.

We're blocked because there's no easy way to install rocksdb v6 on mac (with brew). I'll try to find a workaround

brew install alexreg/dev/rocksdb@6

I tried to get the formula accepted in homebrew-core but sadly no luck. At least this should make it easy though.