Using on OS X: 'Symbol not found: __cg_jpeg_resync_to_restart'

[whodidthis]

I tried using diesel but something gives the following error on a mac from just adding the pq-sys dependency:

> cat Cargo.toml
[package]
name = "testing"
version = "0.1.0"
[dependencies]
pq-sys = "0.2.1"

> cat src/lib.rs
extern crate pq_sys;

> cargo test --verbose
       Fresh libc v0.2.13
       Fresh pq-sys v0.2.1
       Fresh testing v0.1.0 (file:///Users/user/testing)
     Running `/Users/user/testing/target/debug/testing-0688888a7aeac232`
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
error: Process didn't exit successfully: `/Users/user/testing/target/debug/testing-0688888a7aeac232` (signal: 5, SIGTRAP: trace/breakpoint trap)

Caused by:
  Process didn't exit successfully: `/Users/user/testing/target/debug/testing-0688888a7aeac232` (signal: 5, SIGTRAP: trace/breakpoint trap)

[whodidthis]

Also doing this weird stuff from here seems to fix the problem if that helps:

cd /usr/local/lib
mv libjpeg.dylib libjpeg.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libJPEG.dylib libJPEG.dylib
mv libtiff.dylib libtiff.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libTIFF.dylib libTIFF.dylib
mv libpng.dylib libpng.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libPng.dylib libPNG.dylib

Edit: Not recommended as breaks things!

[compressed]

I've run into the same problem and traced it back to here as well...

Running:

DYLD_LIBRARY_PATH=/System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/ cargo run

Seems to make it compile. In my case, some packages I installed using homebrew have made these symlinks in /usr/local/lib. It seems though that somehow /usr/local/lib is added first to the DYLD_LIBRARY_PATH env var when compiling. I've checked my shell though and I don't have that environment variable set anywhere...

It seems since it includes /usr/local/lib first then cargo ends up linking to the wrong files (https://discussions.apple.com/thread/2084183)...

[compressed]

The other strange thing, if I compile with rust beta (1.11.0-beta.1), I get the linking error. If I compile with rust stable (1.10.0), it builds correctly.

[compressed]

So after going down the rabbit hole, here's what's going on:

1. There was a bug in cargo (Correctly record multiple native dirs per package rust-lang/cargo#2763) that was fixed on June 6th. This bug actually prevented this line from having any impact on mac because it wasn't formed correctly when insert into DYLD_LIBRARY_PATH env var; it had native=/usr/local/lib added instead of /usr/local/lib.
2. After June 6th, when the bug was fixed, the build line actually inserted /usr/local/lib into the DYLD_LIBRARY_PATH env var which messes up the mac loader. From what I've read, setting/modifying this env var on macs is generally not a good idea for these exact reasons.
3. Stable rust uses a cargo version prior to June 6th so no one using that version would see this problem. Beta uses a newer cargo version so it winds up failing because it's loading the wrong libjpeg.dylib.

So now.. how do we fix this? As @whodidthis mentioned, you could change the symlinks in /usr/local/lib, but that isn't really a good fix because now any packages that you've installed using homebrew with a different version of libjpeg won't work anymore.

You could install libpq in a non-standard search path, like I guess windows does, but that seems like a bad idea.

The other option that could work would be just to add some conditional compilation here. If windows is the only platform with libpq outside its search path, then we can have that build logic only run for windows.

How about this for build.rs?

#[cfg(windows)]
fn main() {
    use std::process::Command;

    for output in Command::new("pg_config").arg("--libdir").output() {
        if output.status.success() {
            for path in String::from_utf8(output.stdout) {
                println!("cargo:rustc-link-search=native={}", &path);
            }
        }
    }
}

#[cfg(not(windows))]
fn main() {}

[compressed]

Ping @sgrif

[sgrif]

Hrm. I've always assumed this was a Rust bug. brew uninstall libjpeg has fixed this for me in the past (and then you can reinstall afterwards IIRC)

[compressed]

Ah, I hadn't tried that. However, it doesn't seem to help. I needed to do a brew uninstall libjpeg libpng giflib libtiff and then pq-sys will run, but now all the homebrew packages I have that use them (e.g. vips) are broken. If I then do brew install libjpeg libpng giflib libtiff to get them back, then the homebrew packages will work, but now I'm back with pq-sys loading the wrong dependencies (from /usr/local/lib).

[sgrif]

What the hell is even requiring libjpeg in the first place? I'd be fine with changing the build script to make sure that we only specifically change the linking for libpq, but I'd prefer not to make the change you've proposed as it will break anyone who has libpq installed in a nonstandard location.

I'm also wondering if this is actually something that needs to get looked at upstream? Surely anybody who ends up doing cargo:rustc-link-search=native=/usr/local/lib will break mac users for this reason?

[compressed]

To be honest, I'm not exactly sure how libjpeg, etc. is coming into the picture. Homebrew isn't listing it as a dependency:

~/D/pq-sys git:master ❯❯❯ brew deps postgresql                                                    ⏎ ⬆ ✱
openssl
readline

I still get the error if I reduce lib.rs to just this:

#[link(name = "pq")]
extern "C" {}

To your second point, I'm a bit torn on it. On one hand, it's obviously useful to specify it for non-standard locations, but now when we go to run, we're saying we're going to specifically override the loader's search path to have a priority for /usr/local/lib. So it's technically doing what we've asked, but it just won't work in this scenario. But you're correct, adding /usr/local/lib to DYLD_LIBRARY_PATH is probably going to break for mac users with homebrew packages installed (bunch of issues related to this on stackoverflow, etc.).

I was originally thinking it could be possible to only specify that rust-link-search override if that location isn't in the platform's linker path already, but there doesn't seem to be a straight-forward way to do so (e.g. something like gcc -Xlinker -v).

[sgrif]

Is there a way for us to tell the linker to additionally look in a path but
continuing to give the system path priority? (That was what I thought -L
did)

On Mon, Jul 18, 2016, 11:18 AM compressed notifications@github.com wrote:

To be honest, I'm not exactly sure how libjpeg, etc. is coming into the
picture. Homebrew isn't listing it as a dependency:

~/D/pq-sys git:master ❯❯❯ brew deps postgresql ⏎ ⬆ ✱
openssl
readline

I still get the error if I reduce lib.rs to just this:

#[link(name = "pq")]
extern "C" {}

To your second point, I'm a bit torn on it. On one hand, it's obviously
useful to specify it for non-standard locations, but now when we go to run,
we're saying we're going to specifically override the loader's search path
to have a priority for /usr/local/lib. So it's technically doing what
we've asked, but it just won't work in this scenario. But you're correct,
adding /usr/local/lib to DYLD_LIBRARY_PATH is probably going to break for
mac users with homebrew packages installed (bunch of issues related to this
on stackoverflow, etc.).

I was originally thinking it could be possible to only specify that
rust-link-search override if that location isn't in the platform's linker
path already, but there doesn't seem to be a straight-forward way to do so
(e.g. something like gcc -Xlinker -v).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABdWK57R_Z1aaDFEzWLEhLH07x-vIsoXks5qW5lcgaJpZM4JIyln
.

[compressed]

I'm not sure there is a way to do that. We would probably need to add the system paths first.

I remembered today that openssl-sys requires environment vars to install correctly on mac, see: https://github.com/sfackler/rust-openssl and https://github.com/sfackler/rust-openssl/blob/master/openssl-sys/build.rs. We could follow a similar approach (e.g. PQ_LIB_DIR) that would work for all platforms. If you wanted to go further, we could use this pkg-config crate it seems to use to try to find the library; I haven't had any experience with that.

Interestingly, brew --prefix postgresql returns (surprise) yet another path (/usr/local/opt/postgresql, which is a symlink) but would also work, since it's not /usr/local/lib anymore (/usr/local/opt/postgresql/lib). Actually though, for mac I don't believe we would even need to set this path since it's already on the mac loader path.

What do you think of this approach?

[compressed]

I submitted a PR for the above idea. I think this seems to be the safest way to move forward. I didn't bother with pulling in pkg-config since it seemed to add more complexity than we probably need given the existing way is working for most users already.