Linking issue on FreeBSD

[granalberto]

Describe the bug

When compiling from source on FreeBSD 11.2-RELEASE, the linker is called with -llibmaxminddb and it makes it fail. The correct option would be -lmaxminddb.

Logs and dumps

/usr/bin/ld: cannot find -llibmaxminddb

[zimmerle]

Hi @granalberto

Thanks for the report. I will have a look. Just installed a FreeBSD machine here to run a couple of tests.

[zimmerle]

Build bots are running on v3/dev/issue2131. A small change but may broken compilation for others, so let's see what the buildbots are going to tell us.

[airween]

I can't reproduce this issue.

./configure --without-lmdb --enable-parser-generation --enable-mutex-on-pm --prefix=/usr
...
ModSecurity - v3.0.3-95-gd5b93c10 for FreeBSD
 
 Mandatory dependencies
   + libInjection                                  ....v3.0.3-95-gd5b93c10
   + SecLang tests                                 ....d5b93c10
 
 Optional dependencies
   + GeoIP/MaxMind                                 ....found 
      * (MaxMind) v1.3.2
         -lmaxminddb , -DWITH_MAXMIND -I/usr/local/include 
   + LibCURL                                       ....found v7.65.1 
      -L/usr/local/lib -lcurl, -I/usr/local/include -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....found v2.1.0
      -lyajl , -DWITH_YAJL -I/usr/local/include 
   + LMDB                                          ....disabled
   + LibXML2                                       ....found v2.9.9
      -L/usr/local/lib -lxml2 -lz -L/usr/lib -llzma -L/usr/lib -lm, -I/usr/local/include/libxml2 -I/usr/include -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....found 
      -llua-5.3 -lm  -L/usr/local/lib , -DWITH_LUA -I/usr/local/include/lua53 
 
 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....enabled
   + Treating pm operations as critical section    ....enabled

additional info's:

## --------- ##
## Platform. ##
## --------- ##

hostname = freebsd
uname -m = amd64
uname -r = 11.2-RELEASE
uname -s = FreeBSD
uname -v = FreeBSD 11.2-RELEASE #0 r335510: Fri Jun 22 04:32:14 UTC 2018     [email protected]:/usr/obj/usr/src/sys/GENERIC 

/usr/bin/uname -p = amd64
/bin/uname -X     = unknown

Of course, libmaxmindb had been installed:

# pkg info libmaxminddb-1.3.2_1
libmaxminddb-1.3.2_1
Name           : libmaxminddb
Version        : 1.3.2_1
Installed on   : Thu Jul 11 11:37:54 2019 UTC
Origin         : net/libmaxminddb
Architecture   : FreeBSD:11:amd64
Prefix         : /usr/local
Categories     : net
Licenses       : APACHE20
Maintainer     : [email protected]
WWW            : https://github.com/maxmind/libmaxminddb
Comment        : Library for the MaxMind DB file format used for GeoIP2
Shared Libs provided:
	libmaxminddb.so.0
Annotations    :
	FreeBSD_version: 1102000
	repo_type      : binary
	repository     : FreeBSD
Flat size      : 93.6KiB
Description    :
The libmaxminddb library provides a C library for reading MaxMind DB
files, including the GeoIP2 databases from MaxMind. This is a custom
binary format designed to facilitate fast lookups of IP addresses while
allowing for great flexibility in the type of data associated with an
address.

WWW: https://github.com/maxmind/libmaxminddb

And finally, the build flow and all test passed.

[granalberto]

@airween the configure stage passes OK for me too. Then, I run gmake. Default make doesn't work (you need to install gmake, of course). After a few minutes, the stage fails with the reported error.

[airween]

@granalberto - as I wrote, the build flow and all test (gmake/make test) are successfully passed.

[airween@freebsd ~/src/ModSecurity]$ time gmake
Making all in others
gmake[1]: Entering directory '/usr/home/airween/src/ModSecurity/others'
gmake[1]: Nothing to be done for 'all'.
gmake[1]: Leaving directory '/usr/home/airween/src/ModSecurity/others'
Making all in src
...
gmake[1]: Entering directory '/usr/home/airween/src/ModSecurity'
gmake[1]: Nothing to be done for 'all-am'.
gmake[1]: Leaving directory '/usr/home/airween/src/ModSecurity'

real    0m13.788s
user    0m10.908s
sys     0m1.557s

(Note, this time is not the correct value, the binaries are built now...)

[airween@freebsd ~/src/ModSecurity]$ time gmake check
....
============================================================================
Testsuite summary for modsecurity 3.0
============================================================================
# TOTAL: 4842
# PASS:  4821
# SKIP:  18
# XFAIL: 0
# FAIL:  3
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
Please report to [email protected]
============================================================================

The failed tests caused by SSL error (Github needs sslv3, and looks like this release doesn't support it) - I didn't investigate it...

So, I think you forgot to install libmaxmindb, or built from source and linker doesn't found....?

[airween]

Could you share with us your configure summary (as you can see mine above)?

[zimmerle]

The second attempt to the fix is on the go.

[zimmerle]

@airween I don't think the configuration summary is going to help here. The configuration script shall not link (or use) the wrong library regardless of how broken (or not) is the environment of the user. The main reason for having a configuration script is to detect and avoid nuances such as this. That said, works for me is not applied in this scenario.

Something that the configure does, for instance, is to compile o piece of code with the desired library being used. If the compilation fails, it marks the library as not supported.

If I didn't have a suspicious I would ask for the conifiguration log files, but I do have; Is on the buildbots -

Likely @granalberto has libmaxmind being detected by the files on the file system, not pkg-config (or alternatives), therefore he has the lib- prefix in the front of the library.

Nevertheless, I would go for LLVM on FreeBSD not gcc.

[airween]

hi @zimmerle

@airween I don't think the configuration summary is going to help here. The configuration script shall not link (or use) the wrong library

yes, it doesn't help to detect the library,

Nevertheless, I would go for LLVM on FreeBSD not gcc.

but can helps to detect other parts of environment :).

[zimmerle]

hi @zimmerle

@airween I don't think the configuration summary is going to help here. The configuration script shall not link (or use) the wrong library

yes, it doesn't help to detect the library,

You meant: if it helps to identify if the library exists or not?

If so, let me explain how it works -

The build has a set of scripts at the build directory, here.
In this specific case, the m4 script that looks to the libmaxmind is here -
https://github.com/SpiderLabs/ModSecurity/blob/v3/master/build/libmaxmind.m4

The logging will help us to identify whenever there was a conflict of geoIP vs Maxmind or even garbage left for a not workable Maxmind. The summary on another hand will just present just a summary.

Nevertheless, I would go for LLVM on FreeBSD not gcc.

but can helps to detect other parts of environment :).

Yes, most certainly yes. As far as you go from what is considered standard as like you be to identify not tested scenarios, therefore possible issues.

[zimmerle]

Just an update: the script is looking for the library in /usr/local and /opt which may or may not be on the library path.

[granalberto]

This is the summary of configure script:

ModSecurity -  for FreeBSD
 
 Mandatory dependencies
   + libInjection                                  ....v3.9.2-30-gbf234eb
   + SecLang tests                                 ....5d85f36
 
 Optional dependencies
   + GeoIP/MaxMind                                 ....found 
      * (MaxMind) v
         /usr/local/lib//libmaxminddb.so, /usr/local/include, -DWITH_MAXMIND -I/usr/local/include
   + LibCURL                                       ....found v7.65.1 
      -L/usr/local/lib -lcurl, -I/usr/local/include -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....not found
   + LMDB                                          ....not found
   + LibXML2                                       ....found v2.9.9
      -L/usr/local/lib -lxml2 -lz -L/usr/lib -llzma -L/usr/lib -lm, -I/usr/local/include/libxml2 -I/usr/include -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....not found
 
 Other Options
   + Test Utilities                                ....disabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

These are the installed packages:

# pkg info
apache24-2.4.39_1              Version 2.4.x of Apache web server
apr-1.6.5.1.6.1_1              Apache Portability Library
autoconf-2.69_2                Automatically configure source code on many Un*x platforms
autoconf-wrapper-20131203      Wrapper script for GNU autoconf
automake-1.16.1_1              GNU Standards-compliant Makefile generator
binutils-2.32_1,1              GNU binary tools
ca_root_nss-3.44.1             Root certificate bundle from the Mozilla Project
curl-7.65.1                    Command line tool and library for transferring data with URLs
db5-5.3.28_7                   Oracle Berkeley DB, revision 5.3
expat-2.2.6_1                  XML 1.0 parser written in C
gcc8-8.3.0_2                   GNU Compiler Collection 8
gdbm-1.18.1_1                  GNU database manager
gettext-runtime-0.20.1         GNU gettext runtime libraries and programs
git-lite-2.22.0                Distributed source code management tool (lite package)
gmake-4.2.1_3                  GNU version of 'make' utility
gmp-6.1.2_1                    Free library for arbitrary precision arithmetic
indexinfo-0.3.1                Utility to regenerate the GNU info page index
libmaxminddb-1.3.2_1           Library for the MaxMind DB file format used for GeoIP2
libnghttp2-1.39.1              HTTP/2.0 C Library
libtool-2.4.6_1                Generic shared library support script
libxml2-2.9.9                  XML parser library for GNOME
m4-1.4.18_1,1                  GNU M4
modsecurity3-3.0.3_2           Intrusion detection and prevention engine
modsecurity3-apache-0.0.9.b1.19 Intrusion detection and prevention engine / Apache Wrapper
mpc-1.1.0_2                    Library of complex numbers with arbitrarily high precision
mpfr-4.0.2                     Library for multiple-precision floating-point computations
pcre-8.43_1                    Perl Compatible Regular Expressions library
perl5-5.28.2                   Practical Extraction and Report Language
pkg-1.11.1                     Package manager
readline-8.0.0                 Library for editing command lines as they are typed
sudo-1.8.27_1                  Allow others to run commands as root
yajl-2.1.0                     Portable JSON parsing and serialization library in ANSI C

I need to go through the whole compilation process so eventually I can make the modsecurity3-nginx port and package for FreeBSD and perhaps Pkgsrc.

Let me know if you need something else. I cannot respond fast these days but I'll do my best.

[airween]

@granalberto it doesn't visible how do you pass the arguments to the configure script.

Here is how I did:
./configure --without-lmdb --enable-parser-generation --enable-mutex-on-pm --prefix=/usr

I didn't pass any maxmindb info. It's interesting that your script detect it as without version, and shows full path instead if linker flag.

Note, that you don't need to pass the --withoud-lmdb if you didn't installed, and I think you also don't need the --enable-parser-generation nor --enable-mutex-on-pm. The prefix is your choice :).

Anyway, as you can see @zimmerle gave a possible solution - you can grab it from here.

[granalberto]

Thank you all very much. Branch v3/dev/issue/2131 works when you pass --prefix=/usr/local to ./configure.

[ffontaine]

I don't understand why libmaxminddb was removed from MAXMIND_POSSIBLE_LIB_NAMES. libmaxminddb.pc is the pkg-config filename used by upstream libmaxminddb: https://github.com/maxmind/libmaxminddb/blob/main/src/libmaxminddb.pc.in. So, as a result, build is broken with libmodsecurity 3.0.5 and any up-to-date buildsystems such as buildroot.

[arnout]

It's because MAXMIND_POSSIBLE_LIB_NAMES is used both for pkg-config and for matching the library name itself.

I think the solution is not to use it for pkg-config, because it was never called anything other than libmaxminddb. So replace

                for x in ${MAXMIND_POSSIBLE_LIB_NAMES}; do
                    if ${PKG_CONFIG} --exists ${x}; then
                        MAXMIND_PKG_NAME="$x"
                        break
                    fi
                done

with

                if ${PKG_CONFIG} --exists libmaxminddb; then
                    MAXMIND_PKG_NAME="libmaxminddb"
                    break
                fi