Build fails on Ubuntu 16.04

[kkiningh]

I tried building on Ubuntu 16.04 using ninja and it failed with the following error:

CMake Error at tools/clif/backend/CMakeLists.txt:15 (include_directories):
  include_directories given empty-string as include directory.

I installed protobuf from source following the directions here.

Here's the output from the full run:

kkiningh@mew:~/Workspace/clif/clif$ ./INSTALL.sh
+ INSTALL_DIR=/home/kkiningh/opt
+ CLIFSRC_DIR=/home/kkiningh/Workspace/clif/clif
+ LLVM_DIR=/home/kkiningh/Workspace/clif/clif/../clif_backend
+ BUILD_DIR=/home/kkiningh/Workspace/clif/clif/../clif_backend/build_matcher
++ cmake --version
++ head -1
++ cut -f3 '-d '
+ CV=3.5.1
+ CV=(${CV//./ })
+ ((  CV[0] < 3 || CV[0] == 3 && CV[1] < 5  ))
++ protoc --version
++ cut -f2 '-d '
+ PV=3.3.0
+ PV=(${PV//./ })
+ ((  PV[0] < 3 || PV[0] == 3 && PV[1] < 2  ))
++++ which protoc
+++ dirname /usr/bin/protoc
++ dirname /usr/bin
+ PROTOC_PREFIX_PATH=/usr
+ declare -a CMAKE_G_FLAG
+ declare -a MAKE_PARALLELISM
+ which ninja
/usr/bin/ninja
+ CMAKE_G_FLAGS=(-G Ninja)
+ MAKE_OR_NINJA=ninja
+ MAKE_PARALLELISM=()
+ echo 'Using ninja for the clif backend build.'
Using ninja for the clif backend build.
+ [[ '' =~ ^-?-h ]]
+ PYTHON=python
+ [[ -n '' ]]
+ echo -n 'Using Python interpreter: '
Using Python interpreter: + which python
/usr/bin/python
+ CLIF_VIRTUALENV=/home/kkiningh/opt/clif
+ CLIF_PIP=/home/kkiningh/opt/clif/bin/pip
+ virtualenv -p python /home/kkiningh/opt/clif
Running virtualenv with interpreter /usr/bin/python
New python executable in /home/kkiningh/opt/clif/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
+ /home/kkiningh/opt/clif/bin/pip install --upgrade pip
Requirement already up-to-date: pip in /home/kkiningh/opt/clif/lib/python2.7/site-packages
+ /home/kkiningh/opt/clif/bin/pip install --upgrade setuptools
Requirement already up-to-date: setuptools in /home/kkiningh/opt/clif/lib/python2.7/site-packages
Requirement already up-to-date: appdirs>=1.4.0 in /home/kkiningh/opt/clif/lib/python2.7/site-packages (from setuptools)
Requirement already up-to-date: packaging>=16.8 in /home/kkiningh/opt/clif/lib/python2.7/site-packages (from setuptools)
Requirement already up-to-date: six>=1.6.0 in /home/kkiningh/opt/clif/lib/python2.7/site-packages (from setuptools)
Requirement already up-to-date: pyparsing in /home/kkiningh/opt/clif/lib/python2.7/site-packages (from packaging>=16.8->setuptools)
+ mkdir -p /home/kkiningh/Workspace/clif/clif/../clif_backend
+ cd /home/kkiningh/Workspace/clif/clif/../clif_backend
+ svn co https://llvm.org/svn/llvm-project/llvm/trunk@299535 llvm
Checked out revision 299535.
+ cd llvm/tools
+ svn co https://llvm.org/svn/llvm-project/cfe/trunk@299535 clang
Checked out revision 299535.
+ '[' '!' '(' -e clif ')' ']'
+ mkdir -p /home/kkiningh/Workspace/clif/clif/../clif_backend/build_matcher
+ cd /home/kkiningh/Workspace/clif/clif/../clif_backend/build_matcher
+ cmake -DCMAKE_INSTALL_PREFIX=/home/kkiningh/opt/clif/clang -DCMAKE_PREFIX_PATH=/usr -DLLVM_INSTALL_TOOLCHAIN_ONLY=true -DCMAKE_BUILD_TYPE=Release -G Ninja /home/kkiningh/Workspace/clif/clif/../clif_backend/llvm
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Check for working C compiler using: Ninja
-- Check for working C compiler using: Ninja -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler using: Ninja
-- Check for working CXX compiler using: Ninja -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for dirent.h
-- Looking for dirent.h - found
-- Looking for dlfcn.h
-- Looking for dlfcn.h - found
-- Looking for errno.h
-- Looking for errno.h - found
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- Looking for fcntl.h
-- Looking for fcntl.h - found
-- Looking for inttypes.h
-- Looking for inttypes.h - found
-- Looking for link.h
-- Looking for link.h - found
-- Looking for malloc.h
-- Looking for malloc.h - found
-- Looking for malloc/malloc.h
-- Looking for malloc/malloc.h - not found
-- Looking for ndir.h
-- Looking for ndir.h - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for signal.h
-- Looking for signal.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for sys/dir.h
-- Looking for sys/dir.h - found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - found
-- Looking for sys/mman.h
-- Looking for sys/mman.h - found
-- Looking for sys/ndir.h
-- Looking for sys/ndir.h - not found
-- Looking for sys/param.h
-- Looking for sys/param.h - found
-- Looking for sys/resource.h
-- Looking for sys/resource.h - found
-- Looking for sys/stat.h
-- Looking for sys/stat.h - found
-- Looking for sys/time.h
-- Looking for sys/time.h - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for sys/uio.h
-- Looking for sys/uio.h - found
-- Looking for termios.h
-- Looking for termios.h - found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for valgrind/valgrind.h
-- Looking for valgrind/valgrind.h - not found
-- Looking for zlib.h
-- Looking for zlib.h - found
-- Looking for fenv.h
-- Looking for fenv.h - found
-- Looking for FE_ALL_EXCEPT
-- Looking for FE_ALL_EXCEPT - found
-- Looking for FE_INEXACT
-- Looking for FE_INEXACT - found
-- Looking for mach/mach.h
-- Looking for mach/mach.h - not found
-- Looking for histedit.h
-- Looking for histedit.h - not found
-- Looking for CrashReporterClient.h
-- Looking for CrashReporterClient.h - not found
-- Looking for linux/magic.h
-- Looking for linux/magic.h - found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Looking for pthread_getspecific in pthread
-- Looking for pthread_getspecific in pthread - found
-- Looking for pthread_rwlock_init in pthread
-- Looking for pthread_rwlock_init in pthread - found
-- Looking for pthread_mutex_lock in pthread
-- Looking for pthread_mutex_lock in pthread - found
-- Looking for dlopen in dl
-- Looking for dlopen in dl - found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Looking for compress2 in z
-- Looking for compress2 in z - found
-- Looking for setupterm in tinfo
-- Looking for setupterm in tinfo - found
-- Looking for xar_open in xar
-- Looking for xar_open in xar - not found
-- Looking for arc4random
-- Looking for arc4random - not found
-- Looking for backtrace
-- Looking for backtrace - found
-- Looking for _Unwind_Backtrace
-- Looking for _Unwind_Backtrace - found
-- Looking for getpagesize
-- Looking for getpagesize - found
-- Looking for sysconf
-- Looking for sysconf - found
-- Looking for getrusage
-- Looking for getrusage - found
-- Looking for setrlimit
-- Looking for setrlimit - found
-- Looking for isatty
-- Looking for isatty - found
-- Looking for futimens
-- Looking for futimens - found
-- Looking for futimes
-- Looking for futimes - found
-- Looking for posix_fallocate
-- Looking for posix_fallocate - found
-- Looking for sigaltstack
-- Looking for sigaltstack - found
-- Looking for writev
-- Looking for writev - found
-- Looking for lseek64
-- Looking for lseek64 - found
-- Looking for mallctl
-- Looking for mallctl - not found
-- Looking for mallinfo
-- Looking for mallinfo - found
-- Looking for malloc_zone_statistics
-- Looking for malloc_zone_statistics - not found
-- Looking for mkdtemp
-- Looking for mkdtemp - found
-- Looking for mkstemp
-- Looking for mkstemp - found
-- Looking for mktemp
-- Looking for mktemp - found
-- Looking for getcwd
-- Looking for getcwd - found
-- Looking for gettimeofday
-- Looking for gettimeofday - found
-- Looking for getrlimit
-- Looking for getrlimit - found
-- Looking for posix_spawn
-- Looking for posix_spawn - found
-- Looking for pread
-- Looking for pread - found
-- Looking for realpath
-- Looking for realpath - found
-- Looking for sbrk
-- Looking for sbrk - found
-- Looking for strtoll
-- Looking for strtoll - found
-- Looking for strerror
-- Looking for strerror - found
-- Looking for strerror_r
-- Looking for strerror_r - found
-- Looking for strerror_s
-- Looking for strerror_s - not found
-- Looking for setenv
-- Looking for setenv - found
-- Looking for dlopen
-- Looking for dlopen - found
-- Looking for dladdr
-- Looking for dladdr - not found
-- Looking for __GLIBC__
-- Looking for __GLIBC__ - found
-- Looking for pthread_getname_np in pthread
-- Looking for pthread_getname_np in pthread - found
-- Looking for pthread_setname_np in pthread
-- Looking for pthread_setname_np in pthread - found
-- Performing Test HAVE_INT64_T
-- Performing Test HAVE_INT64_T - Success
-- Performing Test HAVE_UINT64_T
-- Performing Test HAVE_UINT64_T - Success
-- Performing Test HAVE_U_INT64_T
-- Performing Test HAVE_U_INT64_T - Success
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB - Success
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB - Success
-- Performing Test LLVM_HAS_ATOMICS
-- Performing Test LLVM_HAS_ATOMICS - Success
-- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG
-- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG - Success
-- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG
-- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG - Failed
-- Performing Test HAS_MAYBE_UNINITIALIZED
-- Performing Test HAS_MAYBE_UNINITIALIZED - Success
-- Native target architecture is X86
-- Threads enabled.
-- Doxygen disabled.
-- Sphinx disabled.
-- Go bindings disabled.
-- Could NOT find OCaml (missing:  OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH)
-- Could NOT find OCaml (missing:  OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH)
-- OCaml bindings disabled.
-- LLVM host triple: x86_64-unknown-linux-gnu
-- LLVM default target triple: x86_64-unknown-linux-gnu
-- Performing Test C_SUPPORTS_FPIC
-- Performing Test C_SUPPORTS_FPIC - Success
-- Performing Test CXX_SUPPORTS_FPIC
-- Performing Test CXX_SUPPORTS_FPIC - Success
-- Building with -fPIC
-- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
-- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG - Success
-- Performing Test C_SUPPORTS_WERROR_DATE_TIME
-- Performing Test C_SUPPORTS_WERROR_DATE_TIME - Success
-- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME
-- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME - Success
-- Performing Test CXX_SUPPORTS_CXX11
-- Performing Test CXX_SUPPORTS_CXX11 - Success
-- Performing Test CXX_SUPPORTS_MISSING_FIELD_INITIALIZERS_FLAG
-- Performing Test CXX_SUPPORTS_MISSING_FIELD_INITIALIZERS_FLAG - Success
-- Performing Test C_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test C_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG - Failed
-- Performing Test CXX_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test CXX_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG - Failed
-- Performing Test C_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG
-- Performing Test C_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG - Failed
-- Performing Test CXX_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG
-- Performing Test CXX_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG - Success
-- Performing Test C_WCOMMENT_ALLOWS_LINE_WRAP
-- Performing Test C_WCOMMENT_ALLOWS_LINE_WRAP - Failed
-- Performing Test C_SUPPORTS_STRING_CONVERSION_FLAG
-- Performing Test C_SUPPORTS_STRING_CONVERSION_FLAG - Failed
-- Performing Test CXX_SUPPORTS_STRING_CONVERSION_FLAG
-- Performing Test CXX_SUPPORTS_STRING_CONVERSION_FLAG - Failed
-- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS
-- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS - Failed
-- Performing Test C_SUPPORTS_FNO_FUNCTION_SECTIONS
-- Performing Test C_SUPPORTS_FNO_FUNCTION_SECTIONS - Success
-- Performing Test C_SUPPORTS_FFUNCTION_SECTIONS
-- Performing Test C_SUPPORTS_FFUNCTION_SECTIONS - Success
-- Performing Test CXX_SUPPORTS_FFUNCTION_SECTIONS
-- Performing Test CXX_SUPPORTS_FFUNCTION_SECTIONS - Success
-- Performing Test C_SUPPORTS_FDATA_SECTIONS
-- Performing Test C_SUPPORTS_FDATA_SECTIONS - Success
-- Performing Test CXX_SUPPORTS_FDATA_SECTIONS
-- Performing Test CXX_SUPPORTS_FDATA_SECTIONS - Success
-- Found PythonInterp: /usr/bin/python2.7 (found version "2.7.12")
-- Constructing LLVMBuild project information
-- Found Subversion: /usr/bin/svn (found version "1.9.3")
-- Targeting AArch64
-- Targeting AMDGPU
-- Targeting ARM
-- Targeting BPF
-- Targeting Hexagon
-- Targeting Lanai
-- Targeting Mips
-- Targeting MSP430
-- Targeting NVPTX
-- Targeting PowerPC
-- Targeting RISCV
-- Targeting Sparc
-- Targeting SystemZ
-- Targeting X86
-- Targeting XCore
-- Could NOT find Z3 (missing:  Z3_LIBRARIES Z3_INCLUDE_DIR) (Required is at least version "4.5")
-- Looking for sys/resource.h
-- Looking for sys/resource.h - found
-- Clang version: 5.0.0
-- Performing Test CXX_SUPPORTS_NO_NESTED_ANON_TYPES_FLAG
-- Performing Test CXX_SUPPORTS_NO_NESTED_ANON_TYPES_FLAG - Failed
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'protobuf'
--   Found protobuf, version 3.3.0
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found version "2.7.12")
CMake Error at tools/clif/backend/CMakeLists.txt:15 (include_directories):
  include_directories given empty-string as include directory.

[mrovner]

Please share /usr/lib/pkgconfig/protobuf.pc (if it's not there we have a problem).
We suspect it lacks the line includedir=${prefix}/include
What is pkg-config --cflags-only-I protobuf output?

[gpshead]

This also happens for me on Debian 9.0 when I'm using a protobuf package I've built myself and installed in my own location rather than system wide (where it could clobber the distro managed stuff). This appears to fix it:

pkg-config --cflags-only-I protobuf is empty. But if I set PKG_CONFIG_PATH to point to the location where I installed protobuf properly, it works.

~/oss/clif$ export PKG_CONFIG_PATH=/home/greg/oss/tool_env/lib/pkgconfig
~/oss/clif$ pkg-config --cflags-only-I protobuf
-I/home/greg/oss/tool_env/include

But cmake doesn't appear to be picking that up from the environment and still fails the same way. Perhaps it needs another flag?

[gpshead]

Ah... nevermind. Solved for me: The ../clif_backend/build_matcher/CMakeConfig.txt file was caching the old empty value so cmake wasn't re-running pkg-config protobuf. Delete that before re-running INSTALL.sh (or rerunning cmake if done manually) and it found everything via the PKG_CONFIG_PATH environment variable and is happily on its way to building.

[gpshead]

Context from an in person discussion: We found that pkg-config is being too smart for our own good vs our current cmake setup... When you have protobuf installed in /usr on the system, pkg-config reads the protobuf.pc file and determines that the necessary -I flag is -I/usr/include which it then removes (thus the empty output) as that is already an implied default include path. Triggering the "empty-string" error.

We should be able to fix that by modifying tools/clif/backend/CMakeLists.txt or something related.

[kkiningh]

@gpshead Gotcha - thanks for looking into it for me. I think your assessment is correct. For completeness, here's the output @mrovner asked for:

kkiningh@mew:~$ pkg-config --cflags-only-I protobuf

kkiningh@mew:~$ cat /usr/lib/pkgconfig/protobuf.pc
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: Protocol Buffers
Description: Google's Data Interchange Format
Version: 3.3.0
Libs: -L${libdir} -lprotobuf -pthread  -lpthread
Libs.private: -lz
Cflags: -I${includedir} -pthread
Conflicts: protobuf-lite

Also, for anyone who comes across this thread later, I was able to workaround this by commenting out lines containing "{GOOGLE_PROTOBUF_INCLUDE_DIRS}" in the relevant CMakeLists.txt (specifically, lines 33 and 37 in tools/clif/backend/CMakeLists.txt).

[suvayu]

I'm facing the same problem on F25, with a fresh clone of clif. Here are my outputs for what @mrovner asked for:

$ cat /usr/lib64/pkgconfig/protobuf.pc 
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib64
includedir=/usr/include

Name: Protocol Buffers
Description: Google's Data Interchange Format
Version: 2.6.1
Libs: -L${libdir} -lprotobuf  -lpthread
Libs.private: -lz 
Cflags: -I${includedir} 
# Commented out because it crashes pkg-config *sigh*:
#   http://bugs.freedesktop.org/show_bug.cgi?id=13265
# Conflicts: protobuf-lite

The output of pkg-config --cflags-only-I protobuf is empty, understandably so since the headers are in /usr/include.

[mrovner]

Sorry about the breakage. This will be fixed in the next "release" this month. Was busy at PyCon.
Meanwhile "workaround this by commenting out lines containing
"{GOOGLE_PROTOBUF_INCLUDE_DIRS}" in the relevant CMakeLists.txt (specifically, lines 33 and 37 in tools/clif/backend/CMakeLists.txt)."

[suvayu]

@mrovner Thanks! Took me a while to find the right CMakeLists.txt file. I tend to get very confused with cmake :-p; and now I'm noticing the previous comment linked to the appropriate lines in the source!

Sorry about the noise.

[gpshead]

I suggest opening a new issue against the current version if build issues are still happening. I run into a few myself depending on what platform I am on, I'm still trying to sort through them to find things that are my own errors vs things we could improve in INSTALL.sh.

[rwgk]

This repo was unmaintained from December 2017 to September 2019.
It was recently updated, see #36 (comment) for some background.

If this issue is still relevant please repopen it or ask questions at https://github.com/google/clif/discussions/.