Skip to content

Commit 0cbec5c

Browse files
dean0x7dwjakob
dean0x7d
authored and
wjakob
committed
Add new options and docs for pybind11_add_module
See the documentation for a description of the options.
1 parent b0f3885 commit 0cbec5c

File tree

4 files changed

+145
-67
lines changed

4 files changed

+145
-67
lines changed

‎docs/compiling.rst

Lines changed: 93 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -39,30 +39,88 @@ extension module can be created with just a few lines of code:
3939
4040
This assumes that the pybind11 repository is located in a subdirectory named
4141
:file:`pybind11` and that the code is located in a file named :file:`example.cpp`.
42-
The CMake command ``add_subdirectory`` will import a function with the signature
43-
``pybind11_add_module(<name> source1 [source2 ...])``. It will take care of all
44-
the details needed to build a Python extension module on any platform.
45-
46-
The target Python version can be selected by setting the ``PYBIND11_PYTHON_VERSION``
47-
variable before adding the pybind11 subdirectory. Alternatively, an exact Python
48-
installation can be specified by setting ``PYTHON_EXECUTABLE``.
42+
The CMake command ``add_subdirectory`` will import the pybind11 project which
43+
provides the ``pybind11_add_module`` function. It will take care of all the
44+
details needed to build a Python extension module on any platform.
4945

5046
A working sample project, including a way to invoke CMake from :file:`setup.py` for
5147
PyPI integration, can be found in the [cmake_example]_ repository.
5248

5349
.. [cmake_example] https://github.com/pybind/cmake_example
5450
55-
For CMake-based projects that don't include the pybind11
56-
repository internally, an external installation can be detected
57-
through `find_package(pybind11 ... CONFIG ...)`. See the `Config file
58-
<https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in>`_
59-
docstring for details of relevant CMake variables.
51+
pybind11_add_module
52+
-------------------
53+
54+
To ease the creation of Python extension modules, pybind11 provides a CMake
55+
function with the following signature:
56+
57+
.. code-block:: cmake
58+
59+
pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
60+
[NO_EXTRAS] [THIN_LTO] source1 [source2 ...])
61+
62+
This function behaves very much like CMake's builtin ``add_library`` (in fact,
63+
it's a wrapper function around that command). It will add a library target
64+
called ``<name>`` to be built from the listed source files. In addition, it
65+
will take care of all the Python-specific compiler and linker flags as well
66+
as the OS- and Python-version-specific file extension. The produced target
67+
``<name>`` can be further manipulated with regular CMake commands.
68+
69+
``MODULE`` or ``SHARED`` may be given to specify the type of library. If no
70+
type is given, ``MODULE`` is used by default which ensures the creation of a
71+
Python-exclusive module. Specifying ``SHARED`` will create a more traditional
72+
dynamic library which can also be linked from elsewhere. ``EXCLUDE_FROM_ALL``
73+
removes this target from the default build (see CMake docs for details).
74+
75+
Since pybind11 is a template library, ``pybind11_add_module`` adds compiler
76+
flags to ensure high quality code generation without bloat arising from long
77+
symbol names and duplication of code in different translation units. The
78+
additional flags enable LTO (Link Time Optimization), set default visibility
79+
to *hidden* and strip unneeded symbols. See the :ref:`FAQ entry <faq:symhidden>`
80+
for a more detailed explanation. These optimizations are never applied in
81+
``Debug`` mode. If ``NO_EXTRAS`` is given, they will always be disabled, even
82+
in ``Release`` mode. However, this will result in code bloat and is generally
83+
not recommended.
84+
85+
As stated above, LTO is enabled by default. Some newer compilers also support
86+
different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
87+
the function to prefer this flavor if available. The function falls back to
88+
regular LTO if ``-flto=thin`` is not available.
89+
90+
.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html
91+
92+
Configuration variables
93+
-----------------------
94+
95+
By default, pybind11 will compile modules with the latest C++ standard
96+
available on the target compiler. To override this, the standard flag can
97+
be given explicitly in ``PYBIND11_CPP_STANDARD``:
98+
99+
.. code-block:: cmake
100+
101+
set(PYBIND11_CPP_STANDARD -std=c++11)
102+
add_subdirectory(pybind11) # or find_package(pybind11)
103+
104+
Note that this and all other configuration variables must be set **before** the
105+
call to ``add_subdiretory`` or ``find_package``. The variables can also be set
106+
when calling CMake from the command line using the ``-D<variable>=<value>`` flag.
107+
108+
The target Python version can be selected by setting ``PYBIND11_PYTHON_VERSION``
109+
or an exact Python installation can be specified with ``PYTHON_EXECUTABLE``.
110+
For example:
60111

61-
Once detected, and after setting any variables to guide Python and
62-
C++ standard detection, the aforementioned ``pybind11_add_module``
63-
wrapper to ``add_library`` can be employed as described above (after
64-
``include(pybind11Tools)``). This procedure is available when using CMake
65-
>= 2.8.12. A working example can be found at [test_installed_module]_ .
112+
.. code-block:: bash
113+
114+
cmake -DPYBIND11_PYTHON_VERSION=3.6 ..
115+
# or
116+
cmake -DPYTHON_EXECUTABLE=path/to/python ..
117+
118+
find_package vs. add_subdirectory
119+
---------------------------------
120+
121+
For CMake-based projects that don't include the pybind11 repository internally,
122+
an external installation can be detected through ``find_package(pybind11)``.
123+
See the `Config file`_ docstring for details of relevant CMake variables.
66124

67125
.. code-block:: cmake
68126
@@ -72,27 +130,32 @@ wrapper to ``add_library`` can be employed as described above (after
72130
find_package(pybind11 REQUIRED)
73131
pybind11_add_module(example example.cpp)
74132
75-
.. [test_installed_module] https://github.com/pybind/pybind11/blob/master/tests/test_installed_module/CMakeLists.txt
133+
Once detected, the aforementioned ``pybind11_add_module`` can be employed as
134+
before. The function usage and configuration variables are identical no matter
135+
if pybind11 is added as a subdirectory or found as an installed package. You
136+
can refer to the same [cmake_example]_ repository for a full sample project
137+
-- just swap out ``add_subdirectory`` for ``find_package``.
138+
139+
.. _Config file: https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in
140+
141+
Advanced: interface library target
142+
----------------------------------
76143

77-
When using a version of CMake greater than 3.0, pybind11 can
78-
additionally be used as a special *interface library* following the
79-
call to ``find_package``. CMake variables to guide Python and C++
80-
standard detection should be set *before* ``find_package``. When
81-
``find_package`` returns, the target ``pybind11::pybind11`` is
82-
available with pybind11 headers, Python headers and libraries as
83-
needed, and C++ compile definitions attached. This target is suitable
84-
for linking to an independently constructed (through ``add_library``,
85-
not ``pybind11_add_module``) target in the consuming project. A working
86-
example can be found at [test_installed_target]_ .
144+
When using a version of CMake greater than 3.0, pybind11 can additionally
145+
be used as a special *interface library* . The target ``pybind11::pybind11``
146+
is available with pybind11 headers, Python headers and libraries as needed,
147+
and C++ compile definitions attached. This target is suitable for linking
148+
to an independently constructed (through ``add_library``, not
149+
``pybind11_add_module``) target in the consuming project.
87150

88151
.. code-block:: cmake
89152
90153
cmake_minimum_required(VERSION 3.0)
91154
project(example)
92155
93-
add_library(example MODULE main.cpp)
156+
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
94157
95-
find_package(pybind11 REQUIRED)
158+
add_library(example MODULE main.cpp)
96159
target_link_libraries(example PRIVATE pybind11::pybind11)
97160
set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}"
98161
SUFFIX "${PYTHON_MODULE_EXTENSION}")
@@ -111,6 +174,3 @@ example can be found at [test_installed_target]_ .
111174
(``-fvisibility=hidden``) and .OBJ files with many sections on Visual Studio
112175
(``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an
113176
explanation on why these are needed.
114-
115-
.. [test_installed_target] https://github.com/pybind/pybind11/blob/master/tests/test_installed_target/CMakeLists.txt
116-

‎tests/test_cmake_build/installed_function/CMakeLists.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ set(CMAKE_MODULE_PATH "")
66
find_package(pybind11 CONFIG REQUIRED)
77
message(STATUS "Found pybind11 v${pybind11_VERSION}: ${pybind11_INCLUDE_DIRS}")
88

9-
pybind11_add_module(test_cmake_build SHARED ../main.cpp)
9+
pybind11_add_module(test_cmake_build SHARED NO_EXTRAS ../main.cpp)
1010

1111
add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build>
1212
${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME})

‎tests/test_cmake_build/subdirectory_function/CMakeLists.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 2.8.12)
22
project(test_subdirectory_module CXX)
33

44
add_subdirectory(${PYBIND11_PROJECT_DIR} pybind11)
5-
pybind11_add_module(test_cmake_build ../main.cpp)
5+
pybind11_add_module(test_cmake_build THIN_LTO ../main.cpp)
66

77
add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build>
88
${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME})

‎tools/pybind11Tools.cmake

Lines changed: 50 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ set(Python_ADDITIONAL_VERSIONS 3.7 3.6 3.5 3.4)
1414
find_package(PythonLibsNew ${PYBIND11_PYTHON_VERSION} REQUIRED)
1515

1616
include(CheckCXXCompilerFlag)
17+
include(CMakeParseArguments)
1718

1819
function(select_cxx_standard)
1920
if(NOT MSVC AND NOT PYBIND11_CPP_STANDARD)
@@ -33,29 +34,48 @@ function(select_cxx_standard)
3334
endif()
3435
endfunction()
3536

37+
# Internal: find the appropriate LTO flag for this compiler
38+
macro(_pybind11_find_lto_flag output_var prefer_thin_lto)
39+
if(${prefer_thin_lto})
40+
# Check for ThinLTO support (Clang)
41+
check_cxx_compiler_flag("-flto=thin" HAS_THIN_LTO_FLAG)
42+
set(${output_var} $<${HAS_THIN_LTO_FLAG}:-flto=thin>)
43+
endif()
44+
45+
if(NOT ${prefer_thin_lto} OR NOT HAS_THIN_LTO_FLAG)
46+
if(NOT CMAKE_CXX_COMPILER_ID MATCHES "Intel")
47+
# Check for Link Time Optimization support (GCC/Clang)
48+
check_cxx_compiler_flag("-flto" HAS_LTO_FLAG)
49+
set(${output_var} $<${HAS_LTO_FLAG}:-flto>)
50+
else()
51+
# Intel equivalent to LTO is called IPO
52+
check_cxx_compiler_flag("-ipo" HAS_IPO_FLAG)
53+
set(${output_var} $<${HAS_IPO_FLAG}:-ipo>)
54+
endif()
55+
endif()
56+
endmacro()
57+
3658
# Build a Python extension module:
37-
# pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL] source1 [source2 ...])
59+
# pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
60+
# [NO_EXTRAS] [THIN_LTO] source1 [source2 ...])
3861
#
3962
function(pybind11_add_module target_name)
40-
set(lib_type "MODULE")
41-
set(do_lto True)
42-
set(exclude_from_all "")
43-
set(sources "")
44-
45-
set(_args_to_try "${ARGN}")
46-
foreach(_ex_arg IN LISTS _args_to_try)
47-
if(${_ex_arg} STREQUAL "MODULE")
48-
set(lib_type "MODULE")
49-
elseif(${_ex_arg} STREQUAL "SHARED")
50-
set(lib_type "SHARED")
51-
elseif(${_ex_arg} STREQUAL "EXCLUDE_FROM_ALL")
52-
set(exclude_from_all "EXCLUDE_FROM_ALL")
53-
else()
54-
list(APPEND sources "${_ex_arg}")
55-
endif()
56-
endforeach()
63+
set(options MODULE SHARED EXCLUDE_FROM_ALL NO_EXTRAS THIN_LTO)
64+
cmake_parse_arguments(ARG "${options}" "" "" ${ARGN})
65+
66+
if(ARG_MODULE AND ARG_SHARED)
67+
message(FATAL_ERROR "Can't be both MODULE and SHARED")
68+
elseif(ARG_SHARED)
69+
set(lib_type SHARED)
70+
else()
71+
set(lib_type MODULE)
72+
endif()
5773

58-
add_library(${target_name} ${lib_type} ${exclude_from_all} ${sources})
74+
if(ARG_EXCLUDE_FROM_ALL)
75+
set(exclude_from_all EXCLUDE_FROM_ALL)
76+
endif()
77+
78+
add_library(${target_name} ${lib_type} ${exclude_from_all} ${ARG_UNPARSED_ARGUMENTS})
5979

6080
target_include_directories(${target_name}
6181
PRIVATE ${PYBIND11_INCLUDE_DIR} # from project CMakeLists.txt
@@ -86,7 +106,7 @@ function(pybind11_add_module target_name)
86106

87107
target_link_libraries(${target_name} PRIVATE "-undefined dynamic_lookup")
88108

89-
if(${lib_type} STREQUAL "SHARED")
109+
if(ARG_SHARED)
90110
# Suppress CMake >= 3.0 warning for shared libraries
91111
set_target_properties(${target_name} PROPERTIES MACOSX_RPATH ON)
92112
endif()
@@ -96,23 +116,21 @@ function(pybind11_add_module target_name)
96116
if(NOT MSVC)
97117
# Make sure C++11/14 are enabled
98118
target_compile_options(${target_name} PUBLIC ${PYBIND11_CPP_STANDARD})
119+
endif()
99120

121+
if(ARG_NO_EXTRAS)
122+
return()
123+
endif()
124+
125+
if(NOT MSVC)
100126
# Enable link time optimization and set the default symbol
101127
# visibility to hidden (very important to obtain small binaries)
102128
string(TOUPPER "${CMAKE_BUILD_TYPE}" U_CMAKE_BUILD_TYPE)
103129
if (NOT ${U_CMAKE_BUILD_TYPE} MATCHES DEBUG)
104-
# Check for Link Time Optimization support (GCC/Clang)
105-
check_cxx_compiler_flag("-flto" HAS_LTO_FLAG)
106-
if(HAS_LTO_FLAG AND NOT CYGWIN)
107-
target_compile_options(${target_name} PRIVATE -flto)
108-
endif()
109-
110-
# Intel equivalent to LTO is called IPO
111-
if(CMAKE_CXX_COMPILER_ID MATCHES "Intel")
112-
check_cxx_compiler_flag("-ipo" HAS_IPO_FLAG)
113-
if(HAS_IPO_FLAG)
114-
target_compile_options(${target_name} PRIVATE -ipo)
115-
endif()
130+
# Link Time Optimization
131+
if(NOT CYGWIN)
132+
_pybind11_find_lto_flag(lto_flag ARG_THIN_LTO)
133+
target_compile_options(${target_name} PRIVATE ${lto_flag})
116134
endif()
117135

118136
# Default symbol visibility

0 commit comments

Comments
 (0)