[C++20][Modules] '#pragma once' is not well supported

[CCCptH]

I am trying to write some codes in C++20 standard. I write some simple code in C++ module way. And I write #include <iostream> because import <iostream> can not be compiled in clang-15.0.2 compiler. Then I meet compile error.

The code is:

// md.ixx
module;
#include <iostream>
export module md;

export int func() {
    return 114514;
}

// main.cpp
#include <iostream>
import md;

using namespace std;

int main() {
    auto val = func();
    cout<<val<<endl;
    return 0;
}

Compile error is:

error: main.cpp:10:5: error: missing '#include <iostream>'; 'cout' must be declared before it is used
    cout<<val<<endl;
    ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\iostream:40:57: note: declaration here is not visible
__PURE_APPDOMAIN_GLOBAL extern _CRTDATA2_IMPORT ostream cout;
                                                        ^
main.cpp:10:16: error: missing '#include <ostream>'; 'endl' must be declared before it is used
    cout<<val<<endl;
               ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:1000:51: note: declaration here is not visible
basic_ostream<_Elem, _Traits>& __CLRCALL_OR_CDECL endl(
                                                  ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:269:9: error: missing '#include <xiosbase>'; 'ios_base' must be declared before it is used
        ios_base::iostate _State = ios_base::goodbit;
        ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\xiosbase:170:28: note: declaration here is not visible
class _CRTIMP2_PURE_IMPORT ios_base : public _Iosb<int> { // base class for ios

And If I delete #include<iostream> in file md.ixx. It can be successfully compiled.

So is there any solution to using <iostream> in both main.cpp and md.ixx?

[llvmbot]

@llvm/issue-subscribers-clang-frontend

[llvmbot]

@llvm/issue-subscribers-clang-modules

[aaronmondal]

@CCCptH Could you post the command lines used to compile these files?

There are a few cases around lambdas and template resolution that don't yet fully work with modules. If I am not mistaken, your example uses the Microsoft C++ standard library. You would probably have to apply a workaround in the headers for that library.

[CCCptH]

@CCCptH Could you post the command lines used to compile these files?

There are a few cases around lambdas and template resolution that don't yet fully work with modules. If I am not mistaken, your example uses the Microsoft C++ standard library. You would probably have to apply a workaround in the headers for that library.

My env is :

• windows11
• clang15.0.2

I use xmake to build the program. The command line that is used is :

clang -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -fmodules -fbuiltin-module-map -fimplicit-modules -fno-implicit-module-maps -fno-ms-compatibility -fmodule-map-file=./module.modulemap -DNDEBUG -fmodule-file=build\.gens\md\windows\x64\release\rules\modules\cache\md.pcm -o build\.objs\md\windows\x64\release\main.cpp.obj main.cpp

Thanks a lot.

[aaronmondal]

@CCCptH Hmm something looks wrong with these commands.

C++20 modules should not be confused with the very similar, but compiler-specific Clang modules. Standard C++ modules do not rely on a module.modulemap file. See https://clang.llvm.org/docs/Modules.html for (non-standard, compiler-specific) Clang modules, and https://clang.llvm.org/docs/StandardCPlusPlusModules.html for standard C++ modules. If I am not mistaken the .pcm files for standard and non-standard modules differ, so that may be a cause of this error. I may be wrong here though.

I think I vaguely remember similar errors to the one you describe being caused by incorrect mixture of the two module variants.

[CCCptH]

@CCCptH Hmm something looks wrong with these commands.

C++20 modules should not be confused with the very similar, but compiler-specific Clang modules. Standard C++ modules do not rely on a module.modulemap file. See https://clang.llvm.org/docs/Modules.html for (non-standard, compiler-specific) Clang modules, and https://clang.llvm.org/docs/StandardCPlusPlusModules.html for standard C++ modules. If I am not mistaken the .pcm files for standard and non-standard modules differ, so that may be a cause of this error. I may be wrong here though.

I think I vaguely remember similar errors to the one you describe being caused by incorrect mixture of the two module variants.

It is still not working. Env is windows11 and clang15.0.2.

The code is :

// main.cpp
import md;
#include <iostream>
using namespace std;

int main() {
    auto val = func();
    cout<<val<<endl;
    return 0;
}

// md.cppm
module;
#include <iostream>
export module md;

export int func() {
    return 0;
}

The command line I used is:

clang++ -std=c++20 ./md.cppm --precompile -o ./md.pcm                  
clang++ -std=c++20 main.cpp -fprebuilt-module-path=. md.pcm -o main.exe

The compile error is:

main.cpp:4:17: warning: using directive refers to implicitly-defined namespace 'std'
using namespace std;
                ^
main.cpp:8:5: error: missing '#include <iostream>'; 'cout' must be declared before it is used
    cout<<val<<endl;
    ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\iostream:40:57: note: declaration here is not visible
__PURE_APPDOMAIN_GLOBAL extern _CRTDATA2_IMPORT ostream cout;
                                                        ^
main.cpp:8:16: error: missing '#include <ostream>'; 'endl' must be declared before it is used
    cout<<val<<endl;
               ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:1000:51: note: declaration here is not visible
basic_ostream<_Elem, _Traits>& __CLRCALL_OR_CDECL endl(
                                                  ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:273:39: error: no member named 'std' in the global namespace
            const _Nput& _Nput_fac  = _STD use_facet<_Nput>(this->getloc());
                                      ^~~~
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\yvals_core.h:1443:22: note: expanded from macro '_STD'
#define _STD       ::std::
                   ~~^
In module 'md' imported from main.cpp:2:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:273:54: error: unexpected type name '_Nput': expected expression
            const _Nput& _Nput_fac  = _STD use_facet<_Nput>(this->getloc());
                                                     ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:269:9: error: use of undeclared identifier 'ios_base'
        ios_base::iostate _State = ios_base::goodbit;
        ^
main.cpp:8:9: note: in instantiation of member function 'std::basic_ostream<char>::operator<<' requested here
    cout<<val<<endl;
            ^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:1003:11: note: in instantiation of member function 'std::basic_ostream<char>::flush' requested here
    _Ostr.flush();
          ^
main.cpp:8:16: note: in instantiation of function template specialization 'std::endl<char, std::char_traits<char>>' requested here
    cout<<val<<endl;
               ^
In module 'md' imported from main.cpp:2:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\include\ostream:184:16: error: use of undeclared identifier '_Pfn'
        return _Pfn(*this);
               ^
main.cpp:8:14: note: in instantiation of member function 'std::basic_ostream<char>::operator<<' requested here
    cout<<val<<endl;
             ^
1 warning and 15 errors generated.

It seems that I can not #include <iostream> in both md.cppm and main.cpp

[aaronmondal]

Ok it seems like this is indeed an incompatibility with the Microsoft C++ library. It works with libcxx and this patch: #57222 (comment).

[aaronmondal]

This appears to not be Windows related. I'm now hitting similar issues (regarding visibility) when #includeing spdlog and building for linux. I'll try to create a minimal reproducer.

I'm speculating that this has something to do with nested namespaces and/or using directives, but I couldn't pinpoint it yet.

[aaronmondal]

This seems to be caused by a bug that occurs when using #pragma once. Related issues are #38532 and #38554.

If there is an error like some_symbol not visible or unknown type SOME_MACRO, adding include guards to the files that declare some_typename or SOME_MACRO before the first occurrence of #pragma once will work around this bug.

[ChuanqiXu9]

It looks like the problem exists after #58716, doesn't it? I think a minimal reproducer in linux would be pretty helpful. (It would be much better if the minimal reproducer doesn't contain #include to std headers since there are too many things in std headers.... The most expected minimal reproducer would be something like: the one in #58716 (comment))

[aaronmondal]

@ChuanqiXu9 Yes the issue still persists after #58716. However, this seems to be the last remaining issue I could find when trying to build libraries like spdlog and nlohmann/json. So I think as soon as this is resolved, most libraries will be buildable (or at least raise less intimidating errors 😆). I'll try to get a reproducer ready 👍

[aaronmondal]

@ChuanqiXu9 I think I could pin it down. It seems like every #pragma once, when included in both an interface and an implementation unit will only be, well, included once 😄, but the symbols don't make it into the implementation unit. I assume that since the BMI for the interface already includes the header with the pragma, the implementation unit will register the header itself as already present, but the contents of that header don't have the correct visibility. This leads to the various kinds of invisibility errors that users have described.

// invisible.h
#pragma once // This breaks things.
const int kInvisibleSymbol = 0;
struct invisible_struct
{};
#define INVISIBLE_DEFINE

// visible.h
#include "invisible.h"
const int kSadlyUndeclaredSymbol = kInvisibleSymbol;
using unfortunately_still_invisible_struct = invisible_struct;
#ifndef INVISIBLE_DEFINE
#    error "Still not defined."
#endif

// implementation.cpp
module;
#include "visible.h"
module mymodule;

// interface.cppm
module;
#include "visible.h"
export module mymodule;

The step that fails is the compilation of implementation.cpp with errors like

visible.h:3:41: error: missing '#include "invisible.h"'; 'kInvisibleSymbol' must be declared before it is used
const int kSadlyUndeclaredSymbol = kInvisibleSymbol;
                                        ^
invisible.h:3:11: note: declaration here is not visible
const int kInvisibleSymbol = 0;
          ^
In file included from implementation.cpp:3:
visible.h:5:46: error: missing '#include "invisible.h"'; 'invisible_struct' must be declared before it is used
using unfortunately_still_invisible_struct = invisible_struct;
                                             ^
invisible.h:5:8: note: declaration here is not visible
struct invisible_struct
       ^
In file included from implementation.cpp:3:
visible.h:8:6: error: "Still not defined."
#    error "Still not defined."
     ^
3 errors generated.

[zygoloid]

#pragma once basically doesn't work in Clang's modules implementation -- if Clang can see any inclusion of the header in any loaded AST file, it won't include it, even if the modules in those AST files aren't imported.

It seems undesirable to implement another system for tracking which included headers are transitively visible through inclusions in order to support #pragma once, so I wonder if we can leverage / generalize the existing system for tracking macro definitions. For example, I wonder if we could do something akin to inventing a unique macro name for each #pragma once, and then preventing a re-inclusion of a header if its unique "macro name" has a visible "macro definition".

[ChuanqiXu9]

if Clang can see any inclusion of the header in any loaded AST file, it won't include it, even if the modules in those AST files aren't imported.

Oh, it makes sense if you refer the header as the one with #pragma once. So visible.h is included in implementation.cpp but invisible.h is not included in implementation.cpp. And implementation.cpp can't find the entities in GMF of other TU. This is the problem. So we can explain every thing here.

It seems undesirable to implement another system for tracking which included headers are transitively visible through inclusions in order to support #pragma once, so I wonder if we can leverage / generalize the existing system for tracking macro definitions. For example, I wonder if we could do something akin to inventing a unique macro name for each #pragma once, and then preventing a re-inclusion of a header if its unique "macro name" has a visible "macro definition".

Cool, it looks like a possible solution.

And I am wondering if we can solve the problem by treating every included header in a named module as invisible. I feel like this is workable since named modules have different semantics than header modules.

[davidstone]

Copying my minimal reproducer from #59708

// a.hpp
#pragma once
using a = int;

// b.hpp
#pragma once
#include "a.hpp"
a b;

// c.cpp
module;
#include "b.hpp"
export module c;

Causes Clang to error with

In file included from c.cpp:3:
b.hpp:4:1: error: unknown type name 'a'
a b;
^

[davidstone]

I think this is in fact a duplicate of #38554.