Missing required module '_FoundationCollections' when building with the Static Linux SDK

[barnard-b]

When building a Swift package using the September 11, 2024 Swift 6 toolchain and matching Static Linux SDK, compilation fails with error: missing required module '_FoundationCollections'

% xcrun --toolchain swift swift build --swift-sdk aarch64-swift-linux-musl   
Building for debugging...
error: emit-module command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'

System info

OS: macOS 14.6.1
Xcode: 16.0 public release, 16.0 release candidate, 15.4
Swift: Apple Swift version 6.0-dev (LLVM fc1036139c5e562, Swift 5bbf725fd958d34) Target: arm64-apple-macosx14.0
Static Linux SDK swift-6.0-DEVELOPMENT-SNAPSHOT-2024-09-11-a_static-linux-0.0.1

Package

Package.swift

// swift-tools-version: 6.0

import PackageDescription

let package = Package(
    name: "FoundationTest",
    targets: [
        .executableTarget(name: "FoundationTest")
    ]
)

Sources/FoundationTest/FoundationTest.swift

import Foundation

@main
struct App {
    static func main() async throws {
        print("Hello World!")
    }
}

[jmschonfeld]

@etcwilde @al45tair any chance you have any insight on what's going on here? For reference, in the standard Linux toolchain we have:

/usr/lib/swift_static/linux/_FoundationCollections.swiftmodule
/usr/lib/swift_static/linux/lib_FoundationCollections.a
/usr/lib/swift/linux/_FoundationCollections.swiftmodule

(in the non-static build, lib_FoundationCollections is statically linked into libFoundationEssentials.so)

but in the static swift Linux SDK, I don't see lib_FoundationCollections.a or _FoundationCollections.swiftmodule, but I do see

musl-1.2.5.sdk/x86_64/usr/lib/swift_static/linux-static/FoundationEssentials.swiftmodule
musl-1.2.5.sdk/x86_64/usr/lib/swift_static/linux-static/libFoundationEssentials.a

Do we have an issue of the packaging of the static SDK where we're missing these files, or is there an issue on the CMake side with swift-collections (that CMake file is at https://github.com/apple/swift-collections/blob/main/Sources/CMakeLists.txt and COLLECTIONS_SINGLE_MODULE and COLLECTIONS_FOUNDATION_TOOLCHAIN_MODULE are defined by swift-foundation)

[finagolfin]

I can reproduce with the just-released static linux SDK 6.0 when simply trying to build swift-argument-parser on Fedora 40 x86_64:

# This file requires XCTest, which isn't in the static linux SDK
> rm Sources/ArgumentParserTestHelpers/TestHelpers.swift
> ../swift-6.0-RELEASE-fedora39/usr/bin/swift build --swift-sdk x86_64-swift-linux-musl
Building for debugging...
error: emit-module command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'
<unknown>:0: error: missing required module '_FoundationCollections'

Simply building with the regular glibc-based static stdlib works fine though, ../swift-6.0-RELEASE-fedora39/usr/bin/swift build --static-swift-stdlib, as it has those missing _FoundationCollections files.

Looking at Evan's last change to the static linux SDK build script to add support for this Foundation rewrite, he added two CMake flags for this, but Jeremy previously added a couple more to the official build-script.

@jmschonfeld, is the _SwiftCollections_SourceDIR flag required for these files to be installed?

[al45tair]

I think @etcwilde is looking at this, but my guess is that the static SDK build script needs tweaking slightly to get CMake to install the additional library files.

[jmschonfeld]

@jmschonfeld, is the _SwiftCollections_SourceDIR flag required for these files to be installed?

Foundation needs some copy of swift-collections to build, but the parameters determine where that source copy comes from. With _SwiftCollections_SourceDIR provided (like in build-script-impl/build.ps1) it will build the swift-collections copy at the given path. Without that parameter provided, it will checkout the necessary tag from GitHub, clone a local copy, and build that copy. In a full toolchain/SDK build environment, I'd expect it to pass _SwiftCollections_SourceDIR but it indeed looks like that's missing from the linked build.sh. However I wouldn't expect that to cause this failure, since it'll just clone a copy from GitHub. Sounds like that's a separate, unrelated issue.

[finagolfin]

Foundation needs some copy of swift-collections to build

Right, but that wasn't the question, ie is it "required for these files to be installed" in the final static SDK? I figured it's somehow getting the source and building it, otherwise the dependency wouldn't be fulfilled and you'd get a compilation failure, but perhaps the variable has to be set for CMake to then finally install that static library and modules?

[jmschonfeld]

perhaps the variable has to be set for CMake to then finally install that static library and modules

No, the variable strictly defines the location of the source checkout for swift-collections

[al45tair]

Having said that I thought @etcwilde was working on it, I went looking. The problem is that swift-collections is trying to work out the system name by lowercasing the CMAKE_SYSTEM_NAME, but that fails for the Static SDK for Linux because the system name there needs to be linux-static. The other Foundation components accept that via the CMake variable SWIFT_SYSTEM_NAME, which gets set by Swift when it's doing the build, but swift-collections ignores that.

I'll have some PRs up tomorrow my time to fix that and a couple of other things I found along the way.

[jmschonfeld]

@al45tair should we move this issue over to the swift-collections repo if we need to make the CMake fix there?

cc @lorentey since this likely entails tagging a new swift-collections version for the toolchain build to pickup with CMake-only changes

[al45tair]

Yes, this probably should be on swift-collections. The other issues I found weren't related to this, but made getting to a fix take a little longer than necessary; I'll deal with those separately.