ldProgram is misleading

[bgamari]

Cabal has long had ldProgram :: Program, which gets resolved to the ld executable. One would be forgiven for thinking that Cabal would use this program for linking in some cases. Furthermore, LocalBuildInfo has an ldOptions field which one might think should get passed to ldProgram. Both of these beliefs are mistaken. The reality is much more complicated:

• typically (read: almost always) the Haskell compiler is responsible for linking
• Cabal passes the options of programOverrideArgs ldProgram to the Haskell compiler during when linking
• in the case of GHC (and, I would guess, most implementations) the Haskell compiler uses the C compiler to link, not ld
• When probing foreign dependencies, Cabal does do some linking on its own. However, it, like GHC, uses the C compiler to link (and passes LBI.ldOptions, but not programOverrideArgs ldProgram, when doing so.

All of this is understandable; GHC was also quite confused about linking for quite a long time. However, the status quo is becoming increasingly problematic as cross-compilation becomes more common. In particular, recently GHC found that Emscripten's toolchain happily accepts the --target flag when invoked as a linker yet not during compilation (see ghc#23744). To manage this, GHC had to begin making a distinction between cc and cc-ld tools (that is, the C compiler used for compilation and the C compiler as used for linking).

Ultimately I'm not familiar enough with Cabal's design vision to know what a better design might look like but I think it's something that we may want to consider.

[angerman]

Cabal should just defer linking to GHC. At least for GHC. It's the same issue we have with excessive libraries. Of which both GHC and cabal need to implement the logic instead of fixing it in GHC only and relying on Cabal to delegate linking to GHC. Maintaining the functionality across two codebases and all the permitted combinations thereof is just exhausting.

[andreabedini]

When probing foreign dependencies, Cabal does do some linking on its own

I guess the next step would be to figure out when and where we have to do this.

[dcoutts]

Historically Cabal had to use ld directly for some things, even with ghc. IIRC, it was for creating GHCi .o files.

Also note that Cabal in theory is multi-compiler, and so knowing about ld can still make sense for other compilers, even if we don't use it for ghc.

And yes, Cabal doesn't really know about toolchains, and thus not about different sets of toolchains, as needed for proper cross-compilation.

The Programs abstraction is not a toolchain abstraction. It's just a mechanism to call programs, and be somewhat consistent about how to find them, use any extra flags etc etc.

What Cabal needs for cross-compilation support is a new abstraction (lets call it Toolchain!) which would bundle together all the tools, and know about the distinction between host, target etc. And of course then we can have multiple values of such.

[bgamari]

My sense here is that @angerman is correct: Cabal needn't (and shouldn't) worry about the toolchain. GHC already has all of the knowledge necessary to drive the native toolchain (including a complete Toolchain abstraction of its own) and exposes the necessary abstractions to expose this functionality to Cabal (including object merging with the introduction of --merge-objs).

[angerman]

I'm not saying cabal didn't need this in the past. But looking at the problem today, as much as we delegate linking of c programs often to the respective c compiler cc or cxx, I believe we should do the same with GHC too. At least for GHC. The whole mess with Darwin's load command limit meant implementing the same logic in GHC and in cabal. Imo for all things linking concerned cabal should delegate to GHC as the linker. And yes we have flags for --merge-objects to create pre-linked ghci files.

So now we are hacking around this in cabal. And then need to hack around it in GHC again. For cross compilation I also believe cabal should just delegate that to GHC and not come up with its own--just so slightly different--logic.

[dcoutts]

I was unclear by calling the proposed abstraction "toolchain". I didn't mean specifically the native toolchain, just the set of programs that Cabal needs to know about, with separate sets for the host and target (which of course might be the same programs, with different flags).

So Cabal absolutely has to worry about its own toolchain and how that differs from host to target. And Cabal does still need to do things like compile and link C code. And it needs to pass linking flags to appropriate phases. But none of those things are directly related to whether Cabal has to call gcc directly or indirectly.