[CFP] Clang vendor options: control new Clang and LLVM changes in a release

[hyp]

Title

Clang vendor options: control new Clang and LLVM changes in a release

Author

Alex Lorenz, arphaman@gmail.com / aleksei_lorenz@apple.com, Apple

Distribution

Apple's Xcode toolchain, used to build Apple's operating systems like iOS and macOS, and also by the App Store developers

Abstract (optional)

Clang and LLVM are constantly improving and are getting 100s of commits per week! Benefiting from all these changes in existing projects comes with unique challenges, for example, the existing code might not be compatible with newly added errors and warnings.

This talk presents clang vendor options and vendor flags. They can be used to selectively control how new Clang and LLVM changes are enabled. For example, a downstream vendor can use them to disable certain new features and/or diagnostics, to ensure that their compiler can continue compiling their existing code, while waiting for the software changes needed to fix the build failures caused by the new compiler changes. They can also be applied selectively to a subset of the user code. The new vendor options allow Apple’s clang team to test the main-tracking branch of Apple Clang on a continuous basis, and also simplifies the process of creating and managing new release branches.

What's unique about the environment you package LLVM for? (optional)

What makes your distribution of LLVM unique? (optional)

What might others learn from your experience? (optional)

What could be improved in upstream LLVM to make working with it easier as a downstream packager? (optional)

---

Reminder that this is meant to be a 15 minute lightning talk; enough to pique
interests but follow up should be done after. Slides can always include links
to more info; we will ask that you send a PR to this repo with your slides when
they are ready.

[nickdesaulniers]

Thanks for taking the time to write up a CFP; we'd be overjoyed to have you present at LLVM Distributors Conf 2021! If you still plan on presenting, this is a reminder to get started on your slides for next week. Once they're done, we will contact you about submitting a PDF of your slides as either a pull request to this repository or via email to the organizer. We hope to have a schedule finalized by EOW; we may iterate on the schedule based on whether presenters have conflicts. Please keep this issue open for attendees to ask questions, or close this issue if you no longer plan on attending. Reminder to keep your talk concise (15 minutes); we wont be doing time for questions in order to fit as much content as possible. Attendees should ask questions here in this github issue.

Also, @tstellar suggested this week gating -Wall on a release number, like -Wall=14 or something, since newly added warnings for new major compiler revisions can sometimes cause a lot of pain (especially if -Werror was enabled).

[jrtc27]

Some build systems already have a hard enough time detecting feature and diagnostic support (often to turn them off) in compiler versions, not helped by the fact that Apple's Clang lies in its version numbers such that you have to know it's Apple's Clang and thus have a completely separate set of version number thresholds for it. How do you intend to mitigate the risk that providing such a mechanism to vendors will lead to a mess for downstreams trying to make sure their software supports all versions of Clang that exist in the wild? Will there be __has_feature or equivalent functions that can be used to detect the features vendors can choose to enable/disable, for example?

[nickdesaulniers]

not helped by the fact that Apple's Clang lies in its version numbers such that you have to know it's Apple's Clang and thus have a completely separate set of version number thresholds for it

At least for AOSP's distro of clang, we often pick up the major version bump when rebasing on upstream. We need to ship features ASAP in the toolchain that various features under development rely on. I think adding a -prerelease suffix to the version number would help developers (and tooling) understand that this isn't equivalent to the actual major version number release from upstream.

[stephenhines]

not helped by the fact that Apple's Clang lies in its version numbers such that you have to know it's Apple's Clang and thus have a completely separate set of version number thresholds for it

At least for AOSP's distro of clang, we often pick up the major version bump when rebasing on upstream. We need to ship features ASAP in the toolchain that various features under development rely on. I think adding a -prerelease suffix to the version number would help developers (and tooling) understand that this isn't equivalent to the actual major version number release from upstream.

We're worse than this, and also have version numbering problems (maybe not as bad though). We do change the major version number once upstream changes it, so ours is labeled 14 right now, even though 14 isn't done yet. It also means that should someone try to use this particular toolchain after 14 is official, it might be missing an expected feature that landed later. It used to be that the version info would append an svn suffix. I agree that having that makes things way easier to understand where it is at (and also that 13 < 14-prerelease < 14 looks a whole lot nicer, even if you have to parse more of a string instead of just an integer).

[hyp]

suggested this week gating -Wall on a release number, like -Wall=14 or something, since newly added warnings for new major compiler revisions can sometimes cause a lot of pain (especially if -Werror was enabled).

That's a good idea. For us, I think that we don't really want to rely on version-gating diagnostics because our versions don't align with the upstream LLVM versions, although I think it might be a good idea to still support that upstream.

How do you intend to mitigate the risk that providing such a mechanism to vendors will lead to a mess for downstreams trying to make sure their software supports all versions of Clang that exist in the wild? Will there be __has_feature or equivalent functions that can be used to detect the features vendors can choose to enable/disable, for example?

I'm not sure I understand your specific concern here. For us specifically our internal software only supports the version of clang we control and adopt for our internal software, so we don't care about supporting all versions of Clang. Some of these flags can apply to the public Apple clang shipped in Xcode, but typically they should have limited practical effect on existing software, since essentially we're delaying enabling new upstream features in Apple clang, so if the software already built with the previous clang, it should hopefully still build without issues with the new release of Apple clang.

I agree that Apple clang versioning scheme is not ideal but just to clarify, it's not meant to lie and pretend its LLVM 11 or 12 or 13 and the numbers don't align with the upstream versions at all. It's been aligned with the Xcode versioning scheme so far, but now that LLVM itself reached versions 11 , 12 and 13, the numbers started to collide causing confusion. Unfortunately I don't think there's a good resolution for this issue as the versioning conflicts already occurred.