[BOLT] Detect Linux kernel version if the binary is a Linux kernel #119088

FLZ101 · 2024-12-07T19:01:23Z

This makes it easier to handle differences (e.g. of exception table
entry size) between versions of Linux kernel

llvmbot · 2024-12-07T19:01:58Z

@llvm/pr-subscribers-bolt

Author: Franklin (FLZ101)

Changes

Full diff: https://github.com/llvm/llvm-project/pull/119088.diff

3 Files Affected:

(modified) bolt/include/bolt/Core/BinaryContext.h (+3)
(modified) bolt/lib/Rewrite/RewriteInstance.cpp (+22)
(added) bolt/test/X86/linux-version.s (+30)

diff --git a/bolt/include/bolt/Core/BinaryContext.h b/bolt/include/bolt/Core/BinaryContext.h
index 115e59ca0697e5..052a145efe4a4b 100644
--- a/bolt/include/bolt/Core/BinaryContext.h
+++ b/bolt/include/bolt/Core/BinaryContext.h
@@ -670,6 +670,9 @@ class BinaryContext {
   /// Indicates if the binary is Linux kernel.
   bool IsLinuxKernel{false};
 
+  /// Linux kernel version (major, minor, reversion)
+  std::tuple<unsigned, unsigned, unsigned> LinuxKernelVersion;
+
   /// Indicates if relocations are available for usage.
   bool HasRelocations{false};
 
diff --git a/bolt/lib/Rewrite/RewriteInstance.cpp b/bolt/lib/Rewrite/RewriteInstance.cpp
index 76e1f0156f828d..b88da11701b354 100644
--- a/bolt/lib/Rewrite/RewriteInstance.cpp
+++ b/bolt/lib/Rewrite/RewriteInstance.cpp
@@ -61,6 +61,7 @@
 #include <fstream>
 #include <memory>
 #include <optional>
+#include <regex>
 #include <system_error>
 
 #undef  DEBUG_TYPE
@@ -1030,6 +1031,27 @@ void RewriteInstance::discoverFileObjects() {
       continue;
     }
 
+    if (BC->IsLinuxKernel && SymName == "linux_banner") {
+      const StringRef SectionContents =
+          cantFail(Section->getContents(), "can not get section contents");
+      const std::string S =
+          SectionContents
+              .substr(SymbolAddress - Section->getAddress(), SymbolSize)
+              .str();
+
+      const std::regex Re(R"---(Linux version ((\d+)\.(\d+)(\.(\d+))?))---");
+      std::smatch Match;
+      if (std::regex_search(S, Match, Re)) {
+        unsigned Major = std::stoi(Match[2].str());
+        unsigned Minor = std::stoi(Match[3].str());
+        unsigned Rev = Match.size() > 5 ? std::stoi(Match[5].str()) : 0;
+        BC->LinuxKernelVersion = std::make_tuple(Major, Minor, Rev);
+        BC->outs() << "BOLT-INFO: Linux kernel version is " << Match[1].str();
+      } else {
+        BC->errs() << "BOLT-WARNING: Linux kernel version is unknown\n";
+      }
+    }
+
     if (!Section->isText()) {
       assert(SymbolType != SymbolRef::ST_Function &&
              "unexpected function inside non-code section");
diff --git a/bolt/test/X86/linux-version.s b/bolt/test/X86/linux-version.s
new file mode 100644
index 00000000000000..079910be931cdb
--- /dev/null
+++ b/bolt/test/X86/linux-version.s
@@ -0,0 +1,30 @@
+# REQUIRES: system-linux
+
+## Check that BOLT correctly detects the Linux kernel version
+
+# RUN: llvm-mc -filetype=obj -triple x86_64-unknown-unknown %s -o %t.o
+# RUN: %clang %cflags -nostdlib %t.o -o %t.exe \
+# RUN:   -Wl,--image-base=0xffffffff80000000,--no-dynamic-linker,--no-eh-frame-hdr
+
+# RUN: llvm-bolt %t.exe -o %t.out | FileCheck %s
+
+# CHECK: BOLT-INFO: Linux kernel version is 6.6.61
+
+	.text
+	.globl	f
+	.type	f, @function
+f:
+	ret
+	.size	f, .-f
+
+	.globl	linux_banner
+	.section	.rodata
+	.align 16
+	.type	linux_banner, @object
+	.size	linux_banner, 22
+linux_banner:
+	.string	"Linux version 6.6.61\n"
+
+## Fake Linux Kernel sections.
+  .section __ksymtab,"a",@progbits
+  .section __ksymtab_gpl,"a",@progbits

This makes it easier to handle differences (e.g. of exception table entry size) between versions of Linux kernel

dcci

I don't mind the direction in general, but if you can add a subsequent patch showing how you want to use the functionality, that would be great.

dcci · 2024-12-08T11:56:27Z

bolt/include/bolt/Core/BinaryContext.h

@@ -670,6 +670,9 @@ class BinaryContext {
  /// Indicates if the binary is Linux kernel.
  bool IsLinuxKernel{false};

+  /// Linux kernel version (major, minor, reversion)
+  std::tuple<unsigned, unsigned, unsigned> LinuxKernelVersion;


This maybe should be a struct instead of a tuple

dcci · 2024-12-08T11:56:54Z

bolt/lib/Rewrite/RewriteInstance.cpp

@@ -1205,6 +1225,9 @@ void RewriteInstance::discoverFileObjects() {
    PreviousFunction = BF;
  }

+  if (BC->IsLinuxKernel && !std::get<0>(BC->LinuxKernelVersion))
+    BC->errs() << "BOLT-WARNING: Linux kernel version is unknown\n";


If the version of the kernel is unknown, you just emit a warning instead of erroring out?

If BOLT errors out here, then all Linux kernel related tests need to specify a proper version.

It would be better to do that later when functions like readAltInstructions() and readStaticKeysJumpTable() have been modified to utilize LinuxKernelVersion to handle differences between Linux kernel versions.

I've added a Linux kernel version for each related test and changed the behavior to erroring out.

Use struct instead of tuple to represent Linux kernel version

maksfb

Thank you for the PR.

maksfb · 2024-12-09T02:56:14Z

bolt/include/bolt/Core/BinaryContext.h

@@ -670,6 +698,8 @@ class BinaryContext {
  /// Indicates if the binary is Linux kernel.
  bool IsLinuxKernel{false};

+  LKVersion LinuxKernelVersion;


If we don't expect to use the kernel version outside of the LinuxKernelRewriter, I suggest we move it in there.

maksfb · 2024-12-09T03:00:16Z

bolt/lib/Rewrite/RewriteInstance.cpp

@@ -1030,6 +1031,25 @@ void RewriteInstance::discoverFileObjects() {
      continue;
    }

+    if (BC->IsLinuxKernel && SymName == "linux_banner") {


The reading part also should be in the LinuxKernelRewriter. You can access symbols via BinaryData objects.

maksfb · 2024-12-09T03:00:37Z

bolt/lib/Rewrite/RewriteInstance.cpp

+        unsigned Major = std::stoi(Match[2].str());
+        unsigned Minor = std::stoi(Match[3].str());
+        unsigned Rev = Match.size() > 5 ? std::stoi(Match[5].str()) : 0;


nit: constify.

Move LinuxKernelVersion to LinuxKernelRewriter

FLZ101 · 2024-12-09T18:39:56Z

Multiple symbols with the same address but different sizes are mapped to the same BinaryData, which has a size of the first symbol.

So we usually can not be certain about the size of a BinaryData we get from a symbol, since there might be a symbol with the same address but different size registered first. I think it is not good.

Is it by design?

maksfb · 2024-12-09T19:24:38Z

Multiple symbols with the same address but different sizes are mapped to the same BinaryData, which has a size of the first symbol.

So we usually can not be certain about the size of a BinaryData we get from a symbol, since there might be a symbol with the same address but different size registered first. I think it is not good.

Is it by design?

BinaryData concept was introduced for the purpose of data reordering optimizations some time ago. We didn't pursue the optimization any further (for a number of reasons), but the framework remains. Regarding the sizes, I vaguely recall there was a support for embedded objects that start at the same address and all data objects could be accessed. But I could be wrong. In any case, we are open to changing the BinaryData handling.

maksfb · 2024-12-11T01:01:57Z

Thank you for refactoring the code. Please add tests for the Linux version and include non-standard versions.

FLZ101 · 2024-12-11T13:58:30Z

Thank you for refactoring the code. Please add tests for the Linux version and include non-standard versions.

There is a failed test case, which I have created a PR #119557 to fix.

I'll add missing tests soon.

Add tests

Fix linux-version.S

FLZ101 · 2024-12-16T11:40:57Z

Thank you for refactoring the code. Please add tests for the Linux version and include non-standard versions.

Tests are added.

maksfb

LGTM. Thanks! Please address the nit before committing.

maksfb · 2024-12-17T03:58:34Z

bolt/test/X86/linux-version.S

+linux_banner:
+
+#ifdef A
+  .string  "Linux version 6.6.61\n"


nit: could you please place CHECKs next to the object you are checking against?

Fix evaluation order problem identified in llvm#119088.

Fix evaluation order problem identified in #119088.

FLZ101 · 2024-12-23T02:45:35Z

I don't mind the direction in general, but if you can add a subsequent patch showing how you want to use the functionality, that would be great.

@dcci An example usage of LinuxKernelVersion is included in #120929.

It will be used more in [BOLT][AArch64] Basic support for Linux kernel.

dcci

LGTM

FLZ101 · 2024-12-25T01:49:46Z

@maksfb @dcci Can you merge this PR? I have no write access to this repository.

…FCI (#120491) Fix evaluation order problem identified in llvm/llvm-project#119088.

FLZ101 requested review from aaupov, maksfb, rafaelauler, ayermolo and dcci as code owners December 7, 2024 19:01

llvmbot added the BOLT label Dec 7, 2024

FLZ101 force-pushed the feature-bolt-linux-kernel-version branch 2 times, most recently from 4e99abe to 782c464 Compare December 7, 2024 19:16

[BOLT] Detect Linux kernel version if the binary is a Linux kernel

69554a8

This makes it easier to handle differences (e.g. of exception table entry size) between versions of Linux kernel

FLZ101 force-pushed the feature-bolt-linux-kernel-version branch from 782c464 to 69554a8 Compare December 8, 2024 04:17

dcci reviewed Dec 8, 2024

View reviewed changes

[BOLT] Detect Linux kernel version if the binary is a Linux kernel

6b931b3

Use struct instead of tuple to represent Linux kernel version

FLZ101 force-pushed the feature-bolt-linux-kernel-version branch from 7dc58da to 6b931b3 Compare December 8, 2024 17:04

maksfb reviewed Dec 9, 2024

View reviewed changes

FLZ101 force-pushed the feature-bolt-linux-kernel-version branch 3 times, most recently from 816d1aa to bb683eb Compare December 9, 2024 18:11

[BOLT] Detect Linux kernel version if the binary is a Linux kernel

bb683eb

Move LinuxKernelVersion to LinuxKernelRewriter

FLZ101 added 3 commits December 14, 2024 00:10

[BOLT] Detect Linux kernel version if the binary is a Linux kernel

2268ec6

Add tests

[BOLT] Detect Linux kernel version if the binary is a Linux kernel

e932882

Fix linux-version.S

Merge branch 'main' into feature-bolt-linux-kernel-version

27fd36b

maksfb approved these changes Dec 17, 2024

View reviewed changes

[BOLT] Detect Linux kernel version if the binary is a Linux kernel

035668e

maksfb added a commit to maksfb/llvm-project that referenced this pull request Dec 18, 2024

[BOLT][Linux] Refactor reading of PC-relative addresses. NFCI

057f202

Fix evaluation order problem identified in llvm#119088.

maksfb mentioned this pull request Dec 18, 2024

[BOLT][Linux] Refactor reading of PC-relative addresses. NFCI #120491

Merged

maksfb added a commit that referenced this pull request Dec 19, 2024

[BOLT][Linux] Refactor reading of PC-relative addresses. NFCI (#120491)

21684e3

Fix evaluation order problem identified in #119088.

FLZ101 requested a review from dcci December 24, 2024 01:49

dcci approved these changes Dec 24, 2024

View reviewed changes

Merge branch 'main' into feature-bolt-linux-kernel-version

89204cc

FLZ101 requested a review from yota9 as a code owner December 26, 2024 01:14

FLZ101 requested review from maksfb and dcci December 26, 2024 11:07

dcci merged commit 6e8a1a4 into llvm:main Dec 26, 2024
7 checks passed

github-actions bot pushed a commit to arm/arm-toolchain that referenced this pull request Jan 10, 2025

Automerge: [BOLT][Linux] Refactor reading of PC-relative addresses. N…

71ec66d

…FCI (#120491) Fix evaluation order problem identified in llvm/llvm-project#119088.

[BOLT] Detect Linux kernel version if the binary is a Linux kernel #119088

[BOLT] Detect Linux kernel version if the binary is a Linux kernel #119088

Uh oh!

Conversation

FLZ101 commented Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Dec 7, 2024

Uh oh!

dcci left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FLZ101 Dec 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maksfb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FLZ101 commented Dec 9, 2024

Uh oh!

maksfb commented Dec 9, 2024

Uh oh!

maksfb commented Dec 11, 2024

Uh oh!

FLZ101 commented Dec 11, 2024

Uh oh!

FLZ101 commented Dec 16, 2024

Uh oh!

maksfb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FLZ101 commented Dec 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcci left a comment

Choose a reason for hiding this comment

Uh oh!

FLZ101 commented Dec 25, 2024

Uh oh!

Uh oh!

Uh oh!

FLZ101 commented Dec 7, 2024 •

edited

Loading

FLZ101 Dec 8, 2024 •

edited

Loading

FLZ101 commented Dec 23, 2024 •

edited

Loading