[PAC][AArch64] Lower ptrauth constants in code #94241
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tschuett
reviewed
Jun 3, 2024
| @@ -110,6 +110,12 @@ def G_GLOBAL_VALUE : GenericInstruction { | |||
| let hasSideEffects = false; | |||
| } | |||
|
|
|||
| def G_PTRAUTH_GLOBAL_VALUE : GenericInstruction { | |||
There was a problem hiding this comment.
Could you please add an entry to:
https://github.com/llvm/llvm-project/blob/main/llvm/docs/GlobalISel/GenericOpcode.rst
There was a problem hiding this comment.
Thanks for suggestion! Done
|
You added a new GlobalIsel named opcode. You may want to check it's invariants in the MachineVerifier: |
Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. Co-authored-by: Ahmed Bougacha <[email protected]>
kovdan01
force-pushed
the
ptrauth-constants-in-code
branch
from
9207e12 to
14664b5
Compare
Thanks for suggestion! I've added some basic invariant checking. |
|
@ahmedbougacha Please note that changes in this PR are a bit different from the https://github.com/ahmedbougacha/llvm-project/tree/eng/arm64e-upstream-llvmorg branch. Some MachO-specific changes are not included and should be submitted as subsequent patches. Here are some of the differences.
|
llvmbot
added
backend:AArch64
llvm:globalisel
llvm:support
llvm:SelectionDAG
|
@llvm/pr-subscribers-llvm-support @llvm/pr-subscribers-llvm-selectiondag Author: Daniil Kovalev (kovdan01) ChangesDepends on #94240. Define the following pseudos for lowering ptrauth constants in code:
Co-authored-by: Ahmed Bougacha <[email protected]> Patch is 73.19 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/94241.diff 25 Files Affected:
diff --git a/llvm/docs/GlobalISel/GenericOpcode.rst b/llvm/docs/GlobalISel/GenericOpcode.rst
index 5c28c6fcd30fb..19adb20f70030 100644
--- a/llvm/docs/GlobalISel/GenericOpcode.rst
+++ b/llvm/docs/GlobalISel/GenericOpcode.rst
@@ -60,6 +60,16 @@ The address of a global value.
%0(p0) = G_GLOBAL_VALUE @var_local
+G_PTRAUTH_GLOBAL_VALUE
+^^^^^^^^^^^^^^^^^^^^^^
+
+The signed address of a global value. Operands: address to be signed (pointer),
+key (32-bit imm), address for address discrimination (zero if not needed) and
+an extra discriminator (64-bit imm).
+
+.. code-block:: none
+ %0:_(p0) = G_PTRAUTH_GLOBAL_VALUE %1:_(p0), s32, %2:_(p0), s64
+
G_BLOCK_ADDR
^^^^^^^^^^^^
diff --git a/llvm/include/llvm/CodeGen/ISDOpcodes.h b/llvm/include/llvm/CodeGen/ISDOpcodes.h
index c8c86ed5eef29..a6f4a03ec5de2 100644
--- a/llvm/include/llvm/CodeGen/ISDOpcodes.h
+++ b/llvm/include/llvm/CodeGen/ISDOpcodes.h
@@ -83,6 +83,12 @@ enum NodeType {
ExternalSymbol,
BlockAddress,
+ /// A ptrauth constant.
+ /// ptr, key, addr-disc, disc
+ /// Note that the addr-disc can be a non-constant value, to allow representing
+ /// a constant global address signed using address-diversification, in code.
+ PtrAuthGlobalAddress,
+
/// The address of the GOT
GLOBAL_OFFSET_TABLE,
diff --git a/llvm/include/llvm/CodeGen/MachineModuleInfoImpls.h b/llvm/include/llvm/CodeGen/MachineModuleInfoImpls.h
index f8a328f13eded..64d841d86c7c4 100644
--- a/llvm/include/llvm/CodeGen/MachineModuleInfoImpls.h
+++ b/llvm/include/llvm/CodeGen/MachineModuleInfoImpls.h
@@ -61,10 +61,20 @@ class MachineModuleInfoMachO : public MachineModuleInfoImpl {
/// MachineModuleInfoELF - This is a MachineModuleInfoImpl implementation
/// for ELF targets.
class MachineModuleInfoELF : public MachineModuleInfoImpl {
+public:
+ struct AuthStubInfo {
+ const MCExpr *AuthPtrRef;
+ };
+
+private:
/// GVStubs - These stubs are used to materialize global addresses in PIC
/// mode.
DenseMap<MCSymbol *, StubValueTy> GVStubs;
+ /// AuthPtrStubs - These stubs are used to materialize signed addresses for
+ /// extern_weak symbols.
+ DenseMap<MCSymbol *, AuthStubInfo> AuthPtrStubs;
+
virtual void anchor(); // Out of line virtual method.
public:
@@ -75,9 +85,19 @@ class MachineModuleInfoELF : public MachineModuleInfoImpl {
return GVStubs[Sym];
}
+ AuthStubInfo &getAuthPtrStubEntry(MCSymbol *Sym) {
+ assert(Sym && "Key cannot be null");
+ return AuthPtrStubs[Sym];
+ }
+
/// Accessor methods to return the set of stubs in sorted order.
SymbolListTy GetGVStubList() { return getSortedStubs(GVStubs); }
+
+ using AuthStubPairTy = std::pair<MCSymbol *, AuthStubInfo>;
+ typedef std::vector<AuthStubPairTy> AuthStubListTy;
+
+ AuthStubListTy getAuthGVStubList();
};
/// MachineModuleInfoCOFF - This is a MachineModuleInfoImpl implementation
diff --git a/llvm/include/llvm/Support/TargetOpcodes.def b/llvm/include/llvm/Support/TargetOpcodes.def
index 559a588c25148..2bb4a0ffd43fe 100644
--- a/llvm/include/llvm/Support/TargetOpcodes.def
+++ b/llvm/include/llvm/Support/TargetOpcodes.def
@@ -294,6 +294,9 @@ HANDLE_TARGET_OPCODE(G_FRAME_INDEX)
/// Generic reference to global value.
HANDLE_TARGET_OPCODE(G_GLOBAL_VALUE)
+/// Generic ptrauth-signed reference to global value.
+HANDLE_TARGET_OPCODE(G_PTRAUTH_GLOBAL_VALUE)
+
/// Generic instruction to materialize the address of an object in the constant
/// pool.
HANDLE_TARGET_OPCODE(G_CONSTANT_POOL)
diff --git a/llvm/include/llvm/Target/GenericOpcodes.td b/llvm/include/llvm/Target/GenericOpcodes.td
index c40498e554215..6846a7e008f3a 100644
--- a/llvm/include/llvm/Target/GenericOpcodes.td
+++ b/llvm/include/llvm/Target/GenericOpcodes.td
@@ -110,6 +110,12 @@ def G_GLOBAL_VALUE : GenericInstruction {
let hasSideEffects = false;
}
+def G_PTRAUTH_GLOBAL_VALUE : GenericInstruction {
+ let OutOperandList = (outs type0:$dst);
+ let InOperandList = (ins unknown:$addr, i32imm:$key, type1:$addrdisc, i64imm:$disc);
+ let hasSideEffects = 0;
+}
+
def G_CONSTANT_POOL : GenericInstruction {
let OutOperandList = (outs type0:$dst);
let InOperandList = (ins unknown:$src);
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index 7efcf21460260..0b59cd836fcc9 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -3486,7 +3486,16 @@ bool IRTranslator::translate(const Constant &C, Register Reg) {
EntryBuilder->buildConstant(Reg, 0);
else if (auto GV = dyn_cast<GlobalValue>(&C))
EntryBuilder->buildGlobalValue(Reg, GV);
- else if (auto CAZ = dyn_cast<ConstantAggregateZero>(&C)) {
+ else if (auto CPA = dyn_cast<ConstantPtrAuth>(&C)) {
+ Register Addr = getOrCreateVReg(*CPA->getPointer());
+ Register AddrDisc = getOrCreateVReg(*CPA->getAddrDiscriminator());
+ EntryBuilder->buildInstr(TargetOpcode::G_PTRAUTH_GLOBAL_VALUE)
+ .addDef(Reg)
+ .addUse(Addr)
+ .addImm(CPA->getKey()->getZExtValue())
+ .addUse(AddrDisc)
+ .addImm(CPA->getDiscriminator()->getZExtValue());
+ } else if (auto CAZ = dyn_cast<ConstantAggregateZero>(&C)) {
if (!isa<FixedVectorType>(CAZ->getType()))
return false;
// Return the scalar if it is a <1 x Ty> vector.
diff --git a/llvm/lib/CodeGen/MachineModuleInfoImpls.cpp b/llvm/lib/CodeGen/MachineModuleInfoImpls.cpp
index 9c3b31935f6d6..f114f1ecc0bae 100644
--- a/llvm/lib/CodeGen/MachineModuleInfoImpls.cpp
+++ b/llvm/lib/CodeGen/MachineModuleInfoImpls.cpp
@@ -13,6 +13,7 @@
#include "llvm/CodeGen/MachineModuleInfoImpls.h"
#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/STLExtras.h"
#include "llvm/MC/MCSymbol.h"
using namespace llvm;
@@ -41,3 +42,25 @@ MachineModuleInfoImpl::SymbolListTy MachineModuleInfoImpl::getSortedStubs(
Map.clear();
return List;
}
+
+template <typename MachineModuleInfoTarget>
+static typename MachineModuleInfoTarget::AuthStubListTy getAuthGVStubListHelper(
+ DenseMap<MCSymbol *, typename MachineModuleInfoTarget::AuthStubInfo>
+ &AuthPtrStubs) {
+ typename MachineModuleInfoTarget::AuthStubListTy List(AuthPtrStubs.begin(),
+ AuthPtrStubs.end());
+
+ if (!List.empty())
+ llvm::sort(List.begin(), List.end(),
+ [](const typename MachineModuleInfoTarget::AuthStubPairTy &LHS,
+ const typename MachineModuleInfoTarget::AuthStubPairTy &RHS) {
+ return LHS.first->getName() < RHS.first->getName();
+ });
+
+ AuthPtrStubs.clear();
+ return List;
+}
+
+MachineModuleInfoELF::AuthStubListTy MachineModuleInfoELF::getAuthGVStubList() {
+ return getAuthGVStubListHelper<MachineModuleInfoELF>(AuthPtrStubs);
+}
diff --git a/llvm/lib/CodeGen/MachineVerifier.cpp b/llvm/lib/CodeGen/MachineVerifier.cpp
index 9ea238c61ed91..0c8a0f2b24a1e 100644
--- a/llvm/lib/CodeGen/MachineVerifier.cpp
+++ b/llvm/lib/CodeGen/MachineVerifier.cpp
@@ -2066,6 +2066,12 @@ void MachineVerifier::verifyPreISelGenericInstruction(const MachineInstr *MI) {
report("Dst operand 0 must be a pointer", MI);
break;
}
+ case TargetOpcode::G_PTRAUTH_GLOBAL_VALUE: {
+ const MachineOperand &AddrOp = MI->getOperand(1);
+ if (!AddrOp.isReg() || !MRI->getType(AddrOp.getReg()).isPointer())
+ report("addr operand must be a pointer", &AddrOp, 1);
+ break;
+ }
default:
break;
}
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 8838cce9810f8..3adaa6e7f5564 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -1802,6 +1802,13 @@ SDValue SelectionDAGBuilder::getValueImpl(const Value *V) {
if (const GlobalValue *GV = dyn_cast<GlobalValue>(C))
return DAG.getGlobalAddress(GV, getCurSDLoc(), VT);
+ if (const ConstantPtrAuth *CPA = dyn_cast<ConstantPtrAuth>(C)) {
+ return DAG.getNode(ISD::PtrAuthGlobalAddress, getCurSDLoc(), VT,
+ getValue(CPA->getPointer()), getValue(CPA->getKey()),
+ getValue(CPA->getAddrDiscriminator()),
+ getValue(CPA->getDiscriminator()));
+ }
+
if (isa<ConstantPointerNull>(C)) {
unsigned AS = V->getType()->getPointerAddressSpace();
return DAG.getConstant(0, getCurSDLoc(),
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
index 12a7b7f11778d..1459c2bf00ed3 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp
@@ -75,6 +75,7 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const {
}
return "<<Unknown Node #" + utostr(getOpcode()) + ">>";
+ // clang-format off
#ifndef NDEBUG
case ISD::DELETED_NODE: return "<<Deleted Node!>>";
#endif
@@ -126,6 +127,7 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const {
case ISD::ConstantFP: return "ConstantFP";
case ISD::GlobalAddress: return "GlobalAddress";
case ISD::GlobalTLSAddress: return "GlobalTLSAddress";
+ case ISD::PtrAuthGlobalAddress: return "PtrAuthGlobalAddress";
case ISD::FrameIndex: return "FrameIndex";
case ISD::JumpTable: return "JumpTable";
case ISD::JUMP_TABLE_DEBUG_INFO:
@@ -168,8 +170,6 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const {
return "OpaqueTargetConstant";
return "TargetConstant";
- // clang-format off
-
case ISD::TargetConstantFP: return "TargetConstantFP";
case ISD::TargetGlobalAddress: return "TargetGlobalAddress";
case ISD::TargetGlobalTLSAddress: return "TargetGlobalTLSAddress";
diff --git a/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp b/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
index da11539eab348..611c3b579ff15 100644
--- a/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
+++ b/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
@@ -133,6 +133,13 @@ class AArch64AsmPrinter : public AsmPrinter {
unsigned emitPtrauthDiscriminator(uint16_t Disc, unsigned AddrDisc,
unsigned &InstsEmitted);
+ // Emit the sequence for LOADauthptrstatic
+ void LowerLOADauthptrstatic(const MachineInstr &MI);
+
+ // Emit the sequence for LOADgotPAC/MOVaddrPAC (either GOT adrp-ldr or
+ // adrp-add followed by PAC sign)
+ void LowerMOVaddrPAC(const MachineInstr &MI);
+
/// tblgen'erated driver function for lowering simple MI->MC
/// pseudo instructions.
bool emitPseudoExpansionLowering(MCStreamer &OutStreamer,
@@ -840,6 +847,15 @@ void AArch64AsmPrinter::emitHwasanMemaccessSymbols(Module &M) {
}
}
+template <typename MachineModuleInfoTarget>
+static void emitAuthenticatedPointer(
+ MCStreamer &OutStreamer, MCSymbol *StubLabel,
+ const typename MachineModuleInfoTarget::AuthStubInfo &StubInfo) {
+ // sym$auth_ptr$key$disc:
+ OutStreamer.emitLabel(StubLabel);
+ OutStreamer.emitValue(StubInfo.AuthPtrRef, /*size=*/8);
+}
+
void AArch64AsmPrinter::emitEndOfAsmFile(Module &M) {
emitHwasanMemaccessSymbols(M);
@@ -853,6 +869,25 @@ void AArch64AsmPrinter::emitEndOfAsmFile(Module &M) {
OutStreamer->emitAssemblerFlag(MCAF_SubsectionsViaSymbols);
}
+ if (TT.isOSBinFormatELF()) {
+ // Output authenticated pointers as indirect symbols, if we have any.
+ MachineModuleInfoELF &MMIELF = MMI->getObjFileInfo<MachineModuleInfoELF>();
+
+ auto Stubs = MMIELF.getAuthGVStubList();
+
+ if (!Stubs.empty()) {
+ const TargetLoweringObjectFile &TLOF = getObjFileLowering();
+ OutStreamer->switchSection(TLOF.getDataSection());
+ emitAlignment(Align(8));
+
+ for (const auto &Stub : Stubs)
+ emitAuthenticatedPointer<MachineModuleInfoELF>(*OutStreamer, Stub.first,
+ Stub.second);
+
+ OutStreamer->addBlankLine();
+ }
+ }
+
// Emit stack and fault map information.
FM.serializeToFaultMapSection();
@@ -1623,6 +1658,214 @@ AArch64AsmPrinter::lowerConstantPtrAuth(const ConstantPtrAuth &CPA) {
CPA.hasAddressDiscriminator(), Ctx);
}
+void AArch64AsmPrinter::LowerLOADauthptrstatic(const MachineInstr &MI) {
+ unsigned DstReg = MI.getOperand(0).getReg();
+ const MachineOperand &GAOp = MI.getOperand(1);
+ const uint64_t KeyC = MI.getOperand(2).getImm();
+ assert(KeyC <= AArch64PACKey::LAST &&
+ "key is out of range [0, AArch64PACKey::LAST]");
+ const auto Key = (AArch64PACKey::ID)KeyC;
+ const uint64_t Disc = MI.getOperand(3).getImm();
+ assert(isUInt<16>(Disc) &&
+ "constant discriminator is out of range [0, 0xffff]");
+
+ // Emit instruction sequence like the following:
+ // ADRP x16, symbol$auth_ptr$key$disc
+ // LDR x16, [x16, :lo12:symbol$auth_ptr$key$disc]
+ //
+ // Where the $auth_ptr$ symbol is the stub slot containing the signed pointer
+ // to symbol.
+ assert(TM.getTargetTriple().isOSBinFormatELF() &&
+ "LOADauthptrstatic is implemented only for ELF");
+ const auto &TLOF =
+ static_cast<const AArch64_ELFTargetObjectFile &>(getObjFileLowering());
+
+ assert(GAOp.getOffset() == 0 &&
+ "non-zero offset for $auth_ptr$ stub slots is not supported");
+ const MCSymbol *GASym = TM.getSymbol(GAOp.getGlobal());
+ MCSymbol *AuthPtrStubSym =
+ TLOF.getAuthPtrSlotSymbol(TM, &MF->getMMI(), GASym, Key, Disc);
+
+ MachineOperand StubMOHi =
+ MachineOperand::CreateMCSymbol(AuthPtrStubSym, AArch64II::MO_PAGE);
+ MachineOperand StubMOLo = MachineOperand::CreateMCSymbol(
+ AuthPtrStubSym, AArch64II::MO_PAGEOFF | AArch64II::MO_NC);
+ MCOperand StubMCHi, StubMCLo;
+
+ MCInstLowering.lowerOperand(StubMOHi, StubMCHi);
+ MCInstLowering.lowerOperand(StubMOLo, StubMCLo);
+
+ EmitToStreamer(
+ *OutStreamer,
+ MCInstBuilder(AArch64::ADRP).addReg(DstReg).addOperand(StubMCHi));
+
+ EmitToStreamer(*OutStreamer, MCInstBuilder(AArch64::LDRXui)
+ .addReg(DstReg)
+ .addReg(DstReg)
+ .addOperand(StubMCLo));
+}
+
+void AArch64AsmPrinter::LowerMOVaddrPAC(const MachineInstr &MI) {
+ unsigned InstsEmitted = 0;
+ auto EmitAndIncrement = [this, &InstsEmitted](const MCInst &Inst) {
+ EmitToStreamer(*OutStreamer, Inst);
+ ++InstsEmitted;
+ };
+
+ const bool IsGOTLoad = MI.getOpcode() == AArch64::LOADgotPAC;
+ MachineOperand GAOp = MI.getOperand(0);
+ const uint64_t KeyC = MI.getOperand(1).getImm();
+ assert(KeyC <= AArch64PACKey::LAST &&
+ "key is out of range [0, AArch64PACKey::LAST]");
+ const auto Key = (AArch64PACKey::ID)KeyC;
+ const unsigned AddrDisc = MI.getOperand(2).getReg();
+ const uint64_t Disc = MI.getOperand(3).getImm();
+ assert(isUInt<16>(Disc) &&
+ "constant discriminator is out of range [0, 0xffff]");
+
+ const int64_t Offset = GAOp.getOffset();
+ GAOp.setOffset(0);
+
+ // Emit:
+ // target materialization:
+ // - via GOT:
+ // adrp x16, :got:target
+ // ldr x16, [x16, :got_lo12:target]
+ // add offset to x16 if offset != 0
+ //
+ // - direct:
+ // adrp x16, target
+ // add x16, x16, :lo12:target
+ // add offset to x16 if offset != 0
+ //
+ // add offset to x16:
+ // - abs(offset) fits 24 bits:
+ // add/sub x16, x16, #<offset>[, #lsl 12] (up to 2 instructions)
+ // - abs(offset) does not fit 24 bits:
+ // - offset < 0:
+ // movn+movk sequence filling x17 register with the offset (up to 4
+ // instructions)
+ // add x16, x16, x17
+ // - offset > 0:
+ // movz+movk sequence filling x17 register with the offset (up to 4
+ // instructions)
+ // add x16, x16, x17
+ //
+ // signing:
+ // - 0 discriminator:
+ // paciza x16
+ // - Non-0 discriminator, no address discriminator:
+ // mov x17, #Disc
+ // pacia x16, x17
+ // - address discriminator (with potentially folded immediate discriminator):
+ // pacia x16, xAddrDisc
+
+ MachineOperand GAMOHi(GAOp), GAMOLo(GAOp);
+ MCOperand GAMCHi, GAMCLo;
+
+ GAMOHi.setTargetFlags(AArch64II::MO_PAGE);
+ GAMOLo.setTargetFlags(AArch64II::MO_PAGEOFF | AArch64II::MO_NC);
+ if (IsGOTLoad) {
+ GAMOHi.addTargetFlag(AArch64II::MO_GOT);
+ GAMOLo.addTargetFlag(AArch64II::MO_GOT);
+ }
+
+ MCInstLowering.lowerOperand(GAMOHi, GAMCHi);
+ MCInstLowering.lowerOperand(GAMOLo, GAMCLo);
+
+ EmitAndIncrement(
+ MCInstBuilder(AArch64::ADRP).addReg(AArch64::X16).addOperand(GAMCHi));
+
+ if (IsGOTLoad) {
+ EmitAndIncrement(MCInstBuilder(AArch64::LDRXui)
+ .addReg(AArch64::X16)
+ .addReg(AArch64::X16)
+ .addOperand(GAMCLo));
+ } else {
+ EmitAndIncrement(MCInstBuilder(AArch64::ADDXri)
+ .addReg(AArch64::X16)
+ .addReg(AArch64::X16)
+ .addOperand(GAMCLo)
+ .addImm(0));
+ }
+
+ if (Offset != 0) {
+ const uint64_t AbsOffset = (Offset > 0 ? Offset : -((uint64_t)Offset));
+ const bool IsNeg = Offset < 0;
+ if (isUInt<24>(AbsOffset)) {
+ for (int BitPos = 0; BitPos != 24 && (AbsOffset >> BitPos);
+ BitPos += 12) {
+ EmitAndIncrement(
+ MCInstBuilder(IsNeg ? AArch64::SUBXri : AArch64::ADDXri)
+ .addReg(AArch64::X16)
+ .addReg(AArch64::X16)
+ .addImm((AbsOffset >> BitPos) & 0xfff)
+ .addImm(AArch64_AM::getShifterImm(AArch64_AM::LSL, BitPos)));
+ }
+ } else {
+ constexpr uint64_t Mask16 = 0xffff;
+ const uint64_t UOffset = Offset;
+ EmitAndIncrement(MCInstBuilder(IsNeg ? AArch64::MOVNXi : AArch64::MOVZXi)
+ .addReg(AArch64::X17)
+ .addImm((IsNeg ? ~UOffset : UOffset) & Mask16)
+ .addImm(/*shift=*/0));
+ auto NeedMovk = [Mask16, IsNeg, UOffset](int BitPos) -> bool {
+ assert(BitPos == 16 || BitPos == 32 || BitPos == 48);
+ uint64_t Shifted = UOffset >> BitPos;
+ if (!IsNeg)
+ return Shifted != 0;
+ for (int I = 0; I != 64 - BitPos; I += 16)
+ if (((Shifted >> I) & Mask16) != Mask16)
+ return true;
+ return false;
+ };
+ for (int BitPos = 16; BitPos != 64 && NeedMovk(BitPos); BitPos += 16) {
+ EmitAndIncrement(MCInstBuilder(AArch64::MOVKXi)
+ .addReg(AArch64::X17)
+ .addReg(AArch64::X17)
+ .addImm((UOffset >> BitPos) & Mask16)
+ .addImm(/*shift=*/BitPos));
+ }
+ EmitAndIncrement(MCInstBuilder(AArch64::ADDXrs)
+ .addReg(AArch64::X16)
+ .addReg(AArch64::X16)
+ .addReg(AArch64::X17)
+ .addImm(/*shift=*/0));
+ }
+ }
+
+ unsigned DiscReg = AddrDisc;
+ if (Disc != 0) {
+ if (AddrDisc != AArch64::XZR) {
+ EmitAndIncrement(MCInstBuilder(AArch64::ORRXrs)
+ .addReg(AArch64::X17)
+ .addReg(AArch64::XZR)
+ .addReg(AddrDisc)
+ .addImm(0));
+ EmitAndIncrement(MCInstBuilder(AArch64::MOVKXi)
+ .addReg(AArch64::X17)
+ .addReg(AArch64::X17)
+ .addImm(Disc)
+ .addImm(/*shift=*/48));
+ } else {
+ EmitAndIncrement(MCInstBuilder(AArch64::MOVZXi)
+ .addReg(AArch64::X17)
+ .addImm(Disc)
+ .addImm(/*shift=*/0));
+ }
+ DiscReg = AArch64::X17;
+ }
+
+ auto MIB = MCInstBuilder(getPACOpcodeForKey(Key, DiscReg == AArch64::XZR))
+ .addReg(AArch64::X16)
+ .addReg(AArch64::X16);
+ if (DiscReg != AArch64::XZR)
+ MIB.addReg(DiscReg);
+ EmitAndIncrement(MIB);
+
+ assert(STI->getInstrInfo()->getInstSizeInBytes(MI) >= InstsEmitted * 4);
+}
+
// Simple pseudo-instructions have their lowering (with expansion to real
// in...
[truncated]
|
|
In the IRTranslator, you build a G_PTRAUTH_GLOBAL_VALUE by hand. Would an entry to |
ahmedbougacha
approved these changes
Jun 26, 2024
| an extra discriminator (64-bit imm). | ||
|
|
||
| .. code-block:: none | ||
| %0:_(p0) = G_PTRAUTH_GLOBAL_VALUE %1:_(p0), s32, %2:_(p0), s64 |
There was a problem hiding this comment.
I think this really does need the extra newline after .. code-block:: none
There was a problem hiding this comment.
Fixed, thanks, see 2b12acb
@tschuett I can't say that it simplifies the build, but it definitely makes things more consistent since for many opcodes there are corresponding entries in MachineIRBuilder. I've added a one for |
kovdan01
added a commit
to kovdan01/llvm-project
that referenced
this pull request
Jun 27, 2024
This reverts commit b5cc19e. See buildbot failure https://lab.llvm.org/buildbot/#/builders/51/builds/570
kovdan01
added a commit
that referenced
this pull request
Jun 27, 2024
) This reverts #94241. See buildbot failure https://lab.llvm.org/buildbot/#/builders/51/builds/570
arsenm
pushed a commit
that referenced
this pull request
Jun 27, 2024
Depends on #94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
kovdan01
added a commit
to kovdan01/llvm-project
that referenced
this pull request
Jun 27, 2024
This re-applies llvm#94241 after fixing buildbot failure, see https://lab.llvm.org/buildbot/#/builders/51/builds/570 According to standard, `constexpr` variables can be used in lambdas w/o capturing - see https://en.cppreference.com/w/cpp/language/lambda. However, MSVC used on buildkite seems to ignore that rule and does not allow using uncaptured `constexpr` variables in lambdas: we have "error C3493: 'Mask16' cannot be implicitly captured because no default capture mode has been specified" - see https://buildkite.com/llvm-project/github-pull-requests/builds/73238 Explicitly capturing a `constexpr` variable, however, makes buildbot fail with "error: lambda capture 'Mask16' is not required to be captured for this use [-Werror,-Wunused-lambda-capture]" - see https://lab.llvm.org/buildbot/#/builders/51/builds/570. Fix both cases by using `const` instead of `constexpr` and explicitly capturing the variable. Original PR description below. Depends on llvm#94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
arsenm
pushed a commit
that referenced
this pull request
Jun 27, 2024
Depends on #94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
kovdan01
added a commit
that referenced
this pull request
Jun 28, 2024
This re-applies #94241 after fixing buildbot failure, see https://lab.llvm.org/buildbot/#/builders/51/builds/570 According to standard, `constexpr` variables and `const` variables initialized with constant expressions can be used in lambdas w/o capturing - see https://en.cppreference.com/w/cpp/language/lambda. However, MSVC used on buildkite seems to ignore that rule and does not allow using such uncaptured variables in lambdas: we have "error C3493: 'Mask16' cannot be implicitly captured because no default capture mode has been specified" - see https://buildkite.com/llvm-project/github-pull-requests/builds/73238 Explicitly capturing such a variable, however, makes buildbot fail with "error: lambda capture 'Mask16' is not required to be captured for this use [-Werror,-Wunused-lambda-capture]" - see https://lab.llvm.org/buildbot/#/builders/51/builds/570. Fix both cases by using `0xffff` value directly instead of giving a name to it. Original PR description below. Depends on #94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
lravenclaw
pushed a commit
to lravenclaw/llvm-project
that referenced
this pull request
Jul 3, 2024
Depends on llvm#94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
lravenclaw
pushed a commit
to lravenclaw/llvm-project
that referenced
this pull request
Jul 3, 2024
…lvm#96865) This reverts llvm#94241. See buildbot failure https://lab.llvm.org/buildbot/#/builders/51/builds/570
lravenclaw
pushed a commit
to lravenclaw/llvm-project
that referenced
this pull request
Jul 3, 2024
This re-applies llvm#94241 after fixing buildbot failure, see https://lab.llvm.org/buildbot/#/builders/51/builds/570 According to standard, `constexpr` variables and `const` variables initialized with constant expressions can be used in lambdas w/o capturing - see https://en.cppreference.com/w/cpp/language/lambda. However, MSVC used on buildkite seems to ignore that rule and does not allow using such uncaptured variables in lambdas: we have "error C3493: 'Mask16' cannot be implicitly captured because no default capture mode has been specified" - see https://buildkite.com/llvm-project/github-pull-requests/builds/73238 Explicitly capturing such a variable, however, makes buildbot fail with "error: lambda capture 'Mask16' is not required to be captured for this use [-Werror,-Wunused-lambda-capture]" - see https://lab.llvm.org/buildbot/#/builders/51/builds/570. Fix both cases by using `0xffff` value directly instead of giving a name to it. Original PR description below. Depends on llvm#94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
AlexisPerry
pushed a commit
to llvm-project-tlp/llvm-project
that referenced
this pull request
Jul 9, 2024
Depends on llvm#94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]>
AlexisPerry
pushed a commit
to llvm-project-tlp/llvm-project
that referenced
this pull request
Jul 9, 2024
…lvm#96865) This reverts llvm#94241. See buildbot failure https://lab.llvm.org/buildbot/#/builders/51/builds/570
searlmc1
pushed a commit
to ROCm/llvm-project
that referenced
this pull request
Sep 6, 2024
This re-applies llvm#94241 after fixing buildbot failure, see https://lab.llvm.org/buildbot/#/builders/51/builds/570 According to standard, `constexpr` variables and `const` variables initialized with constant expressions can be used in lambdas w/o capturing - see https://en.cppreference.com/w/cpp/language/lambda. However, MSVC used on buildkite seems to ignore that rule and does not allow using such uncaptured variables in lambdas: we have "error C3493: 'Mask16' cannot be implicitly captured because no default capture mode has been specified" - see https://buildkite.com/llvm-project/github-pull-requests/builds/73238 Explicitly capturing such a variable, however, makes buildbot fail with "error: lambda capture 'Mask16' is not required to be captured for this use [-Werror,-Wunused-lambda-capture]" - see https://lab.llvm.org/buildbot/#/builders/51/builds/570. Fix both cases by using `0xffff` value directly instead of giving a name to it. Original PR description below. Depends on llvm#94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]> Change-Id: Ic59980fe9466834f29d0818436149acfd1ebc1da
searlmc1
pushed a commit
to ROCm/llvm-project
that referenced
this pull request
Sep 23, 2024
add back -fallow-half-arguments-and-returns for hipRuntime builds. ---------------------------------------------------------------------- Revert "[PAC][AArch64] Lower ptrauth constants in code (llvm#96879)" This reverts commit 88dd10c. ---------------------------------------------------------------------- [PAC][AArch64] Lower ptrauth constants in code (llvm#96879) This re-applies llvm#94241 after fixing buildbot failure, see https://lab.llvm.org/buildbot/#/builders/51/builds/570 According to standard, `constexpr` variables and `const` variables initialized with constant expressions can be used in lambdas w/o capturing - see https://en.cppreference.com/w/cpp/language/lambda. However, MSVC used on buildkite seems to ignore that rule and does not allow using such uncaptured variables in lambdas: we have "error C3493: 'Mask16' cannot be implicitly captured because no default capture mode has been specified" - see https://buildkite.com/llvm-project/github-pull-requests/builds/73238 Explicitly capturing such a variable, however, makes buildbot fail with "error: lambda capture 'Mask16' is not required to be captured for this use [-Werror,-Wunused-lambda-capture]" - see https://lab.llvm.org/buildbot/#/builders/51/builds/570. Fix both cases by using `0xffff` value directly instead of giving a name to it. Original PR description below. Depends on llvm#94240. Define the following pseudos for lowering ptrauth constants in code: - non-`extern_weak`: - no GOT load needed: `MOVaddrPAC` - similar to `MOVaddr`, with added PAC; - GOT load needed: `LOADgotPAC` - similar to `LOADgot`, with added PAC; - `extern_weak`: `LOADauthptrstatic` - similar to `LOADgot`, but use a special stub slot named `sym$auth_ptr$key$disc` filled by dynamic linker during relocation resolving instead of a GOT slot. --------- Co-authored-by: Ahmed Bougacha <[email protected]> (cherry picked from commit 1488fb4) ---------------------------------------------------------------------- [AArch64][PAC] Lower ptrauth constants in code for MachO. (llvm#97665) This also adds support for auth stubs on MachO using __DATA,__auth_ptr. Some of the machinery for auth stubs is already implemented; this generalizes that a bit to support MachO, and moves some of the shared logic into MMIImpls. In particular, this originally had an AuthStubInfo struct, but we no longer need it beyond a single MCExpr. So this provides variants of the symbol stub helper type declarations and functions for "expr stubs", where a stub points at an arbitrary MCExpr, rather than a simple MCSymbol (and a bit). (cherry picked from commit 5f1bb62) ---------------------------------------------------------------------- [AArch64][PAC] Sign block addresses used in indirectbr. (llvm#97647) Enabled in clang using: -fptrauth-indirect-gotos and at the IR level using function attribute: "ptrauth-indirect-gotos" Signing uses IA and a per-function integer discriminator. The discriminator isn't ABI-visible, and is currently: ptrauth_string_discriminator("<function_name> blockaddress") A sufficiently sophisticated frontend could benefit from per-indirectbr discrimination, which would need additional machinery, such as allowing "ptrauth" bundles on indirectbr. For our purposes, the simple scheme above is sufficient. This approach doesn't support subtracting label addresses and using the result as offsets, because each label address is signed. Pointer arithmetic on signed pointers corrupts the signature bits, and because label address expressions aren't typed beyond void*, we can't do anything reliably intelligent on the arithmetic exprs. Not signing addresses when used to form offsets would allow easily hijacking control flow by overwriting the offset. This diagnoses the basic cases (`&&lbl2 - &&lbl1`) in the frontend, while we evaluate either alternative implementations (e.g., lowering blockaddress to a bb number, and indirectbr to a checked jump-table), or better diagnostics (both at the frontend level and on unencodable IR constants). (cherry picked from commit b8721fa) ---------------------------------------------------------------------- [AArch64][PAC] Lower auth/resign into checked sequence. (llvm#79024) This introduces 3 hardening modes in the authentication step of auth/resign lowering: - unchecked, which uses the AUT instructions as-is - poison, which detects authentication failure (using an XPAC+CMP sequence), explicitly yielding the XPAC result rather than the AUT result, to avoid leaking - trap, which additionally traps on authentication failure, using BRK #0xC470 + key (IA C470, IB C471, DA C472, DB C473.) Not all modes are necessarily useful in all contexts, and there are more performant alternative lowerings in specific contexts (e.g., when I/D TBI enablement is a target ABI guarantee.) These will be implemented separately. This is controlled by the `ptrauth-auth-traps` function attributes, and can be overridden using `-aarch64-ptrauth-auth-checks=`. This also adds the FPAC extension, which we haven't needed before, to improve isel when we can rely on HW checking. (cherry picked from commit d7e8a74) ---------------------------------------------------------------------- [Clang][Arm] Convert -fallow-half-arguments-and-returns to a target option. NFC This cc1 option -fallow-half-arguments-and-returns allows __fp16 to be passed by argument and returned, without giving an error. It is currently always enabled for Arm and AArch64, by forcing the option in the driver. This means any cc1 tests (especially those needing arm_neon.h) need to specify the option too, to prevent the error from being emitted. This changes it to a target option instead, set to true for Arm and AArch64. This allows the option to be removed. Previously it was implied by -fnative_half_arguments_and_returns, which is set for certain languages like open_cl, renderscript and hlsl, so that option now too controls the errors. There were are few other non-arm uses of -fallow-half-arguments-and-returns but I believe they were unnecessary. The strictfp_builtins.c tests were converted from __fp16 to _Float16 to avoid the issues. Differential Revision: https://reviews.llvm.org/D133885 (cherry picked from commit 9ef11036505c0ae6cdb56ff49f39ab7abcded3cf) ---------------------------------------------------------------------- [clang] XFAIL a few tests due to 'noundef' etc Not all, but most of these are failing due to the presence of a 'noundef' call return attribute on some intrinsics. This is not present on upstream 'main' due to the AlwaysInliner pass being run. See commit 1a2e77c. ---------------------------------------------------------------------- [DebugInfo] Restore missing disabled ptrauth support See "[DebugInfo] Teach LLVM and LLDB about ptrauth in DWARF": commit a8c3d98 Author: Jonas Devlieghere <[email protected]> Date: Wed Jul 27 10:44:15 2022 -0700 ---------------------------------------------------------------------- Apply simple-do.ll test change from b46c085 ---------------------------------------------------------------------- Adjust ptrauth.s test for ptrauth_authentication_mode encoding ---------------------------------------------------------------------- Fix dwarf-eh-prepare-dbg.ll test: dwarfAddressSpace=>addressSpace ---------------------------------------------------------------------- Update some SLPVectorizer/AArch64 tests from upstream ---------------------------------------------------------------------- Regenerate assertions in arm_mult_q15.ll ---------------------------------------------------------------------- [AsmPrinter] Handle null extracted addr class ---------------------------------------------------------------------- [PowerPC] Account for custom LLVM moniker in aix tests ---------------------------------------------------------------------- [LoongArch] Add "Verify Heterogeneous Debug Preconditions" to pipeline test ---------------------------------------------------------------------- [JITLink][RISCV] Un-XFAIL ELF_pc_indirect.s ---------------------------------------------------------------------- Change-Id: Ie6ab500b2451b3ed070dfad0bc16d003e5e2fe10
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends on #94240.
Define the following pseudos for lowering ptrauth constants in code:
extern_weak:MOVaddrPAC- similar toMOVaddr, with added PAC;LOADgotPAC- similar toLOADgot, with added PAC;extern_weak:LOADauthptrstatic- similar toLOADgot, but use a special stub slot namedsym$auth_ptr$key$discfilled by dynamic linker during relocation resolving instead of a GOT slot.Co-authored-by: Ahmed Bougacha [email protected]