Skip to content

[LLVM][CodeGen] Fix register lane liveness tracking in RegisterPressure #88892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 17, 2024
Merged

[LLVM][CodeGen] Fix register lane liveness tracking in RegisterPressure #88892

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c720523
[LLVM][CodeGen] Fix register lane liveness tracking in RegisterPressure
kparzysz Apr 12, 2024
14c05f6
Fix comment
kparzysz Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 21 additions & 18 deletions llvm/lib/CodeGen/RegisterPressure.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ static void increaseSetPressure(std::vector<unsigned> &CurrSetPressure,
static void decreaseSetPressure(std::vector<unsigned> &CurrSetPressure,
const MachineRegisterInfo &MRI, Register Reg,
LaneBitmask PrevMask, LaneBitmask NewMask) {
//assert((NewMask & !PrevMask) == 0 && "Must not add bits");
assert((NewMask & ~PrevMask).none() && "Must not add bits");
if (NewMask.any() || PrevMask.none())
return;

Expand Down Expand Up @@ -617,17 +617,11 @@ void RegisterOperands::adjustLaneLiveness(const LiveIntervals &LIS,
++I;
}
}
for (auto *I = Uses.begin(); I != Uses.end();) {
LaneBitmask LiveBefore = getLiveLanesAt(LIS, MRI, true, I->RegUnit,
Pos.getBaseIndex());
LaneBitmask LaneMask = I->LaneMask & LiveBefore;
if (LaneMask.none()) {
I = Uses.erase(I);
} else {
I->LaneMask = LaneMask;
++I;
}
}

// For uses just copy the information from LIS.
for (auto &[RegUnit, LaneMask] : Uses)
LaneMask = getLiveLanesAt(LIS, MRI, true, RegUnit, Pos.getBaseIndex());

if (AddFlagsMI != nullptr) {
for (const RegisterMaskPair &P : DeadDefs) {
Register RegUnit = P.RegUnit;
Expand Down Expand Up @@ -1060,18 +1054,27 @@ void RegPressureTracker::bumpUpwardPressure(const MachineInstr *MI) {
// Kill liveness at live defs.
for (const RegisterMaskPair &P : RegOpers.Defs) {
Register Reg = P.RegUnit;
LaneBitmask LiveLanes = LiveRegs.contains(Reg);
LaneBitmask LiveAfter = LiveRegs.contains(Reg);
LaneBitmask UseLanes = getRegLanes(RegOpers.Uses, Reg);
LaneBitmask DefLanes = P.LaneMask;
LaneBitmask LiveAfter = (LiveLanes & ~DefLanes) | UseLanes;
decreaseRegPressure(Reg, LiveLanes, LiveAfter);
LaneBitmask LiveBefore = (LiveAfter & ~DefLanes) | UseLanes;

// There may be parts of the register that were dead before the
// instruction, but became live afterwards. Similarly, some parts
// may have been killed in this instruction.
decreaseRegPressure(Reg, LiveAfter, LiveAfter & LiveBefore);
increaseRegPressure(Reg, LiveAfter, ~LiveAfter & LiveBefore);
}
// Generate liveness for uses.
for (const RegisterMaskPair &P : RegOpers.Uses) {
Register Reg = P.RegUnit;
LaneBitmask LiveLanes = LiveRegs.contains(Reg);
LaneBitmask LiveAfter = LiveLanes | P.LaneMask;
increaseRegPressure(Reg, LiveLanes, LiveAfter);
// If this register was also in a def operand, we've handled it
// with defs.
if (getRegLanes(RegOpers.Defs, Reg).any())
continue;
LaneBitmask LiveAfter = LiveRegs.contains(Reg);
LaneBitmask LiveBefore = LiveAfter | P.LaneMask;
increaseRegPressure(Reg, LiveAfter, LiveBefore);
}
}

Expand Down