-
Notifications
You must be signed in to change notification settings - Fork 13.6k
[VPlan] Add VPPhiAccessors to provide interface for phi recipes (NFC) #129388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
b357af4
9113022
c61233f
a73a137
90b2d4d
f768b97
d6d6bb5
8dfd569
595b057
a2b4ebb
46780ab
54981bf
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2883,11 +2883,8 @@ void InnerLoopVectorizer::fixNonInductionPHIs(VPTransformState &State) { | |
PHINode *NewPhi = cast<PHINode>(State.get(VPPhi)); | ||
// Make sure the builder has a valid insert point. | ||
Builder.SetInsertPoint(NewPhi); | ||
for (unsigned Idx = 0; Idx < VPPhi->getNumOperands(); ++Idx) { | ||
VPValue *Inc = VPPhi->getIncomingValue(Idx); | ||
VPBasicBlock *VPBB = VPPhi->getIncomingBlock(Idx); | ||
for (const auto &[Inc, VPBB] : VPPhi->incoming_values_and_blocks()) | ||
NewPhi->addIncoming(State.get(Inc), State.CFG.VPBB2IRBB[VPBB]); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this return (in simplification?) worth the investment. Would an API of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There will be additional users when updating more recipes to use the accessors; they are more conveiient, but There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about first wrapping There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah yes, done for now, thanks |
||
} | ||
} | ||
} | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1171,6 +1171,59 @@ class VPIRInstruction : public VPRecipeBase { | |
void extractLastLaneOfFirstOperand(VPBuilder &Builder); | ||
}; | ||
|
||
/// Helper type to provide functions to access incoming values and blocks for | ||
/// phi-like recipes. RecipeTy must be a sub-class of VPRecipeBase. | ||
template <typename RecipeTy> class VPPhiAccessors { | ||
/// Return a VPRecipeBase* to the current object. | ||
const VPRecipeBase *getAsRecipe() const { | ||
return static_cast<const RecipeTy *>(this); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the templating really needed, can There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep it seems so, problem is that w/o the template there's no inheritance relation here, and static_casts are rejected (Curiously Recurring Template Pattern). For some reason, we need to cast exactly to the type. But the template parameter may cause problems in the future, so I updated There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The use case enabled by getting rid of the template argument is supporting There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, would having getAsRecipe() do a dyn_cast to VPRecipeBase be ok? Possibly followed by an assert... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we could |
||
} | ||
|
||
public: | ||
/// Returns the incoming VPValue with index \p Idx. | ||
VPValue *getIncomingValue(unsigned Idx) const { | ||
return getAsRecipe()->getOperand(Idx); | ||
} | ||
Comment on lines
+1180
to
+1182
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All recipes are single defs, but we now unfortunately have some recipes (e.g. VPIRPhi) where the base class Alternatively we could manually add definitions of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, who are all the relevant classes - expected to inherit directly from If In any case, good to implement both There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we need it for VPWidenPHIRecipe, VPHeaderPHIRecipe, VPIRPhi, VPEVLBasedPhi and VPPhi (scalar phis via VPInstruction, probably via a new specialization). Define both There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Another one is VPPredInstPhi There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. VPWidenPHIRecipe, VPHeaderPHIRecipe (base class of VPEVLBasedPhi), and VPPredInstPhi all inherit from VPSingleDefRecipe. So could inherit from VPSingleDefPhiRecipe instead, which could take care of implementing these pure virtual methods for them. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Made the |
||
|
||
/// Returns an interator range over the incoming values | ||
VPUser::const_operand_range incoming_values() const { | ||
return getAsRecipe()->operands(); | ||
} | ||
|
||
/// Returns the incoming block with index \p Idx. | ||
const VPBasicBlock *getIncomingBlock(unsigned Idx) const; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Inline as well? Good to see its implementation next to that of getIncomingValue() and getNumIncoming(). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It needs access to There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, probably better done separately, if at all. |
||
|
||
using const_incoming_block_iterator = | ||
mapped_iterator<detail::index_iterator, | ||
std::function<const VPBasicBlock *(size_t)>>; | ||
using const_incoming_blocks_range = | ||
iterator_range<const_incoming_block_iterator>; | ||
|
||
const_incoming_block_iterator incoming_block_begin() const { | ||
return const_incoming_block_iterator( | ||
detail::index_iterator(0), | ||
[this](size_t Idx) { return getIncomingBlock(Idx); }); | ||
} | ||
const_incoming_block_iterator incoming_block_end() const { | ||
return const_incoming_block_iterator( | ||
detail::index_iterator(getAsRecipe()->getNumOperands()), | ||
[this](size_t Idx) { return getIncomingBlock(Idx); }); | ||
} | ||
|
||
/// Returns an iterator range over the incoming blocks. | ||
const_incoming_blocks_range incoming_blocks() const { | ||
return make_range(incoming_block_begin(), incoming_block_end()); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If/when predecessors already have a natural iterator, would building an index_iterator from retrieving each block get folded into the former? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can check once we bring those iterators back |
||
|
||
/// Returns an iterator range over pairs of incoming values and corresponding | ||
/// incoming blocks. | ||
detail::zippy<llvm::detail::zip_shortest, VPUser::const_operand_range, | ||
const_incoming_blocks_range> | ||
incoming_values_and_blocks() const { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suffice to keep this method public, as There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. for now lets just add |
||
return zip(incoming_values(), incoming_blocks()); | ||
} | ||
}; | ||
|
||
/// An overlay for VPIRInstructions wrapping PHI nodes enabling convenient use | ||
/// cast/dyn_cast/isa and execute() implementation. A single VPValue operand is | ||
/// allowed, and it is used to add a new incoming value for the single | ||
|
@@ -1976,7 +2029,8 @@ class VPWidenPointerInductionRecipe : public VPWidenInductionRecipe, | |
/// recipe is placed in an entry block to a (non-replicate) region, it must have | ||
/// exactly 2 incoming values, the first from the predecessor of the region and | ||
/// the second from the exiting block of the region. | ||
class VPWidenPHIRecipe : public VPSingleDefRecipe { | ||
class VPWidenPHIRecipe : public VPSingleDefRecipe, | ||
public VPPhiAccessors<VPWidenPHIRecipe> { | ||
/// Name to use for the generated IR instruction for the widened phi. | ||
lukel97 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
std::string Name; | ||
|
||
|
@@ -2007,12 +2061,6 @@ class VPWidenPHIRecipe : public VPSingleDefRecipe { | |
void print(raw_ostream &O, const Twine &Indent, | ||
VPSlotTracker &SlotTracker) const override; | ||
#endif | ||
|
||
/// Returns the \p I th incoming VPBasicBlock. | ||
VPBasicBlock *getIncomingBlock(unsigned I); | ||
|
||
/// Returns the \p I th incoming VPValue. | ||
VPValue *getIncomingValue(unsigned I) { return getOperand(I); } | ||
}; | ||
|
||
/// A recipe for handling first-order recurrence phis. The start value is the | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1205,6 +1205,32 @@ void VPIRPhi::print(raw_ostream &O, const Twine &Indent, | |
} | ||
#endif | ||
|
||
/// Returns the incoming block at index \p Idx for \p R. This handles both | ||
/// recipes placed in entry blocks of loop regions (incoming blocks are the | ||
/// region's predecessor and the region's exit) and other locations (incoming | ||
/// blocks are the direct predecessors). | ||
static const VPBasicBlock *getIncomingBlockForRecipe(const VPRecipeBase *R, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it worth using the same There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated to use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should this be OTOH, if used only by This provides the CFG predecessor basic-blocks of a given block (rather than recipe), which could be in CFG mode (in which case they are held explicitly, can cast them from block to basic-block) or HCFG mode (in which case region header blocks need to collect their region's predecessor('s exiting) basic-block and exiting basic-block. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep had a similar thought, but wasn't sure what a good name would be. Added as |
||
unsigned Idx) { | ||
const VPBasicBlock *Parent = R->getParent(); | ||
const VPBlockBase *Pred = nullptr; | ||
if (Parent->getNumPredecessors() > 0) { | ||
Pred = Parent->getPredecessors()[Idx]; | ||
} else { | ||
auto *Region = Parent->getParent(); | ||
assert(Region && !Region->isReplicator() && Region->getEntry() == Parent && | ||
"must be in the entry block of a non-replicate region"); | ||
assert( | ||
Idx < 2 && R->getNumOperands() == 2 && | ||
"when placed in an entry block, only 2 incoming blocks are available"); | ||
|
||
// Idx == 0 selects the predecessor of the region, Idx == 1 selects the | ||
// region itself whose exiting block feeds the phi across the backedge. | ||
Pred = Idx == 0 ? Region->getSinglePredecessor() : Region; | ||
} | ||
|
||
return Pred->getExitingBasicBlock(); | ||
} | ||
|
||
void VPIRMetadata::applyMetadata(Instruction &I) const { | ||
for (const auto &[Kind, Node] : Metadata) | ||
I.setMetadata(Kind, Node); | ||
|
@@ -3694,25 +3720,10 @@ void VPReductionPHIRecipe::print(raw_ostream &O, const Twine &Indent, | |
} | ||
#endif | ||
|
||
VPBasicBlock *VPWidenPHIRecipe::getIncomingBlock(unsigned I) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Better keep delegating the implementation of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is now providing a specialized definition of |
||
VPBasicBlock *Parent = getParent(); | ||
VPBlockBase *Pred = nullptr; | ||
if (Parent->getNumPredecessors() > 0) { | ||
Pred = Parent->getPredecessors()[I]; | ||
} else { | ||
auto *Region = Parent->getParent(); | ||
assert(Region && !Region->isReplicator() && Region->getEntry() == Parent && | ||
"must be in the entry block of a non-replicate region"); | ||
assert( | ||
I < 2 && getNumOperands() == 2 && | ||
"when placed in an entry block, only 2 incoming blocks are available"); | ||
|
||
// I == 0 selects the predecessor of the region, I == 1 selects the region | ||
// itself whose exiting block feeds the phi across the backedge. | ||
Pred = I == 0 ? Region->getSinglePredecessor() : Region; | ||
} | ||
|
||
return Pred->getExitingBasicBlock(); | ||
template <> | ||
const VPBasicBlock * | ||
VPPhiAccessors<VPWidenPHIRecipe>::getIncomingBlock(unsigned Idx) const { | ||
return getIncomingBlockForRecipe(getAsRecipe(), Idx); | ||
} | ||
|
||
void VPWidenPHIRecipe::execute(VPTransformState &State) { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Perhaps better to rename Inc to Idx? It sounds like a short form of increment or incoming.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a short form of incoming value, while Idx would refer to an Index?