Handling tied operands in ternary pseudos in RISCVVLOptimizer

[lukel97]

TL;DR: we can't express tail undefined with ternary pseudos

I've been exploring using RISCVVLOptimizer to reduce VL toggles with EVL tail folding. In some of the benchmarks we fail to reduce the AVL of a ternary instruction, causing a vl toggle e.g.:

        vsetvli a4, zero, e32, m2, ta, ma
        vsext.vf2       v14, v10
        vsetvli zero, a1, e32, m2, ta, ma
        vmadd.vv        v14, v12, v8

Although we're able to reduce the AVL of the vmadd.vv itself, we can't reduce the AVL of the vsext.vf2 because its user, vmadd.vv, is a tied operand:

Trying to reduce VL for early-clobber %43:vrm2 = PseudoVSEXT_VF2_M2 $noreg(tied-
def 0), killed %42:vr, -1, 5, 3

  Checking user: %49:vrm2 = nsw PseudoVMADD_VV_M2 %43:vrm2(tied-def 0), killed %
47:vrm2, %48:vrm2, %40:gprnox0, 5, 1

    Abort because user used as tied operand

We don't allow users that are passthrus because the tail elements past vl will depend on them.

std::optional<MachineOperand> RISCVVLOptimizer::checkUsers(MachineInstr &MI) {
  for (auto &UserOp : MRI->use_operands(MI.getOperand(0).getReg())) {
    ...
    // Tied operands might pass through.
    if (UserOp.isTied()) {
      LLVM_DEBUG(dbgs() << "    Abort because user used as tied operand\n");
      return std::nullopt;
    }

At first glance we might think that we can relax this restriction if the user's policy is ta/ma. However I don't think this is correct because tail agnostic/mask agnostic != tail undefined/mask undefined, so we still need to preserve the passthru to get "previous or -1" for each element.

I think the problem comes down to how for binary pseudos we represent tail undefined/mask undefined by using NoRegister for the passthru.

For ternary pseudos the third operand is simultaneously also the passthru, so it's not possible to represent a tail undefined/mask undefined pseudo with a non-undefined third operand.

This means that we lose the information that the inactive elements are undefined during instruction selection. One possible solution could be to add new _INACTIVE_UNDEF pseudos with a TSFlag that marks this fact, and use these for regular SDNode patterns. Then RISCVVLOptimizer (and RISCVInsertVSETVLI) could pick up the TSFlag and allow it as a user.

[llvmbot]

@llvm/issue-subscribers-backend-risc-v

Author: Luke Lau (lukel97)

TL;DR: we can't express tail undefined with ternary pseudos

I've been exploring using RISCVVLOptimizer to reduce VL toggles with EVL tail folding. In some of the benchmarks we fail to reduce the AVL of a ternary instruction, causing a vl toggle e.g.:

        vsetvli a4, zero, e32, m2, ta, ma
        vsext.vf2       v14, v10
        vsetvli zero, a1, e32, m2, ta, ma
        vmadd.vv        v14, v12, v8

Although we're able to reduce the AVL of the vmadd.vv itself, we can't reduce the AVL of the vsext.vf2 because its user, vmadd.vv, is a tied operand:

Trying to reduce VL for early-clobber %43:vrm2 = PseudoVSEXT_VF2_M2 $noreg(tied-
def 0), killed %42:vr, -1, 5, 3

  Checking user: %49:vrm2 = nsw PseudoVMADD_VV_M2 %43:vrm2(tied-def 0), killed %
47:vrm2, %48:vrm2, %40:gprnox0, 5, 1

    Abort because user used as tied operand

We don't allow users that are passthrus because the tail elements past vl will depend on them.

std::optional<MachineOperand> RISCVVLOptimizer::checkUsers(MachineInstr &MI) {
  for (auto &UserOp : MRI->use_operands(MI.getOperand(0).getReg())) {
    ...
    // Tied operands might pass through.
    if (UserOp.isTied()) {
      LLVM_DEBUG(dbgs() << "    Abort because user used as tied operand\n");
      return std::nullopt;
    }

At first glance we might think that we can relax this restriction if the user's policy is ta/ma. However I don't think this is correct because tail agnostic/mask agnostic != tail undefined/mask undefined, so we still need to preserve the passthru to get "previous or -1" for each element.

I think the problem comes down to how for binary pseudos we represent tail undefined/mask undefined by using NoRegister for the passthru.

For ternary pseudos the third operand is simultaneously also the passthru, so it's not possible to represent a tail undefined/mask undefined pseudo with a non-undefined third operand.

This means that we lose the information that the inactive elements are undefined during instruction selection. One possible solution could be to add new _INACTIVE_UNDEF pseudos with a TSFlag that marks this fact, and use these for regular SDNode patterns. Then RISCVVLOptimizer (and RISCVInsertVSETVLI) could pick up the TSFlag and allow it as a user.