Skip to content

[DAGCombiner][RISCV] Handle truncating splats in isNeutralConstant #87338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 4, 2024
Merged

[DAGCombiner][RISCV] Handle truncating splats in isNeutralConstant #87338

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ec4dadb
[RISCV] Add tests where we aren't folding select with identity consta…
lukel97 Apr 2, 2024
1e69ba4
[DAGCombiner][RISCV] Handle truncating splats in isNeutralConstant
lukel97 Apr 2, 2024
b301127
Add argument comments
lukel97 Apr 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11549,30 +11549,32 @@ bool llvm::isNeutralConstant(unsigned Opcode, SDNodeFlags Flags, SDValue V,
unsigned OperandNo) {
// NOTE: The cases should match with IR's ConstantExpr::getBinOpIdentity().
// TODO: Target-specific opcodes could be added.
if (auto *Const = isConstOrConstSplat(V)) {
if (auto *ConstV = isConstOrConstSplat(V, /*AllowUndefs*/ false,
/*AllowTruncation*/ true)) {
APInt Const = ConstV->getAPIntValue().trunc(V.getScalarValueSizeInBits());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch handles truncating splats by getting the APInt value and truncating it. We almost don't need to do this since most of the neutral elements are either one/zero/all ones, but it will make a difference for smax and smin.

Can you add some tests for smax/smin?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried but I couldn't think of how to. We need to create a splat_vector that implicitly truncates, but if you create it via a regular splat in IR it will arrive in SelectionDAG with a matching type.

It will eventually get type legalized to a truncating splat_vector but by then the fold will have already happened in the first round of DAG combine:

Initial selection DAG: %bb.0 'add:'
SelectionDAG has 19 nodes:
  t0: ch,glue = EntryToken
  t4: nxv2i32,ch = CopyFromReg t0, Register:nxv2i32 %1
  t10: nxv2i32 = insert_vector_elt undef:nxv2i32, Constant:i32<1>, Constant:i64<0>
      t2: nxv2i32,ch = CopyFromReg t0, Register:nxv2i32 %0
        t6: nxv2i1,ch = CopyFromReg t0, Register:nxv2i1 %2
        t11: nxv2i32 = splat_vector Constant:i32<1>
        t13: nxv2i32 = splat_vector Constant:i32<0>
      t14: nxv2i32 = vselect t6, t11, t13
    t15: nxv2i32 = add t2, t14
  t17: ch,glue = CopyToReg t0, Register:nxv2i32 $v8, t15
  t18: ch = RISCVISD::RET_GLUE t17, Register:nxv2i32 $v8, t17:1



Optimized lowered selection DAG: %bb.0 'add:'
SelectionDAG has 13 nodes:
  t0: ch,glue = EntryToken
      t6: nxv2i1,ch = CopyFromReg t0, Register:nxv2i1 %2
        t11: nxv2i32 = splat_vector Constant:i32<1>
      t20: nxv2i32 = add t19, t11
    t21: nxv2i32 = vselect t6, t20, t19
  t17: ch,glue = CopyToReg t0, Register:nxv2i32 $v8, t21
    t2: nxv2i32,ch = CopyFromReg t0, Register:nxv2i32 %0
  t19: nxv2i32 = freeze t2
  t18: ch = RISCVISD::RET_GLUE t17, Register:nxv2i32 $v8, t17:1



Type-legalized selection DAG: %bb.0 'add:'
SelectionDAG has 13 nodes:
  t0: ch,glue = EntryToken
      t6: nxv2i1,ch = CopyFromReg t0, Register:nxv2i1 %2
        t11: nxv2i32 = splat_vector Constant:i64<1>
      t20: nxv2i32 = add t19, t11
    t21: nxv2i32 = vselect t6, t20, t19
  t17: ch,glue = CopyToReg t0, Register:nxv2i32 $v8, t21
    t2: nxv2i32,ch = CopyFromReg t0, Register:nxv2i32 %0
  t19: nxv2i32 = freeze t2
  t18: ch = RISCVISD::RET_GLUE t17, Register:nxv2i32 $v8, t17:1

Hence why the tests are in that zext form, since they get created during vector legalization. But the zext form only allows us to choose constants of zero or one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

combineBinOpToReduce also uses isNeutralConstant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it also runs before type legalization so I presume it will have the same issue

switch (Opcode) {
case ISD::ADD:
case ISD::OR:
case ISD::XOR:
case ISD::UMAX:
return Const->isZero();
return Const.isZero();
case ISD::MUL:
return Const->isOne();
return Const.isOne();
case ISD::AND:
case ISD::UMIN:
return Const->isAllOnes();
return Const.isAllOnes();
case ISD::SMAX:
return Const->isMinSignedValue();
return Const.isMinSignedValue();
case ISD::SMIN:
return Const->isMaxSignedValue();
return Const.isMaxSignedValue();
case ISD::SUB:
case ISD::SHL:
case ISD::SRA:
case ISD::SRL:
return OperandNo == 1 && Const->isZero();
return OperandNo == 1 && Const.isZero();
case ISD::UDIV:
case ISD::SDIV:
return OperandNo == 1 && Const->isOne();
return OperandNo == 1 && Const.isOne();
}
} else if (auto *ConstFP = isConstOrConstSplatFP(V)) {
switch (Opcode) {
Expand Down
21 changes: 8 additions & 13 deletions llvm/test/CodeGen/RISCV/intrinsic-cttz-elts-vscale.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,12 @@ define i32 @ctz_nxv4i32(<vscale x 4 x i32> %a) #0 {
; RV32-NEXT: vmsne.vi v0, v8, 0
; RV32-NEXT: vsetvli zero, zero, e16, m1, ta, ma
; RV32-NEXT: vmv.v.i v8, 0
; RV32-NEXT: vmerge.vim v8, v8, -1, v0
; RV32-NEXT: vand.vv v8, v11, v8
; RV32-NEXT: vmerge.vvm v8, v8, v11, v0
; RV32-NEXT: vredmaxu.vs v8, v8, v8
; RV32-NEXT: vmv.x.s a1, v8
; RV32-NEXT: sub a0, a0, a1
; RV32-NEXT: lui a1, 16
; RV32-NEXT: addi a1, a1, -1
; RV32-NEXT: and a0, a0, a1
; RV32-NEXT: slli a0, a0, 16
; RV32-NEXT: srli a0, a0, 16
; RV32-NEXT: ret
;
; RV64-LABEL: ctz_nxv4i32:
Expand All @@ -41,14 +39,12 @@ define i32 @ctz_nxv4i32(<vscale x 4 x i32> %a) #0 {
; RV64-NEXT: vmsne.vi v0, v8, 0
; RV64-NEXT: vsetvli zero, zero, e16, m1, ta, ma
; RV64-NEXT: vmv.v.i v8, 0
; RV64-NEXT: vmerge.vim v8, v8, -1, v0
; RV64-NEXT: vand.vv v8, v11, v8
; RV64-NEXT: vmerge.vvm v8, v8, v11, v0
; RV64-NEXT: vredmaxu.vs v8, v8, v8
; RV64-NEXT: vmv.x.s a1, v8
; RV64-NEXT: sub a0, a0, a1
; RV64-NEXT: lui a1, 16
; RV64-NEXT: addiw a1, a1, -1
; RV64-NEXT: and a0, a0, a1
; RV64-NEXT: subw a0, a0, a1
; RV64-NEXT: slli a0, a0, 48
; RV64-NEXT: srli a0, a0, 48
; RV64-NEXT: ret
%res = call i32 @llvm.experimental.cttz.elts.i32.nxv4i32(<vscale x 4 x i32> %a, i1 0)
ret i32 %res
Expand Down Expand Up @@ -158,8 +154,7 @@ define i32 @ctz_nxv16i1(<vscale x 16 x i1> %pg, <vscale x 16 x i1> %a) {
; RV64-NEXT: li a1, -1
; RV64-NEXT: vmadd.vx v16, a1, v8
; RV64-NEXT: vmv.v.i v8, 0
; RV64-NEXT: vmerge.vim v8, v8, -1, v0
; RV64-NEXT: vand.vv v8, v16, v8
; RV64-NEXT: vmerge.vvm v8, v8, v16, v0
; RV64-NEXT: vredmaxu.vs v8, v8, v8
; RV64-NEXT: vmv.x.s a1, v8
; RV64-NEXT: subw a0, a0, a1
Expand Down
60 changes: 60 additions & 0 deletions llvm/test/CodeGen/RISCV/rvv/fold-binop-into-select.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 4
; RUN: llc -mtriple=riscv32 -mattr=+v -verify-machineinstrs < %s | FileCheck %s
; RUN: llc -mtriple=riscv64 -mattr=+v -verify-machineinstrs < %s | FileCheck %s

; The following binop x, (zext i1) tests will be vector-legalized into a vselect
; of two splat_vectors, but on RV64 the splat value will be implicitly
; truncated:
;
; t15: nxv2i32 = splat_vector Constant:i64<1>
; t13: nxv2i32 = splat_vector Constant:i64<0>
; t16: nxv2i32 = vselect t2, t15, t13
; t7: nxv2i32 = add t4, t16
;
; Make sure that foldSelectWithIdentityConstant in DAGCombiner.cpp handles the
; truncating splat, so we pull the vselect back and fold it into a mask.

define <vscale x 2 x i32> @i1_zext_add(<vscale x 2 x i1> %a, <vscale x 2 x i32> %b) {
; CHECK-LABEL: i1_zext_add:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetvli a0, zero, e32, m1, ta, mu
; CHECK-NEXT: vadd.vi v8, v8, 1, v0.t
; CHECK-NEXT: ret
%zext = zext <vscale x 2 x i1> %a to <vscale x 2 x i32>
%add = add <vscale x 2 x i32> %b, %zext
ret <vscale x 2 x i32> %add
}

define <vscale x 2 x i32> @i1_zext_add_commuted(<vscale x 2 x i1> %a, <vscale x 2 x i32> %b) {
; CHECK-LABEL: i1_zext_add_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetvli a0, zero, e32, m1, ta, mu
; CHECK-NEXT: vadd.vi v8, v8, 1, v0.t
; CHECK-NEXT: ret
%zext = zext <vscale x 2 x i1> %a to <vscale x 2 x i32>
%add = add <vscale x 2 x i32> %zext, %b
ret <vscale x 2 x i32> %add
}

define <vscale x 2 x i32> @i1_zext_sub(<vscale x 2 x i1> %a, <vscale x 2 x i32> %b) {
; CHECK-LABEL: i1_zext_sub:
; CHECK: # %bb.0:
; CHECK-NEXT: li a0, 1
; CHECK-NEXT: vsetvli a1, zero, e32, m1, ta, mu
; CHECK-NEXT: vsub.vx v8, v8, a0, v0.t
; CHECK-NEXT: ret
%zext = zext <vscale x 2 x i1> %a to <vscale x 2 x i32>
%sub = sub <vscale x 2 x i32> %b, %zext
ret <vscale x 2 x i32> %sub
}

define <vscale x 2 x i32> @i1_zext_or(<vscale x 2 x i1> %a, <vscale x 2 x i32> %b) {
; CHECK-LABEL: i1_zext_or:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetvli a0, zero, e32, m1, ta, mu
; CHECK-NEXT: vor.vi v8, v8, 1, v0.t
; CHECK-NEXT: ret
%zext = zext <vscale x 2 x i1> %a to <vscale x 2 x i32>
%or = or <vscale x 2 x i32> %b, %zext
ret <vscale x 2 x i32> %or
}
Loading