Silent wrong result on FusionTransposeViewSelfMapping_CUDA

[zasdfgbnm]

🐛 Describe the bug

Our codegen fails on fusion like

x = torch.zeros(2, 3)
x.transpose(0, 1) + x.view(3, 2)

i.e.:

t0 = torch.zeros(2, 3);
t1 = t0.view({3,2})
t2 = t0.transpose(0, 1)
t3 = t1 + t2

We should:

1. Figure out why our codegen doesn't detect this and report an error
2. Find a way to detect this in schedulers so the DAG is segmented correctly

See discussion: #1954 (comment)

Versions

TOT devel

[csarofeen]

I think the easiest thing to do but most conservative is to not take view-transpose fusions where there's any path:
(1) starting at any producer (all the way to inputs) of a transpose
(2) ending at any consumer (all the way to outputs) of the transpose in (1)
(3) does not go through the transpose in (1) (or an exact matching transpose)
(4) has a view operation

Less conservative, but I don't know how to make it very generic is for your example...
(I'm uncertain if this approach is similar or what you were saying, had some trouble following.)

The consumer of the view operation is:
t1 root=[i0{2}, i1{3}] => merge(0,1)->split(0, 3, false) => rfactor=[i2{3}, i3{2}]

IterDomains can only be consumed once. i0 and i1 were consumed to produce i2 and i3. Which is directly stating: i2{3} is not the same as i1{3} and i3{2} is not the same as i0{2}

The last expression is:
t3 = t1[i2{3}, i3{2}] + t2[i1{3}, i0{2}]
Which is directly stating: i2{3} is the same as i1{3} and i3{2} is the same as i0{2}

I assume this could get more complicated when there's multiple views, and transposes on each path. However, maybe we could always detect these by looking at the compute at map and finding discrepancies where things are exact mapped that shouldn't be.