[SYCL][CUDA] Add basic sub-group functionality

[Pennycook]

Implements:

• Sub-group local id
• Sub-group id
• Number of sub-groups
• Sub-group size
• Max sub-group size

The implementations are functionally correct, but may benefit from
additional optimization.

Signed-off-by: John Pennycook [email protected]

---

The implementation is different to the one proposed in https://intel.github.io/llvm-docs/cuda/opencl-subgroup-vs-cuda-crosslane-op.html, because I don't think sreg.warpid and sreg.nwarpid have the correct semantics for sub-groups. The mapping from work-items to sub-groups is invariant during a kernel's execution, which isn't true of the warp ID in PTX. As far as I can tell, the number of warp IDs represents the maximum number of warps that can execute in a CTA rather than the number of warps in a CTA.

NVIDIA's PTX documentation says that tid should be used to compute a "virtual warp ID" if one is required, which is what I've implemented. I convinced myself that we have to compute the sub-group IDs and sizes from the linear size of the work-group, and couldn't find a simpler way to express this.

Ideally, we wouldn't have to re-compute each of these values on every call. It would be sufficient to compute them once at the start of the kernel and then re-use them, but I don't have enough knowledge of Clang/LLVM/libclc to implement that.

[bader]

Tagging @Naghasan.

[illuhad]

NVIDIA's PTX documentation says that tid should be used to compute a "virtual warp ID" if one is required, which is what I've implemented. I convinced myself that we have to compute the sub-group IDs and sizes from the linear size of the work-group, and couldn't find a simpler way to express this.

Yes, I also stumbled across that - that's also how it works in hipSYCL. Additionally, my understanding is that the laneid register may be less performant than computing the subgroup local id yourself (which is what hipSYCL does), see e.g. https://stackoverflow.com/questions/44337309/whats-the-most-efficient-way-to-calculate-the-warp-id-lane-id-in-a-1-d-grid

[Pennycook]

Thanks, @illuhad.

Additionally, my understanding is that the laneid register may be less performant than computing the subgroup local id yourself (which is what hipSYCL does), see e.g. https://stackoverflow.com/questions/44337309/whats-the-most-efficient-way-to-calculate-the-warp-id-lane-id-in-a-1-d-grid

We should definitely revisit this if benchmarks suggest that manually computing the lane id is always better. I found some references to special registers potentially having high latency, but nothing about the relative cost of different special registers. Since we'd need to read the tid and ntid special registers to compute things manually, I wasn't confident that I understood the trade-off here.