[SYCL][RTC] Use program manager

[jopperm]

The idea is to assign a per-compilation prefix (e.g., rtc_42$) to all offload entries in the sycl_device_binaries datastructure before feeding them into ProgramManager::addImages, resulting in unique kernel_ids as far as the PM is concerned. When querying the PM for the device images to construct the kernel bundle, I look for kernel IDs starting with the current prefix, which should reliably return only the device images corresponding to current compilation request.

Note that the actual kernel names don't change, i.e. __sycl_kernel_foo keeps that name even though the PM might know it as rtc_42$__sycl_kernel_foo. The prefix is stored inside the bundle, and prepended to the requested kernel name in the ext_onapi_[has|get]_kernel(string) methods.

Kernel objects are also obtained via the program manager. Compared to creating the UR kernel from the selected device image's UR program directly, this approach ensures eliminated arguments are handled correctly. Hence, I was able to drop the previously mandatory -fno-sycl-dead-args-optimization from the pipeline.

The compilation pipeline and extended kernel bundle now support multiple device images. To test this, I added support for the -fsycl-device-code-split= option, and apply it to one of the compilations in the E2E test.

The persistent cache is circumvented for now for the sycl_jit language (lack of suitable on-disk format), but should be brought back in the future.

[sommerlukas]

Looks good apart from one minor nit.

[sommerlukas]

Do we still need this comment?
https://github.com/intel/llvm/pull/16316/files#diff-01975a59586f8efe5e76699c7ee2ff4ff69d9a394134a2bb888124d0c54049bfR500-R501

[jopperm]

Do we still need this comment?

No, I'll drop it!

[cperkinsintel]

cool!

[jopperm]

A bit of additional context for the last commit which marks the constructed kernel bundle as interop:

llvm/sycl/source/detail/kernel_bundle_impl.hpp

Lines 387 to 392 in 5a729ba

	// Mark this bundle explicitly as "interop" to ensure that its kernels are
	// enqueued with the info from the kernel object passed by the application,
	// cf. `enqueueImpKernel` in `commands.cpp`. While runtime-compiled kernels
	// loaded via the program manager have `kernel_id`s, they can't be looked up
	// from the (unprefixed) kernel name.
	MIsInterop = true;

I'm doing this to get into the second branch here (there are more occurrences of this pattern in the codebase):

llvm/sycl/source/detail/scheduler/commands.cpp

Lines 2731 to 2765 in 5a729ba

	// Use kernel_bundle if available unless it is interop.
	// Interop bundles can't be used in the first branch, because the kernels
	// in interop kernel bundles (if any) do not have kernel_id
	// and can therefore not be looked up, but since they are self-contained
	// they can simply be launched directly.
	if (KernelBundleImplPtr && !KernelBundleImplPtr->isInterop()) {
	kernel_id KernelID =
	detail::ProgramManager::getInstance().getSYCLKernelID(KernelName);
	kernel SyclKernel =
	KernelBundleImplPtr->get_kernel(KernelID, KernelBundleImplPtr);
	SyclKernelImpl = detail::getSyclObjImpl(SyclKernel);
	Kernel = SyclKernelImpl->getHandleRef();
	DeviceImageImpl = SyclKernelImpl->getDeviceImage();
	Program = DeviceImageImpl->get_ur_program_ref();
	EliminatedArgMask = SyclKernelImpl->getKernelArgMask();
	KernelMutex = SyclKernelImpl->getCacheMutex();
	} else if (nullptr != MSyclKernel) {
	assert(MSyclKernel->get_info<info::kernel::context>() ==
	Queue->get_context());
	Kernel = MSyclKernel->getHandleRef();
	Program = MSyclKernel->getProgramRef();
	// Non-cacheable kernels use mutexes from kernel_impls.
	// TODO this can still result in a race condition if multiple SYCL
	// kernels are created with the same native handle. To address this,
	// we need to either store and use a ur_native_handle_t -> mutex map or
	// reuse and return existing SYCL kernels from make_native to avoid
	// their duplication in such cases.
	KernelMutex = &MSyclKernel->getNoncacheableEnqueueMutex();
	EliminatedArgMask = MSyclKernel->getKernelArgMask();
	} else {

We can't take the first branch because we can't look up the correct kernel_id from the kernel name without adding the RTC-specific prefix. At a first glance, it doesn't seem feasible to also use the prefix in context of kernel objects, handler, commands, etc., because, IIUC, names there are tied to the actual function names, and hence shouldn't be modified. (The overall idea in this PR is to use unique names wrt. the PM's bookkeeping, but don't change the compiled programs.)

Note that the kernel bundle constructed before this PR was also marked as interop (by choice of the parent constructor), hence my commit just restores the situation as before. I'm not a 100% sure I understand all implications of the bundle being interop or not, so please let me know if I'm doing this wrong 🙏

[sergey-semenov]

The approach looks good to me.