CMAKE CUDA fix: Unknown option '-Wall'

[clibdev]

Closes #5542

[phymbert]

I was also facing this issue with CLion CMake import, thanks a lot @clibdev

[slaren]

I think we can merge this to solve the immediate problem, but the way CXX_FLAGS and the function get_flags is being used looks completely wrong to me. get_flags is only used with LLAMA_ALL_WARNINGS, yet it also contains optimization flags for the Intel compiler. get_flags is also used with CUDA to set the host compiler flags, and it actually checks what is the host compiler instead of assuming that it is the same than for the rest of the C++ code, which is great, but there are still flags from CXX_FLAGS that should also be passed to the host compiler. Adding CXX_FLAGS to CUDA_CXX_FLAGS is not correct either, because CXX_FLAGS also includes the flags from get_flags, but checked for the wrong compiler.

[ggerganov]

@slaren Thanks for the review

get_flags is only used with LLAMA_ALL_WARNINGS, yet it also contains optimization flags for the Intel compiler

@abhilash1910 Could you take a look into this - it does look incorrect to add Intel compiler flags here

Will wait for @cebtenzzre to review get_flags usage and see if we can improve

[cebtenzzre]

get_flags is only used with LLAMA_ALL_WARNINGS, yet it also contains optimization flags for the Intel compiler.

This was added by @ngxson in #5068, I didn't review that PR because the title and commit messages only mentioned Docker.

Adding CXX_FLAGS to CUDA_CXX_FLAGS is not correct either, because CXX_FLAGS also includes the flags from get_flags, but checked for the wrong compiler.

The idea is that CXX_FLAGS contains generic flags, and GF_CXX_FLAGS contains the compiler-specific flags - they must both be combined to get the full list of flags. For the host this is done in the add_compile_options step, for CUDA this is currently not done consistently (mainly because I've never personally seen nvcc choke on e.g. -Wall - I'm not sure what causes this).

[cebtenzzre]

I propose this solution: #5570

[abhilash1910]

@slaren Thanks for the review

get_flags is only used with LLAMA_ALL_WARNINGS, yet it also contains optimization flags for the Intel compiler

@abhilash1910 Could you take a look into this - it does look incorrect to add Intel compiler flags here

Will wait for @cebtenzzre to review get_flags usage and see if we can improve

Yes it does not appear to be correct as that is in the scope of LLAMA_ALL_WARNINGS. I will remove this. Thanks for pointing this out.

[ngxson]

@cebtenzzre Thanks for pointing it out, #5068 was a trial to make the compilation with intel dpcpp yield more performance, but this is pretty much deprecated now after the introduction of sycl. I'm planning to remove these flags.