Skip to content

Enable Fused-Multiply-Add (FMA) and F16C/CVT16 vector extensions on MSVC #375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 28, 2023
Merged

Enable Fused-Multiply-Add (FMA) and F16C/CVT16 vector extensions on MSVC #375

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
50ea002
Enable Fused-Multiply-Add (FMA) instructions on MSVC
anzz1 Mar 22, 2023
c43d456
Enable F16C/CVT16 vector extensions on MSVC
anzz1 Mar 22, 2023
67ad7c4
MSVC cvt intrinsics
anzz1 Mar 22, 2023
b8a80f9
Add __SSE3__ macro for MSVC too because why not
anzz1 Mar 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions ggml.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,19 @@ static int sched_yield (void) {
typedef void* thread_ret_t;
#endif

// __FMA__ and __F16C__ are not defined in MSVC, however they are implied with AVX2/AVX512
#if defined(_MSC_VER) && (defined(__AVX2__) || defined(__AVX512F__))
#ifndef __FMA__
#define __FMA__
#endif
#ifndef __F16C__
#define __F16C__
#endif
#ifndef __SSE3__
#define __SSE3__
#endif
#endif

#ifdef __HAIKU__
#define static_assert(cond, msg) _Static_assert(cond, msg)
#endif
Expand Down Expand Up @@ -172,8 +185,13 @@ typedef double ggml_float;

#ifdef __F16C__

#ifdef _MSC_VER
#define GGML_COMPUTE_FP16_TO_FP32(x) _mm_cvtss_f32(_mm_cvtph_ps(_mm_cvtsi32_si128(x)))
#define GGML_COMPUTE_FP32_TO_FP16(x) _mm_extract_epi16(_mm_cvtps_ph(_mm_set_ss(x), 0), 0)
#else
#define GGML_COMPUTE_FP16_TO_FP32(x) _cvtsh_ss(x)
#define GGML_COMPUTE_FP32_TO_FP16(x) _cvtss_sh(x, 0)
#endif

#elif defined(__POWER9_VECTOR__)

Expand Down