-
Notifications
You must be signed in to change notification settings - Fork 12.4k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
While trying to quantize Huginn-22b-Prototype to Q5_0, I ran into this assertion failure while quantizing the output tensor:
[ 331/ 363] output.weight - [ 6656, 32000, 1, 1], type = f16, quantizing to q6_K .. quantize: k_quants.c:73: nearest_int: Assertion `fval <= 4194303.f' failed.
quantize: k_quants.c:73: nearest_int: Assertion `fval <= 4194303.f' failed.
It happens here:
#5 0x00007fed68032d26 in __assert_fail (assertion=0x557583a83303 "fval <= 4194303.f", file=0x557583a832f8 "k_quants.c",
line=73, function=0x557583a83338 <__PRETTY_FUNCTION__.31> "nearest_int") at assert.c:101
#6 0x0000557583a6c991 in nearest_int (fval=-nan(0x400000)) at k_quants.c:73
#7 0x0000557583a7142c in quantize_row_q6_K_reference (x=0x7fed18577010, y=0x7fecdd38c210, k=16384) at k_quants.c:1092
#8 0x0000557583a71cad in ggml_quantize_q6_K (src=0x7fed18577010, dst=0x7fecdd38c210, n=16384, k=16384, hist=0x7feccc000b70)
at k_quants.c:1200
#9 0x00005575839dad38 in ggml_quantize_chunk (type=GGML_TYPE_Q6_K, src=0x7fed18537010, dst=0x7fecdd37f010, start=65536,
n=16384, hist=0x7feccc000b70) at ggml.c:19527
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working