Skip to content

quantize: k_quants.c:73: nearest_int: Assertion `fval <= 4194303.f' failed. #2982

@cebtenzzre

Description

@cebtenzzre

While trying to quantize Huginn-22b-Prototype to Q5_0, I ran into this assertion failure while quantizing the output tensor:

[ 331/ 363]                        output.weight - [ 6656, 32000,     1,     1], type =    f16, quantizing to q6_K .. quantize: k_quants.c:73: nearest_int: Assertion `fval <= 4194303.f' failed.
quantize: k_quants.c:73: nearest_int: Assertion `fval <= 4194303.f' failed.

It happens here:

#5  0x00007fed68032d26 in __assert_fail (assertion=0x557583a83303 "fval <= 4194303.f", file=0x557583a832f8 "k_quants.c", 
    line=73, function=0x557583a83338 <__PRETTY_FUNCTION__.31> "nearest_int") at assert.c:101
#6  0x0000557583a6c991 in nearest_int (fval=-nan(0x400000)) at k_quants.c:73
#7  0x0000557583a7142c in quantize_row_q6_K_reference (x=0x7fed18577010, y=0x7fecdd38c210, k=16384) at k_quants.c:1092
#8  0x0000557583a71cad in ggml_quantize_q6_K (src=0x7fed18577010, dst=0x7fecdd38c210, n=16384, k=16384, hist=0x7feccc000b70)
    at k_quants.c:1200
#9  0x00005575839dad38 in ggml_quantize_chunk (type=GGML_TYPE_Q6_K, src=0x7fed18537010, dst=0x7fecdd37f010, start=65536, 
    n=16384, hist=0x7feccc000b70) at ggml.c:19527

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions