Description
LocalAI version:
Latest
Environment, CPU architecture, OS, and Version:
Linux deb11-local 5.10.0-25-amd64 #1 SMP Debian 5.10.191-1 (2023-08-16) x86_64 GNU/Linux
KVM hosted VM
32GB Ram NVIDIA RTX3090
Docker Version 20
NVidia Container Toolkit
Describe the bug
Running the lunademo curl request from the howtos
Output:
'{"error":{"code":500,"message":"could not load model: rpc error: code = Unavailable desc = error reading from server: EOF","type":""}}
'
Docker Logs: `@@@@@
Skipping rebuild
@@@@@
If you are experiencing issues with the pre-compiled builds, try setting REBUILD=true
If you are still experiencing issues with the build, try setting CMAKE_ARGS and disable the instructions set as needed:
CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF"
see the documentation at: https://localai.io/basics/build/index.html
Note: See also #288
@@@@@
CPU info:
model name : Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq vmx ssse3 cx16 pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cpuid_fault pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust smep erms xsaveopt arat umip md_clear arch_capabilities
CPU: AVX found OK
CPU: no AVX2 found
CPU: no AVX512 found
@@@@@
7:14AM INF Starting LocalAI using 8 threads, with models path: /models
7:14AM INF LocalAI version: 704323b (704323b)
7:14AM DBG Model: lunademo (config: {PredictionOptions:{Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.2 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:lunademo F16:false Threads:0 Debug:false Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0}})
7:14AM DBG Extracting backend assets files to /tmp/localai/backend_data
┌───────────────────────────────────────────────────┐
│ Fiber v2.48.0 │
│ http://127.0.0.1:8080 │
│ (bound on host 0.0.0.0 and port 8080) │
│ │
│ Handlers ............ 59 Processes ........... 1 │
│ Prefork ....... Disabled PID ................ 14 │
└───────────────────────────────────────────────────┘
7:15AM DBG Request received:
7:15AM DBG Configuration read: &{PredictionOptions:{Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:lunademo F16:false Threads:8 Debug:true Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0}}
7:15AM DBG Parameters: &{PredictionOptions:{Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:lunademo F16:false Threads:8 Debug:true Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0}}
7:15AM DBG Prompt (before templating): How are you?
7:15AM DBG Template found, input modified to: How are you?
Response:
7:15AM DBG Prompt (after templating): How are you?
Response:
7:15AM DBG Loading model llama from wizardlm-13b-v1.2.ggmlv3.q4_0.bin
7:15AM DBG Loading model in memory from file: /models/wizardlm-13b-v1.2.ggmlv3.q4_0.bin
7:15AM DBG Loading GRPC Model llama: {backendString:llama model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin threads:8 assetDir:/tmp/localai/backend_data context:0xc00003c0b0 gRPCOptions:0xc0002c2a00 externalBackends:map[autogptq:/build/extra/grpc/autogptq/autogptq.py bark:/build/extra/grpc/bark/ttsbark.py diffusers:/build/extra/grpc/diffusers/backend_diffusers.py exllama:/build/extra/grpc/exllama/exllama.py huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false}
7:15AM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama
7:15AM DBG GRPC Service for wizardlm-13b-v1.2.ggmlv3.q4_0.bin will be running at: '127.0.0.1:42467'
7:15AM DBG GRPC Service state dir: /tmp/go-processmanager1225292084
7:15AM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42467: connect: connection refused"
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr 2023/08/24 07:15:07 gRPC Server listening at 127.0.0.1:42467
7:15AM DBG GRPC Service Ready
7:15AM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:} sizeCache:0 unknownFields:[] Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin ContextSize:1024 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/wizardlm-13b-v1.2.ggmlv3.q4_0.bin Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 Tokenizer:}
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr SIGILL: illegal instruction
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr PC=0x87a09c m=4 sigcode=2
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr signal arrived during cgo execution
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr instruction bytes: 0xc4 0xe3 0x7d 0x39 0x8c 0x24 0x18 0x3 0x0 0x0 0x1 0x66 0x89 0x84 0x24 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 19 [syscall]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.cgocall(0x85c6c0, 0xc0003496c0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000349698 sp=0xc000349660 pc=0x424a9c
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/go-llama%2ecpp._Cfunc_load_model(0x7f71b8000b60, 0x400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x200, ...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr _cgo_gotypes.go:266 +0x4d fp=0xc0003496c0 sp=0xc000349698 pc=0x852bad
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/go-llama%2ecpp.New({0xc000340090, 0x29}, {0xc00034c140, 0x7, 0x950400?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/go-llama/llama.go:28 +0x297 fp=0xc000349830 sp=0xc0003496c0 pc=0x8533b7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/backend/llm/llama.(*LLM).Load(0xc00006de90, 0xc00035a000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/backend/llm/llama/llama.go:73 +0x8e7 fp=0xc000349960 sp=0xc000349830 pc=0x857ce7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/grpc.(*server).LoadModel(0xc00006df20, {0xc00035a000?, 0x51f086?}, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/grpc/server.go:50 +0xf7 fp=0xc000349a10 sp=0xc000349960 pc=0x85a6d7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/grpc/proto._Backend_LoadModel_Handler({0x9bd280?, 0xc00006df20}, {0xaa01c0, 0xc00030a3c0}, 0xc00034e070, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/grpc/proto/backend_grpc.pb.go:277 +0x170 fp=0xc000349a68 sp=0xc000349a10 pc=0x8463f0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).processUnaryRPC(0xc00019a1e0, {0xaa2df8, 0xc00033c000}, 0xc000142120, 0xc0001910e0, 0x1156150, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:1360 +0xe23 fp=0xc000349e48 sp=0xc000349a68 pc=0x82e283
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).handleStream(0xc00019a1e0, {0xaa2df8, 0xc00033c000}, 0xc000142120, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:1737 +0xa36 fp=0xc000349f68 sp=0xc000349e48 pc=0x8333d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).serveStreams.func1.1()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:982 +0x98 fp=0xc000349fe0 sp=0xc000349f68 pc=0x82bc58
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000349fe8 sp=0xc000349fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc.(*Server).serveStreams.func1
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:980 +0x18c
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 1 [IO wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00018fb68 sp=0xc00018fb48 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.netpollblock(0xc00018fbf8?, 0x42412f?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc00018fba0 sp=0xc00018fb68 pc=0x451017
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.runtime_pollWait(0x7f71c179eef8, 0x72)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc00018fbc0 sp=0xc00018fba0 pc=0x482109
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).wait(0xc0000b0680?, 0x0?, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00018fbe8 sp=0xc00018fbc0 pc=0x4f1a12
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).waitRead(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*FD).Accept(0xc0000b0680)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc00018fc90 sp=0xc00018fbe8 pc=0x4f731d
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*netFD).accept(0xc0000b0680)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc00018fd48 sp=0xc00018fc90 pc=0x65b655
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*TCPListener).accept(0xc0000126f0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc00018fd70 sp=0xc00018fd48 pc=0x673ec5
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*TCPListener).Accept(0xc0000126f0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc00018fda0 sp=0xc00018fd70 pc=0x672fbd
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).Serve(0xc00019a1e0, {0xa9fa20?, 0xc0000126f0})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:844 +0x475 fp=0xc00018fee8 sp=0xc00018fda0 pc=0x82a875
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/grpc.StartServer({0x7fff590cfc4c?, 0xc000024190?}, {0xaa3960?, 0xc00006de90})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/grpc/server.go:178 +0x125 fp=0xc00018ff50 sp=0xc00018fee8 pc=0x85c185
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr main.main()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/cmd/grpc/llama/main.go:22 +0x85 fp=0xc00018ff80 sp=0xc00018ff50 pc=0x85c2e5
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.main()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc00018ffe0 sp=0xc00018ff80 pc=0x4582a7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00018ffe8 sp=0xc00018ffe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 2 [force gc (idle)]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000056fb0 sp=0xc000056f90 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goparkunlock(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:387
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.forcegchelper()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000056fe0 sp=0xc000056fb0 pc=0x458510
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000056fe8 sp=0xc000056fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.init.6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:293 +0x25
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 3 [GC sweep wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000057780 sp=0xc000057760 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goparkunlock(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:387
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.bgsweep(0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc0000577c8 sp=0xc000057780 pc=0x4448ce
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gcenable.func1()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000577e0 sp=0xc0000577c8 pc=0x439b86
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000577e8 sp=0xc0000577e0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.gcenable
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:178 +0x6b
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 4 [GC scavenge wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0xc000080000?, 0xa98988?, 0x1?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000057f70 sp=0xc000057f50 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goparkunlock(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:387
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.(*scavengerState).park(0x11a2fe0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000057fa0 sp=0xc000057f70 pc=0x4427f3
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.bgscavenge(0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc000057fc8 sp=0xc000057fa0 pc=0x442dc5
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gcenable.func2()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000057fe0 sp=0xc000057fc8 pc=0x439b26
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000057fe8 sp=0xc000057fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.gcenable
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:179 +0xaa
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 5 [finalizer wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x1a0?, 0x11a3500?, 0x60?, 0x78?, 0xc000056770?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000056628 sp=0xc000056608 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.runfinq()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000567e0 sp=0xc000056628 pc=0x438bc7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000567e8 sp=0xc0000567e0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.createfing
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mfinal.go:163 +0x45
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 35 [select]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0xc000357f00?, 0x2?, 0xc3?, 0x51?, 0xc000357ed4?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000357d60 sp=0xc000357d40 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.selectgo(0xc000357f00, 0xc000357ed0, 0x67e3e9?, 0x0, 0xc000322000?, 0x1)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/select.go:327 +0x7be fp=0xc000357ea0 sp=0xc000357d60 pc=0x4682be
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0003040a0, 0x1)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:418 +0x115 fp=0xc000357f30 sp=0xc000357ea0 pc=0x79fa35
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00034e000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:552 +0x91 fp=0xc000357f90 sp=0xc000357f30 pc=0x7a01b1
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func2()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:341 +0xda fp=0xc000357fe0 sp=0xc000357f90 pc=0x7b7b9a
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000357fe8 sp=0xc000357fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:338 +0x1bb3
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 36 [select]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0xc000052f70?, 0x4?, 0x10?, 0x0?, 0xc000052ec0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000052d08 sp=0xc000052ce8 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.selectgo(0xc000052f70, 0xc000052eb8, 0x0?, 0x0, 0x0?, 0x1)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/select.go:327 +0x7be fp=0xc000052e48 sp=0xc000052d08 pc=0x4682be
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00033c000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:1155 +0x233 fp=0xc000052fc8 sp=0xc000052e48 pc=0x7bf273
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func4()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:344 +0x26 fp=0xc000052fe0 sp=0xc000052fc8 pc=0x7b7a86
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000052fe8 sp=0xc000052fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:344 +0x1bf8
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 37 [IO wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x447a60?, 0xb?, 0x0?, 0x0?, 0x6?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000314aa0 sp=0xc000314a80 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.netpollblock(0x4d5da5?, 0x42412f?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc000314ad8 sp=0xc000314aa0 pc=0x451017
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.runtime_pollWait(0x7f71c179ee08, 0x72)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc000314af8 sp=0xc000314ad8 pc=0x482109
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).wait(0xc000308000?, 0xc00031a000?, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000314b20 sp=0xc000314af8 pc=0x4f1a12
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).waitRead(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*FD).Read(0xc000308000, {0xc00031a000, 0x8000, 0x8000})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000314bb8 sp=0xc000314b20 pc=0x4f2df9
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*netFD).Read(0xc000308000, {0xc00031a000?, 0x1060100000000?, 0x8?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc000314c00 sp=0xc000314bb8 pc=0x6594c9
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*conn).Read(0xc00030c000, {0xc00031a000?, 0x60?, 0x0?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/net.go:183 +0x45 fp=0xc000314c48 sp=0xc000314c00 pc=0x66b005
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*TCPConn).Read(0x800010601?, {0xc00031a000?, 0x0?, 0xc000314ca8?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr :1 +0x29 fp=0xc000314c78 sp=0xc000314c48 pc=0x67e0e9
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr bufio.(*Reader).Read(0xc000318000, {0xc000332040, 0x9, 0x0?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/bufio/bufio.go:237 +0x1bb fp=0xc000314cb0 sp=0xc000314c78 pc=0x5d2dfb
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr io.ReadAtLeast({0xa9c3e0, 0xc000318000}, {0xc000332040, 0x9, 0x9}, 0x9)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/io/io.go:332 +0x9a fp=0xc000314cf8 sp=0xc000314cb0 pc=0x4cf89a
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr io.ReadFull(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/io/io.go:351
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr golang.org/x/net/http2.readFrameHeader({0xc000332040?, 0x9?, 0xc000136048?}, {0xa9c3e0?, 0xc000318000?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:237 +0x6e fp=0xc000314d48 sp=0xc000314cf8 pc=0x78b20e
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr golang.org/x/net/http2.(*Framer).ReadFrame(0xc000332000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:498 +0x95 fp=0xc000314df8 sp=0xc000314d48 pc=0x78ba55
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc00033c000, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:642 +0x167 fp=0xc000314f10 sp=0xc000314df8 pc=0x7baec7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).serveStreams(0xc00019a1e0, {0xaa2df8?, 0xc00033c000})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:969 +0x162 fp=0xc000314f80 sp=0xc000314f10 pc=0x82b9a2
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).handleRawConn.func1()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:912 +0x46 fp=0xc000314fe0 sp=0xc000314f80 pc=0x82b246
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000314fe8 sp=0xc000314fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc.(*Server).handleRawConn
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:911 +0x185
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rax 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rbx 0xac06e0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rcx 0x7f71c30121a0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rdx 0x7f7215b1e6d8
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rdi 0x7f7215b1e6c8
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rsi 0x7f7215b16e38
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rbp 0x7f71c30122c0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rsp 0x7f71c3011f40
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r8 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r9 0x7f71b8000080
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r10 0xfffffffffffffa9e
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r11 0x7f7215a21990
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r12 0x1
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r13 0x7f71c3012060
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r14 0x7f71c3011ff0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r15 0x7f71c3012160
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rip 0x87a09c
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rflags 0x10246
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr cs 0x33
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr fs 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr gs 0x0
[172.18.0.1]:43586 500 - POST /v1/chat/completions
`
To Reproduce
See before
Expected behavior
Response from the model
Logs
See before
Additional context
I noticed in the logs that it says "SchedulerType: CUDA:false" Not sure whether this means anything.