llama-cpp manually is noticeably faster than running it through the python API

[rnwang04]

similar to #71 (comment), I found on my machine, LLaMA CPP is 27ms per token, and the python bindings are 68ms per token so it's much slower.
I wonder is there any idea for this performance degradation? Is such degradation normal? Or is there any way to close this gap?
Thanks a lot！

[gpake]

llama.cpp is 40ms for native build, using more thread will make it faster (4 vs 8 threads, got ~2x faster)
python version is ~86ms per token, using more thread will not get a significant speed change (5(seems the default value) vs 8, got almost 0% faster)

[gjmulder]

Any update?

[gjmulder]

#232 looks to be the best issue to discuss performance currently