Implementation details of speculative decoding

[exhyy]

Speculative decoding is described at https://arxiv.org/pdf/2302.01318
In the paper, speculative decoding requires calculations between draft probabilities and target probabilities, as shown in the figure below. However, I cannot find the code corresponding to the formula. Based on my understanding, llama.cpp simply directly compares the tokens sampled by the draft model and the tokens sampled by the target model. If they differ, it resamples from the target distribution. If my understanding is incorrect, please correct me.

[ggerganov]

We had this implemented at some point (see #5625), but I think I decided it's not worth the complication (#10362). Greedy sampling is good enough and the logic is much simpler.