[core][optimization] use a pool of numpy ndarray to hold seq data

[youkaichao]

similar to #5584

the same benchmark command:

python benchmarks/benchmark_throughput.py --output-len 256 --input 256 --model meta-llama/Llama-2-7b-hf -tp 8

the same machine: 8*H100

before (current main): Throughput: 38.07 requests/s, 19493.23 tokens/s

after (this PR): Throughput: 38.94 requests/s, 19939.65 tokens/s

let's see if it breaks anything. we need to make sure, we only use python list when receiving/sending user's request. elsewhere, we should keep numpy array, where slicing is only a view operation. Never copy the whole sequence.

[youkaichao]

close as it is separated into #5882 and #5942