test flake in weighted random picker

[nirrozenbaum]

What happened:
test flakes in random weighted picker.
example here:
https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_gateway-api-inference-extension/1537/pull-gateway-api-inference-extension-test-unit-main/1963873589643972608
What you expected to happen:
tests to pass
How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Inference extension version (use git describe --tags --dirty --always):
• Cloud provider or hardware configuration:
• Install tools:
• Others:

[rockswe]

/assign

[nirrozenbaum]

@rockswe any update on this issue?
unfortunately it seems like this flake is hitting us on every other PR.
we need a fix asap.

please let me know if you don't have the capacity so we can take care of this issue soon.

[nirrozenbaum]

since I got no response and this is an urgent fix that blocks incoming PRs (flake hits us almost on every PR), I have to push a fix.
apologize @rockswe, if you want we can find more good first issues to work on 🙏

[rockswe]

I ran into some delays and couldn’t prioritize it properly, so I couldn’t deliver the fix as quickly as needed. Sorry about that.