Properly handle delete events in reconcilers

[kfswain]

This cropped up in: #286 (comment)

That comment thread also contains the context in how to adequately resolve delete events with controller-runtime. We should have proper delete handling for our inferenceModel reconciliation (possibly/probably for our pod reconcilier also)

[ahg-g]

/kind bug

[ahg-g]

We probably better use finer grained event handlers for handle Create/Update/Delete events for both InferenceModel and Pod: see example https://github.com/kubernetes-sigs/kueue/blob/96eb2e59692ae2bf1bb7341a073facbdbb0de12f/pkg/controller/core/localqueue_controller.go#L132

Those handlers should do minimal work (updating our local datastore) since they are invoked in the watch callback; work that requires any heavy lifting, like status updates, should continue to be done in Reconcile which is called by the worker threads.