Skip to content

Source-controller Leader Election Errors - Timeout/Connection Issues (v1.5.0) #1815

@GorCorl

Description

@GorCorl

We're encountering issues with the source-controller in our Azure Kubernetes Service (AKS - 1.29.4v) cluster, and I'm hoping someone can offer some guidance. We are running source-controller version 1.5.0.

We're seeing the following errors repeatedly in the source-controller pod logs:

{"level":"error","ts":"2025-05-24T08:51:05.665Z","logger":"runtime","msg":"Failed to update lock optimistically: Put \"https://10.0.0.1:443/apis/coordination.k8s.io/v1/namespaces/flux-system/leases/source-controller-leader-election?timeout=15s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers), falling back to slow path"}
{"level":"error","ts":"2025-05-24T08:51:20.652Z","logger":"runtime","msg":"error retrieving resource lock flux-system/source-controller-leader-election: Get \"https://10.0.0.1:443/apis/coordination.k8s.io/v1/namespaces/flux-system/leases/source-controller-leader-election?timeout=15s\": context deadline exceeded"}
{"level":"error","ts":"2025-05-24T08:51:30.653Z","logger":"runtime","msg":"Failed to release lock: Put \"https://10.0.0.1:443/apis/coordination.k8s.io/v1/namespaces/flux-system/leases/source-controller-leader-election?timeout=15s\": http2: client connection lost"}
{"level":"error","ts":"2025-05-24T08:51:30.659Z","logger":"setup","msg":"problem running manager","error":"leader election lost"}

Are there any known issues or configuration recommendations specific to AKS that might be relevant?
What are the recommended ways to diagnose network connectivity issues between pods and the API server within AKS?
Are there any specific metrics we should be monitoring to help identify the root cause of these errors?

Thanks in advance for any help you can provide!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions