Skip to content

Got "There is not enough space on the disk" error when testing the Kubernetes client in a Windows based CI pipeline #1189

@shenglol

Description

@shenglol

Describe the bug
We have a Windows based CI pipeline to test deploying K8s resource to AKS using the Kubernetes client. The pipeline is invoked many times a day and has been running for months but now we are getting "There is not enough space on the disk" error. Below is the call stack:

   at Internal.Cryptography.Pal.CertificatePal.FilterPFXStore(ReadOnlySpan`1 rawData, SafePasswordHandle password, PfxCertStoreFlags pfxCertStoreFlags)
   at Internal.Cryptography.Pal.CertificatePal.FromBlobOrFile(ReadOnlySpan`1 rawData, String fileName, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(ReadOnlySpan`1 data)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData)
   at k8s.CertUtils.GeneratePfx(KubernetesClientConfiguration config) in /_/src/KubernetesClient/CertUtils.cs:line 105
   at k8s.CertUtils.GetClientCert(KubernetesClientConfiguration config) in /_/src/KubernetesClient/CertUtils.cs:line 203
   at k8s.Kubernetes.InitializeFromConfig(KubernetesClientConfiguration config) in /_/src/KubernetesClient/Kubernetes.ConfigInit.cs:line 90
   at k8s.Kubernetes..ctor(KubernetesClientConfiguration config, DelegatingHandler[] handlers) in /_/src/KubernetesClient/Kubernetes.ConfigInit.cs:line 28

After checking the code, it seems to me that the root cause is that each Kubernetes() constructor call creates a one-time use X509Certificate2 file on the disk that is never removed:

var cert = X509Certificate2.CreateFromPem(certData, keyData);
// see https://github.com/kubernetes-client/csharp/issues/737
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
if (config.ClientCertificateKeyStoreFlags.HasValue)
{
cert = new X509Certificate2(cert.Export(X509ContentType.Pkcs12), "", config.ClientCertificateKeyStoreFlags.Value);
}
else
{
cert = new X509Certificate2(cert.Export(X509ContentType.Pkcs12));
}
}

Should line 94 above be using var cert = X509Certificate2.CreateFromPem(certData, keyData);?

Kubernetes C# SDK Client Version
e.g. 9.0.38

Server Kubernetes Version
N/A

Dotnet Runtime Version
6.0

To Reproduce
Call new Kubernetes(KubernetesClientConfiguration) many times until the disk is full.

Expected behavior
The X509Certificate2 files should be cleaned up.

KubeConfig
N/A

Where do you run your app with Kubernetes SDK (please complete the following information):

  • OS: Windows
  • Environment [e.g. container]
  • Cloud [e.g. Azure]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions