Installation Failure due to Blobfuse Proxy Container #1658
Description
What happened:
I'm encountering an issue while setting up the Azure Blob CSI driver with the install-blobfuse-proxy container.
It seems the installation script fails because it tries to reference a package that does not exist within the container, specifically /blobfuse-proxy/packages-microsoft-prod-18.04.deb
What you expected to happen:
The installation script should complete without errors, and Blobfuse/Blobfuse2 should be installed properly in the container.
How to reproduce it:
- Install csi driver using helm chart using
helm install blob-csi-driver blob-csi-driver/blob-csi-driver --set node.enableBlobfuseProxy=true --set controller.replicas=1 --set linux.kubelet="/var/snap/microk8s/common/var/lib/kubelet" --namespace kube-system --version v1.25.0
- Check for logs for install-blobfuse-proxy container
kubectl logs pod/csi-blob-node-k6fvw -n kube-system -c install-blobfuse-proxy
delphix@vk-hs:~/hyperscale-poc$ kubectl logs pod/csi-blob-node-k6fvw -n kube-system -c install-blobfuse-proxy
+ INSTALL_BLOBFUSE_PROXY=true
+ DISABLE_UPDATEDB=true
+ SET_MAX_OPEN_FILE_NUM=true
+ SET_READ_AHEAD_SIZE=true
+ READ_AHEAD_KB=15380
+ KUBELET_PATH=/var/snap/microk8s/common/var/lib/kubelet
+ [ /var/snap/microk8s/common/var/lib/kubelet != /var/lib/kubelet ]
+ echo kubelet path is /var/snap/microk8s/common/var/lib/kubelet, update blobfuse-proxy.service....
+ sed -i s#--blobfuse-proxy-endpoint[^ ]*#--blobfuse-proxy-endpoint=unix://var/snap/microk8s/common/var/lib/kubelet/plugins/blob.csi.azure.com/blobfuse-proxy.sock# /blobfuse-proxy/blobfuse-proxy.service
kubelet path is /var/snap/microk8s/common/var/lib/kubelet, update blobfuse-proxy.service....
+ echo blobfuse-proxy endpoint is updated to unix://var/snap/microk8s/common/var/lib/kubelet/plugins/blob.csi.azure.com/blobfuse-proxy.sock
+ HOST_CMD=nsenter --mount=/proc/1/ns/mnt
blobfuse-proxy endpoint is updated to unix://var/snap/microk8s/common/var/lib/kubelet/plugins/blob.csi.azure.com/blobfuse-proxy.sock
+ nsenter --mount=/proc/1/ns/mnt cat /etc/os-release
+ grep ^ID=
+ + cut -d= -f2
tr -d "
+ DISTRIBUTION=ubuntu
+ nsenter --mount=/proc/1/ns/mnt uname -m
Linux distribution: ubuntu, Arch: x86_64
+ ARCH=x86_64
+ echo Linux distribution: ubuntu, Arch: x86_64
+ . ./blobfuse-proxy/install-proxy.sh
+ set -xe
+ [ ubuntu != ubuntu ]
+ [ true = true ]
+ echo start to install blobfuse/blobfuse2....
start to install blobfuse/blobfuse2....
+ nsenter --mount=/proc/1/ns/mnt lsb_release -rs
Ubuntu release: 20.04
+ release=20.04
+ echo Ubuntu release: 20.04
+ expr 20.04 < 22.04
+ [ 1 -eq 1 ]
+ cp /blobfuse-proxy/packages-microsoft-prod-18.04.deb /host/etc/packages-microsoft-prod.deb
cp: cannot stat '/blobfuse-proxy/packages-microsoft-prod-18.04.deb': No such file or directory
Anything else we need to know?:
The same error also occurred when on setting '--set node.enableBlobfuseProxy=false'.
However, after uninstalling the current veriion, installing with an older version seemed to work
helm install blob-csi-driver blob-csi-driver/blob-csi-driver --set node.enableBlobfuseProxy=true --set linux.kubelet="/var/snap/microk8s/common/var/lib/kubelet" --set controller.replicas=1 --namespace kube-system --version v1.24.3
On checking the events of the container:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 28s default-scheduler Successfully assigned kube-system/csi-blob-node-k6fvw to vk-hs
Normal Created 16s (x2 over 24s) kubelet Created container install-blobfuse-proxy
Normal Started 16s (x2 over 24s) kubelet Started container install-blobfuse-proxy
Warning BackOff 14s (x2 over 15s) kubelet Back-off restarting failed container install-blobfuse-proxy in pod csi-blob-node-k6fvw_kube-system(d3e83bd2-bb18-411c-9d26-d596f63814f4)
Warning DNSConfigForming 3s (x6 over 28s) kubelet Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.16.101.11 172.16.105.2 8.8.8.8
Normal Pulled 3s (x3 over 28s) kubelet Container image "mcr.microsoft.com/oss/kubernetes-csi/blob-csi:v1.25.0" already present on machine
Environment:
CSI Driver version: v1.25.0
Kubernetes version (use kubectl version): v1.27.4
OS (e.g. from /etc/os-release): "Ubuntu", (20.04.6 LTS) (Focal Fossa)"
Kernel (e.g. uname -a): Linux vk-azure 5.15.0-1073-dx2024092516-650d167a5-azure #82 SMP Wed Sep 25 16:38:55 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Install tools: snap (for microk8 installation)