Skip to content

Kicbase/ISO: update crio to 1.32 #20630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Kicbase/ISO: update crio to 1.32 #20630

wants to merge 13 commits into from

Conversation

ComradeProgrammer
Copy link
Member

@ComradeProgrammer ComradeProgrammer commented Apr 15, 2025 •
edited
Loading

FIX #20608

  • manual update crio in kicbase
  • manual update crio in iso
  • modify update automation for crictl and crio

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 15, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ComradeProgrammer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 15, 2025
@ComradeProgrammer
Copy link
Member Author

/ok-to-build-image

@ComradeProgrammer ComradeProgrammer removed the request for review from spowelljr April 15, 2025 22:28
@ComradeProgrammer
Copy link
Member Author

/ok-to-build-image

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 18, 2025
@ComradeProgrammer
Copy link
Member Author

/ok-to-build-iso

@minikube-bot
Copy link
Collaborator

Hi @ComradeProgrammer, we have updated your PR with the reference to newly built kicbase image. Pull the changes locally if you want to test with them or update your PR further.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 18, 2025
@ComradeProgrammer ComradeProgrammer changed the title WIP: Kicbase/ISO: update crio to 1.32 Kicbase/ISO: update crio to 1.32 Apr 18, 2025
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 18, 2025
@ComradeProgrammer
Copy link
Member Author

/ok-to-build-iso

@ComradeProgrammer
Copy link
Member Author

/ok-to-build-image

1 similar comment
@ComradeProgrammer
Copy link
Member Author

/ok-to-build-image

@minikube-bot
Copy link
Collaborator

Hi @ComradeProgrammer, we have updated your PR with the reference to newly built kicbase image. Pull the changes locally if you want to test with them or update your PR further.

@kubernetes kubernetes deleted a comment from minikube-bot Apr 29, 2025
@ComradeProgrammer
Copy link
Member Author

/ok-to-build-image

@ComradeProgrammer
Copy link
Member Author

/ok-to-build-iso

@minikube-bot
Copy link
Collaborator

Hi @ComradeProgrammer, we have updated your PR with the reference to newly built kicbase image. Pull the changes locally if you want to test with them or update your PR further.

@minikube-bot
Copy link
Collaborator

Hi @ComradeProgrammer, we have updated your PR with the reference to newly built ISO. Pull the changes locally if you want to test with them or update your PR further.

@ComradeProgrammer
Copy link
Member Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Apr 30, 2025
@minikube-pr-bot

This comment has been minimized.

Sign in to view
@medyagh medyagh mentioned this pull request Apr 30, 2025
Open
@ComradeProgrammer
Copy link
Member Author

ComradeProgrammer commented May 6, 2025 •
edited
Loading

I am looking at the CI failures

  • For kicbase I forgot to install crio-runc
  • For KVM2_crio, I also need to update conmon path
  • Still don't know why in KVM_Linux why crictl cannot talk with crio-dockerd
    I will try to fix the first two problems...

@ComradeProgrammer
Copy link
Member Author

/ok-to-test
/ok-to-build-iso
/ok-to-build-image

@minikube-pr-bot

This comment has been minimized.

Sign in to view
@ComradeProgrammer
Copy link
Member Author

I have run the test locally and found the reason why tests fails on kvm
It seems that somehow now cri-dockerd will be started before docker daemon is started, and then it fails.
related log from journalctl -u cri-docker.service

May 06 23:01:24 minikube systemd[1]: Starting CRI Interface for Docker Application Container Engine...
May 06 23:01:24 minikube cri-dockerd[399]: time="2025-05-06T23:01:24Z" level=info msg="Starting cri-dockerd 0.3.15 (c1c566e)"
May 06 23:01:24 minikube cri-dockerd[399]: time="2025-05-06T23:01:24Z" level=info msg="Connecting to docker on the Endpoint unix:///var/run/docker.sock"
May 06 23:01:24 minikube cri-dockerd[399]: time="2025-05-06T23:01:24Z" level=info msg="Start docker client with request timeout 0s"
May 06 23:01:24 minikube cri-dockerd[399]: time="2025-05-06T23:01:24Z" level=fatal msg="failed to get docker version from dockerd: Cannot connect to the Docker daemon at unix:///var/run/
May 06 23:01:24 minikube systemd[1]: cri-docker.service: Main process exited, code=exited, status=1/FAILURE
May 06 23:01:24 minikube systemd[1]: :5:185m:5:185mcri-docker.service: Failed with result 'exit-code'.
May 06 23:01:24 minikube systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.
May 06 23:01:26 minikube systemd[1]: cri-docker.service: Scheduled restart job, restart counter is at 1.
May 06 23:01:26 minikube systemd[1]: Stopped CRI Interface for Docker Application Container Engine.
May 06 23:01:26 minikube systemd[1]: Starting CRI Interface for Docker Application Container Engine...
May 06 23:01:26 minikube cri-dockerd[453]: time="2025-05-06T23:01:26Z" level=info msg="Starting cri-dockerd 0.3.15 (c1c566e)"
May 06 23:01:26 minikube cri-dockerd[453]: time="2025-05-06T23:01:26Z" level=info msg="Connecting to docker on the Endpoint unix:///var/run/docker.sock"
May 06 23:01:26 minikube cri-dockerd[453]: time="2025-05-06T23:01:26Z" level=info msg="Start docker client with request timeout 0s"
May 06 23:01:26 minikube cri-dockerd[453]: time="2025-05-06T23:01:26Z" level=fatal msg="failed to get docker version from dockerd: Cannot connect to the Docker daemon at unix:///var/run/
May 06 23:01:26 minikube systemd[1]: cri-docker.service: Main process exited, code=exited, status=1/FAILURE
May 06 23:01:26 minikube systemd[1]: :5:185m:5:185mcri-docker.service: Failed with result 'exit-code'.
May 06 23:01:26 minikube systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.
May 06 23:01:29 minikube systemd[1]: cri-docker.service: Scheduled restart job, restart counter is at 2.
May 06 23:01:29 minikube systemd[1]: Stopped CRI Interface for Docker Application Container Engine.
May 06 23:01:29 minikube systemd[1]: Starting CRI Interface for Docker Application Container Engine...
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Starting cri-dockerd 0.3.15 (c1c566e)"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Connecting to docker on the Endpoint unix:///var/run/docker.sock"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Start docker client with request timeout 0s"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Hairpin mode is set to none"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Loaded network plugin cni"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Docker cri networking managed by network plugin cni"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Setting cgroupDriver systemd"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Docker cri received runtime config &RuntimeConfig{NetworkConfig:&NetworkConfig{PodCidr:,},}"
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Starting the GRPC backend for the Docker CRI interface."
May 06 23:01:29 minikube cri-dockerd[804]: time="2025-05-06T23:01:29Z" level=info msg="Start cri-dockerd grpc backend"
May 06 23:01:29 minikube systemd[1]: Started CRI Interface for Docker Application Container Engine.
May 06 23:01:30 minikube systemd[1]: Stopping CRI Interface for Docker Application Container Engine...
May 06 23:01:30 minikube systemd[1]: cri-docker.service: Deactivated successfully.
May 06 23:01:30 minikube systemd[1]: Stopped CRI Interface for Docker Application Container Engine.
May 06 23:01:31 minikube systemd[1]: :5:185m:5:185mcri-docker.service: Start request repeated too quickly.
May 06 23:01:31 minikube systemd[1]: :5:185m:5:185mcri-docker.service: Failed with result 'start-limit-hit'.
May 06 23:01:31 minikube systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.

Obviously when cri-dockerd tries to start, the docker daemon is not ready at all. Simply restart the cri-dockerd can resolve all problems.

I am still thinking about how to solve this, and why this is caused in this PR

@ComradeProgrammer
Copy link
Member Author

/ok-to-build-iso

@ComradeProgrammer
Copy link
Member Author

/ok-to-build-image

@ComradeProgrammer
Copy link
Member Author

/ok-to-test

@minikube-pr-bot

This comment has been minimized.

Sign in to view
@minikube-bot
Copy link
Collaborator

Hi @ComradeProgrammer, we have updated your PR with the reference to newly built kicbase image. Pull the changes locally if you want to test with them or update your PR further.

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime
error downloading artifacts: artifact download start: exit status 90docker driver with docker runtime
error downloading artifacts: artifact download start: exit status 81docker driver with containerd runtime
error downloading artifacts: artifact download start: exit status 81

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2025
@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ComradeProgrammer ComradeProgrammer mentioned this pull request May 10, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@medyagh medyagh Awaiting requested review from medyagh

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Bump Crio to minikube's Default Kubernetes version
4 participants
@ComradeProgrammer @k8s-ci-robot @minikube-bot @minikube-pr-bot