Skip to content

ci: Add Binary Signing Task #3649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 52 commits into
base: master
Choose a base branch
from
Open

Conversation

sheylatrudo
Copy link
Contributor

@sheylatrudo sheylatrudo commented May 10, 2025

Reason for Change:
This PR enables us to build signed binaries.
Scripts part out the builder portions of the dockerfiles to allow us to sign the binaries that are internal to the images.
The packages are then added to the image during build time. This results in signed binaries as an integrated part of our built images.
Also updated the build time image name to switch the docker image path.

Issue Fixed:

Requirements:

Notes:

@sheylatrudo
Copy link
Contributor Author

/acn run ACN Test Unofficial

@sheylatrudo sheylatrudo changed the title Smtrudo/binary signing ci: Add Binary Signing Task May 12, 2025
@sheylatrudo sheylatrudo marked this pull request as ready for review May 12, 2025 21:31
@Copilot Copilot AI review requested due to automatic review settings May 12, 2025 21:31
@sheylatrudo sheylatrudo requested a review from a team as a code owner May 12, 2025 21:31
@sheylatrudo sheylatrudo requested a review from rohitams May 12, 2025 21:31
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a CI task to sign build binaries across multiple platforms.

  • Adds a new pipeline job block in run-pipeline.yaml for binary signing with OS-specific details.
  • Extends binaries.jobs.yaml with a conditional branch to execute signing when the "official" parameter is true.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
.pipelines/run-pipeline.yaml Adds new signing jobs for Linux/AMD64, Windows/AMD64, and Linux/ARM64 using a shared template.
.pipelines/build/binaries.jobs.yaml Introduces a conditional branch that executes signing tasks when the official flag is set.

@@ -43,3 +43,30 @@ jobs:
target: $(name)
os: $(OS)
arch: $(ARCH)


Copy link
Preview

Copilot AI May 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Please add a comment explaining the conditional branch logic for triggering the signing tasks when the 'official' parameter is true.

Suggested change
# Trigger signing tasks only when the action is 'sign' and the 'official' parameter is true.

Copilot uses AI. Check for mistakes.

Comment on lines 344 to 352
azure_ipam:
artifact: azure-ipam
cni:
artifact: cni
cns:
artifact: cns
ipv6_hp_bpf:
artifact: ipv6-hp-bpf
npm:
Copy link
Preview

Copilot AI May 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider adding inline comments in the matrix section to clarify the purpose of each artifact mapping, which would improve the readability and long-term maintenance of this configuration.

Suggested change
azure_ipam:
artifact: azure-ipam
cni:
artifact: cni
cns:
artifact: cns
ipv6_hp_bpf:
artifact: ipv6-hp-bpf
npm:
azure_ipam: # Azure IPAM (IP Address Management) component
artifact: azure-ipam
cni: # Container Network Interface plugin
artifact: cni
cns: # Container Networking Service
artifact: cns
ipv6_hp_bpf: # IPv6 High-Performance BPF program
artifact: ipv6-hp-bpf
npm: # Network Policy Manager

Copilot uses AI. Check for mistakes.

@sheylatrudo
Copy link
Contributor Author

/azp run ACN Test Unofficial

Copy link

No pipelines are associated with this pull request.

@sheylatrudo
Copy link
Contributor Author

/azp run Azure Container Networking PR

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Contributor

@jpayne3506 jpayne3506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the Dockerfiles in /build/dockerfiles/* there to give us reproducibility? I thought we had a script that was generating these per run.

@@ -43,3 +43,37 @@ jobs:
target: $(name)
os: $(OS)
arch: $(ARCH)


- ${{ elseif and(eq(job_data.templateContext.action, 'sign'), job_data.templateContext.isOfficial) }}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this template get called multiple times? Any reason why wouldn't we want to run this job on every run that generates binaries?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand the conditional differences between lines 7,8 and 48

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signing is really only necessary for official builds. Have 2 different actions allows me to separate the two jobs, but also ensure that the dependencies are populated as needed.



jobs:
- ${{ each job_data in parameters.images }}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What unique information do we need to have to call this template correctly? Technically every single line here is a parameter

- job: linux_amd64
displayName: "Linux/AMD64"
templateContext:
repositoryArtifact: drop_setup_env_source
pkgArtifact: drop_build_pkg_$(os)_$(arch)_$(name)
buildScript: .pipelines/build/scripts/$(name).sh
obDockerfile: .pipelines/build/dockerfiles/$(name).Dockerfile
strategy:
maxParallel: 5
matrix:
azure_ipam:
name: azure-ipam
extraArgs: ''
archiveName: azure-ipam
archiveVersion: $(AZURE_IPAM_VERSION)
imageTag: $(Build.BuildNumber)
packageWithDropGZ: True
cni:
name: cni
extraArgs: '--build-arg CNI_AI_PATH=$(CNI_AI_PATH) --build-arg CNI_AI_ID=$(CNI_AI_ID)'
archiveName: azure-cni
archiveVersion: $(CNI_VERSION)
imageTag: $(Build.BuildNumber)
packageWithDropGZ: True
cns:
name: cns
extraArgs: '--build-arg CNS_AI_PATH=$(CNS_AI_PATH) --build-arg CNS_AI_ID=$(CNS_AI_ID)'
archiveName: azure-cns
archiveVersion: $(CNS_VERSION)
imageTag: $(Build.BuildNumber)
ipv6_hp_bpf:
name: ipv6-hp-bpf
extraArgs: "--build-arg DEBUG=$(System.Debug)"
archiveName: ipv6-hp-bpf
archiveVersion: $(IPV6_HP_BPF_VERSION)
imageTag: $(Build.BuildNumber)
npm:
name: npm
extraArgs: '--build-arg NPM_AI_PATH=$(NPM_AI_PATH) --build-arg NPM_AI_ID=$(NPM_AI_ID)'
archiveName: azure-npm
archiveVersion: $(NPM_VERSION)
imageTag: $(Build.BuildNumber)
- job: windows_amd64
displayName: "Windows"
templateContext:
repositoryArtifact: drop_setup_env_source
pkgArtifact: drop_build_pkg_$(os)_$(arch)_$(name)
buildScript: .pipelines/build/scripts/$(name).sh
obDockerfile: .pipelines/build/dockerfiles/$(name).Dockerfile
strategy:
maxParallel: 5
matrix:
azure_ipam:
name: azure-ipam
extraArgs: ''
archiveName: azure-ipam
archiveVersion: $(OS)-$(ARCH)-$(AZURE_IPAM_VERSION)
imageTag: $(Build.BuildNumber)
packageWithDropGZ: True
cni:
name: cni
extraArgs: '--build-arg CNI_AI_PATH=$(CNI_AI_PATH) --build-arg CNI_AI_ID=$(CNI_AI_ID)'
archiveName: azure-cni
archiveVersion: $(CNI_VERSION)
imageTag: $(Build.BuildNumber)
packageWithDropGZ: True
cns:
name: cns
extraArgs: '--build-arg CNS_AI_PATH=$(CNS_AI_PATH) --build-arg CNS_AI_ID=$(CNS_AI_ID)'
archiveName: azure-cns
archiveVersion: $(CNS_VERSION)
imageTag: $(Build.BuildNumber)
npm:
name: npm
extraArgs: '--build-arg NPM_AI_PATH=$(NPM_AI_PATH) --build-arg NPM_AI_ID=$(NPM_AI_ID)'
archiveName: azure-npm
archiveVersion: $(NPM_VERSION)
imageTag: $(Build.BuildNumber)
- job: linux_arm64
displayName: "Linux/ARM64"
templateContext:
repositoryArtifact: drop_setup_env_source
pkgArtifact: drop_build_pkg_$(os)_$(arch)_$(name)
buildScript: .pipelines/build/scripts/$(name).sh
obDockerfile: .pipelines/build/dockerfiles/$(name).Dockerfile
strategy:
maxParallel: 3
matrix:
azure_ipam:
name: azure-ipam
archiveName: azure-ipam
archiveVersion: $(AZURE_IPAM_VERSION)
extraArgs: ''
imageTag: $(Build.BuildNumber)
packageWithDropGZ: True
cni:
name: cni
extraArgs: '--build-arg CNI_AI_PATH=$(CNI_AI_PATH) --build-arg CNI_AI_ID=$(CNI_AI_ID)'
archiveName: azure-cni
archiveVersion: $(CNI_VERSION)
imageTag: $(Build.BuildNumber)
packageWithDropGZ: True
cns:
name: cns
extraArgs: '--build-arg CNS_AI_PATH=$(CNS_AI_PATH) --build-arg CNS_AI_ID=$(CNS_AI_ID)'
archiveName: azure-cns
archiveVersion: $(CNS_VERSION)
imageTag: $(Build.BuildNumber)
#ipv6_hp_bpf:
# name: ipv6-hp-bpf
# extraArgs: "--build-arg DEBUG=$(System.Debug)"
# archiveName: ipv6-hp-bpf
# archiveVersion: $(IPV6_HP_BPF_VERSION)
# imageTag: $(Build.BuildNumber)
npm:
name: npm
extraArgs: '--build-arg NPM_AI_PATH=$(NPM_AI_PATH) --build-arg NPM_AI_ID=$(NPM_AI_ID)'
archiveName: azure-npm
archiveVersion: $(NPM_VERSION)
imageTag: $(Build.BuildNumber)

As a nit, I feel like this could have been pulled up one level

jobs:
- template: /.pipelines/build/images.jobs.yaml
parameters:
images:

and iterated.

Instead we are passing a normal job.strategy.matrix into a jobList and parsing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to focus on the configuration for someone looking to add/change basic stuff for each image. The items in the templateContext are variables that are like additional information that isn't expected to change in format. The only unique information needed is what is added to the matrix.

@@ -51,7 +51,7 @@ steps:
echo "##vso[task.setvariable variable=Tag;isOutput=true]$TAG"
echo "Tag: $TAG"

IMAGEREPOPATH="artifact/dd590928-4e04-48cb-9d3d-ee06c5f0e17f/buddy"
IMAGEREPOPATH="artifact/dd590928-4e04-48cb-9d3d-ee06c5f0e17f/$BUILD_TYPE"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: ACR cleanup task for both repo paths

Comment on lines +12 to +19
CNI_NET_DIR="$REPO_ROOT"/cni/network/plugin
pushd "$CNI_NET_DIR"
go build -v -a -trimpath \
-o "$OUT_DIR"/bin/azure-vnet"$FILE_EXT" \
-ldflags "-X main.version="$CNI_VERSION"" \
-gcflags="-dwarflocationlists=true" \
./main.go
popd
Copy link
Contributor

@jpayne3506 jpayne3506 May 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we continuously using pushd and popd? Can we just use it once for ACN root and then call the respective make commands or just azure-cni-plugin?

If not, why not just use relative pathing as we do in the Dockerfile?

WORKDIR /azure-container-networking
COPY . .
RUN GOOS=$OS CGO_ENABLED=0 go build -a -o /go/bin/azure-vnet -trimpath -ldflags "-X main.version="$VERSION"" -gcflags="-dwarflocationlists=true" cni/network/plugin/main.go
RUN GOOS=$OS CGO_ENABLED=0 go build -a -o /go/bin/azure-vnet-telemetry -trimpath -ldflags "-X main.version="$VERSION" -X "$CNI_AI_PATH"="$CNI_AI_ID"" -gcflags="-dwarflocationlists=true" cni/telemetry/service/telemetrymain.go
RUN GOOS=$OS CGO_ENABLED=0 go build -a -o /go/bin/azure-vnet-ipam -trimpath -ldflags "-X main.version="$VERSION"" -gcflags="-dwarflocationlists=true" cni/ipam/plugin/main.go
RUN GOOS=$OS CGO_ENABLED=0 go build -a -o /go/bin/azure-vnet-stateless -trimpath -ldflags "-X main.version="$VERSION"" -gcflags="-dwarflocationlists=true" cni/network/stateless/main.go

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic goes to all build/scripts/*

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relative pathing can be flaky and some utilities ask for non-relative paths. Absolute paths are much easier to debug in logs. They are also less subject to issues when repo paths change on a higher level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants