fix: allow HTTP egress traffic to all destinations

[jonpas]

Description

Allow HTTP (port 80) traffic by default to allow commonly-used resources to be accessible from the docker-autoscaler runner workers.

Fix #1302.

Migrations required

No

Verification

See #1302 for test cases.

[github-actions]

Hey @jonpas! 👋

Thank you for your contribution to the project. Please refer to the contribution rules for a quick overview of the process.

Make sure that this PR clearly explains:

• the problem being solved
• the best way a reviewer and you can test your changes

With submitting this PR you confirm that you hold the rights of the code added and agree that it will published under this LICENSE.

The following ChatOps commands are supported:

• /help: notifies a maintainer to help you out

Simply add a comment with the command in the first line. If you need to pass more information, separate it with a blank line from the command.

This message was generated automatically. You are welcome to improve it.

[philvarner]

I can't comment on the appropriateness of this for the default configuration, but I had to add this to my deployment to fix an issue with some installation steps that hit http endpoints.

[karthikholla]

The interesting part is that this issue only occurs when the runners are public (i.e., behind an Internet Gateway). In this case, apt-get update with http:// fails.
As a workaround, I had to add the following:
Based on Debian variants

- sed -i 's/http:/https:/g' /etc/apt/sources.list.d/ubuntu.sources  
- echo 'Acquire::https::Verify-Peer "false";' | tee /etc/apt/apt.conf.d/99verify-peer.conf

- sed -i 's/http:/https:/g' /etc/apt/sources.list.d/debian.sources
- echo 'Acquire::https::Verify-Peer "false";' | tee /etc/apt/apt.conf.d/99verify-peer.conf

- sed -i 's/http:/https:/g' /etc/apt/sources.list
- echo 'Acquire::https::Verify-Peer "false";' | tee /etc/apt/apt.conf.d/99verify-peer.conf

However, when the runners are private (i.e., behind a NAT Gateway), it works without any issues.

[jonpas]

@karthikholla you are correct. I completely forgot to mention our runners are public (Internet Gateway) to reduce costs with NAT Gateway.

[jessedobbelaere]

I'm using private runners, though with the following setup with a self-hosted NAT instance (int128/nat-instance/aws) instead of an expensive NAT gateway.

GitLab Runners (private subnet) 
    ↓ (outbound internet traffic)
NAT EC2 Instance (public subnet)
    ↓ (via route table with IGW)
Internet Gateway
    ↓
Internet

Spent a few hours debugging. After updating terraform-aws-gitlab-runner, any apt-get or apk started to fail in my docker builds. Fix was indeed adding port 80 back to the egress security group 👍

Would be nice to merge this PR to avoid headaches for other users 👌

---

Docker build logs output

$ docker buildx bake --progress=plain
#0 building with "laughing_hopper" instance using docker-container driver
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.0s done
#1 creating container buildx_buildkit_laughing_hopper0
#1 creating container buildx_buildkit_laughing_hopper0 0.8s done
#1 DONE 2.8s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile:
#2 transferring dockerfile: 314B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/ubuntu:latest
#3 DONE 0.3s
#4 [internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s
#5 [1/3] FROM docker.io/library/ubuntu:latest@sha256:a08e551cb33850e4740772b38217fc1796a66da2506d312abe51acda354ff061
#5 resolve docker.io/library/ubuntu:latest@sha256:a08e551cb33850e4740772b38217fc1796a66da2506d312abe51acda354ff061 0.0s done
#5 sha256:e3bd89a9dac501ff564b39359113adad7c3d2813d5e04eab53ee10e20a6793a7 12.58MB / 28.86MB 0.2s
#5 sha256:e3bd89a9dac501ff564b39359113adad7c3d2813d5e04eab53ee10e20a6793a7 28.86MB / 28.86MB 0.3s done
#5 extracting sha256:e3bd89a9dac501ff564b39359113adad7c3d2813d5e04eab53ee10e20a6793a7
#5 extracting sha256:e3bd89a9dac501ff564b39359113adad7c3d2813d5e04eab53ee10e20a6793a7 0.7s done
#5 DONE 1.1s
#6 [2/3] RUN apt-get update && apt-get install -y curl gcc build-essential
#6 31.34 Ign:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease
#6 31.34 Ign:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease
#6 31.34 Ign:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease
#6 31.34 Ign:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease
#6 32.34 Ign:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease
#6 32.34 Ign:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease
#6 32.34 Ign:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease
#6 32.34 Ign:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease
#6 34.34 Ign:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease
#6 34.34 Ign:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease
#6 34.34 Ign:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease
#6 34.35 Ign:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease
#6 38.34 Err:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease
#6 38.34   Could not connect to ports.ubuntu.com:80 (91.189.91.103), connection timed out Could not connect to ports.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to ports.ubuntu.com:80 (91.189.91.102), connection timed out Could not connect to ports.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to ports.ubuntu.com:80 (91.189.91.104), connection timed out
#6 38.35 Err:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease
#6 38.35   Unable to connect to ports.ubuntu.com:80:
#6 38.35 Err:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease
#6 38.35   Unable to connect to ports.ubuntu.com:80:
#6 38.35 Err:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease
#6 38.35   Unable to connect to ports.ubuntu.com:80:
#6 38.35 Reading package lists...
#6 38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble/InRelease  Could not connect to ports.ubuntu.com:80 (91.189.91.103), connection timed out Could not connect to ports.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to ports.ubuntu.com:80 (91.189.91.102), connection timed out Could not connect to ports.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to ports.ubuntu.com:80 (91.189.91.104), connection timed out
#6 38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble-updates/InRelease  Unable to connect to ports.ubuntu.com:80:
#6 38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble-backports/InRelease  Unable to connect to ports.ubuntu.com:80:
#6 38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble-security/InRelease  Unable to connect to ports.ubuntu.com:80:
#6 38.36 W: Some index files failed to download. They have been ignored, or old ones used instead.
#6 38.37 Reading package lists...
#6 38.37 Building dependency tree...
#6 38.38 Reading state information...
#6 38.38 E: Unable to locate package curl
#6 38.38 E: Unable to locate package gcc
#6 38.38 E: Unable to locate package build-essential
#6 ERROR: process "/bin/sh -c apt-get update && apt-get install -y curl gcc build-essential" did not complete successfully: exit code: 100
------
 > [2/3] RUN apt-get update && apt-get install -y curl gcc build-essential:
38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble-updates/InRelease  Unable to connect to ports.ubuntu.com:80:
38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble-backports/InRelease  Unable to connect to ports.ubuntu.com:80:
38.36 W: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/noble-security/InRelease  Unable to connect to ports.ubuntu.com:80:
38.36 W: Some index files failed to download. They have been ignored, or old ones used instead.
38.37 Reading package lists...
38.37 Building dependency tree...
38.38 Reading state information...
38.38 E: Unable to locate package curl
38.38 E: Unable to locate package gcc
38.38 E: Unable to locate package build-essential
------
WARNING: buildx: git was not found in the system. Current commit information was not captured by the build
Dockerfile:2
--------------------
   1 |     FROM ubuntu:latest AS cavif
   2 | >>> RUN apt-get update && apt-get install -y curl gcc build-essential
   3 |     SHELL ["/bin/bash", "-o", "pipefail", "-c"]
   4 |     RUN curl https://sh.rustup.rs -sSf | sh -s -- -y && \
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install -y curl gcc build-essential" did not complete successfully: exit code: 100

[kayman-mk]

Thanks for pointing this out.