fix: eip race condition when updating ASG

Add a second EIP, when we decide to use EIP for the manager. This way, when we update the ASG, a free EIP is available to pick by the `aws-ec2-assign-elastic-ip` tool.

This fix the rolling update, and allow for a smoother transition (instead of trying to work around the removing of the EIP association).

The caveats is that we require two EIP for this use case, but I guess it's okay, as long the user is aware of it.

Author: blackillzone

main.tf

@@ -359,7 +359,7 @@ resource "aws_iam_instance_profile" "instance" {
 
 resource "aws_eip" "gitlab_runner" {
   # checkov:skip=CKV2_AWS_19:We can't use NAT gateway here as we are contacted from the outside.
-  count = var.runner_instance.use_eip ? 1 : 0
+  count = var.runner_instance.use_eip ? 2 : 0
 
   tags = local.tags
 }