EKS kubelet-extra-args are not set when using managedmachinepool

[felipeweb]

/kind bug

What steps did you take and what happened:

apiVersion: exp.cluster.x-k8s.io/v1alpha3
kind: MachinePool
metadata:
  name: undistro-quickstart-mp-0
  namespace: default
spec:
  clusterName: undistro-quickstart
  minReadySeconds: 0
  replicas: 1
  template:
    metadata: {}
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: EKSConfig
          name: undistro-quickstart-mp-033dcec9-adbf-4e6d-b493-6db22e2d0e30-0
          namespace: default
      clusterName: undistro-quickstart
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
        kind: AWSManagedMachinePool
        name: undistro-quickstart-mp-0
        namespace: default
      version: v1.18.9
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
kind: AWSManagedMachinePool
metadata:
  name: undistro-quickstart-mp-0
  namespace: default
spec:
  additionalTags:
    infra-node: "true"
  amiType: AL2_x86_64
  eksNodegroupName: undistro-quickstart-mp-0
  instanceType: t3.medium
  roleName: nodes.cluster-api-provider-aws.sigs.k8s.io
  scaling:
    maxSize: 5
    minSize: 1
---
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
kind: EKSConfig
metadata:
  name: undistro-quickstart-mp-033dcec9-adbf-4e6d-b493-6db22e2d0e30-0
  namespace: default:
spec:
  kubeletExtraArgs:
    node-labels: node-role.undistro.io/infra=true
    register-with-taints: dedicated=infra:NoSchedule

is using wrong userdata

/etc/eks/bootstrap.sh default_undistro-quickstart --kubelet-extra-args '--node-labels=eks.amazonaws.com/nodegroup-image=ami-0dd0589ee7a07c236,eks.amazonaws.com/capacityType=ON_DEMAND,eks.amazonaws.com/nodegroup=undistro-quickstart-mp-0' --b64-cluster-ca $B64_CLUSTER_CA --apiserver-endpoint $API_SERVER_URL --dns-cluster-ip $K8S_CLUSTER_DNS_IP

What did you expect to happen:

/etc/eks/bootstrap.sh default_undistro-quickstart --kubelet-extra-args '--node-labels=node-role.undistro.io/infra=true,eks.amazonaws.com/nodegroup-image=ami-0dd0589ee7a07c236,eks.amazonaws.com/capacityType=ON_DEMAND,eks.amazonaws.com/nodegroup=undistro-quickstart-mp-0 --register-with-taints=dedicated=infra:NoSchedule' --b64-cluster-ca $B64_CLUSTER_CA --apiserver-endpoint $API_SERVER_URL --dns-cluster-ip $K8S_CLUSTER_DNS_IP

Anything else you would like to add:

Environment:

• Cluster-api-provider-aws version: v0.6.4
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[richardcase]

/area provider/eks
/help

[k8s-ci-robot]

@richardcase:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/area provider/eks
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[felipeweb]

@richardcase the problem is that the machinepool is not converting template object into a concrete object. I sent a PR to CAPI, but it seems to do not work with concrete objects too. I will make a more deep investigation

[felipeweb]

We are using the default launch template for eks, I am assuming this because we are not passing any launch template

cluster-api-provider-aws/pkg/cloud/services/eks/nodegroup.go

Lines 162 to 171 in b7294ce

	input := &eks.CreateNodegroupInput{
	ScalingConfig: s.scalingConfig(),
	ClusterName: aws.String(eksClusterName),
	NodegroupName: aws.String(nodegroupName),
	Subnets: aws.StringSlice(s.subnets()),
	NodeRole: roleArn,
	Labels: aws.StringMap(managedPool.Labels),
	Tags: aws.StringMap(tags),
	RemoteAccess: remoteAccess,
	}

There are any reasons for that?

According to AWS Console the userdata is configured into launch template so we need to reconcile and pass a launch template in node group creation.

@richardcase could you give me the basic paths about where start to fix this? I'm newbie into capa codebase.

I am assuming the starting point is here

cluster-api-provider-aws/exp/controllers/awsmanagedmachinepool_controller.go

Line 190 in 634f30e

if err := ekssvc.ReconcilePool(); err != nil {

[richardcase]

@felipeweb - there isn't a specific reason the launch template support isn't there for AWSManagedMachinePool....it was really to do with timing and us wanting to get initial functionality into the provider (which meant using the default launch template). We have an issue created for adding the launch template support ( #2055)....... @michaelbeaumont are you working on this or is this something @felipeweb can help with?

There is launch template support on the non-managed machine pool (AWSMachinePool) that may be worth looking at......and potentially refactoring so that it can be reused.

[richardcase]

@felipeweb - also feel free to ping me on the slack channel if needed

[michaelbeaumont]

@richardcase I am working on it, I'll have a PR up today. We can gladly collaborate on it! @felipeweb

[richardcase]

Great @michaelbeaumont 👍