Unable to get pod names in webhook notification.

[IvanDechovsky]

As per the PR #388, we should be able to get the Pod names by including the following in the webhook template: Pods: {{ .Pods }}. However, it returns an empty array Pods: [].
Slack notification:

EventID: spot-itn-xxxx
Kind: SPOT_ITN
Description: Spot ITN received. Instance will be interrupted at 2021-05-03T08:30:49Z
Start Time: 2021-05-03 08:30:49 +0000 UTC
Node: ip-10-xx-xx-xx.eu-west-1.compute.internal
Pods: []

This is tested on tag: v1.13.0
Helm values used:

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/os
          operator: In
          values:
          - linux
        - key: kubernetes.io/arch
          operator: In
          values:
          - amd64
        - key: eks.amazonaws.com/compute-type
          operator: NotIn
          values:
          - fargate
deleteLocalData: true
enablePrometheusServer: true
enableScheduledEventDraining: true
enableSpotInterruptionDraining: true
ignoreDaemonSets: true
jsonLogging: true
nodeTerminationGracePeriod: 120
podAnnotations:
  co.elastic.logs/enabled: "true"
podMonitor:
  create: true
podTerminationGracePeriod: 60
priorityClassName: system-node-critical
rbac:
  pspEnabled: true
updateStrategy:
  rollingUpdate:
    maxUnavailable: 100%
  type: RollingUpdate
webhookTemplate: |
  {
    "blocks": [
      {
        "type": "section",
        "text": {
          "type": "mrkdwn",
          "text": ":boom: Instance interruption on cluster :boom:"
        }
      },
      {
        "type": "divider"
      },
      {
        "type": "context",
        "elements": [
          {
            "type": "mrkdwn",
            "text": "*EventID:* {{ .EventID }}\n*Kind:* {{ .Kind }}\n*Description:* {{ .Description }}\n*Start Time:* {{ .StartTime }}\n*Node:* {{ .NodeName }}\n*Pods:* {{ .Pods }}"
          }
        ]
      }
    ]
  }
webhookURL: xxx

[imuqtadir]

Let me check and get back to you on this.

[imuqtadir]

This PR #411 undo some of the changes which is why you are not able to see any Pods anymore. I will get it fixed.

[vermaabhay]

I am still not able to get pods list in the webhook notification.

This is tested against main branch.

Logs -

2021/06/02 06:44:12 INF Adding new event to the event store event={"AutoScalingGroupName":"","Description":"Spot ITN received. Instance will be interrupted at 2021-06-02T06:46:10Z \n","EndTime":"0001-01-01T00:00:00Z","EventID":"spot-itn-6705023f947c4d5b409435ac29ba4f683235a53be5f632f3b351e15648178113","InProgress":false,"InstanceID":"","Kind":"SPOT_ITN","NodeLabels":null,"NodeName":"ip-10-101-5-66.ap-southeast-1.compute.internal","NodeProcessed":false,"Pods":null,"StartTime":"2021-06-02T06:46:10Z","State":""}
2021/06/02 06:44:12 INF Error when trying to list Nodes w/ label, falling back to direct Get lookup of node
2021/06/02 06:44:12 INF Cordoning the node
2021/06/02 06:44:12 INF Error when trying to list Nodes w/ label, falling back to direct Get lookup of node
2021/06/02 06:44:12 INF Draining the node
2021/06/02 06:44:13 INF Error when trying to list Nodes w/ label, falling back to direct Get lookup of node
--

Though evicting pods with pod-name message is present in NTH logs but webhook notification contains an empty list.

[bwagner5]

The problem is that we are only adding pods when cordonOnly is enabled. We should be adding it when for the default Cordon and Drain too. I'll work on making this fix.

[iusergii]

@imuqtadir should it works with v.1.13.1? Still have an empty list there.