Document the metadata endpoints/paths used

[leosunmo]

First of all, thanks for this project! It's a must for anyone running Kubernetes on AWS with Spot instances!

Now, I use Kiam to handle AWS IAM credentials in the cluster, and as part of its operation it redirects/blocks access to the metadata endpoint on all nodes. It does have the ability to pass through a specified list of endpoints/paths. So what I need to know is the exact list of endpoints/paths that the NTH uses to make sure I add them to the Kiam access list. I can see some endpoints in the code but am hoping for a definite list from a maintainer.

I can see these constants in https://github.com/aws/aws-node-termination-handler/blob/master/pkg/ec2metadata/ec2metadata.go#L29 being used.
http://169.254.169.254/latest/meta-data/spot/instance-action
http://169.254.169.254/latest/meta-data/events/maintenance/scheduled
http://169.254.169.254/latest/meta-data/instance-id
http://169.254.169.254/latest/meta-data/instance-type
http://169.254.169.254/latest/meta-data/public-hostname
http://169.254.169.254/latest/meta-data/public-ipv4
http://169.254.169.254/latest/meta-data/local-hostname
http://169.254.169.254/latest/meta-data/local-ipv4

If someone can verify this list I will try to add it to the documentation, including how to add pass-through in Kiam (it's a fairly popular solution for AWS IAM).

Thanks!

[bwagner5]

That list is confirmed as the only metadata paths NTH queries! ✅

Adding documentation on how to setup NTH with Kiam would be awesome! Thanks!!

[leosunmo]

Excellent, thanks for the quick reply!
I will work on the Kiam bit and then send through a documentation PR.

[leosunmo]

Okay, so looking at https://github.com/uswitch/kiam/blob/6c1999c40337a7c5d8c02880f756d5ada5e7a569/pkg/aws/metadata/handler_proxy.go#L47 we should be able to verify that this will indeed work.

Regexp slapped together here: https://regex101.com/r/ctbD5m/2 (please let me know if there's a nicer way to do this, I didn't want to resort to any wildcards for security reasons).

Go playground verifying it (hopefully using the same logic as Kiam, please double check the link above) here: https://goplay.space/#GoCn67SAiMj

Final config for KIAM agents:
--whitelist-route-regexp="^\/latest\/meta-data\/(spot\/instance-action|events\/maintenance\/scheduled|instance-(id|type)|public-(hostname|ipv4)|local-(hostname|ipv4))$"

A bit of a mouthful, but should be set-and-forget as long as the requirements of NTH doesn't change.

[leosunmo]

I'm going to deploy this config in my own cluster first to verify then I will add documentation.

[leosunmo]

Done. Works well in my staging cluster. Tested two nodes, one with the new Kiam config and one with the old. The vital endpoints are blocked on the old config, works fine on the new.

[bwagner5]

thanks for the docs and testing!