-
Notifications
You must be signed in to change notification settings - Fork 274
Closed
Labels
staleIssues / PRs with no activityIssues / PRs with no activity
Description
Describe the bug
When NTH receives a spot interruption notice and logs the interruption time, it logs the current time rather than the actual interruption time.
See also #606
Steps to reproduce
- Set up NTH in Deployment/SQS mode
- Configure EventBridge to send spot termination notices
- Wait for a spot termination notice
- Observe log with "Spot Interruption event received"
Expected outcome
Spot interruption time should be roughly +2 minutes from when the event was received. Ideally we can use the actual interruption time as supplied by AWS rather than guestimating it though.
Application Logs
2023-01-23T00:18:31.086824792Z aws-node-termination-handler-6574867b74-z7l57 {"level":"info","event":{"EventID":"spot-itn-event-62363231313539382d666634352d343631612d646537642d346232353565353832386463","Kind":"SPOT_ITN","Monitor":"SQS_MONITOR","Description":"Spot Interruption event received. Instance i-099447d7bc6f9dde7 will be interrupted at 2023-01-23 00:18:29 +0000 UTC \n","State":"","AutoScalingGroupName":"eks-main-20220713005232834100000009","NodeName":"ip-10-9-139-214.ec2.internal","NodeLabels":null,"Pods":null,"InstanceID":"i-099447d7bc6f9dde7","ProviderID":"aws:///us-east-1b/i-099447d7bc6f9dde7","IsManaged":true,"StartTime":"2023-01-23T00:18:29Z","EndTime":"0001-01-01T00:00:00Z","NodeProcessed":false,"InProgress":false},"time":"2023-01-23T00:18:31Z","message":"Adding new event to the event store"}
2023-01-23T00:20:18.474299920Z aws-node-termination-handler-6574867b74-z7l57 {"level":"info","node_name":"ip-10-9-139-214.ec2.internal","reason":"Spot Interruption event received. Instance i-099447d7bc6f9dde7 will be interrupted at 2023-01-23 00:18:29 +0000 UTC \n","time":"2023-01-23T00:20:18Z","message":"Node successfully cordoned and drained"}
Environment
- NTH App Version: v1.18.2 (Helm chart version 0.18.5)
- NTH Mode (IMDS/Queue processor): Queue processor
- OS/Arch: Amazon Linux 2, x86
- Kubernetes version: 1.24
- Installation method: Helm
Metadata
Metadata
Assignees
Labels
staleIssues / PRs with no activityIssues / PRs with no activity