Publish by-HandlerErrorCode, by-OperationStatus and top-level InternalFailure metrics

[clemengi]

Hi,

We would like to request to publish by-ErrorCode and by-OperationStatus metrics to allow the configuration of Cloudwatch alarms.

Problem

The current LambdaWrapper implementation publishes two sets of metrics related to anomalies:

  Namespace: AWS/CloudFormation/<FullyQualifiedResourceNameWithSlashes>
  Metric name: "HandlerException"
  Dimensions: Action, ExceptionType, HandlerErrorCode, ResourceType

  Namespace: AWS/CloudFormation/<AWS_ACCOUNT_ID>/<FullyQualifiedResourceNameWithSlashes>
  Metric name: HandlerException
  Dimensions: Action, ExceptionType, HandlerErrorCode, ResourceType

These metrics are great for providing a quick insight into a problem but they are not a good option for alarms, because the values of the ExceptionType dimension are not known in advance.

We would like to have additional metrics of the form:

Namespace:  <does not really matter>
Metric name: "HandlerException"
Dimensions: Action, HandlerErrorCode, ResourceType # with HandlerErrorCode in https://docs.aws.amazon.com/cloudformation-cli/latest/userguide/resource-type-test-contract-errors.html

Namespace:  <does not really matter>
Metric name: "HandlerException"
Dimensions: Action, OperationStatus, ResourceType # with OperationStatus in software.amazon.cloudformation.proxy.OperationStatus.values()

Namespace:  <does not really matter>
Metric name: "HandlerException"
Dimensions: Action, ResourceType

The first set of metrics would allow to set up alarms on a closed set of metrics, broken down by the HandlerErrorCode taxonomy.

The second metric would be useful for broad alarms on Progress status (e.g. spikes in FAILED or IN_PROGRESS calls).

The third metric should assume value 1 anytime a call to the Resource's Handler throws an exception, and zero otherwise. It would allow to alarm on any uncaught Exception, regardless of the type.

Thank you!