Skip to content

A failure when an instrumented WebClient records metrics causes the request to fail #30978

@ChristianLMI

Description

@ChristianLMI

I recently ran into an issue with a misconfigured MeterFilter. The filter threw exceptions causing in requests failing. I'm adding the filter like this:

@Bean
public MeterFilter taggingMeterFilter(MetricsProperties props) {
    return new TaggingMeterFilter(props);
}

static class TaggingMeterFilter implements MeterFilter {

I first observed that this is not causing issues (other than a cluttered log) when recording http.server.requests metrics. This is due to the MetricsWebFilter catching Exceptions and logging them away:

private void record(ServerWebExchange exchange, Throwable cause, long start) {
    try {
[...]
        AutoTimer.apply(this.autoTimer, this.metricName, annotations,
                (builder) -> builder.tags(tags).register(this.registry).record(duration, TimeUnit.NANOSECONDS));
    }
    catch (Exception ex) {
        logger.warn("Failed to record timer metrics", ex);
        // Allow exchange to continue, unaffected by metrics problem
    }
}

I then discovered that when the exception occurs in recording http.client.requests this is not handled similarly and I get failed requests. Apparently this is due to code in MetricsWebClientFilterFunction that doesn't do any exception handling.

private void recordTimer(Iterable<Tag> tags, Long startTime) {
    this.autoTimer.builder(this.metricName).tags(tags).description("Timer of WebClient operation")
	.register(this.meterRegistry).record(System.nanoTime() - startTime, TimeUnit.NANOSECONDS);
}

While I will add additional resilience to my code, I'd also suggest to add exception handling here. Any thoughts?

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions