Missing RabbitMQ metrics if bean is defined as a ConnectionFactory

[Huo12345]

Expected behaviour

When deploying an application with a RabbitMQ connection and prometheus metrics enabled to cloud foundy, the prometheus endpoint should expose metrics for RabbitMQ as documented in Spring Boot Reference Documentation similar to when run locally.

Observed behaviour

When deployed to cloud foundry, no RabbitMQ metrics are exposed on the prometheus endpoint.

Steps to reproduce

1. Download sample src attached demo-rabbitmq-metrics-issue.zip
2. Start RabbitMQ service (locally installed service or from docker-compose file in the source)
3. Run the application with the profile local and check prometheus endpoint for entries with rabbit -> Should have multiple
4. Deploy the app to cloud foundry with a RabbitMQ service attached and check the prometheus endpoint there for entries with rabbit -> Should have none.
   4.1 If there is no cloud foundry instance available for testing, you can run the app with profile ab to see the metrics and if to not see them. More details in section Probable Cause.

Probable Cause

When investigating this issue I found that RabbitMetricsAutoConfiguration class has the condition @ConditionalOnBean({ AbstractConnectionFactory.class, MeterRegistry.class }) attached. From code snippets in the documentation I assume that cloud foundry exposes the connection as ConnectionFactory, an interface implemented by AbstractConnectionFactory. To test this theory I manually configured a CachingConnectionFactory and exposed it in a bean of type ConnectionFactory and AbstractConnectionFactory (see class RabbitConfig in attached code, profiles ab for AbstractConnectionFactory and if for ConnectionFactory). When exposing it as ConnectionFactory no metrics were available, but when using AbstractConnectionFactory the metrics showed up in the prometheus endpoint. Is it possible to change the conditional on RabbitMetricsAutoConfiguration to @ConditionalOnBean({ ConnectionFactory.class, MeterRegistry.class }) to fix this issue?

[scottfrederick]

@Huo12345 I don't see anything in your application that configures the RabbitMQ connection on Cloud Foundry, so it appears that the ConnectionFactory is being created and contributed to the application context by Java buildpack auto-reconfiguration. This approach is not compatible with Spring Boot auto-configuration in all cases. This is the main reason why the library that enables buildpack auto-reconfiguration has been deprecated.

A better approach is to include the Java CFEnv library in your application as shown in the CFEnv documentation and disable buildpack auto-reconfiguration when pushing the app to CF. There is no additional code to write, just adding the CFEnv dependency is sufficient. This will result in Boot auto-configuration creating the ConnectionFactory in a way that is compatible with the metrics auto-configuration.

Can you try this approach and see if it addresses your issue?

[Huo12345]

@scottfrederick I can confirm that using the Java CFEnv library fixes the issue mentioned above. So my issue is fixed, thanks.
Despite that, I question the use of AbstractConnectionFactory as a condition for the RabbitMetricsAutoConfiguration since I've seen some instances in documentation (i.e. Spring AMQP Documentation) where the Java configuration exposed a bean of type ConnectionFactory, leading to the same behaviour as described above. Especially because I see no reason not to use ConnectionFactory as the condition.

[wilkinsona]

Arguably this is a bug in the CloudFoundry code that defines the bean as a ConnectionFactory as it's recommended that beans are defined with as much type information as possible. /cc @garyrussell as, IMO, the Spring AMQP documentation should be updated here too.

The metrics auto-configuration requires an AbstractConnectionFactory as it calls AbstractConnectionFactory.getRabbitConnectionFactory(). While we could define the rabbitConnectionFactoryMetricsPostProcessor when there's a ConnectionFactory bean, it would then sometimes do nothing in the (perhaps unlikely) event that the org.springframework.amqp.rabbit.connection.ConnectionFactory isn't an AbstractConnectionFactory.

[garyrussell]

IMO, the Spring AMQP documentation should be updated here too.

@wilkinsona Which documentation? And, to what? Any boot examples generally rely on the auto-configured beans and don't define any.

Or, do you mean change all the examples that do define a connection factory @Bean to ACF to encourage using narrower types? We can do that.

The metrics BPP checks the bean type anyway, so making it conditional on the interface wouldn't break anything.

[wilkinsona]

Sorry for not being clear, Gary. I was referring to the documentation that @Huo12345 linked to above. rabbitTemplate and myQueue both declare a return type that matches what they return but connectionFactory and amqpAdmin do not. I would change them to declare that they return CachingConnectionFactory and RabbitAdmin respectively.

The metrics BPP checks the bean type anyway, so making it conditional on the interface wouldn't break anything.

Yeah, indeed, it won't break anything. It may end up being defined unnecessarily but I think that's unlikely and it certainly wouldn't be the end of the world if it was.

[garyrussell]

Sorry; I missed that comment.

wouldn't be the end of the world

Yes, and as you said, it is [extremely] unlikely it wouldn't be an ACF, except perhaps in tests where it might be a mock CF.

[scottfrederick]

There is an issue to address the usage of Spring Cloud Connectors in the Java buildpack, and the Java CFEnv documentation covers this use case in the interim. I'll close this issue.

[wilkinsona]

We've decided to tweak the condition so the post-processor will be in place if the bean is defined as a ConnectionFactory rather than as an AbstractConnectionFactory or one of its sub-classes.