Percentile values (perc90, perc95, perc99) available in the TestPlanStats sometimes differ from those in the JMeter Report Dashboard

[uberlannunes]

Hello,
I have observed that the percentile values (perc90, perc95, perc99) available in the TestPlanStats sometimes differ from those in the JMeter Report Dashboard.
It seams that different algorithms are being used to calculate these values:

• Abstracta JMeter uses PSquarePercentile (https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/stat/descriptive/rank/PSquarePercentile.html)

• JMeter Report Dashboard uses Percentile.EstimationType.R_3 (https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/stat/descriptive/rank/PSquarePercentile.html) as referenced here: https://jmeter.apache.org/usermanual/generating-dashboard.html

Could you please confirm whether this is indeed the case?
If so, is there a way to configure both to use the same percentile calculation algorithm?
Thank you!

[andreig09]

Hi there!

Yes, JMeterDSL uses a different algorithm, and this was a deliberate choice.

Long story short, the algorithm in JMeterDSL is more accurate and efficient compared to the ones in JMeter.

First of all, JMeter uses at least two different methods to calculate percentiles. One of them, for example, only considers the last 20k results to calculate the percentile. So, if your sample size is larger, it trims the data.

Anyways, JMeter’s algorithms can give inconsistent results with small sample sizes, but for larger samples, the difference shouldn’t be significant.

There are even better algorithms out there than the one chosen for JMeterDSL, but they tend to be slower or consume more resources. PSquarePercentile strikes a good balance between accuracy and performance.

If you want to switch to one of the algorithms used in JMeter, you can definitely do that. For example, you could use a JSR223PostProcessor with your custom logic.