From ddbace4b99ac640140dd0630a07a787cf34671a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Thu, 24 Apr 2025 17:04:00 +0200 Subject: [PATCH 01/11] feat(om2): add native histograms to OpenMetrics2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- content/docs/specs/om/open_metrics_spec_2_0.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index bc8b98382..d40e9f1a2 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -64,7 +64,15 @@ This section MUST be read together with the ABNF section. In case of disagreemen #### Values -Metric values in OpenMetrics MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero. +Metric values in OpenMetrics MUST be either numbers or complex data types. + +Numbers MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero. + +Complex data types MUST contain all information necessary to recreate a Metric Type, with the exception of Created time and Exemplars. + +List of complex data types: +- Integer (counter) native histograms, +- Integer gauge native histograms. ##### Booleans @@ -220,6 +228,8 @@ Histograms measure distributions of discrete events. Common examples are the lat A Histogram MetricPoint MUST contain at least one bucket, and SHOULD contain Sum, and Created values. Every bucket MUST have a threshold and a value. + + Histogram MetricPoints MUST have one bucket with an +Inf threshold. Buckets MUST be cumulative. As an example for a metric representing request latency in seconds its values for buckets with thresholds 1, 2, 3, and +Inf MUST follow value_1 <= value_2 <= value_3 <= value_+Inf. If ten requests took 1 second each, the values of the 1, 2, 3, and +Inf buckets MUST equal 10. The +Inf bucket counts all requests. If present, the Sum value MUST equal the Sum of all the measured event values. Bucket thresholds within a MetricPoint MUST be unique. From 2f4113fa6968a88d9d2df62bc8ff83ebc5bcd55f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Fri, 25 Apr 2025 14:59:31 +0200 Subject: [PATCH 02/11] define the counter histogram model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../docs/specs/om/open_metrics_spec_2_0.md | 73 ++++++++++++++++--- 1 file changed, 61 insertions(+), 12 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index d40e9f1a2..59ad58ca5 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -226,28 +226,77 @@ MetricFamilies of type Info MUST have an empty Unit string. Histograms measure distributions of discrete events. Common examples are the latency of HTTP requests, function runtimes, or I/O request sizes. -A Histogram MetricPoint MUST contain at least one bucket, and SHOULD contain Sum, and Created values. Every bucket MUST have a threshold and a value. +A Histogram MetricPoint MAY contain any number of buckets, and SHOULD contain Count, Sum, and Created values. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Count and bucket values MUST be integers. +Semantically, Count, and buckets values are counters so MUST NOT be NaN or negative. +The Sum is only a counter semantically as long as there are no negative event values measured by the Histogram MetricPoint. The Sum MUST NOT be NaN. If present, the Sum value MUST equal the Sum of all the measured event values. -Histogram MetricPoints MUST have one bucket with an +Inf threshold. Buckets MUST be cumulative. As an example for a metric representing request latency in seconds its values for buckets with thresholds 1, 2, 3, and +Inf MUST follow value_1 <= value_2 <= value_3 <= value_+Inf. If ten requests took 1 second each, the values of the 1, 2, 3, and +Inf buckets MUST equal 10. +A Histogram MetricPoint SHOULD have a Timestamp value called Created. This can help ingestors discern between new metrics and long-running ones it did not see before. -The +Inf bucket counts all requests. If present, the Sum value MUST equal the Sum of all the measured event values. Bucket thresholds within a MetricPoint MUST be unique. +A Histogram's Metric's LabelSet MUST NOT have a "le" label name. -Semantically, Sum, and buckets values are counters so MUST NOT be NaN or negative. -Negative threshold buckets MAY be used, but then the Histogram MetricPoint MUST NOT contain a sum value as it would no longer be a counter semantically. Bucket thresholds MUST NOT equal NaN. Count and bucket values MUST be integers. +A Histogram MetricPoint MUST include either classic buckets or exponential buckets or both. -A Histogram MetricPoint SHOULD have a Timestamp value called Created. This can help ingestors discern between new metrics and long-running ones it did not see before. +##### Classic buckets -A Histogram's Metric's LabelSet MUST NOT have a "le" label name. +Every classic bucket MUST have a threshold. Classic bucket thresholds within a MetricPoint MUST be unique. -Bucket values MAY have exemplars. Buckets are cumulative to allow monitoring systems to drop any non-+Inf bucket for performance/anti-denial-of-service reasons in a way that loses granularity but is still a valid Histogram. +A classic bucket MUST cover every measured value less or equal to its threshold, or to put it another way, buckets MUST be cumulative. Classic buckets are cumulative to allow monitoring systems to drop any non-+Inf bucket for performance/anti-denial-of-service reasons in a way that loses granularity but is still a valid Histogram. - +As an example for a metric representing request latency in seconds its values for classic buckets with thresholds 1, 2, 3, and +Inf MUST follow value_1 <= value_2 <= value_3 <= value_+Inf. If ten requests took 1 second each, the values of the 1, 2, 3, and +Inf buckets MUST equal 10. -Each bucket covers the values less and or equal to it, and the value of the exemplar MUST be within this range. Exemplars SHOULD be put into the bucket with the highest value. A bucket MUST NOT have more than one exemplar. +Histogram MetricPoints with classic buckets MUST have one classic bucket with an +Inf threshold. The +Inf bucket counts all requests. + +The Count value MUST equal the value of the +Inf bucket. + +Negative threshold classic buckets MAY be used. + +Classic bucket values MAY have exemplars. The value of the exemplar MUST be within the classic bucket. Exemplars SHOULD be put into the classic bucket with the highest threshold. A classic bucket MUST NOT have more than one exemplar. + +##### Native buckets + +Histogram MetricPoints with native buckets MUST have a Schema value. The Schema is an 8 bit signed integer between -4 and 127. Schema values between -4 and 127 are also called Standard Schemas. + +For any Standard Schema n, the Histogram MetricPoint MAY contain positive, negative native buckets and possibly a single zero native bucket. It is valid to have no native buckets at all. + +The boundaries of a positive or negative native bucket with index i MUST BE calculated as follows (using Python syntax): + +The upper inclusive limit of a positive native bucket: `(2**2**-n)**i` + +The lower exclusive limit of a positive native bucket: `(2**2**-n)**(i-1)` + +The lower inclusive limit of a negative native bucket: `-((2**2**-n)**i)` + +The upper exclusive limit of a negative native bucket: `-((2**2**-n)**(i-1))` + +i is an integer number that MAY be negative. + +There are exceptions to the rules above concerning the largest and smallest finite values representable as a float64 (called MaxFloat64 and MinFloat64 in the following) and the positive and negative infinity values (+Inf and -Inf): + +The positive native bucket that contains MaxFloat64 (according to the boundary formulas above) has an upper inclusive limit of MaxFloat64 (rather than the limit calculated by the formulas above, which would overflow float64). + +The next positive native bucket (index i+1 relative to the bucket from the previous item) has a lower exclusive limit of MaxFloat64 and an upper inclusive limit of +Inf. (It could be called a positive native overflow bucket.) + +The negative native bucket that contains MinFloat64 (according to the boundary formulas above) has a lower inclusive limit of MinFloat64 (rather than the limit calculated by the formulas above, which would underflow float64). + +The next negative native bucket (index i+1 relative to the bucket from the previous item) has an upper exclusive limit of MinFloat64 and an lower inclusive limit of -Inf. (It could be called a negative native overflow bucket.) + +Native buckets beyond the +Inf and -Inf buckets described above MUST NOT be used. + +If the zero native bucket is present, the Historam MetricPoint MUST have a Zero threshold. The Zero threshold is a non-negative float64 value (threshold >= 0.0). The boundaries of the Zero native bucket are `[-threshold, threshold]` inclusive. + +If the zero native bucket is present, any measured value that falls into the zero native bucket MUST BE counted towards the Zero bucket and MUST NOT be counted in any other native bucket. The Zero threshold SHOULD be equal to a lower limit of an arbitraty Standard native bucket. + +The Count value MUST equal the sum of the values of the positive, negative and the zero bucket. + +A Histogram MetricPoint with native buckets MAY contain exemplars. + +Exemplars associated with a Histogram MetricPoint with native buckets MUST have a timestamp. + +The values of exemplars in a Histogram MetricPoint with native buckets MUST fall into one of the native buckets. + +The values of exemplars in a Histogram MetricPoint with native buckets SHOULD be evenly distributed to avoid only representing the bucket with the highest value and therefore most common case. #### GaugeHistogram From a47ce0c0f46993497e1fbe352ad41f3c82f35e36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Fri, 25 Apr 2025 15:46:33 +0200 Subject: [PATCH 03/11] small fixes to histogram model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- content/docs/specs/om/open_metrics_spec_2_0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 59ad58ca5..766797173 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -226,7 +226,9 @@ MetricFamilies of type Info MUST have an empty Unit string. Histograms measure distributions of discrete events. Common examples are the latency of HTTP requests, function runtimes, or I/O request sizes. -A Histogram MetricPoint MAY contain any number of buckets, and SHOULD contain Count, Sum, and Created values. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Count and bucket values MUST be integers. +A Histogram MetricPoint MUST contain either classic buckets or exponential buckets or both. + +A Histogram MetricPoint SHOULD contain Count, Sum, and Created values. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Count and bucket values MUST be integers. Semantically, Count, and buckets values are counters so MUST NOT be NaN or negative. @@ -236,17 +238,15 @@ A Histogram MetricPoint SHOULD have a Timestamp value called Created. This can h A Histogram's Metric's LabelSet MUST NOT have a "le" label name. -A Histogram MetricPoint MUST include either classic buckets or exponential buckets or both. - ##### Classic buckets Every classic bucket MUST have a threshold. Classic bucket thresholds within a MetricPoint MUST be unique. -A classic bucket MUST cover every measured value less or equal to its threshold, or to put it another way, buckets MUST be cumulative. Classic buckets are cumulative to allow monitoring systems to drop any non-+Inf bucket for performance/anti-denial-of-service reasons in a way that loses granularity but is still a valid Histogram. +A classic bucket MUST cover every measured value less or equal to its threshold, or to put it another way, classic buckets MUST be cumulative. Classic buckets are cumulative to allow monitoring systems to drop any non-+Inf bucket for performance/anti-denial-of-service reasons in a way that loses granularity but is still a valid Histogram. As an example for a metric representing request latency in seconds its values for classic buckets with thresholds 1, 2, 3, and +Inf MUST follow value_1 <= value_2 <= value_3 <= value_+Inf. If ten requests took 1 second each, the values of the 1, 2, 3, and +Inf buckets MUST equal 10. -Histogram MetricPoints with classic buckets MUST have one classic bucket with an +Inf threshold. The +Inf bucket counts all requests. +Histogram MetricPoints with classic buckets MUST have one classic bucket with a +Inf threshold. The +Inf bucket counts all requests. The Count value MUST equal the value of the +Inf bucket. From 10dc43d9597c0363163c1d406fe14ec14ba32b7b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Fri, 25 Apr 2025 15:46:43 +0200 Subject: [PATCH 04/11] update gauge histogram model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- content/docs/specs/om/open_metrics_spec_2_0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 766797173..85244277d 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -302,17 +302,17 @@ The values of exemplars in a Histogram MetricPoint with native buckets SHOULD be GaugeHistograms measure current distributions. Common examples are how long items have been waiting in a queue, or size of the requests in a queue. -A GaugeHistogram MetricPoint MUST have one bucket with an +Inf threshold, and SHOULD contain a Gsum value. Every bucket MUST have a threshold and a value. +A GaugeHistogram MetricPoint MUST contain either classic buckets or exponential buckets or both. -The buckets for a GaugeHistogram follow all the same rules as for a Histogram. +A GaugeHistogram MetricPoint SHOULD contain Gcount, Gsum. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Gcount and bucket values MUST be integers. -The bucket and Gsum of a GaugeHistogram are conceptually gauges, however bucket values MUST NOT be negative or NaN. If negative threshold buckets are present, then sum MAY be negative. Gsum MUST NOT be NaN. Bucket values MUST be integers. +The bucket and Gsum of a GaugeHistogram are conceptually gauges, however bucket values MUST NOT be negative or NaN. If negative threshold buckets are present, then Gsum MAY be negative. Gsum MUST NOT be NaN. Bucket values MUST be integers. A GaugeHistogram's Metric's LabelSet MUST NOT have a "le" label name. -Bucket values can have exemplars. +The buckets for a GaugeHistogram follow all the same rules as for a Histogram, with Gcount playing the same role as Count. -Each bucket covers the values less and or equal to it, and the value of the exemplar MUST be within this range. Exemplars SHOULD be put into the bucket with the highest value. A bucket MUST NOT have more than one exemplar. +The exemplars for a GaugeHistogram follow all the same rules as for a Histogram. #### Summary From d12dfeb80c837e3d2ded71bd0c2b9af3794912f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Thu, 1 May 2025 12:56:17 +0200 Subject: [PATCH 05/11] wip: wip MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- content/docs/specs/om/open_metrics_spec_2_0.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 85244277d..983ac9bd2 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -382,7 +382,7 @@ metric = *sample metric-type = counter / gauge / histogram / gaugehistogram / stateset metric-type =/ info / summary / unknown -sample = metricname [labels] SP number [SP timestamp] [exemplar] LF +sample = metricname [labels] SP value [SP timestamp] [exemplar] LF exemplar = SP HASH SP labels SP number [SP timestamp] @@ -390,6 +390,8 @@ labels = "{" [label *(COMMA label)] "}" label = label-name EQ DQUOTE escaped-string DQUOTE +value = number / complextype- + number = realnumber ; Case insensitive number =/ [SIGN] ("inf" / "infinity") @@ -444,6 +446,11 @@ escaped-char =/ BS normal-char ; Any unicode character, except newline, double quote, and backslash normal-char = %x00-09 / %x0B-21 / %x23-5B / %x5D-D7FF / %xE000-10FFFF + +; Complex types +complextype = "{" complextype "}" + +nativehistogram ``` #### Overall Structure From 471a7dd9a30a67330b11d28736b3afba5ce8ded5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Thu, 22 May 2025 10:05:28 +0200 Subject: [PATCH 06/11] adjust wording MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../docs/specs/om/open_metrics_spec_2_0.md | 46 +++++++++---------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 983ac9bd2..55c09d55e 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -68,10 +68,10 @@ Metric values in OpenMetrics MUST be either numbers or complex data types. Numbers MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero. -Complex data types MUST contain all information necessary to recreate a Metric Type, with the exception of Created time and Exemplars. +Complex data types MUST contain all information necessary to recreate a sample of a Metric Type, with the exception of Created time and Exemplars. List of complex data types: -- Integer (counter) native histograms, +- Integer counter native histograms, - Integer gauge native histograms. ##### Booleans @@ -226,11 +226,11 @@ MetricFamilies of type Info MUST have an empty Unit string. Histograms measure distributions of discrete events. Common examples are the latency of HTTP requests, function runtimes, or I/O request sizes. -A Histogram MetricPoint MUST contain either classic buckets or exponential buckets or both. +A Histogram MetricPoint MUST contain either [classic buckets](#classic-buckets) or [exponential buckets](#exponential-buckets) or both. A Histogram MetricPoint SHOULD contain Count, Sum, and Created values. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Count and bucket values MUST be integers. -Semantically, Count, and buckets values are counters so MUST NOT be NaN or negative. +Semantically, Count and bucket values are counters so MUST NOT be NaN or negative. The Sum is only a counter semantically as long as there are no negative event values measured by the Histogram MetricPoint. The Sum MUST NOT be NaN. If present, the Sum value MUST equal the Sum of all the measured event values. @@ -254,49 +254,49 @@ Negative threshold classic buckets MAY be used. Classic bucket values MAY have exemplars. The value of the exemplar MUST be within the classic bucket. Exemplars SHOULD be put into the classic bucket with the highest threshold. A classic bucket MUST NOT have more than one exemplar. -##### Native buckets +##### Exponential buckets -Histogram MetricPoints with native buckets MUST have a Schema value. The Schema is an 8 bit signed integer between -4 and 127. Schema values between -4 and 127 are also called Standard Schemas. +Histogram MetricPoints with exponential buckets MUST have a Schema value. The Schema is an 8 bit signed integer between -4 and 127. Schema values between -4 and 127 are also called Standard Schemas. -For any Standard Schema n, the Histogram MetricPoint MAY contain positive, negative native buckets and possibly a single zero native bucket. It is valid to have no native buckets at all. +For any Standard Schema n, the Histogram MetricPoint MAY contain positive, negative exponential buckets and a single zero bucket. It is valid to have no exponentual buckets at all. -The boundaries of a positive or negative native bucket with index i MUST BE calculated as follows (using Python syntax): +The boundaries of a positive or negative exponential bucket with index i MUST BE calculated as follows (using Python syntax): -The upper inclusive limit of a positive native bucket: `(2**2**-n)**i` +The upper inclusive limit of a positive exponential bucket: `(2**2**-n)**i` -The lower exclusive limit of a positive native bucket: `(2**2**-n)**(i-1)` +The lower exclusive limit of a positive exponential bucket: `(2**2**-n)**(i-1)` -The lower inclusive limit of a negative native bucket: `-((2**2**-n)**i)` +The lower inclusive limit of a negative exponential bucket: `-((2**2**-n)**i)` -The upper exclusive limit of a negative native bucket: `-((2**2**-n)**(i-1))` +The upper exclusive limit of a negative exponential bucket: `-((2**2**-n)**(i-1))` i is an integer number that MAY be negative. There are exceptions to the rules above concerning the largest and smallest finite values representable as a float64 (called MaxFloat64 and MinFloat64 in the following) and the positive and negative infinity values (+Inf and -Inf): -The positive native bucket that contains MaxFloat64 (according to the boundary formulas above) has an upper inclusive limit of MaxFloat64 (rather than the limit calculated by the formulas above, which would overflow float64). +The positive exponential bucket that contains MaxFloat64 (according to the boundary formulas above) has an upper inclusive limit of MaxFloat64 (rather than the limit calculated by the formulas above, which would overflow float64). -The next positive native bucket (index i+1 relative to the bucket from the previous item) has a lower exclusive limit of MaxFloat64 and an upper inclusive limit of +Inf. (It could be called a positive native overflow bucket.) +The next positive exponential bucket (index i+1 relative to the bucket from the previous item) has a lower exclusive limit of MaxFloat64 and an upper inclusive limit of +Inf. (It could be called a positive exponential overflow bucket.) -The negative native bucket that contains MinFloat64 (according to the boundary formulas above) has a lower inclusive limit of MinFloat64 (rather than the limit calculated by the formulas above, which would underflow float64). +The negative exponential bucket that contains MinFloat64 (according to the boundary formulas above) has a lower inclusive limit of MinFloat64 (rather than the limit calculated by the formulas above, which would underflow float64). -The next negative native bucket (index i+1 relative to the bucket from the previous item) has an upper exclusive limit of MinFloat64 and an lower inclusive limit of -Inf. (It could be called a negative native overflow bucket.) +The next negative exponential bucket (index i+1 relative to the bucket from the previous item) has an upper exclusive limit of MinFloat64 and an lower inclusive limit of -Inf. (It could be called a negative exponential overflow bucket.) -Native buckets beyond the +Inf and -Inf buckets described above MUST NOT be used. +Exponential buckets beyond the +Inf and -Inf buckets described above MUST NOT be used. -If the zero native bucket is present, the Historam MetricPoint MUST have a Zero threshold. The Zero threshold is a non-negative float64 value (threshold >= 0.0). The boundaries of the Zero native bucket are `[-threshold, threshold]` inclusive. +If the zero bucket is present, the Historam MetricPoint MUST have a Zero threshold. The Zero threshold MUST BE a non-negative float64 value (threshold >= 0.0). The boundaries of the Zero native bucket are `[-threshold, threshold]` inclusive. -If the zero native bucket is present, any measured value that falls into the zero native bucket MUST BE counted towards the Zero bucket and MUST NOT be counted in any other native bucket. The Zero threshold SHOULD be equal to a lower limit of an arbitraty Standard native bucket. +If the zero bucket is present, any measured value that falls into the zero bucket MUST BE counted towards the zero bucket and MUST NOT be counted in any other exponential bucket. The Zero threshold SHOULD be equal to a lower limit of an arbitraty exponential bucket. The Count value MUST equal the sum of the values of the positive, negative and the zero bucket. -A Histogram MetricPoint with native buckets MAY contain exemplars. +A Histogram MetricPoint with exponential buckets MAY contain exemplars. -Exemplars associated with a Histogram MetricPoint with native buckets MUST have a timestamp. +Exemplars associated with a Histogram MetricPoint with exponential buckets MUST have a timestamp. -The values of exemplars in a Histogram MetricPoint with native buckets MUST fall into one of the native buckets. +The values of exemplars in a Histogram MetricPoint with exponential buckets MUST fall into one of the exponential buckets. -The values of exemplars in a Histogram MetricPoint with native buckets SHOULD be evenly distributed to avoid only representing the bucket with the highest value and therefore most common case. +The values of exemplars in a Histogram MetricPoint with exponential buckets SHOULD be evenly distributed to avoid only representing the bucket with the highest value and therefore most common case. #### GaugeHistogram From ec991a4599fc5eadf82787ded5aa23443a173294 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Thu, 22 May 2025 16:25:41 +0200 Subject: [PATCH 07/11] add abnf and presentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../docs/specs/om/open_metrics_spec_2_0.md | 75 ++++++++++++++++++- 1 file changed, 71 insertions(+), 4 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 55c09d55e..6b6b3f214 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -390,7 +390,7 @@ labels = "{" [label *(COMMA label)] "}" label = label-name EQ DQUOTE escaped-string DQUOTE -value = number / complextype- +value = number / "{" complextype "}" number = realnumber ; Case insensitive @@ -448,9 +448,42 @@ escaped-char =/ BS normal-char normal-char = %x00-09 / %x0B-21 / %x23-5B / %x5D-D7FF / %xE000-10FFFF ; Complex types -complextype = "{" complextype "}" +complextype = nativehistogram -nativehistogram +nativehistogram = nh-count "," nh-sum "," nh-schema "," nh-zero-threshold "," nh-zero-count [ "," nh-negative-spans "," nh-negative-deltas ] [ "," nh-positive-spans "," nh-positive-deltas ] +; count:n +nh-count = %d99.111.117.110.116 ":" non-negative-integer +; sum:f Does not allow +-Inf and NaN +nh-sum = %d115.117.109 ":" realnumber +; schema:i +nh-schema = %d115.99.104.101.109.97 ":" integer +; zero_threshold:f +nh-zero-threshold = %d122.101.114.111 "_" %d116.104.114.101.115.104.111.108.100 ":" realnumber +; zero_count:n +nh-zero-count = %d122.101.114.111 "_" %d99.111.117.110.116 ":" non-negative-integer +; negative_spans:[1:2,3:4] +nh-negative-spans = %d110.101.103.97.116.105.118.101 "_" %d115.112.97.110.115 ":" "[" nh-spans "]" +nh-positive-spans = %d112.111.115.105.116.105.118.101 "_" %d115.112.97.110.115 ":" "[" nh-spans "]" +; Spans can start from any index, even negative, however subsequent spans +; can only advance the index, not decrease it. +nh-spans = nh-start-span *("," nh-span) +nh-start-span = integer ":" positive-integer +nh-span = non-negative-integer ":" positive-integer + +nh-negative-deltas = %d110.101.103.97.116.105.118.101 "_" %d100.101.108.116.97.115 ":" "[" nh-deltas "]" +nh-positive-deltas = %d112.111.115.105.116.105.118.101 "_" %d100.101.108.116.97.115 ":" "[" nh-deltas "]" + +; Bucket counts are non-negative, thus the first absolute count must be non-negative. +nh-deltas = non-negative-integer *("," integer) + +; Integers, does not allow -0, or +n. +integer = non-negative-integer / "-" positive-integer + +non-negative-integer = "0" / positive-integer + +positive-integer = positive-digit [ *DIGIT ] + +positive-digit = "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9" ``` #### Overall Structure @@ -484,6 +517,16 @@ go_goroutines 69 # UNIT process_cpu_seconds seconds # HELP process_cpu_seconds Total user and system CPU time spent in seconds. process_cpu_seconds_total 4.20072246e+06 +# TYPE acme_http_request_seconds histogram +# UNIT acme_http_request_seconds seconds +# HELP acme_http_request_seconds Latency histogram of all of ACME's HTTP requests. +acme_http_request_seconds{path="/api/v1",method="GET"} {count:123,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2,3:4],positive_deltas:[5,2,3]} +acme_http_request_seconds_count{path="/api/v1",method="GET"} 123 +acme_http_request_seconds_sum{path="/api/v1",method="GET"} 1.2e2 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 0 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 1 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 1 +acme_http_request_seconds_created{path="/api/v1",method="GET"} 1605281325.0 # EOF ``` @@ -806,7 +849,7 @@ foo{quantile="0.99"} 150.0 Quantiles MAY be in any order. -##### Histogram +##### Histogram with class buckets The MetricPoint's Bucket Values Sample MetricNames MUST have the suffix `_bucket`. If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. If present, the MetricPoint's Created Value Sample MetricName MUST have the suffix `_created`. If and only if a Sum Value is present in a MetricPoint, then the MetricPoint's +Inf Bucket value MUST also appear in a Sample with a MetricName with the suffix "_count". @@ -833,6 +876,30 @@ foo_sum 324789.3 foo_created 1520430000.123 ``` +##### Histogram with exponential buckets + +The MetricPoint's value MUST BE a complex data type. + +Histograms with exponential buckets use the integer native histogram data type. + +The native histogram data type is a JSON like structure with fields. +The native histogram data type MUST include the Count, Sum, Schema, Zero Threshold, Zero bucket value as the fields `count`, `sum`, `schema`, `zero_threshold`, `zero_count`. + +If there are no negative exponential buckets, then the fields `negative_spans` and `negative_deltas` MUST BE omitted. +If there are no positive exponential buckets, then the fields `positive_spans` and `positive_deltas` MUST BE omitted. + +If there are negative (or positive) exponential buckets then the bucket values MUST BE ordered by their index, and their values MUST BE placed in the `negative_deltas` (or `positive_deltas`) field using delta encoding, that is the first bucket value is written as is and the following values only as a delta relative to the previous value. For example bucket values 1, 5, 4 will become 1, 4, -1. + +To map the `negative_deltas` (or `positive_deltas`) back to their indices, the `negative_spans` (or `positive_spans`) field must be constructed in the following way: each span consists of a pair of numbers, a signed 32-bit integer (short: int32) called offset and an unsigned 32-bit integer (short: uint32) called length. Only the first span in each list can have a negative offset. It defines the index of the first bucket in its corresponding `negative_deltas` (or `positive_deltas`). The length defines the number of consecutive buckets the bucket list starts with. The offsets of the following spans define the number of excluded (and thus unpopulated buckets). The lengths define the number of consecutive buckets in the list following the excluded buckets. + +The sum of all length values in each span list MUST be equal to the length of the corresponding bucket list. + +``` +# TYPE acme_http_request_seconds histogram +acme_http_request_seconds{path="/api/v1",method="GET"} {count:123,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2,3:4],positive_deltas:[5,2,3]} +acme_http_request_seconds_created 1520430000.123 +``` + ###### Exemplars Exemplars without Labels MUST represent an empty LabelSet as {}. From dc308e271ad65554a0361a79d809865495b5b37e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Fri, 23 May 2025 10:57:20 +0200 Subject: [PATCH 08/11] updates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add semantic conventions about where complex types may occure. Allow empty spans and deltas. Be more precise. Signed-off-by: György Krajcsovits --- .../docs/specs/om/open_metrics_spec_2_0.md | 73 ++++++++++++++----- 1 file changed, 53 insertions(+), 20 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 6b6b3f214..c785b12ab 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -71,8 +71,10 @@ Numbers MUST be either floating points or integers. Note that ingestors of the f Complex data types MUST contain all information necessary to recreate a sample of a Metric Type, with the exception of Created time and Exemplars. List of complex data types: -- Integer counter native histograms, -- Integer gauge native histograms. +- Integer counter native histograms for the Metric Type Histogram. +- Integer gauge native histograms for the Metric Type GaugeHistogram. + +Complex data types MUST occure only in the corresponding MetricFamily. This means for example that a counter cannot have an integer counter native histogram value. ##### Booleans @@ -461,17 +463,17 @@ nh-schema = %d115.99.104.101.109.97 ":" integer nh-zero-threshold = %d122.101.114.111 "_" %d116.104.114.101.115.104.111.108.100 ":" realnumber ; zero_count:n nh-zero-count = %d122.101.114.111 "_" %d99.111.117.110.116 ":" non-negative-integer -; negative_spans:[1:2,3:4] -nh-negative-spans = %d110.101.103.97.116.105.118.101 "_" %d115.112.97.110.115 ":" "[" nh-spans "]" -nh-positive-spans = %d112.111.115.105.116.105.118.101 "_" %d115.112.97.110.115 ":" "[" nh-spans "]" +; negative_spans:[1:2,3:4] and negative_spans:[] +nh-negative-spans = %d110.101.103.97.116.105.118.101 "_" %d115.112.97.110.115 ":" "[" [nh-spans] "]" +nh-positive-spans = %d112.111.115.105.116.105.118.101 "_" %d115.112.97.110.115 ":" "[" [nh-spans] "]" ; Spans can start from any index, even negative, however subsequent spans ; can only advance the index, not decrease it. nh-spans = nh-start-span *("," nh-span) nh-start-span = integer ":" positive-integer nh-span = non-negative-integer ":" positive-integer -nh-negative-deltas = %d110.101.103.97.116.105.118.101 "_" %d100.101.108.116.97.115 ":" "[" nh-deltas "]" -nh-positive-deltas = %d112.111.115.105.116.105.118.101 "_" %d100.101.108.116.97.115 ":" "[" nh-deltas "]" +nh-negative-deltas = %d110.101.103.97.116.105.118.101 "_" %d100.101.108.116.97.115 ":" "[" [nh-deltas] "]" +nh-positive-deltas = %d112.111.115.105.116.105.118.101 "_" %d100.101.108.116.97.115 ":" "[" [nh-deltas] "]" ; Bucket counts are non-negative, thus the first absolute count must be non-negative. nh-deltas = non-negative-integer *("," integer) @@ -520,12 +522,12 @@ process_cpu_seconds_total 4.20072246e+06 # TYPE acme_http_request_seconds histogram # UNIT acme_http_request_seconds seconds # HELP acme_http_request_seconds Latency histogram of all of ACME's HTTP requests. -acme_http_request_seconds{path="/api/v1",method="GET"} {count:123,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2,3:4],positive_deltas:[5,2,3]} -acme_http_request_seconds_count{path="/api/v1",method="GET"} 123 +acme_http_request_seconds{path="/api/v1",method="GET"} {count:2,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2],positive_deltas:[1,0]} +acme_http_request_seconds_count{path="/api/v1",method="GET"} 2 acme_http_request_seconds_sum{path="/api/v1",method="GET"} 1.2e2 -acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 0 -acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 1 -acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 1 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 1 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 2 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 2 acme_http_request_seconds_created{path="/api/v1",method="GET"} 1605281325.0 # EOF ``` @@ -882,24 +884,55 @@ The MetricPoint's value MUST BE a complex data type. Histograms with exponential buckets use the integer native histogram data type. -The native histogram data type is a JSON like structure with fields. -The native histogram data type MUST include the Count, Sum, Schema, Zero Threshold, Zero bucket value as the fields `count`, `sum`, `schema`, `zero_threshold`, `zero_count`. +The integer native histogram data type is a JSON like structure with fields. There MUST NOT BE any whitespace around fields. +The integer native histogram data type MUST include the Count, Sum, Schema, Zero Threshold, Zero bucket value as the fields `count`, `sum`, `schema`, `zero_threshold`, `zero_count`. + +If there are no negative exponential buckets, then the fields `negative_spans` and `negative_deltas` SHOULD BE omitted. +If there are no positive exponential buckets, then the fields `positive_spans` and `positive_deltas` SHOULD BE omitted. + +If there are negative (and/or positive) exponential buckets then the fields `negative_spans`, `negative_deltas` (and/or `positive_spans`, `positive_deltas`) MUST BE present in this order after the `zero_count` field. + +Exponential bucket values MUST BE ordered by their index, and their values MUST BE placed in the `negative_deltas` (and/or `positive_deltas`) field using delta encoding, that is the first bucket value is written as is and the following values only as a delta relative to the previous value. For example bucket values 1, 5, 4, 4 will become 1, 4, -1, 0. + +To map the `negative_deltas` (and/or `positive_deltas`) back to their indices, the `negative_spans` (and/or `positive_spans`) field MUST BE constructed in the following way: each span consists of a pair of numbers, an integer called offset and an non-negative integer called length. Only the first span in each list can have a negative offset. It defines the index of the first bucket in its corresponding `negative_deltas` (and/or `positive_deltas`). The length defines the number of consecutive buckets the bucket list starts with. The offsets of the following spans define the number of excluded (and thus unpopulated buckets). The lengths define the number of consecutive buckets in the list following the excluded buckets. -If there are no negative exponential buckets, then the fields `negative_spans` and `negative_deltas` MUST BE omitted. -If there are no positive exponential buckets, then the fields `positive_spans` and `positive_deltas` MUST BE omitted. +The sum of all length values in each span list MUST BE equal to the length of the corresponding bucket list. -If there are negative (or positive) exponential buckets then the bucket values MUST BE ordered by their index, and their values MUST BE placed in the `negative_deltas` (or `positive_deltas`) field using delta encoding, that is the first bucket value is written as is and the following values only as a delta relative to the previous value. For example bucket values 1, 5, 4 will become 1, 4, -1. +An example with all fields: -To map the `negative_deltas` (or `positive_deltas`) back to their indices, the `negative_spans` (or `positive_spans`) field must be constructed in the following way: each span consists of a pair of numbers, a signed 32-bit integer (short: int32) called offset and an unsigned 32-bit integer (short: uint32) called length. Only the first span in each list can have a negative offset. It defines the index of the first bucket in its corresponding `negative_deltas` (or `positive_deltas`). The length defines the number of consecutive buckets the bucket list starts with. The offsets of the following spans define the number of excluded (and thus unpopulated buckets). The lengths define the number of consecutive buckets in the list following the excluded buckets. +``` +# TYPE acme_http_request_seconds histogram +acme_http_request_seconds{path="/api/v1",method="GET"} {count:59,sum:1.2e2,schema:7,zero_threshold:1e-4,zero_count:0,negative_spans:[1:2],negative_deltas:[5,2],positive_spans:[-1:2,3:4],positive_deltas:[5,2,3,-1,-1,0]} +acme_http_request_seconds_created 1520430000.123 +``` -The sum of all length values in each span list MUST be equal to the length of the corresponding bucket list. +An example without any buckets in use: ``` # TYPE acme_http_request_seconds histogram -acme_http_request_seconds{path="/api/v1",method="GET"} {count:123,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2,3:4],positive_deltas:[5,2,3]} +acme_http_request_seconds{path="/api/v1",method="GET"} {count:0,sum:0,schema:3,zero_threshold:1e-4,zero_count:0} acme_http_request_seconds_created 1520430000.123 ``` +##### Histogram with both classic and exponential buckets + +If a Histogram MetricPoint has both classic and exponential buckets, the exponential buckets MUST come first and the created time MUST NOT BE duplicated. + +The order ensures that implementations can easily skip the classic buckets if the exponential buckets are preferred. + +``` +# TYPE acme_http_request_seconds histogram +# UNIT acme_http_request_seconds seconds +# HELP acme_http_request_seconds Latency histogram of all of ACME's HTTP requests. +acme_http_request_seconds{path="/api/v1",method="GET"} {count:2,sum:1.2e2,schema:0,zero_threshold:1e-4,zero_count:0,positive_spans:[1:2],positive_deltas:[1,0]} +acme_http_request_seconds_count{path="/api/v1",method="GET"} 2 +acme_http_request_seconds_sum{path="/api/v1",method="GET"} 1.2e2 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 1 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 2 +acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 2 +acme_http_request_seconds_created{path="/api/v1",method="GET"} 1605281325.0 +``` + ###### Exemplars Exemplars without Labels MUST represent an empty LabelSet as {}. From 57c6ab8020c03a858c88829571822c7042b9ece2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Fri, 23 May 2025 11:12:18 +0200 Subject: [PATCH 09/11] Add gauge histogram syntax MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- content/docs/specs/om/open_metrics_spec_2_0.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index c785b12ab..539dd7a42 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -952,7 +952,7 @@ foo_sum 324789.3 foo_created 1520430000.123 ``` -##### GaugeHistogram +##### GaugeHistogram with classic buckets The MetricPoint's Bucket Values Sample MetricNames MUST have the suffix `_bucket`. If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_gsum`. If and only if a Sum Value is present in a MetricPoint, then the MetricPoint's +Inf Bucket value MUST also appear in a Sample with a MetricName with the suffix `_gcount`. @@ -972,6 +972,20 @@ foo_gcount 42.0 foo_gsum 3289.3 ``` +##### GaugeHistogram with exponential buckets + +GaugeHistogram MetricPoints with exponential buckets follow the same syntax as Histogram MetricPoints with exponential buckets. + +``` +# TYPE acme_http_request_seconds gaugehistogram +acme_http_request_seconds{path="/api/v1",method="GET"} {count:59,sum:1.2e2,schema:7,zero_threshold:1e-4,zero_count:0,negative_spans:[1:2],negative_deltas:[5,2],positive_spans:[-1:2,3:4],positive_deltas:[5,2,3,-1,-1,0]} +acme_http_request_seconds_created 1520430000.123 +``` + +##### GaugeHistogram with both classic and exponential buckets + +If a GaugeHistogram MetricPoint has both classic and exponential buckets, the exponential buckets MUST come first and the created time MUST NOT BE duplicated. + ##### Unknown The sample metric name for the value of the MetricPoint for a MetricFamily of type Unknown MUST NOT have a suffix. From 32a65cf5421c68a5ca703b81783a780940dbf357 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Tue, 27 May 2025 15:53:36 +0200 Subject: [PATCH 10/11] clarify NaN and Inf and be permissive MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../docs/specs/om/open_metrics_spec_2_0.md | 22 ++++++++++++++----- 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 539dd7a42..5a924f113 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -228,13 +228,19 @@ MetricFamilies of type Info MUST have an empty Unit string. Histograms measure distributions of discrete events. Common examples are the latency of HTTP requests, function runtimes, or I/O request sizes. +A Histogram MetricPoint MUST contain Count, Sum values. + +The Count value MUST equal the number of measurements taken by the Histogram. The Count is a counter semantically. The Count MUST be an integer and MUST NOT be NaN or negative. + +The Sum value MUST equal the Sum of all the measured event values. The Sum is only a counter semantically as long as there are no negative event values measured by the Histogram MetricPoint. + A Histogram MetricPoint MUST contain either [classic buckets](#classic-buckets) or [exponential buckets](#exponential-buckets) or both. -A Histogram MetricPoint SHOULD contain Count, Sum, and Created values. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Count and bucket values MUST be integers. +Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Bucket values MUST be integers. Semantically, bucket values are counters so MUST NOT be NaN or negative. -Semantically, Count and bucket values are counters so MUST NOT be NaN or negative. +A Histogram SHOULD refuse to measure NaN value as adding NaN to the Sum will make the Sum equal to NaN and mask the sum of the real measurements until the next reset of the counters. If a Histogram does allow NaN, it MUST be counted in the Count and MUST be added to the Sum, resulting in the Sum becoming NaN. -The Sum is only a counter semantically as long as there are no negative event values measured by the Histogram MetricPoint. The Sum MUST NOT be NaN. If present, the Sum value MUST equal the Sum of all the measured event values. +A Histogram MAY refuse to measure +Inf and -Inf values as adding these to the Sum will mask the sum of the real measurements until the next reset of the counters. If a Histogram measures +Inf or -Inf, they MUST be counted in the Count and MUST be added to the Sum, potentially resulting in +Inf, -Inf or NaN in the Sum, the later for example in case of adding +Inf to -Inf. A Histogram MetricPoint SHOULD have a Timestamp value called Created. This can help ingestors discern between new metrics and long-running ones it did not see before. @@ -250,7 +256,9 @@ As an example for a metric representing request latency in seconds its values fo Histogram MetricPoints with classic buckets MUST have one classic bucket with a +Inf threshold. The +Inf bucket counts all requests. -The Count value MUST equal the value of the +Inf bucket. +If the NaN value is not allowed, then the Count value MUST be equal to the value of the +Inf bucket. + +If the NaN value is allowed, it SHOULD be counted in the +Inf bucket, and MUST not be counted in any other bucket. In case the NaN is counted in the +Inf bucket, then the Count MUST be equal to the value of the +Inf bucket, otherwise the Count MUST be greater. The ratonale is that NaN does not belong to any bucket mathematically, however instrumentation libraries traditionally put it into the +Inf bucket, which is why the wording "SHOULD" is used. Negative threshold classic buckets MAY be used. @@ -290,7 +298,9 @@ If the zero bucket is present, the Historam MetricPoint MUST have a Zero thresho If the zero bucket is present, any measured value that falls into the zero bucket MUST BE counted towards the zero bucket and MUST NOT be counted in any other exponential bucket. The Zero threshold SHOULD be equal to a lower limit of an arbitraty exponential bucket. -The Count value MUST equal the sum of the values of the positive, negative and the zero bucket. +If the NaN value is not allowed, then the Count value MUST be equal to the sum of the negative, positive and zero buckets. + +If the NaN value is allowed, it SHOULD NOT be counted in any bucket. The value NaN MAY be counted in the +Inf bucket, and MUST not be counted in any other bucket. In case the NaN is counted in the +Inf bucket, then the Count value MUST be equal to the sum of the negative, positive and zero buckets, otherwise the Count MUST be greater. The ratonale is that NaN does not belong to any bucket mathematically, and instrumentation libraries traditionally don't put it into any bucket, which is why the wording "SHOULD NOT" is used. A Histogram MetricPoint with exponential buckets MAY contain exemplars. @@ -456,7 +466,7 @@ nativehistogram = nh-count "," nh-sum "," nh-schema "," nh-zero-threshold "," nh ; count:n nh-count = %d99.111.117.110.116 ":" non-negative-integer ; sum:f Does not allow +-Inf and NaN -nh-sum = %d115.117.109 ":" realnumber +nh-sum = %d115.117.109 ":" number ; schema:i nh-schema = %d115.99.104.101.109.97 ":" integer ; zero_threshold:f From 6968e00806d62c0a8605d2304ac1a6538cb9fce2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= Date: Tue, 27 May 2025 16:01:46 +0200 Subject: [PATCH 11/11] fix from Arthur's comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: György Krajcsovits --- .../docs/specs/om/open_metrics_spec_2_0.md | 38 +++++++++---------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/content/docs/specs/om/open_metrics_spec_2_0.md b/content/docs/specs/om/open_metrics_spec_2_0.md index 5a924f113..c8bc032cb 100644 --- a/content/docs/specs/om/open_metrics_spec_2_0.md +++ b/content/docs/specs/om/open_metrics_spec_2_0.md @@ -68,13 +68,13 @@ Metric values in OpenMetrics MUST be either numbers or complex data types. Numbers MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero. -Complex data types MUST contain all information necessary to recreate a sample of a Metric Type, with the exception of Created time and Exemplars. +Complex data types MUST contain all information necessary to recreate a sample of a Metric Type, with the exception of Created Value and Exemplars. List of complex data types: - Integer counter native histograms for the Metric Type Histogram. - Integer gauge native histograms for the Metric Type GaugeHistogram. -Complex data types MUST occure only in the corresponding MetricFamily. This means for example that a counter cannot have an integer counter native histogram value. +Complex data types MUST occur only in the corresponding MetricFamily. This means for example that a counter cannot have an integer counter native histogram value. ##### Booleans @@ -230,17 +230,17 @@ Histograms measure distributions of discrete events. Common examples are the lat A Histogram MetricPoint MUST contain Count, Sum values. -The Count value MUST equal the number of measurements taken by the Histogram. The Count is a counter semantically. The Count MUST be an integer and MUST NOT be NaN or negative. +The Count value MUST be equal to the number of measurements taken by the Histogram. The Count is a counter semantically. The Count MUST be an integer and MUST NOT be NaN or negative. -The Sum value MUST equal the Sum of all the measured event values. The Sum is only a counter semantically as long as there are no negative event values measured by the Histogram MetricPoint. +The Sum value MUST be equal to the Sum of all the measured event values. The Sum is only a counter semantically as long as there are no negative event values measured by the Histogram MetricPoint. A Histogram MetricPoint MUST contain either [classic buckets](#classic-buckets) or [exponential buckets](#exponential-buckets) or both. Every bucket MUST have well defined boundaries and a value. Boundaries of a bucket MUST NOT be NaN. Bucket values MUST be integers. Semantically, bucket values are counters so MUST NOT be NaN or negative. -A Histogram SHOULD refuse to measure NaN value as adding NaN to the Sum will make the Sum equal to NaN and mask the sum of the real measurements until the next reset of the counters. If a Histogram does allow NaN, it MUST be counted in the Count and MUST be added to the Sum, resulting in the Sum becoming NaN. +A Histogram SHOULD refuse to measure NaN value as adding NaN to the Sum will make the Sum equal to NaN and mask the sum of the real measurements until the next reset of the counters. If a Histogram does allow NaN, then NaN MUST be counted in the Count and MUST be added to the Sum, resulting in the Sum becoming NaN. -A Histogram MAY refuse to measure +Inf and -Inf values as adding these to the Sum will mask the sum of the real measurements until the next reset of the counters. If a Histogram measures +Inf or -Inf, they MUST be counted in the Count and MUST be added to the Sum, potentially resulting in +Inf, -Inf or NaN in the Sum, the later for example in case of adding +Inf to -Inf. +A Histogram MAY refuse to measure +Inf and -Inf values as adding these to the Sum will mask the sum of the real measurements until the next reset of the counters. If a Histogram measures +Inf or -Inf, then +Inf or -Inf MUST be counted in the Count and MUST be added to the Sum, potentially resulting in +Inf, -Inf or NaN in the Sum, the later for example in case of adding +Inf to -Inf. A Histogram MetricPoint SHOULD have a Timestamp value called Created. This can help ingestors discern between new metrics and long-running ones it did not see before. @@ -270,7 +270,7 @@ Histogram MetricPoints with exponential buckets MUST have a Schema value. The Sc For any Standard Schema n, the Histogram MetricPoint MAY contain positive, negative exponential buckets and a single zero bucket. It is valid to have no exponentual buckets at all. -The boundaries of a positive or negative exponential bucket with index i MUST BE calculated as follows (using Python syntax): +The boundaries of a positive or negative exponential bucket with index i MUST be calculated as follows (using Python syntax): The upper inclusive limit of a positive exponential bucket: `(2**2**-n)**i` @@ -294,9 +294,9 @@ The next negative exponential bucket (index i+1 relative to the bucket from the Exponential buckets beyond the +Inf and -Inf buckets described above MUST NOT be used. -If the zero bucket is present, the Historam MetricPoint MUST have a Zero threshold. The Zero threshold MUST BE a non-negative float64 value (threshold >= 0.0). The boundaries of the Zero native bucket are `[-threshold, threshold]` inclusive. +If the zero bucket is present, the Historam MetricPoint MUST have a Zero threshold. The Zero threshold MUST be a non-negative float64 value (threshold >= 0.0). The boundaries of the Zero native bucket are `[-threshold, threshold]` inclusive. -If the zero bucket is present, any measured value that falls into the zero bucket MUST BE counted towards the zero bucket and MUST NOT be counted in any other exponential bucket. The Zero threshold SHOULD be equal to a lower limit of an arbitraty exponential bucket. +If the zero bucket is present, any measured value that falls into the zero bucket MUST be counted towards the zero bucket and MUST NOT be counted in any other exponential bucket. The Zero threshold SHOULD be equal to a lower limit of an arbitraty exponential bucket. If the NaN value is not allowed, then the Count value MUST be equal to the sum of the negative, positive and zero buckets. @@ -890,23 +890,23 @@ foo_created 1520430000.123 ##### Histogram with exponential buckets -The MetricPoint's value MUST BE a complex data type. +The MetricPoint's value MUST be a complex data type. Histograms with exponential buckets use the integer native histogram data type. -The integer native histogram data type is a JSON like structure with fields. There MUST NOT BE any whitespace around fields. +The integer native histogram data type is a JSON like structure with fields. There MUST NOT be any whitespace around fields. The integer native histogram data type MUST include the Count, Sum, Schema, Zero Threshold, Zero bucket value as the fields `count`, `sum`, `schema`, `zero_threshold`, `zero_count`. -If there are no negative exponential buckets, then the fields `negative_spans` and `negative_deltas` SHOULD BE omitted. -If there are no positive exponential buckets, then the fields `positive_spans` and `positive_deltas` SHOULD BE omitted. +If there are no negative exponential buckets, then the fields `negative_spans` and `negative_deltas` SHOULD be omitted. +If there are no positive exponential buckets, then the fields `positive_spans` and `positive_deltas` SHOULD be omitted. -If there are negative (and/or positive) exponential buckets then the fields `negative_spans`, `negative_deltas` (and/or `positive_spans`, `positive_deltas`) MUST BE present in this order after the `zero_count` field. +If there are negative (and/or positive) exponential buckets then the fields `negative_spans`, `negative_deltas` (and/or `positive_spans`, `positive_deltas`) MUST be present in this order after the `zero_count` field. -Exponential bucket values MUST BE ordered by their index, and their values MUST BE placed in the `negative_deltas` (and/or `positive_deltas`) field using delta encoding, that is the first bucket value is written as is and the following values only as a delta relative to the previous value. For example bucket values 1, 5, 4, 4 will become 1, 4, -1, 0. +Exponential bucket values MUST be ordered by their index, and their values MUST be placed in the `negative_deltas` (and/or `positive_deltas`) field using delta encoding, that is the first bucket value is written as is and the following values only as a delta relative to the previous value. For example bucket values 1, 5, 4, 4 will become 1, 4, -1, 0. -To map the `negative_deltas` (and/or `positive_deltas`) back to their indices, the `negative_spans` (and/or `positive_spans`) field MUST BE constructed in the following way: each span consists of a pair of numbers, an integer called offset and an non-negative integer called length. Only the first span in each list can have a negative offset. It defines the index of the first bucket in its corresponding `negative_deltas` (and/or `positive_deltas`). The length defines the number of consecutive buckets the bucket list starts with. The offsets of the following spans define the number of excluded (and thus unpopulated buckets). The lengths define the number of consecutive buckets in the list following the excluded buckets. +To map the `negative_deltas` (and/or `positive_deltas`) back to their indices, the `negative_spans` (and/or `positive_spans`) field MUST be constructed in the following way: each span consists of a pair of numbers, an integer called offset and an non-negative integer called length. Only the first span in each list can have a negative offset. It defines the index of the first bucket in its corresponding `negative_deltas` (and/or `positive_deltas`). The length defines the number of consecutive buckets the bucket list starts with. The offsets of the following spans define the number of excluded (and thus unpopulated buckets). The lengths define the number of consecutive buckets in the list following the excluded buckets. -The sum of all length values in each span list MUST BE equal to the length of the corresponding bucket list. +The sum of all length values in each span list MUST be equal to the length of the corresponding bucket list. An example with all fields: @@ -926,7 +926,7 @@ acme_http_request_seconds_created 1520430000.123 ##### Histogram with both classic and exponential buckets -If a Histogram MetricPoint has both classic and exponential buckets, the exponential buckets MUST come first and the created time MUST NOT BE duplicated. +If a Histogram MetricPoint has both classic and exponential buckets, the exponential buckets MUST come first and the created time MUST NOT be duplicated. The order ensures that implementations can easily skip the classic buckets if the exponential buckets are preferred. @@ -994,7 +994,7 @@ acme_http_request_seconds_created 1520430000.123 ##### GaugeHistogram with both classic and exponential buckets -If a GaugeHistogram MetricPoint has both classic and exponential buckets, the exponential buckets MUST come first and the created time MUST NOT BE duplicated. +If a GaugeHistogram MetricPoint has both classic and exponential buckets, the exponential buckets MUST come first and the created time MUST NOT be duplicated. ##### Unknown