Statistics request fails for large indices

[andrewdeans]

Elastic.Clients.Elasticsearch version: 8.0.7

Elasticsearch version: 8.6.0

.NET runtime version: Framework 4.8.1

Operating system version: Windows 10 21H1

Description of the problem including expected versus actual behavior:
The API response cannot be deserialized when retrieving statistics for an index with sizes larger than 2 GiB as a result of these values exceeding Int32.MaxValue.

Ultimately, changing the size properties of StoreStats (and possibly other similar types?) to long/ulong should resolve this. The similar TranslogStats class already uses long for its equivalent properties.

Elastic.Transport.UnexpectedTransportException: The JSON value could not be converted to System.Int32. Path: $._all.primaries.store.size_in_bytes | LineNumber: 0 | BytePositionInLine: 176. ---> System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $._all.primaries.store.size_in_bytes | LineNumber: 0 | BytePositionInLine: 176. ---> System.FormatException: Either the JSON value is not in a supported format, or is out of bounds for an Int32.
   at System.Text.Json.ThrowHelper.ThrowFormatException(NumericType numericType)
   at System.Text.Json.Utf8JsonReader.GetInt32()
   at System.Text.Json.Serialization.Converters.Int32Converter.ReadNumberWithCustomHandling(Utf8JsonReader& reader, JsonNumberHandling handling, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
--- End of inner exception stack trace ---
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[TValue](Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo, ReadStack& state)
   at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.<ReadFromStreamAsync>d__76`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Elastic.Transport.DefaultResponseBuilder`1.<SetBodyAsync>d__12`1.MoveNext() in /home/runner/work/elastic-transport-net/elastic-transport-net/src/Elastic.Transport/Components/Pipeline/DefaultResponseBuilder.cs:line 285
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elastic.Transport.DefaultResponseBuilder`1.<ToResponseAsync>d__5`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Elastic.Transport.HttpWebRequestTransportClient.<RequestAsync>d__7`1.MoveNext() in /home/runner/work/elastic-transport-net/elastic-transport-net/src/Elastic.Transport/Components/TransportClient/HttpWebRequestTransportClient.cs:line 178
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elastic.Transport.DefaultRequestPipeline`1.<CallProductEndpointAsync>d__55`1.MoveNext() in /home/runner/work/elastic-transport-net/elastic-transport-net/src/Elastic.Transport/Components/Pipeline/DefaultRequestPipeline.cs:line 271
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Elastic.Transport.DefaultHttpTransport`1.<RequestAsync>d__17`1.MoveNext() in /home/runner/work/elastic-transport-net/elastic-transport-net/src/Elastic.Transport/DefaultHttpTransport.cs:line 280
   --- End of inner exception stack trace ---
   at Elastic.Transport.DefaultHttpTransport`1.<RequestAsync>d__17`1.MoveNext() in /home/runner/work/elastic-transport-net/elastic-transport-net/src/Elastic.Transport/DefaultHttpTransport.cs:line 303
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Elastic.Clients.Elasticsearch.ElasticsearchClient.<>c__DisplayClass32_0`3.<<DoRequestAsync>g__SendRequest|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at ElasticTest.Program.<Run>d__10.MoveNext() in D:\Projects\ElasticTest\Program.cs:line 110

Steps to reproduce:

var nodes = new[]
{
    new Uri("https://node1.example.org:9200"),
    new Uri("https://node2.example.org:9200"),
    new Uri("https://node3.example.org:9200")
};

using (var pool = new StaticNodePool(nodes))
{
    var settings = new ElasticsearchClientSettings(pool)
        .Authentication(new ApiKey("secret"));
    var client = new ElasticsearchClient(settings);
    var request = new IndicesStatsRequest("some-large-index");

    // Exception thrown here.
    var response = await client.Indices.StatsAsync(request);
}

Expected behavior
Successful deserialization of the response.

Provide ConnectionSettings (if relevant): See above.

Provide DebugInformation (if relevant): n/a

[stevejgordon]

Thanks for raising this @andrewdeans. Indeed some of the types in the specification are incorrect and need to be fixed.

[andrewdeans]

Thanks for the quick response! I noticed that this was included in the recent 8.0.8 release, however it looks like the assemblies included in the NuGet package do not reflect this fix.

[stevejgordon]

You're welcome, @andrewdeans. It looks like the backport failed and I missed it. I'll manually backport and get this into 8.0.9.