As a user, I want the API to answer gracefully when OpenSearch unable to answer

[tloubrieu-jpl]

Checked for duplicates

No - I haven't checked

🧑‍🔬 User Persona(s)

API users
web UI users and developers
peppi users and developers

💪 Motivation

...so that I can the API client can understand what is happening and react properly.

📖 Additional Details

We need to identify a HTTP status which would be different than 500.

We would also need to think of follow up actions, but we already have planetPatrol which sends alerts when something does not work.

Acceptance Criteria

Given an opensearch servie with maxed out capacity
When I perform any query on the API
Then I expect HTTP Status 503 AND a clear message, saying the service is overflown and the PDS team is working on it AND an error UUID to report to the support.

⚙️ Engineering Details

Currently OpenSearch queries return time out errors which results currently with this message in log :
2025-11-06T04:49:27.580Z INFO 7 --- [p-nio-80-exec-8] g.n.p.a.r.m.e.RegistryApiException : c23a0ca0-51fc-4c33-8b04-70c38eaee238 Unhandled Exception: Read timed out

And a 500 error for the end-user.

Other food for thoughts:

• when opensearch answers with a time out, there could be 2 cases:
  a) opensearch is temporarilly overloaded but did not reach its full capacity, it is scaling out
  b) opensearch is fully maxed out

In case a) the user could retry almost immediately but in case b the user should not try again before the on-going queries are processed.
A solution would be to have a quicker scale out parameters, but that would come at a cost.

🎉 I&T

No response

[tloubrieu-jpl]

Other food for thoughts:

• when opensearch answers with a time out, there could be 2 cases:
  a) opensearch is temporarilly overloaded but did not reach its full capacity, it is scaling out
  b) opensearch is fully maxed out

In case a) the user could retry almost immediately but in case b the user should not try again before the on-going queries are processed.
A solution would be to have a quicker scale out parameters, but that would come at a cost.