Baseline Testing: Scale testing

[mpstefan]

• Scaling the number of resources (HTTPRoutes, Gateways, Secrets, Pods )
• Ensure NKG can process those resources and handle the traffic
• Ensure a new version doesn't increase resource usage compared with the previous one.

Acceptance

• A process is documented for deploying a large number of Gateway Resources resources with NKG installed to test for when NKG stop working correctly.
• Traffic is generated and sent through NKG to an upstream as a part of this test.
• The resource utilization (CPU, memory, disk) of NKG, errors from the NKG pod(s), and errors from the client, are measured and reported as a result of this test.
• No element of the test is randomized.
• The test is carried out manually and the results are reported in a static document within the NKG repo.

[mpstefan]

See #952 for potential overlap

[vepatel]

• All the scaling tests(incl. NGF, tea/coffee) should ideally be localised to this ticket so as no to have overlap with Longevity tests.
• Baseline Testing: Reload Count #955 can be a subcase of this