Performance testing - Initial Examples #15
Conversation
|
|
||
| ### Solution | ||
|
|
||
| Introduce think time between user actions to simulate real user behavior. This example adds a random delay between 1 to 5 seconds between each request. The bigger the range, the more realistic the simulation. |
There was a problem hiding this comment.
@bubafinder what if we want to stretch the system to test it's limits (load test)?
Do you think this is still the right approach, and if not, is it worth adding a disclaimer?
There was a problem hiding this comment.
Correct me if I'm wrong, but adding random amount of delay time simulates real user interaction. When we want to stress test, we just increase the number of concurrent users until the service starts degrading. This is what the screenshot illustrates: VU (virtual users) are ramped up and then we monitor infrastructure usage and response time to find out how the system responds to increased load.
There was a problem hiding this comment.
You are both right. I need to be more precise about where think time / random waits between actions should be used for performance testing. I will rephrase it.
There was a problem hiding this comment.
@mbareta yes, but sometimes tests will intentionally avoid doing that to achieve different system behavior - for example, to avoid relying on cache to keep the system going.
There was a problem hiding this comment.
@MiroDojkic I didn't understand exactly what are you referring to. Can you explain a bit more?
There was a problem hiding this comment.
I'd rephrase it in a way that either says something like:
- Make tests use the system in a desired way, whether through testing maximum stress capacity of the system, or performance under heavy load. As an example, when testing performance under load, we can add delay between requests to make the flow closer to the real user interaction. This influences how system components that affect performance behave under the hood, such as cache.
- Or add a disclaimer that this can be achieved in different ways and we need to be thoughtful about the desired system behavior such as whether we want to rely on cache or not, to make it close to the use-case we want to test.
Disclaimer 😄 : This is too verbose. I'm sharing my thoughts on it for the directional purpose.
|
|
||
| ## Antipattern 2: Lack of Data Variation | ||
|
|
||
| Using static, hardcoded data for requests can cause caching mechanisms to produce artificially high performance metrics. In this example, the same username is used for every request, which may not represent real-world scenarios. |
bubafinder
changed the title
| @@ -0,0 +1,118 @@ | |||
| # Performance Testing - Examples | |||
There was a problem hiding this comment.
@bubafinder can we link this page somewhere? I suggest we put the link somewhere here https://github.com/ExtensionEngine/tech-guide/blob/master/recipes/automated-testing.md#performance-tests
|
|
||
| Not including think time between user actions can lead to unrealistic performance metrics. In this example, the user actions are executed without any delay, which is unrealistic. | ||
| Excluding think time between user actions can result in unrealistic performance | ||
| metrics for certain types of tests, such as average, stress, and soak tests. |
There was a problem hiding this comment.
I would like to see this in the glossary. Just a few words on what each type means.
Some initial examples for Performance testing are to give a starting overview.
According to feedback, we need to concentrate on antipatterns and solutions.