-
Notifications
You must be signed in to change notification settings - Fork 78
Open
Labels
A-opsArea: Operations (e.g. CI)Area: Operations (e.g. CI)P-highPriority: High. A high-priority issue should be fixed as soon as possible.Priority: High. A high-priority issue should be fixed as soon as possible.
Description
We consistently run and collect performance data for MMTk. It is inevitable that we will see performance change due to changes in the CI environment changes, including:
- Testing hardware may change. We update our testing machines every few years.
- Testing software may change. We update our testing machines regularly for new security patches, kernel changes, dependencies etc.
- Benchmarks used may change. We will update our benchmarks when there is a new version of the benchmarks available.
We should be aware of the environment changes and know the performance impact from the changes. There are a few things we should do:
- Add notes on the graph to note those changes.
- Run a binary (such as an OpenJDK release binary) for a fixed workload to detect any performance change due to environment change.
- Run the performance CI before or after we update the testing machines so we can see the diff between the environment with the same MMTk version.
Metadata
Metadata
Assignees
Labels
A-opsArea: Operations (e.g. CI)Area: Operations (e.g. CI)P-highPriority: High. A high-priority issue should be fixed as soon as possible.Priority: High. A high-priority issue should be fixed as soon as possible.