Run BenchmarkDotNet in GitHub Actions

[jasdefer]

I want to run BenchmarkDotNet in a GitHub action to continously measure the performance of some methods. I like the idea of automating the benchmark and illustrating the results.

Are there any recommendations or examples for this approach and is it even reliable, because the benchmark would not run on my local machine?

I am sorry if I missed any information about this topic in the documentation, but I couldn't find anything.

[AndreyAkinshin]

@jasdefer

I like the idea of automating the benchmark and illustrating the results.

For now, we don't have any out-of-the-box solutions. Please, follow #54.

Are there any recommendations or examples for this approach and is it even reliable

In general case, you will not get reliable and repeatable results. I'm working on a workflow that will allow getting reliable results (when it's possible) using an adequate number of iterations, but it's a pretty hard task which is still work in progress. Follow #155 for updates.

[jasdefer]

Thank you very much, this is exactly the discussion I was looking for. Somehow I didn't find this issue by myself. Sorry for that!

[bonesoul]

is appending data to buildplots.r is a feasible solution for this and then marking the regions with last commit?

[evilguest]

Hello everyone,
I am trying to get exactly this for my project: running the benchmarks as a part of the commit handler in the GitHub workflows: https://github.com/evilguest/linq2d/blob/master/.github/workflows/dotnet-core.yml#L31
(So far I am not expecting any comparisons or failures; it is more of having an access to the history of the benchmark variations over builds for the "manual" review).
However, I am getting stuck with the BDN complaining about missing runtime 2.1, which puzzles me a lot:

It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '2.1.0' was not found.
  - The following frameworks were found:
      3.1.1 at [/opt/hostedtoolcache/dncs/3.1.101/x64/shared/Microsoft.NETCore.App]

(https://github.com/evilguest/linq2d/runs/1116039786?check_suite_focus=true#step:9:7)

Is this expected? How do I fix that?

[adamsitnik]

@evilguest are you using dotnet benchmark command? Please see #1460 (comment)

[evilguest]

@adamsitnik thanks a lot! This did solve it for me.