Document that llvm-lit needs to be run to collect static metrics

[hiraditya]

No description provided.

[ilovepi]

LGTM modulo the one comment.

[hiraditya]

Yes, actually compile_time metrics has already produced by running make/ninja and lit is just aggregating the .o.time per directory. Cf litsupport/modules/compiletime

Just to ask because I didn t find it, how to have a repeated build to have at least a averaged compile_time ? When we use perf there is the -r option but the wrapper script timeit.sh don t allow to add any options.

Does this help:
https://llvm.org/docs/TestSuiteGuide.html#displaying-and-analyzing-results

[v01dXYZ]

@hiraditya sorry i was tryharding I should have open an issue instead. Running lit don t rebuild anything. So the result for compile time are noisy and there is no way to estimate the variance and know if the number is statically significant.
That's quite a PITA for sharing reliable results to others.

[hiraditya]

@hiraditya sorry i was tryharding I should have open an issue instead. Running lit don t rebuild anything. So the result for compile time are noisy and there is no way to estimate the variance and know if the number is statically significant. That's quite a PITA for sharing reliable results to others.

yeah please open a separate issue.