This is part of a series of changes to add metric charts in performance
benchmarks.
All benchmarks now auto-detect if profiling metrics are enabled in the test
runtime. If profiling metrics data is present, it will generate charts of
the data.
PiperOrigin-RevId: 633094699
This CL uses container links rather than going over the host network stack.
This avoids relying on that network stack, making the benchmarks more reliable,
and simplifies benchmark setup code.
#codehealth
PiperOrigin-RevId: 486221868
- Tweak the benchmarks to work with b.N where appropriate. In many cases,
b.N was simply being ignored. This creates an implicit dependency in the
user passing a reasonable benchtime (less than or equal to the actual
runtime of the test, or using the X syntax) otherwise the test runs
forever.
- In cases where the above is impossible, explicitly set benchtime from
the test wrapper, to prevent the above behavior (tensorflow).
- Drop the *Reverse variants, which are simply hey benchmarks. We should
just add a hey benchmark. The platforms benchmarks already include a
native platform, and thus these benchmarks are incredibly confusing.
(In other words, BenchmarkNginxReverse has nothing to do with an nginx
benchmark for runsc.)
- Remove the redunant Harness object, which contains no state, in order
to slightly simplify the code.
- Make Block and Heap profiling actually work, but setting appropriate
runtime parameters (and plumbing them through the config).
- Split the profiling into two phases: start and stop, since some will
need to be started early, and others will need to happen at the end.
PiperOrigin-RevId: 349495377
For "hey", requests >= concurrency. b.N can be set by the
--test.benchtime={b.N}x. The previous setting of b.N * c
can be surprisingly slow for larger c.
Set the requests to max(b.N, c) and log to the user if it is c.
PiperOrigin-RevId: 347053675
Add parser and formatting for golang benchmarks for docker benchmarks.
Change adds a library for printing and parsing Test parameters and metrics.
Benchmarks use the library to print parameters in the Benchmark title
(e.g. the name field in b.Run()), and to report CustomMetrics. Parser
uses the library to parse printed data from benchmark output and
put it into BigQuery structs.
PiperOrigin-RevId: 336365628
Etienne Perot
Ayush Ranjan
Adin Scannell
Zach Koopmans