Hi all. Is there organized data/results kept for performance tests that are automatically run/collected by the OpenCV build pipeline? So that I can inspect and compare across builds/releases performance changes? My intention is to see the performance effect of a code fix and compare it historically across all the platforms the pipeline/builtbot makes.
I can see at e.g. https://pullrequest.opencv.org/buildbot/builders/precommit_windows64/builds/27410/steps/perf_core that performance tests were run. The raw stdio was collected. That 44 seconds elapsed for 2400+ test cases. However, I do not see where the individual test results are collected, verified, or compared across builds. There is no visibility into how a single test within those 2400+ test cases in 44 seconds changes over time. Like summary.py does for isolated builds.
Naturally, I run performance tests on my own computers and I analyze individual test cases for changed code. But my tests run on a minority of the hardware/os platforms that the pipeline already does today. Which could lead to performance increasing on some platforms, and decreasing on others -- which is undesired.