Commit graph

8 commits

Author SHA1 Message Date
Przemek Stekiel
4d13c833da analyze_outcomes.py: remove components and ignore parameters
Use a dictionary to specify optional parameters for each task.
If the task is not specified then all tasks are executed.

Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
2022-10-26 16:12:01 +02:00
Przemek Stekiel
c86dedfdc1 Fix code style
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
2022-10-24 09:16:04 +02:00
Przemek Stekiel
58bbc23ca3 Use coverage analyze as default task
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
2022-10-24 08:10:10 +02:00
Przemek Stekiel
4e95590ae7 analyze_outcomes.py: Add test coverage regresion analyze for driver only builds
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
2022-10-21 13:43:13 +02:00
Gilles Peskine
686c292e8a Move collect_available_test_cases to check_test_cases.py
No behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-01-07 18:34:12 +01:00
Gilles Peskine
3d863f2631 Document the fields of TestCasesOutcomes
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:31:26 +02:00
Gilles Peskine
8d3c70a279 Check test case coverage
Check that every available test case in the test suites and ssl-opt.sh
has been executed at least once.

For the time being, only report a warning, because our coverage is
incomplete. Once we've updated all.sh to have full coverage, this
warning should become an error.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:29:34 +02:00
Gilles Peskine
15c2cbfed5 New script for test outcome analysis
This is a new script designed to analyze test outcomes collected
during a whole CI run.

This commit introduces the script, the code to read the outcome file,
and a very simple framework to report errors. It does not perform any
actual analysis yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:29:34 +02:00