Add test suite framework summary
This commit is contained in:
Azim Khan
Mohammad Azim Khan
parent
63ab501aa7
commit
aee05bbe70
1 changed files with 140 additions and 11 deletions
|
|
@ -19,18 +19,147 @@
|
|||
# This file is part of mbed TLS (https://tls.mbed.org)
|
||||
|
||||
"""
|
||||
This script dynamically generates test suite code for Mbed TLS, by
|
||||
taking following input files.
|
||||
This script is a key part of Mbed TLS test suites framework. For
|
||||
understanding the script it is important to understand the
|
||||
framework. This doc string contains a summary of the framework
|
||||
and explains the function of this script.
|
||||
|
||||
Mbed TLS test suites:
|
||||
=====================
|
||||
Scope:
|
||||
------
|
||||
The test suites focus on unit testing the crypto primitives and also
|
||||
include x509 parser tests. Tests can be added to test any MBED TLS
|
||||
module. However, the framework is not capable of testing SSL
|
||||
protocol, since that requires full stack execution and that is best
|
||||
tested as part of the system test.
|
||||
|
||||
Test case definition:
|
||||
---------------------
|
||||
Tests are defined in a test_suite_<module>[.<optional sub module>].data
|
||||
file. A test definition contains:
|
||||
test name
|
||||
optional build macro dependencies
|
||||
test function
|
||||
test parameters
|
||||
|
||||
Test dependencies are build macros that can be specified to indicate
|
||||
the build config in which the test is valid. For example if a test
|
||||
depends on a feature that is only enabled by defining a macro. Then
|
||||
that macro should be specified as a dependency of the test.
|
||||
|
||||
Test function is the function that implements the test steps. This
|
||||
function is specified for different tests that perform same steps
|
||||
with different parameters.
|
||||
|
||||
Test parameters are specified in string form separated by ':'.
|
||||
Parameters can be of type string, binary data specified as hex
|
||||
string and integer constants specified as integer, macro or
|
||||
as an expression. Following is an example test definition:
|
||||
|
||||
X509 CRL Unsupported critical extension (issuingDistributionPoint)
|
||||
depends_on:MBEDTLS_PEM_PARSE_C:MBEDTLS_RSA_C:MBEDTLS_SHA256_C
|
||||
mbedtls_x509_crl_parse:"data_files/crl-idp.pem":MBEDTLS_ERR_X509_INVALID_EXTENSIONS + MBEDTLS_ERR_ASN1_UNEXPECTED_TAG
|
||||
|
||||
Test functions:
|
||||
---------------
|
||||
Test functions are coded in C in test_suite_<module>.function files.
|
||||
Functions file is itself not compilable and contains special
|
||||
format patterns to specify test suite dependencies, start and end
|
||||
of functions and function dependencies. Check any existing functions
|
||||
file for example.
|
||||
|
||||
Execution:
|
||||
----------
|
||||
Tests are executed in 3 steps:
|
||||
- Generating test_suite_<module>[.<optional sub module>].c file
|
||||
for each corresponding .data file.
|
||||
- Building each source file into executables.
|
||||
- Running each executable and printing report.
|
||||
|
||||
Generating C test source requires more than just the test functions.
|
||||
Following extras are required:
|
||||
- Process main()
|
||||
- Reading .data file and dispatching test cases.
|
||||
- Platform specific test case execution
|
||||
- Dependency checking
|
||||
- Integer expression evaluation
|
||||
- Test function dispatch
|
||||
|
||||
Build dependencies and integer expressions (in the test parameters)
|
||||
are specified as strings in the .data file. Their run time value is
|
||||
not known at the generation stage. Hence, they need to be translated
|
||||
into run time evaluations. This script generates the run time checks
|
||||
for dependencies and integer expressions.
|
||||
|
||||
Similarly, function names have to be translated into function calls.
|
||||
This script also generates code for function dispatch.
|
||||
|
||||
The extra code mentioned here is either generated by this script
|
||||
or it comes from the input files: helpers file, platform file and
|
||||
the template file.
|
||||
|
||||
Helper file:
|
||||
------------
|
||||
Helpers file contains common helper/utility functions and data.
|
||||
|
||||
Platform file:
|
||||
--------------
|
||||
Platform file contains platform specific setup code and test case
|
||||
dispatch code. For example, host_test.function reads test data
|
||||
file from host's file system and dispatches tests.
|
||||
In case of on-target target_test.function tests are not dispatched
|
||||
on target. Target code is kept minimum and only test functions are
|
||||
dispatched. Test case dispatch is done on the host using tools like
|
||||
Greentea.
|
||||
|
||||
Template file:
|
||||
---------
|
||||
Template file for example main_test.function is a template C file in
|
||||
which generated code and code from input files is substituted to
|
||||
generate a compilable C file. It also contains skeleton functions for
|
||||
dependency checks, expression evaluation and function dispatch. These
|
||||
functions are populated with checks and return codes by this script.
|
||||
|
||||
Template file contains "replacement" fields that are formatted
|
||||
strings processed by Python str.format() method.
|
||||
|
||||
This script:
|
||||
============
|
||||
Core function of this script is to fill the template file with
|
||||
code that is generated or read from helpers and platform files.
|
||||
|
||||
This script replaces following fields in the template and generates
|
||||
the test source file:
|
||||
|
||||
{test_common_helpers} <-- All common code from helpers.function
|
||||
is substituted here.
|
||||
{functions_code} <-- Test functions are substituted here
|
||||
from the input test_suit_xyz.function
|
||||
file. C preprocessor checks are generated
|
||||
for the build dependencies specified
|
||||
in the input file. This script also
|
||||
generates wrappers for the test
|
||||
functions with code to expand the
|
||||
string parameters read from the data
|
||||
file.
|
||||
{expression_code} <-- This script enumerates the
|
||||
expressions in the .data file and
|
||||
generates code to handle enumerated
|
||||
expression Ids and return the values.
|
||||
{dep_check_code} <-- This script enumerates all
|
||||
build dependencies and generate
|
||||
code to handle enumerated build
|
||||
dependency Id and return status: if
|
||||
the dependency is defined or not.
|
||||
{dispatch_code} <-- This script enumerates the functions
|
||||
specified in the input test data file
|
||||
and generates the initializer for the
|
||||
function table in the template
|
||||
file.
|
||||
{platform_code} <-- Platform specific setup and test
|
||||
dispatch code.
|
||||
|
||||
test_suite_xyz.function - Test suite functions file contains test
|
||||
functions.
|
||||
test_suite_xyz.data - Contains test case vectors.
|
||||
main_test.function - Template to substitute generated test
|
||||
functions, dispatch code, dependency
|
||||
checking code etc.
|
||||
platform .function - Platform specific initialization and
|
||||
platform code.
|
||||
helpers.function - Common/reusable data and functions.
|
||||
"""
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue