The race condition mentioned in the previous commit
"Stop CMake out of source tests running on 16.04"
has also been observed with test_cmake_as_subdirectory and can presumably
happen with test_cmake_as_package and test_cmake_as_package_install as well.
So skip all of these components on Ubuntu 16.04.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Run some of the test configs twice, enabling MBEDTLS_USE_PSA_CRYPTO
and MBEDTLS_PSA_CRYPTO_C in one of the runs.
Add relevant comments in these configs.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
psa_aead_encrypt_setup() and psa_aead_decrypt_setup() were returning
PSA_ERROR_INVALID_ARGUMENT, while the same failed checks were producing
PSA_ERROR_NOT_SUPPORTED if they happened in psa_aead_encrypt() or
psa_aead_decrypt().
The PSA Crypto API 1.1 spec will specify PSA_ERROR_INVALID_ARGUMENT
in the case that the supplied algorithm is not an AEAD one.
Also move these shared checks to a helper function, to reduce code
duplication and ensure that the functions remain in sync.
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
Update the fork of the compliance test suite, and remove the multipart
AEAD tests from the expected failures list.
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
Fix library references, tests and programs.
Testing is performed in the already present all.sh test.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Pylint errors are fixed.
The Python script is improved to take default arguments when not
passed (eg invoked from root of the tree)
check-generated-files.sh and CMakeLists.sh updated.
Signed-off-by: Archana <archana.madhavan@silabs.com>
Running `generate_ssl_debug_helpers.py` generates both
`ssl_debug_helpers_generated.c` and `ssl_debug_helpers_generated.h`.
List the `.h` file as well as the `.c` file in `check-generated-files.sh` so
that `check-generated-files.sh -u` will complain if it isn't up to date.
List it in `Makefile` and `CMakeLists.txt` so that parallel builds know when
to wait until the `.h` file is present. In `Makefile`, declare the `.c` file
as depending on the `.h` file for order. This way, a dependency for either
will wait until the `.h` file is present, and since the `.h` file is
generated after the `.c` file, this guarantees that the `.c` file is
present.
This fixes random failures of `make -j` from a fresh checkout.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
As we have now a minimal viable implementation of TLS 1.3,
let's remove EXPERIMENTAL from the config option enabling
it.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Run tests with middlebox compatibility enabled but tests
dedicated to middlebox compatibility disabled.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Add ECDSA and RSA signature acceleration testing
with signature capabilitites removed from the
Mbed TLS library.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
This commit removes the test_psa_crypto_config_basic
all.sh component that can no longer work without
adapting it to the separately compiled test driver
library. This component is replaced by several
components in the following commits to test various
type of acceleration independently.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Empty the current line if it's entirely inside a comment.
Don't incorrectly end a block comment at the second line if it doesn't
contain `*/`.
Recognize `/*` to start a multiline comment even if it isn't at the start of
the line.
When stripping off comments, consistently strip off `/*` and `*/`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use '|'.join([comma-separated list]) rather than r'...|' r'...|'. This way
there's less risk of forgetting a '|'. Pylint will yell if we forget a comma
between list elements.
Use match rather than search + mandatory start anchor for EXCLUSION_LINES.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Make parse_identifiers less complex. Pylint was complaining that it had too
many local variables, and it had a point.
* Lift the constants identifier_regex and exclusion_lines to class
constants (renamed to uppercase because they're constants).
* Lift the per-file loop into a new function parse_identifiers_in_file.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix cases like
```
/*short comment*/ /*long
comment */
int mbedtls_foo;
```
where the previous code thought that the second line started outside of a
comment and ended inside of a comment.
I believe that the new code strips comments correctly. It also strips string
literals, just in case.
Fixes#5191.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Running the out of source CMake test on Ubuntu 16.04 using more than one
processor (as the CI does) can create a race condition whereby the build
fails to see a generated file, despite that file actually having been
generated. This problem appears to go away with 18.04 or newer, so make
the out of source tests not supported on Ubuntu 16.04
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
Non-regression for the fix in https://github.com/ARMmbed/mbedtls/pull/5126:
libmbedtls and libmbedx509 did not declare their dependencies on libmbedx509
and libmbedcrypto when built with make.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The new fork was rebased on top of the upstream master, removing the
need for most of the downstream patches we carried.
On the other hand, the new fork includes a couple of fixes to problems
that were not addressed by the original fork, or were introduced with the
new version of psa-arch-tests.
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
It was unmaintained and untested, and the fear of breaking it was holding us
back. Resolves#4934.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The Makefiles already assume that filenames don't contain
special characters anyway, so we don't need to check this
in generate_psa_tests.py.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Use the generate_psa_tests.py script to generate the list
of test data files used as output files by cmake.
Do this by introducing a new option --list-for-cmake
that prints a semicolon-separated list of the data files
with no terminating newline (since this is how a cmake list
is represented).
Replace the hard-coded output file list with a variable
generated by the script using this option.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Genertae test_suite_psa_crypto_generate_key.generated.data.
Use test_suite_psa_crypto_generate_key.function as a test function.
Signed-off-by: Przemyslaw Stekiel <przemyslaw.stekiel@mobica.com>
Set the build type to Release (-O2) when running CPU-intensive tests (ssl-opt,
or unit tests with debug features). A build type of Check (-Os) would be best
when the main objective of the build is to check for build errors or warnings
and there aren't many tests to run; in this commit there are no such test
cases to change. Only use cmake with no build type (which results in not
passing a -O option, and thus missing some GCC warnings) when exercising cmake
features.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use Release mode (-O2) for component_test_full_cmake_clang which runs SSL
tests.
To have some coverage with Check mode (which enables more compiler warnings
but compiles with -Os), change a few other builds that only run unit tests
at most to Check mode.
Don't add any new builds, to keep the total build volume down. We don't need
extensive coverage of all combinations, just a reasonable set.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
SSL testing benefits from faster executables, so use -O2 rather than -O1.
Some builds use -O1, but that's intended for jobs that only run unit tests,
where the build takes longer than the tests.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Don't default to unbridled -j, which causes a load spike and isn't really
faster.
"Number of CPUs" is implemented here as a reasonable compromise between
portability, correctness and simplicity. This is just a default that can be
overridden by setting MAKEFLAGS in the environment.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When not using DEBUG_C, but using the DTLS CID feature -
a null pointer was accessed in ssl_tls.c.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Don't try to enumerate excluded files. List included files, and remove names
from the list if they match an excluded-file pattern.
This resolves the problem that the script could get into an infinite loop
due to the use of recursive globbing. Unfortunately, Python's recursive
globs follows symbolic links to directories, which leads to an infinite loop
if a symbolic link points to an ancestor of the directory that contains it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
To facilitate maintenance and to make it easier to reproduce all.sh builds
manually, remove the long, repeated list of -D options from
component_test_psa_crypto_config_basic and component_test_psa_crypto_drivers
and put it in a header file instead.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When building with make, `make test` runs `run-test-suites.pl` which has a
verbose mode that reports the failing test cases, but it didn't provide a
way to enable this verbose mode. With the present commit, you can run `make
test TEST_FLAGS=-v` to use verbose mode.
Base the default for verbose mode on the same environment variable that
`make test` uses when building with CMake: default off, but enabled if
`CTEST_OUTPUT_ON_FAILURE` is true. In particular, verbose mode will now be
on when building from `all.sh`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The list was trimmed previously according to code coverage, however
this did not really evalute all test cases, e.g in the case of re.sub
or m_cipher.replace. These lines are executed no matter what, so code
coverage is not suitable.
I have gone through each step in the translate functions and made sure
there is at least one ciphersuite per step
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
Having a list of every ciphersuite suggests that it should be maintained with
any new ciphersuites that are added in the future. This in turn almost defeats
the purpose of having translation functions to begin with
Instead, the unit test now only test a much smaller subset of ciphersuite names
that exercise each stage in the OpenSSL and GnuTLS translate functions.
In the future, if a new cipersuite is added that requires an extra stage in
translation, then that cipher can be added to the test suite, otherwise it
should not be necessary.
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
As test_translate_ciphers_format.sh was made as a testing ground before
utilising translate_ciphers.py in compat.sh, once it was translated to
python code - as a unit test, it became redundant.
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
Rather than having the tests seperated into different files, they were integrated
into translate_ciphers.py and can be run from root using:
`python -m unittest tests/scripts/translate_ciphers.py`
test_translate_ciphers_format.sh was originally made as a testing ground before
having the translation tool being implmented into compat.sh. Translating it to
python code makes it redundant and therefore it will be removed.
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
test_translate_ciphers_names.py
- Combined m, o and g ciphers all into one a single list of tuples to
avoid needing to rely on indexes
test_translate_ciphers_format.sh
- Removed redundant test
- Added return errors
compat.sh
- Improved how translate_ciphers.py is called
translate_ciphers.py
- Improve regex and translation to be more intutive and efficient
- change how arguments are taken and handelled to be more reliable
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
To run test_translate_ciphers_names.py and _format.sh in the CI, include
it in all.sh component_check_generate_test_code.
Rename check_generate_test_code to check_test_helpers
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
After improving coding style, pylint suggeted using enumerate but
zip is more appropriate to avoid indexing
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
As per check-python-files.sh, added string documentation for
files and functions.
Modified for loops to use enumerate rather than range(len(
although as the same iteration index is used for multiple
lists it does not seem quite appropriate
Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
In build_arm_none_eabi_gcc_m0plus, use baremetal_size instead of baremetal
as the configuration, i.e. exclude debugging features. This job is the only
one switching to baremetal_size because it's our primary point of reference
for code size evolution, and which is the only job where we display the code
size built with -Os so it's presumably the only job for which we really care
about a meaningful code size report.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
- Problem() is a parent abstract class that should only be used for
subclassing.
- With the help of ABC, implement abstract methods that force
subclasses to implement quiet and verbose outputs.
- The repeated logic of "if self.quiet" is consolidated in Problem.
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
This reverts a previous change where line_no was removed and put into
a triple tuple. It was discovered that re.Match.span() conveniently
returns (start, end), so separating line_no again makes the code cleaner.
The legibility of the code heavily outweighs the issues pointed out by
Pylint (hence disabled).
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
The try/catch was used to catch Exceptions and exit with code 1,
a legacy from check_names.py which uses the pattern to exit with
code 2. But code 1 is the default for the Python runtime anyway,
so it is redundant and can be removed.
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
