This also adds a new task in analyze_outcomes.py for checking
the accelaration coverage against the reference element.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
- the script now only terminates in case of hard faults
- each task is assigned a log
- this log tracks messages, warning and errors
- when task completes, errors and warnings are listed and
messages are appended to the main log
- on exit the main log is printed and the proper return value
is returned
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
In the full configuration MBEDTLS_TEST_HOOKS is
enabled thus the configurations in
test_full_cmake_gcc_asan_new_bignum and
test_full_cmake_gcc_asan_new_bignum_test_hooks
are the same. Keep the component that runs more
tests.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
These are build-only components so this was never supported for sure.
Let's stick to what's really tested for now, and expand later (with
proper testing!) if there's demand for more flexibility.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
These were temporaries to avoid unwanted use of low-level modules. Now
that we have test components with those modules fully disabled (replaced
with drivers), we no longer need these tests - which were out of date
too.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Restore guards from the previous release, instead of the new, more
permissive guards.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
I chose to divide along the lines of Weierstrass vs other curve shapes
(currently just Montgomery), mainly because it's the first thing that
came to mind.
It happened to reveal an issue in the logic for when (deterministic)
ECDSA and ECJPAKE are built-in, which this commit is also fixing.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
By default, we disable ASN1_[PARSE/WRITE]_C in common_tfm_config.
In fact, this is what happens for accelerated p256m driver, which
means all asn1[parse/write] tests are skipped in driver_accel test.
However, those two macros are automatically enabled for built-in
ECDSA via PSA, which means all asn1[parse/write] tests are passed
in tfm_config test.
This commit simply ignores the whole asn1[parse/write] test suite
when analyzing between driver and reference.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
{} are valid characters in test description, but they're not escaped
properly in python string format(). To resolve the bug of KeyError
when it tries to log test description which contains {}, we replace
{XXX} format with {{XXX}} in order to escape {} in python string
format() properly.
In addition, the calls to Results.log() are also handled to avoid
similar potential problems.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Since we have removed PK_C, PK_[WRITE/PARSE]_C, there is no need to
define PK related configurations again. Therefore we removed them
in common_tfm_config to make a simpler.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
We have set MBEDTLS_MD_C in tfm_mbedcrypto_config_profile_medium.h
so there is no need to enable it again.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Before this PR, the tests passed without this, but they shouldn't have.
Accelerators need to declare what curves they support, and if no curve
is declared as accelerated, the built-in implementations of algs and key
types need to be enabled.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Those components were introduced in #7103, resolving #6622: Some PSA
ECC size macros are too small when the largest accelerated curve is
larger than the largest built-in curve.
At that point, it was not possible yet to omit all built-in curves,
so we made these components that had only one (small) curve built-in and
all the others accelerated.
Now that it's possible to disable all ECC built-ins, and we have tests
doing that, we don't need that kind of fiddling any more.
Note: these component disabled RSA in order to make sure max key size
macros were not taken from RSA. We have test components with all of ECC
accelerated and RSA disabled
(component_test_psa_crypto_config_accel_ecc_no_bignum and
component_test_psa_crypto_config_accel_ecc_ffdh_no_bignum), making the
"all curves except one" components really redundant.
Note: removing them was one of the items in #7757.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Components that accelerate an algorithm that uses hashing internally
(such as deterministic ECDSA and RSA-PSS) need the hash algorithms
available in libtestdriver1.
Previously, the omission of SHA-3 in
tests/include/test/drivers/crypto_config_test_driver_extension.h meant
it was enabled in libtestdriver1 when not requesting its acceleration,
and disabled when requesting it. Adding it in a previous commit fixed
the components that asked it accelerated, but broke the component that
didn't ask for it but still needed it.
Fix those components by explicitly requesting SHA-3 as we already do for
the other hash algorithms that are require for the same reason.
Note: this broke test_suite_psa_crypto_storage_format.v0 which is
apparently the only place exercising signatures with SHA-3.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The official spelling of the trade mark changed from all-lowercase "mbed"
to normal proper noun capitalization "Mbed" a few years ago. We've been
using the new spelling in new text but still have the old spelling in a
lot of text. This commit updates most occurrences of "mbed TLS":
```
sed -i -e 's/mbed TLS/Mbed TLS/g' $(git ls-files ':!ChangeLog' ':!tests/data_files/**' ':!tests/suites/*.data' ':!programs/x509/*' ':!configs/tfm*')
```
Justification for the omissions:
* `ChangeLog`: historical text.
* `test/data_files/**`, `tests/suites/*.data`, `programs/x509/*`: many
occurrences are significant names in certificates and such. Changing
the spelling would invalidate many signatures and tests.
* `configs/tfm*`: this is an imported file. We'll follow the upstream
updates.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
These should be shared between ref and accel, for meaningful coverage
comparison.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Builds, but 20 test cases failing in test_suite_psa_crypto, to be
addressed in future commits.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
We should run make clean before we build libraries in armc6_build_test.
On the one hand, this makes sure we do have a clean build directory
initially. On the other hand, we can do extra actions after building
the library with armc6_build_test.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
grep corresponding mbedtls_xxx_setkey_dec and mbedtls_xxx_decrypt
symbols in cipher_only tests to make sure CIPHER_ENCRYPT_ONLY is
enabled as expected.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Conflicts:
* `include/psa/crypto_sizes.h`: the addition of the `u` suffix in this branch
conflicts with the rework of the calculation of `PSA_HASH_MAX_SIZE` and
`PSA_HMAC_MAX_HASH_BLOCK_SIZE` in `development`. Use the new definitions
from `development`, and add the `u` suffix to the relevant constants.
This enables compatibility with older versions of CMake that do not have
the abbreviated switch.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Use CMake's -t option to build only the crypto target. Parameterize the
crypto target to have the right name depending on whether this is Mbed
TLS or PSA Crypto.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Either remove exclusive references to Mbed TLS or accompany them with
references to "PSA Crypto".
Signed-off-by: David Horstmann <david.horstmann@arm.com>
They aren't used anywhere.
Keep the command line options of all.sh to avoid breaking any wrapper
scripts that people might have.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
None of the tests actually need GNUTLS_LEGACY (3.3.8): GNUTLS (3.4.10)
works.
None of the tests actually need OPENSSL_LEGACY (1.0.1j): OPENSSL (1.0.2g)
works.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use CMake to build the library out-of-source (rather than make)
in tests/scripts/test_psa_compliance.py and add a script argument for
the out-of-source build directory.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Introduce changes needed to run all.sh in the psa-crypto repo. Where
behaviour must differ, detect that we are in the psa-crypto repo by
checking for the 'core' directory.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
* Don't break string literals in the allow list definition
* Comment each test that belongs to the allow list is there.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
* Turn the warnings produced when finding non-executed tests that
are not in the allow list into errors.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Some basic test coverage for now:
* Nominal operation.
* Larger output buffer.
* Clone an operation and use it after the original operation stops.
Generate test data automatically. For the time being, only do that for
hashes that Python supports natively. Supporting all algorithms is future
work.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Since most of the code in "ECC+FFDH without BN" scenario was shared
with the "ECC without BN" one, I tried to reuse part of the code in
order to avoid duplications.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
Introduce the --require-full-coverage in analyze_outcomes.py so that
when analyze_outcomes.py --require-full-coverage is called, those
tests that are not executed and are not in the allowed list issue an
error instead of a warning.
Note that it is useful to run analyze_outcomes.py on incomplete test
results, so this error mode needs to remain optional in the long
term.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The allow list explicits which test cases are allowed to not be
executed when testing. This may be, for example, because a feature
is yet to be developed but the test for that feature is already in
our code base.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
component_test_crypto_full_no_cipher doesn't bring any extra value given the
existence of component_test_full_no_cipher.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
- `grep '^flags' /proc/cpuino` does not work in my local machine inside script.
- `make test programs/sleftest ` causes `strings | grep ` fail.
For timebeing, I did not figure out the root cause.
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
There is a precedent for having bigger and less mature options in
mbedtls_config.h (MBEDTLS_USE_PSA_CRYPTO) for an extended period.
Having this option in mbedtls_config.h is simpler and more robust.
Signed-off-by: Janos Follath <janos.follath@arm.com>
This previous test is correct, as it is using make. Switch to using
config.py for robustness and consistency.
Signed-off-by: Janos Follath <janos.follath@arm.com>
These tests weren't working, because they use CMake and can't pass
options with CFLAGS directly. This could be mitigated by adding a CMake
option, but using config.py is less invasive and it is what we normally
use for setting build options anyway.
Signed-off-by: Janos Follath <janos.follath@arm.com>
The latest and earliest clang/GCC compilers are now used through
variables instead of symlinks and also the all.sh script is updated
to support options for overriding the default values.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The component functions in all.sh will be listed using
compgen instead of sed so this check is not needed.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
Explain how PSA_CRYPTO_DRIVER_TEST_ALL works and why we have it. Note that
it is incompatible with MBEDTLS_PSA_CRYPTO_CONFIG. As a consequence, disable
that option in component_test_psa_crypto_drivers.
MBEDTLS_PSA_CRYPTO_BUILTIN_KEYS is in the full config, so there's no need to
add it explicitly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Running clang-16 on mbedtls reports warnings of type "-Wstrict-prototypes".
This patch fixes these warnings by adding void to functions with no
arguments. The generate_test_code.py is modified to insert void into test
functions with no arguments in *.function files.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The Ubuntu 16.04 and 22.04 docker images have been updated with
earliest and latest versions of gcc and clang respectively.
This patch adds the necessary component and support functions
required for the CI to run these compilers.
For FreeBSD we invoke the function by name so a condition is added
to disable the existing test_clang_opt function for linux.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
- RSA was missing the MBEDTLS_ prefix.
- DH needs the same temporary fix (prefix + suffix) for now.
- hack_dependencies_not_implemented() needs to ignore MBEDTLS_PSA_WANT
dependencies.
While at it, make the code currently used for ECC more generic, so that
it's ready to be used for RSA and DH in the near future.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
ShebangIssueTracker implements the rule that scripts must be executable if
and only if they have a shebang line. By removing PermissionIssueTracker, we
now allow files with any extension to be executable (provided they have a
shebang line), and allow *.sh and *.pl to be non-executable modules if they
don't have a shebang line (as was already the case for *.py).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Run the sample program demo scripts in builds with a configuration
that is at least as complete as the default configuration.
Do not run sample programs in all configurations since they are
expected to fail if a required feature is missing.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
run_demos.py is the frontend to a framework for smoke-testing the
sample programs. It runs scripts called programs/*/*_demo.sh
("demo scripts") and check that they succeed. A typical demo script
runs one sample program or a combination of sample programs to
demonstrate their usage.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When MBEDTLS_USE_PSA_CRYPTO is disabled on a base of full, mention it.
Now that full implies MBEDTLS_PSA_CRYPTO_CONFIG, don't mention it, and don't
set it explicitly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Having a wrapper made it harder to use: incompatible with setting
MBEDTLS_CONFIG_FILE, harder to combine with other settings. It was also
surprising since it was the only test config that was structured in that
way.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The original goal (https://github.com/Mbed-TLS/mbedtls/pull/5072) was to run
a test with ChaChaPoly disabled in PSA. It was actually implemented with GCM
also partially disabled (legacy GCM enabled but PSA GCM disabled), which
distracted from the objective. It's actually useful to test both with and
without GCM, so test both. Don't test inconsistencies between legacy and PSA
support because that's not a common case and not one we have particular
reasons to test.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
There were two copies of component_test_psa_crypto_config_accel_pake,
identical except for two typos. Keep the copy without the typos.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
