For now, ignore test suites that don't have parity even is they should.
The purpose is just to prepare the infrastructure and map the work.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This is the basis for future work, we'll want to make sure everything
passes in this component.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Slightly re-organize (accel list at the top).
No need to disable USE_PSA or TLS 1.3 because they're already that way
in the default config.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Some PSA curves' symbols (PSA_WANT_) were not matching the corresponding
MBEDTLS_ECP_DP_. This was fixed together with the removal of extra code
when DEBUG_C is not enabled.
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
mbedtls_test_psa_setup_key_derivation_wrap() returns 1 for success, 0
for error, so the test here was wrong.
This is just a hotfix in order to avoid a testing gap. Larger issues not
addressed here:
- I don't think we should just exit and mark the test as passed; if
we're not doing the actual testing this should be marked as SKIP.
- Returning 1 for success and 0 for failure is a violation of our
documented coding guidelines. We're also supposed to test with == 0 or
!= 0. Having consistent conventions is supposed to help avoid errors
like this.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Otherwise, in builds without PKSC1_V15, tests that are supposed to
accept the certificate will fail, because once the cert is OK they will
move on to checking the CRL and will choke on its non-PSS signature.
Tests that are supposed to reject the cert due to an invalid signature
from the CA will not check the CRL because they don't recognize the CA
as valid, so they have no reason to check the CA's CRL. This was hiding
the problem until the recent commit that added a test where the cert is
supposed to be accepted.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
We've decided not to check it, see
https://github.com/Mbed-TLS/mbedtls/issues/5277
Also add a test that we accept the certificate with USE_PSA.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The code had an earlier version. Update to the new seed that
mpi_core_random_basic has moved to.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
mbedtls_mpi_mod_raw_random() and mbedtls_mpi_mod_random() were producing
output in the Montgomery representation, instead of obeying the
representation chosen in the modulus structure. Fix this.
Duplicate the test cases for mod-random output to have separate test cases
for each representation.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This wasn't reported by pylint due to a pylint bug (apparently):
`pylint A B` doesn't complain about an unused import in B if A happens to
import and use the same module, which happens to be the case when we run
pylint on the CI.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
These variables were both uses to select the default version of OpenSSL
to use for tests:
- when running compat.sh or ssl-opt.sh directly, OPENSSL_CMD was used;
- when running all.sh, OPENSSL was used.
This caused surprising situations if you had one but not the other set
in your environment. For example I used to have OPENSSL_CMD set but not
OPENSSL, so ssl-opt.sh was failing in some all.sh components but passing
when I ran it manually in the same configuration and build, a rather
unpleasant experience.
The natural name would be OPENSSL, and that's what set in the Docker
images used by the CI. However back in the 1.3.x days, that name was
already used in library/Makefile, so it was preferable to pick a
different one, hence OPENSSL_CMD. However the build system has not been
using this name since at least Mbed TLS 2.0.0, so it's now free for use
again (as demonstrated by the fact that it's been set in the CI without
causing any trouble).
So, unify things and use OPENSSL everywhere. Just leave an error message
for the benefit of developers which might have OPENSSL_CMD, not OPENSSL,
set in their environment from the old days.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Ideally the result of the generator would conform to the code style, but
this would be difficult, especially with respect to the placement of line
breaks in long logical lines. So, to avoid surprises when checking the style
of generated files (which happens in releases and in long-time support
branches), systematically skip generated files.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Allow check_names.py to detect declarations of the form:
enum some_enum_name {
This pattern has only just appeared due to code style correction, which
explains why the issue was not previously noticed.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
The script was parsing the output from `make lcov` to extract numbers and
calculate percentages. But everything including the percentages is already
present in the output of `make lcov`, just with a slightly different
presentation. So replace all this by a simple extraction of the relevant
lines from the output of `make lcov`.
This is more robust than the previous code, which relied on `tail -n4` to
extract relevant lines, which broke when `make lcov` started to emit one
extra line at the end.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In the basic/XXX=core test cases, use odd upper bounds, because the mod
version of random() only supports odd upper bounds (the upper bound is a
modulus and the mod modules only support odd moduli).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When including <test/bignum_helpers.h>, the library/ directory now needs to
be on the include path.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Move bignum-related helper functions to their own files under tests/include
and tests/src. The primary motivation is that a subsequent commit will make
bignum_helpers.h include library/bignum*.h, but we want to be able to
include <test/helpers.h> without having the library directory on the include
path (we do this in some programs under programs/ intended for testing).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
mpi_core_random_basic and mpi_random_values must generate the same random
sequences in order to get the expected test coverage (where we know we'll
hit certain numbers of retries). Facilitate this by defining the RNG seed
only once.
Fix the seed to explicitly list all 16 words of the key. This isn't strictly
required (missing initializer fields get the value zero), but it's clearer.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This is meant to adapt to the new library design in which
SHA224 and SHA256 can be built independently from each other.
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
This is meant to adapt to the new library design in which
SHA384 and SHA512 can be built independently from each other.
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
Previously the same test was repeated twice.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
Tests are refactored to generate separate cases for 32-bit and 64-bit
limbs using arch_split. Duplicate arguments and branching in the test
function is removed.
Signed-off-by: Werner Lewis <werner.lewis@arm.com>
This is not new, it had always been the case, just not documented.
Pointed out by depends.py pkalgs (again, now that restartable is part of
full).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
muladd() (restartable or not) is only available when at least one short
weirstrass curve is enabled.
Found by depends.py curves (now that restartable is part of full).
Also, document that restartable only work for short weierstrass curves
(actually unrelated, but this made me think of that).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The previous commit declared that some tests cases in ssl-opt.sh depend
on USE_PSA being disabled, which is the right thing to do.
We had a check that forbade that - it was mainly meant to prevent
accidental re-introduction of such dependencies after we cleaned up a
number of cases where it was not warranted, but already at the time that
was controversial [1]. Now it's preventing us from doing the right
thing, so let's just remove it.
[1]: https://github.com/Mbed-TLS/mbedtls/pull/5742#discussion_r855112412
See also https://github.com/Mbed-TLS/mbedtls/pull/5907/ which also
removes this for a similar reason.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This fixes the two failures in test_suite_x509parse when both
ECP_RESTARTABLE and USE_PSA_CRYPTO are enabled.
The failure happened because the operation is dispatched to PSA when
restart is disabled (max_ops == 0).
Previously it was correct for this test function not to initialize PSA,
because it depends on ECP_RESTARTABLE which used to conflict with
USE_PSA_CRYPTO, but that's no longer the case.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The random-in-nrange test code has auxiliary functions that are common to all
the interfaces (core, mod_raw (upcoming), mod (upcoming), legacy), and does
some differential testing to check that all the layers consume the RNG in
the saame way. Test them all in the same test suite.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Run the main test suites after running code style correction to check
that code style correction does not break these tests.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
It might not be obvious that this option goes beyond adding new
functions, but also automagically modifies the behaviour of TLS
in some circumstances. Moreover, the exact modifications and
circumstances were not documented anywhere outside the ChangeLog.
Fix that.
While at it, adjust the test that checks no restartable behaviour with
other key exchanges, to use a key exchange that allows cert-based client
authentication so that we can check that this is not restartable either.
We don't have any automated test checking that the server is never
affected. That would require adding an ec_max_ops command-line option to
ssl_server2 that never has any effect, just to check that it indeed
doesn't. I'm not sure that's worth it. I tested manually and could
confirm that the server never has restartable behaviour, even for the
parts that are shared between client and server such as cert chain
verification.
Note (from re-reading the code): all restartable behaviour is controlled
by the flag ssl->handshake->ecrs_enabled which is only client-side with
the ECDHE-ECDSA key exchange (TLS 1.2).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Test some cases where mbedtls_mpi_core_random() or mbedtls_mpi_random()
should return MBEDTLS_ERR_MPI_NOT_ACCEPTABLE. These test cases use a very
small range that makes the NOT_ACCEPTABLE case likely. The test code uses a
deterministic RNG whose implementation is in the test framework, so we know
that the tests will pass reproducibly unless the implementation the test
framework changes.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The test function mpi_random_many() is the main function for testing the
get-random-in-range function. It validates that the random generator's
output is within the desired range, and performs some basic statistical
checks including checking that small ranges are covered exhaustively.
Switch this function from testing mbedtls_mpi_random() to testing
mbedtls_mpi_core_random(). This does not reduce the test coverage of
mbedtls_mpi_random() because the same properties are now validated
indirectly via mpi_random_values() which checks that mbedtls_mpi_random()
and mbedtls_mpi_core_random() produce identical values for identical inputs.
As of this commit, mpi_random_many() still uses some legacy mpi functions
internally because the corresponding functions don't exist yet in core.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
For good cases, test that mbedtls_mpi_random() produces the same output as
mbedtls_mpi_core_random().
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Compare a single-limb MPI with a multi-limb MPI. This is rather ad hoc, but
will be useful for mbedtls_mpi_core_random.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Added mixed cases:
- server using opaque password, while client not
- client using opaque password, while server not
Added a test with mismatched passwords in case both server and
client are using opaque passwords (the same test was already
present for the non-opaque case)
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
This is already covered by other already existing cases such as
"component_test_full_cmake_gcc_asan" which build with
"config.py full" and run all "ssl-opt.sh" test cases.
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
The commit "Preserve line breaks in comments before test functions"
only handled block comments. This commit handles line comments.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When testing under Valgrind for constant flow, skip test suites that don't
have any constant-flow annotations, since the testing wouldn't do anything
more that testing with ordinary Valgrind (component_test_valgrind and
component_test_valgrind_psa). This is a significant time saving since
testing with Valgrind is very slow.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This is the first step in arranging that functions from constant_time.c are
tested in test_suite_constant_time.function.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
These are very CPU-intensive, so make it easy to skip them. And conversely,
make it easy to run them without the growing body of SSL tests.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
All builds using ASAN_CFLAGS were with Asan but no optimisation, making
them particularly slow. Indeed, we were overwriting CFLAGS which
defaults to -O2 and not using any -O in the replacement. (CMake already has
optimisations on with ASan.)
While at it, also remove -Wall -Wextra which are redundant as they are
already part of WARNING_CFLAGS which we are not overwriting.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
If the variable SKIP_TEST_SUITES is not defined with -D, but is defined
in an environment variable, tell cmake to get it from there.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Require equality for the number of limbs in the modulus and the residue.
This makes these functions consistent with residue_setup().
Signed-off-by: Janos Follath <janos.follath@arm.com>
The value was overwritten and the length wasn't used either. This latter
could have lead to a buffer overflow as well.
Signed-off-by: Janos Follath <janos.follath@arm.com>
Test macros have goto instructions to the end where everything is freed.
We need to call init before that happens to make calling free functions
safe.
Signed-off-by: Janos Follath <janos.follath@arm.com>
The external representation before included more than just endianness
(like reading in Mongtomery curve scalars or converting hashes to
numbers in a standard compliant way).
These are higher level concepts and are out of scope for Bignum and for
the modulus structure.
Signed-off-by: Janos Follath <janos.follath@arm.com>
The external representation before included more than just endianness
(like reading in Mongtomery curve scalars or converting hashes to
numbers in a standard compliant way).
These are higher level concepts and are out of scope for Bignum and for
the modulus structure.
Passing endianness as a parameter is a step towards removing it from the
modulus structure.
Signed-off-by: Janos Follath <janos.follath@arm.com>
The external representation before included more than just endianness
(like reading in Mongtomery curve scalars or converting hashes to
numbers in a standard compliant way).
These are higher level concepts and are out of scope for Bignum and for
the modulus structure.
Passing endianness as a parameter is a step towards removing it from the
modulus structure.
Signed-off-by: Janos Follath <janos.follath@arm.com>
In theory we could allow residues to have more allocated limbs than the
modulus, but we might or might not need it in the end.
Go for the simpler option for now and we can extend it later if we
really need it.
Signed-off-by: Janos Follath <janos.follath@arm.com>
We want to make sure that the value has at least as many limbs allocated
as the modulus as we need this to be able to do any operations in
constant time.
An invariant of the API is that the residue values are canonical, make
sure that the residue is compared to the entire modulus.
Signed-off-by: Janos Follath <janos.follath@arm.com>
This patch adjusts the I/O methods and the tests.
Documentation has also been updated to be more clear.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
This patch adjusts the logic of the size checking of the method,
and refactors the tests. Documentation has also been updated.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
This patch is inverting the input type checking logic in the method,
in order to ensure that residue < modulus.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
This patch adds the following tests for the high levet IO api:
* mpi_mod_io_neg
* mpi_mod_io
Manually generated test data has also been included.
Signed-off-by: Minos Galanakis <minos.galanakis@arm.com>
CID is now enabled in the default config (as well as full), so it's
already tested in numerous all.sh components, not need to add one for
that.
We need a component for the legacy/compat option though as it's never
enabled in existing components. So, keep that one, but adjust the name
and fix a typo in a message.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
We don't need to have two copies of the test with one of them depending
on legacy/compat CID: we can have just one copy, but make sure we run
ssl-opt.sh both in a build with standard CID and in a build with
legacy/compat - that's the job of all.sh (see next commit).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Per suggestion from Manuel, I removed this redundant test.
Signed-off-by: Hannes Tschofenig <hannes.tschofenig@arm.com>
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Added also extra text.
Signed-off-by: Hannes Tschofenig <hannes.tschofenig@arm.com>
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
OpenSSL may be configured to support features such as cipher suites or
protocol versions that are disabled by default. Enable them all: we're
testing, we don't care about enabling insecure stuff. This is not needed
with the builds of OpenSSL that we're currently using on the Jenkins CI, but
it's needed with more recent versions such as typically found on developer
machines, and with future CI additions.
The syntax to do that was only introduced in OpenSSL 1.1.0; fortunately we
don't need to do anything special with earlier versions.
With OpenSSL 1.1.1f on Ubuntu 20.04, this allows SHA-1 in certificates,
which is still needed for a few test cases in ssl-opt.sh. Curiously, this is
also needed for the cipher suite TLS-DHE-PSK-WITH-ARIA-128-GCM-SHA256 (and
no other, including other DHE-PSK or ARIA cipher suites).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
psa_cipher_encrypt() and psa_cipher_decrypt() sometimes add a zero offset to
a null pointer when the cipher does not use an IV. This is undefined
behavior, although it works as naively expected on most platforms. This
can cause a crash with modern Clang+ASan (depending on compiler optimizations).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The test cases aim to mirror the legacy function, but needed the some
cases to be removed because:
- Null representation is not valid in core
- There are no negative numbers in core
- Bignum core doesn't do parameter checking and there are no promises for
even N
The _size variant of the test has been removed as bignum core doesn't do
parameter checking and there is no promises for inputs that are larger
than MBEDTLS_MPI_MAX_SIZE.
Signed-off-by: Janos Follath <janos.follath@arm.com>
Now that we have a function that calls
mbedtls_mpi_core_ct_uint_table_lookup(), the compiler won't complain if
we make it static.
Signed-off-by: Janos Follath <janos.follath@arm.com>
- external errors are now checked in the specified point. If the
same error happens in another line, then this is not valid and
the test fails
- fixed some inconsistency in which injected error codes were not
taken from the data file. Now all the expected error code are
read from the data file
- added a couple of defines to shrink the code
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
As for ecjpake_setup(), now the test function can handle:
- "external" errors, through parameters set by the data file
- "internal" ones, through enums which inject ad-hoc failures
Similarly also ecjpake_rounds() can handle both type of errors,
but right now there's no erroneous case in the associated ".data"
file.
In both cases, after an error the current test is terminated.
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
Eventually we want it to be enabled by default
when TLS 1.3 is enabled but currently the
feature is on development thus it should not be
enabled by default.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Normally we need all the combinations, unique combinations make sense
only if the operation is commutative.
No changes to generated tests.
Signed-off-by: Janos Follath <janos.follath@arm.com>
The class BaseTarget served two purposes:
- track test cases and target files for generation
- provide an abstract base class for individual test groups
Splitting these allows decoupling these two and to have further common
superclasses across targets.
No intended change in generated test cases.
Signed-off-by: Janos Follath <janos.follath@arm.com>
Add a test case that would fail if all line comments were parsed before
block comments, and a test case that would fail if all block comments were
parsed before line comments.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The value of the first sent signature algorithm is overwritten.
This test forces only a single algorithm to be sent and then
validates that the client received such algorithm.
04 03 is the expected value for SECP256R1_SHA256.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Now when operation holds pointer to dynamically allocated buffer for password key we can't do copy of the operation object in test instead we need to re-initialize operation object after error.
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
Looking for the .data file doesn't work in out-of-tree builds. Use the
.datax file instead. `make clean` removes all .datax files, so this resolves
the issue of executables not present on the current branch being left behind
after a branch change followed by a `make clean`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When x is the most negative value of a two's complement type,
`(unsigned_type)(-x)` has undefined behavior, whereas `-(unsigned_type)x`
has well-defined behavior and does what was intended.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Although negative zero is officially unsupported, we've had bugs related to
it in the past. So do test functions with a negative zero input.
There will likely be cases where we don't want to accept negative zero as if
it was valid, because it's too hard to handle. We'll add exceptions on a
case by case basis.
For the functions that are currently tested by the generated tests, the new
test cases pass.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The bignum module does not officially support "negative zero" (an
mbedtls_mpi object with s=-1 and all limbs zero). However, we have a
history of bugs where a function that should produce an official
zero (with s=1), produces a negative zero in some circumstances. So it's
good to check that the bignum functions are robust when passed a negative
zero as input. And for that, we need a way to construct a negative zero
from test case arguments.
There are checks that functions don't produce negative zeros as output in
the test suite. Skip those checks if there's a negative zero input: we
don't want functions to _create_ negative zeros, but we don't mind if
they _propagate_ negative zeros.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This is to ensure that the MbedTLS based implementation of EC-JPAKE
is compatible with the PSA crypto one
Signed-off-by: Valerio Setti <vsetti@baylibre.com>
