Commit graph

2963 commits

Author SHA1 Message Date
Jaeden Amero
252faff19f
Merge pull request #2966 from dgreen-arm/fix-pylint-warnings
Sideport: Fix some pylint warnings
2019-12-20 16:07:07 +00:00
Darryl Green
fb5faa2582 Fix some pylint warnings
Add docstrings where they were missing and fix a too-long line
2019-12-20 15:14:59 +00:00
Janos Follath
c673c2cd44 Break up the ssl_mock_tcp unit test
Break the test up to three different tests for the sake of
better readability and maintainability.
2019-12-09 09:10:21 +00:00
Janos Follath
3766ba50de Add non-blocking mock TCP callbacks to SSL tests 2019-12-09 09:10:21 +00:00
Janos Follath
031827feba Add mbedtls_mock_socket to SSL unit tests
In a unit test we want to avoid accessing the network. To test the
handshake in the unit test suite we need to implement a connection
between the server and the client. This socket implementation uses
two ring buffers to mock the transport layer.
2019-12-09 09:10:14 +00:00
Janos Follath
6264e66ba4 Add mbedtls_test_buffer to SSL unit tests
In a unit test we want to avoid accessing the network. To test the
handshake in the unit test suite we need to implement a connection
between the server and the client. This ring buffer implementation will
serve as the said connection.
2019-12-06 07:23:49 +00:00
Janos Follath
512fe9673f Fix test assert macro calls
The assert() macro in test is not available anymore. It is superseeded
by TEST_HELPER_ASSERT().
2019-11-29 10:13:32 +00:00
Gilles Peskine
6608e71032 Change ASSERT_ALLOC to take a size in elements, not bytes
`ASSERT_ALLOC(p, length)` now allocates `length` elements, i.e.
`length * sizeof(*p)` bytes.
2019-11-29 10:13:32 +00:00
Gilles Peskine
28405300ee New macro ASSERT_ALLOC to allocate memory in tests
The new macro ASSERT_ALLOC allocates memory with mbedtls_calloc and
fails the test if the allocation fails. It outputs a null pointer if
the requested size is 0. It is meant to replace existing calls to
mbedtls_calloc.
2019-11-29 10:13:32 +00:00
Jaeden Amero
61c8a371e0
Merge pull request #2836 from hanno-arm/x509_crt_policies_tests
X.509: Enhance negative testing for CertificatePolicy extension
2019-11-20 15:45:57 +00:00
Gilles Peskine
43259ce31e Remove unused test data file
Since "Remove component designed to test MAX_SIGNATURE_SIZE",
secp521r1_prv.der is no longer used.

ec_521_prv.pem can be used for the same purpose.
2019-11-14 19:14:40 +01:00
Gilles Peskine
2e86a206f7 Remove component designed to test MAX_SIGNATURE_SIZE
MBEDTLS_PK_SIGNATURE_MAX_SIZE is tested in Mbed Crypto. Its effect on
Mbed TLS is also tested via the X.509 tests. The case of
MBEDTLS_MPI_MAX_SIZE < MBEDTLS_ECDSA_MAX_LEN, for which this component
was added as a regression test, is covered by config-suite-b.h which
is tested via test-ref-configs.pl.
2019-11-13 15:32:11 +01:00
Gilles Peskine
ce35cb3cc7 'make test' must fail if Asan fails
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
  cleanup failed, in particular if a sanitizer reported a memory leak.

Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.

Fix ARMmbed/mbed-crypto#303
2019-10-21 20:10:21 +02:00
Gilles Peskine
5ca393f7b8 Asan make builds: avoid sanitizer recovery
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
2019-10-21 20:10:12 +02:00
Gilles Peskine
2558d30f93 Use UBsan in addition to Asan with 'make test'
When building with make with the address sanitizer enabled, also
enable the undefined behavior sanitizer.
2019-10-21 20:09:22 +02:00
Gilles Peskine
8fd5942229 Unify ASan options in make builds
Use a common set of options when building with Asan without CMake.
2019-10-21 20:09:13 +02:00
Jaeden Amero
719ae74253
Merge pull request #2884 from AndrzejKurek/iotssl-2886-fix-parallel-make-jobs-2
Fix parallel make jobs for shared target
2019-10-18 14:18:55 +01:00
Andrzej Kurek
232e8f959a Enable parallel shared target tests
Now that the dependency issues for shared target are fixed, the feature can be
enabled in tests.
2019-10-07 09:24:43 -04:00
Andrzej Kurek
df2cd9e14f Makefiles: move the dependencies block to be after DLEXT definition
Having it before them resulted in incomplete dependency names, always ending
with a period.
2019-10-07 09:24:22 -04:00
Gilles Peskine
f70d3eb43a Uncomment X509 test that now works
The test failed due to an ASN.1 bug that the latest crypto submodule
update fixed.
2019-10-04 19:24:37 +02:00
Jaeden Amero
74692aeb8c Merge remote-tracking branch 'origin/pr/2488' into development
* origin/pr/2488:
  Change X.509 test cases to not rely on asn1parse limitations
2019-10-02 18:01:57 +01:00
Jaeden Amero
230b87a1ea Merge remote-tracking branch 'origin/pr/2843' into development
* origin/pr/2843: (26 commits)
  Make hyperlink a hyperlink in every markdown flavor
  Update the crypto submodule to be the same as development
  Document test case descriptions
  Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
  ssl-opt.sh: Fix some test case descriptions
  Reject non-ASCII characters in test case descriptions
  Process input files as binary
  Factor description-checking code into a common function
  Fix cosmetic error in warnings
  Fix regex matching run_test calls in ssl-opt.sh
  all.sh: run check-test-cases.py
  Better information messages for quick checks
  Fix configuration short name in key-exchanges.pl
  Make test case descriptions unique
  New test script check-test-cases.py
  Document the test outcome file
  Create infrastructure for architecture documents in Markdown
  all.sh --outcome-file creates an outcome file
  Set meaningful test configuration names when running tests
  ssl-opt: remove semicolons from test case descriptions
  ...
2019-10-02 18:01:32 +01:00
Gilles Peskine
717cd76e8a Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
Since components run in the main process, unsetting
MBEDTLS_TEST_OUTCOME_FILE unset it in subsequent components as well.
To avoid this, save and restore the value.

(Making each component run in a subshell would be a better solution,
but it would be a much bigger change.)
2019-09-27 20:24:33 +02:00
Gilles Peskine
0d8b86a131 ssl-opt.sh: Fix some test case descriptions
Fix copypasta in some test cases with
MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES enabled.

Add unique suffix to the two
"DTLS fragmenting: proxy MTU: auto-reduction" test cases.
2019-09-24 19:21:20 +02:00
Gilles Peskine
57870e8c67 Reject non-ASCII characters in test case descriptions
Don't require that all the tools we use to process test outcomes are
Unicode-clean.
2019-09-24 19:21:20 +02:00
Gilles Peskine
f12ad58a1d Process input files as binary
Don't die if there's a non-ASCII character and we're running in an
ASCII environment.
2019-09-24 19:21:20 +02:00
Gilles Peskine
32b9421f12 Factor description-checking code into a common function
Behavior change: some error messages are slightly different.
2019-09-24 19:21:20 +02:00
Gilles Peskine
283df2e90c Fix cosmetic error in warnings 2019-09-24 19:21:20 +02:00
Gilles Peskine
168858f52d Fix regex matching run_test calls in ssl-opt.sh
No descriptions were processed before due to bugs in the regex.

Support \" inside double-quoted strings.
2019-09-24 19:21:19 +02:00
Gilles Peskine
895868bc82 all.sh: run check-test-cases.py 2019-09-24 19:21:19 +02:00
Gilles Peskine
600bb694ac Better information messages for quick checks
Call them "check" rather than "test" to distinguish them from tests
that build and run code, and for consistency with the component names.
2019-09-24 19:21:19 +02:00
Gilles Peskine
a9478bab08 Fix configuration short name in key-exchanges.pl
This is testing with $kex, not without $kex, so use $kex, not "-$kex".

In test-ref-configs.pl, use $conf rather than "$conf". This is purely
a matter of Perl coding style.
2019-09-24 19:21:19 +02:00
Gilles Peskine
7a020f3d10 Make test case descriptions unique
Remove one test case which was an exact duplicate.

Tweak the description of two test cases that had the same description.
2019-09-24 19:21:19 +02:00
Gilles Peskine
ba94b58127 New test script check-test-cases.py
This script checks test case descriptions in test_suite_*.data and
ssl-opt.sh.

It reports the following issues:
* Error: forbidden character in a test case description.
* Error: Duplicate test description.
* Warning: Test description is too long.
2019-09-24 19:21:19 +02:00
Gilles Peskine
67ffdafde6 all.sh --outcome-file creates an outcome file
By default, remove the outcome file before starting. With
--append-outcome, append to the existing outcome file if there is one.
2019-09-24 19:21:19 +02:00
Jaeden Amero
c21a9f04f9 Merge remote-tracking branch 'origin/pr/2765' into development
* origin/pr/2765: (28 commits)
  Add set+get tests
  Consolidate tests for set with/without values
  config.py testing: also test the get command
  Compatibility redirect: add copyright notice
  Compatibility redirect: if python3 is not available, try python
  Fix config.py output when a symbol has acquired or lost a value
  Remove redundant test case
  cmake: update interpreter requirement for the test suite generator
  cmake: fix Python requirement
  Test script for config.py
  Documentation improvements
  Fix "#define ... not found" error when using the default file name
  Fix "--force set" without a value sneaking a None in
  Fix --force requiring an argument
  Fix Config.unset() making the name known
  Also search config.h near the script
  Report an error if switching to Python fails
  Fix 'config.py set' without --force
  Fix encoding errors
  Print help when invoked with no arguments
  ...
2019-09-23 17:27:44 +01:00
Gilles Peskine
16a25e005d Add set+get tests
The tests were not covering get for a symbol with a value. No symbol
has an uncommented value in the default config.h. (Actually there's
_CRT_SECURE_NO_DEPRECATE, but that's a bit of a hack that this script
is not expected to handle, so don't use it).

Add tests of "get FOO" after "set FOO" and "set FOO value", so that we
have coverage for "get FOO" when "FOO" has a value.
2019-09-19 12:19:24 +02:00
Gilles Peskine
fd7ad33ee9 Consolidate tests for set with/without values
We currently test setting a symbol with a value even if it didn't
originally had one and vice versa. So there's no need to have separate
lists of symbols to test with. Just test everything we want to test
with each symbol.
2019-09-19 12:18:23 +02:00
Gilles Peskine
9004a1768b Set meaningful test configuration names when running tests
Set MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION to meaningful
values in all.sh. These environment variables are used when
writing an outcome file, which happens if MBEDTLS_TEST_OUTCOME_FILE is
also set.

When running one of the try-multiple-configuration scripts, set
MBEDTLS_TEST_CONFIGURATION to a value that uniquely describes the
configuration.
2019-09-18 17:44:29 +02:00
Gilles Peskine
654bab7635 ssl-opt: remove semicolons from test case descriptions
Don't use semicolons in test case descriptions. The test outcome file
is a semicolon-separated CSV file without quotes to keep things
simple, so fields in that file may not contain semicolons.
2019-09-18 17:44:29 +02:00
Gilles Peskine
560280b17d Test outcome file support: ssl-opt.sh
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set, then for
each test case, write a line to the file with the given name, of the
form

    PLATFORM;CONFIGURATION;ssl-opt;TEST CASE DESCRIPTION;PASS/FAIL/SKIP;CAUSE

PLATFORM and CONFIGURATION come from the environment variables
MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION. If these
variables are unset, the script uses some easily-calculated values.
2019-09-18 17:44:29 +02:00
Gilles Peskine
51dcc24998 Test outcome file support: test suites
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set, then for
each test case, write a line to the file with the given name, of the
form

    PLATFORM;CONFIGURATION;TEST SUITE;TEST CASE DESCRIPTION;PASS/FAIL/SKIP;CAUSE

PLATFORM and CONFIGURATION come from the environment variables
MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION.

Errors while writing the test outcome file are not considered fatal,
and are not reported except for an error initially opening the file.
This is in line with other write errors that are not checked.
2019-09-18 17:44:29 +02:00
Gilles Peskine
47b7540fec Give a type name to test_info
Make it possible to pass test_info around rather than always refer to the
global variable.
2019-09-18 17:44:29 +02:00
Gilles Peskine
31fccc80a5 Fix typo in message 2019-09-18 17:44:29 +02:00
Gilles Peskine
3c1c8ea3e7 Prefer unsigned types for non-negative numbers
Use size_t for some variables that are array indices.
Use unsigned for some variables that are counts of "small" things.
2019-09-18 17:44:29 +02:00
Gilles Peskine
a708dae94b Add comment to help syntax highlighting in editors 2019-09-18 17:44:29 +02:00
Jaeden Amero
914a5071b4 Bump Mbed TLS version to 2.19.1 2019-09-18 13:42:36 +01:00
Hanno Becker
6dfa665398 X.509: Add numerous negative parsing tests for CertificatePolicy ext
This commit adds multiple test cases to the X.509 CRT parsing test suite
exercising the stack's behaviour when facing CertificatePolicy extensions
that are malformed for a variety of reasons. It follows the same scheme
as in other negative parsing tests: For each ASN.1 component, have test
cases for (a) unexpected tag, (b) missing length, (c) invalid length
encoding, (d) length out of bounds.
2019-09-17 13:10:42 +01:00
Hanno Becker
c15ff98455 X.509: Adapt negative parsing test for no data in CrtPolicy ext
This commit modifies the test

   X509 CRT ASN1 (TBSCertificate v3, inv CertificatePolicies, data missing)

which exercises the behaviour of the X.509 CRT parser when facing a
CertificatePolicy extension with empty data field.

The following adaptations are made:
- The subject ID and issuer ID are modified to have length 0.
  The previous values `aa` and `bb` are OK, but a generic ASN.1
  parser will try to interpret them as ASN.1 tags and fail. For
  maintainability, it's therefore better to use something that
  can be parsed as ASN.1, and an empty ID is the easiest solution
  here.
- The TBS part of the certificate wasn't followed by signature
  algorithm and signature fields, which makes the test incompatible
  with future changes swapping to breadth-first parsing of
  certificates.
2019-09-17 13:10:42 +01:00
Hanno Becker
7b8330a9c7 X.509: Move negative tests for CertificatePolicy parsing
This commit moves the X.509 negative parsing tests for the
CertificatePolicy extension to the place where negative
testing of other extensions happens.
2019-09-17 13:10:42 +01:00