Commit graph

19 commits

Author SHA1 Message Date
Gilles Peskine
739e08a68d Keystore format stability test strategy
Initial revision.

Save-compare-load approach: the test case data contains attributes of
the object under test and the expected file content. Create the
object, save it, check that the file has the expected content, load
the file and check that the new object has the expected attributes.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-14 18:51:47 +01:00
Gilles Peskine
fb4f933f8e Rename Python scripts to use '_' and not '-'
You can't import a Python script whose name includes '-'.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 14:22:06 +02:00
Gilles Peskine
24ba42cef7 Fix explanation of rules for function substitution
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:07:42 +02:00
Gilles Peskine
688f6cc591 There are test programs, not just unit tests
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:07:40 +02:00
Gilles Peskine
5925183b8a Fix explanation of PSA_DONE
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:06:37 +02:00
Gilles Peskine
2182585776 Introduction: present the top-level sections
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:03:33 +02:00
Gilles Peskine
d04b9ed7dd Spelling
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:03:24 +02:00
Gilles Peskine
fa51820e39 Expand the rule for internal functions exposed for tests only
Clarify that using a header in library/ rather than include/ for
internal functions is a rule, not just a possibility.

As suggested by Manuel, state a rule for functions that need to be
static for best optimization but that we want to unit-test.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:38 +01:00
Gilles Peskine
4b7279e5d9 Write up some solutions
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:31 +01:00
Gilles Peskine
dff10c773b Add a note that TLS requires further consideration
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:22 +01:00
Gilles Peskine
ab4b9b4165 New test strategy document: invasive testing
Evaluate possible approaches for invasive testing.

State some rules.

This commit was originally written for Mbed Crypto only.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:15 +01:00
Gilles Peskine
9afbfdc833 Merge remote-tracking branch 'myfork-public/development' into merge-crypto-development-20191115
First deal with deleted files.

* Files deleted by us: keep them deleted.
* Files deleted by them, whether modified by us or not: keep our version.

```
git rm $(git status -s | sed -n 's/^DU //p')
git reset -- $(git status -s | sed -n 's/^D  //p')
git checkout -- $(git status -s | sed -n 's/^ D //p')
git add -- $(git status -s | sed -n 's/^UD //p')
```

Individual files with conflicts:

* `3rdparty/everest/library/Hacl_Curve25519_joined.c`: spurious conflict because git mistakenly identified this file as a rename. Keep our version.
* `README.md`: conflict due to their change in a paragraph that doesn't exist in our version. Keep our version of this paragraph.
* `docs/architecture/Makefile`: near-identical additions. Adapt the definition of `all_markdown` and include the clean target.
* `doxygen/input/docs_mainpage.h`: conflict in the version number. Keep our version number.
* `include/mbedtls/config.h`: two delete/modify conflicts. Keep the removed chunks out.
* `library/CMakeLists.txt`: discard all their changes as they are not relevant.
* `library/Makefile`:
    * Discard the added chunk about the crypto submodule starting with `INCLUDING_FROM_MBEDTLS:=1`.
    * delete/modify: keep the removed chunk out.
    * library build: This is almost delete/modify. Their changes are mostly not applicable. Do keep the `libmbedcrypto.$(DLEXT): | libmbedcrypto.a` order dependency.
    * `.c.o`: `-o` was added on both sides but in a different place. Change to their place.
* `library/error.c`: to be regenerated.
* `library/version_features.c`: to be regenerated.
* `programs/Makefile`: Most of the changes are not relevant. The one relevant change is in the `clean` target for Windows; adapt it by removing `/S` from our version.
* `programs/test/query_config.c`: to be regenerated.
* `scripts/config.py`: added in parallel on both sides. Keep our version.
* `scripts/footprint.sh`: parallel changes. Keep our version.
* `scripts/generate_visualc_files.pl`: one delete/modify conflict. Keep the removed chunks out.
* `tests/Makefile`: discard all of their changes.
* `tests/scripts/all.sh`:
    * `pre_initialize_variables` add `append_outcome`: add it.
    * `pre_initialize_variables` add `ASAN_CFLAGS`: already there, keep our version.
    * `pre_parse_command_line` add `--no-append-outcome`: add it.
    * `pre_parse_command_line` add `--outcome-file`: add it.
    * `pre_print_configuration`: add `MBEDTLS_TEST_OUTCOME_FILE`.
    * Several changes in SSL-specific components: keep our version without them.
    * Several changes where `config.pl` was changed to `config.py` and there was an adjacent difference: keep our version.
    * Changes regarding the inclusion of `MBEDTLS_MEMORY_xxx`: ignore them here, they will be normalized in a subsequent commit.
    * `component_test_full_cmake_gcc_asan`: add it without the TLS tests.
    * `component_test_no_use_psa_crypto_full_cmake_asan`: keep the fixed `msg`, discard other changes.
    * `component_test_memory_buffer_allocator_backtrace`, `component_test_memory_buffer_allocator`: add them without the TLS tests.
    * `component_test_m32_everest`: added in parallel on both sides. Keep our version.
* `tests/scripts/check-names.sh`, `tests/scripts/list-enum-consts.pl`, `tests/scripts/list-identifiers.sh`, ``tests/scripts/list-macros.sh`: discard all of their changes.
* `tests/scripts/test-ref-configs.pl`: the change in the conflict is not relevant, so keep our version there.
* `visualc/VS2010/*.vcxproj`: to be regenerated.

Regenerate files:

```
scripts/generate_visualc_files.pl
git add visualc/VS2010/*.vcxproj
scripts/generate_errors.pl
git add library/error.c
scripts/generate_features.pl
git add library/version_features.c
scripts/generate_query_config.pl
git add programs/test/query_config.c
```

Rejected changes in non-conflicting files:

* `CMakeLists.txt`: discard their addition which has already been side-ported.
* `doxygen/mbedtls.doxyfile`: keep the version number change. Discard the changes related to `../crypto` paths.

Keep the following changes after examination:

* `.travis.yml`: all of their changes are relevant.
* `include/mbedtls/error.h`: do keep their changes. Even though Crypto doesn't use TLS errors, it must not encroach on TLS's allocated numbers.
* `tests/scripts/check-test-cases.py`: keep the code dealing with `ssl-opt.sh`. It works correctly when the file is not present.
2019-11-15 11:47:14 +01:00
Gilles Peskine
7334462278 Make hyperlink a hyperlink in every markdown flavor 2019-10-01 10:36:10 +02:00
Gilles Peskine
e94bc87ebe Document test case descriptions 2019-09-27 20:24:33 +02:00
Gilles Peskine
508caf528a Document the test outcome file 2019-09-24 19:21:19 +02:00
Gilles Peskine
8b193c10ca Check inputs too 2019-09-05 19:37:01 +02:00
Gilles Peskine
545c28bf70 Fix URL of ATECC driver 2019-09-04 19:41:16 +02:00
Gilles Peskine
92bcfdbb66 Write secure element driver interface test strategy 2019-09-04 19:26:50 +02:00
Gilles Peskine
b26c8d8bb4 Create a driver interface test strategy document
Just the structure for now, no actual content.
2019-09-04 19:26:17 +02:00