Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.
The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:
* `.github/issue_template.md`: completely different. We may want to reconcile
them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
relevant except as a stopgap as mbed-crypto did not have its own product
versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
but the unremoval step restored an old version.
Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```
Resolve the remaining conflicts:
* `library/CMakeLists.txt`:
* Keep the TLS definition of `src_crypto`
* `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
`include` and `crypto/include` in `target_include_directories`, all with
version `2.21.0`.
* `programs/Makefile`:
* Reconcile the APPS lists (add/add from a differently-formatted common
ancestor): insert the `psa/*` from crypto into the tls list.
* Keep the `fuzz` target defined only in tls version.
* Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
stuff for the `clean` target.
* `scripts/config.py`:
* `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
* `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
version in crypto. Take the tls version.
* `component_test_malloc_0_null` and more: take
`component_test_malloc_0_null` from crypto (with `config.py` rather than
`config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
add the call to `ssl-opt.sh` from tls. Take the other components from
crypto.
With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.
There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.
Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
This reverts commit 9afb2e9921.
Conflicts:
* include/CMakeLists.txt
* "Make config.h available" comment: there has been a change
adjacent to where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* compat.sh: there has been a change immediately before where it was
removed. Just re-add what was removed.
This reverts commit 120d571e8e.
Conflicts:
* tests/CMakeLists.txt:
* target_include_directories: the instruction whose addition is to
be reverted has changed. Remove what is there now.
This reverts commit b478bb6ddb.
Conflicts:
* tests/CMakeLists.txt: revert the introduction of exe_name, but keep
the addition of ${CMAKE_SOURCE_DIR}/crypto/library/ to
target_include_directories.
This reverts commit d832f187f7.
Conflicts:
* CMakeLists.txt:
* USE_PKCS11_HELPER_LIBRARY: there has been a change immediately before
where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* USE_PKCS11_HELPER_LIBRARY: there has been a change immediately before
where it was removed. Just re-add what was removed.
This reverts commit d874a1fd14.
Conflicts:
* CMakeLists.txt:
* ENABLE_ZLIB_SUPPORT: there has been a change immediately after
where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* ENABLE_ZLIB_SUPPORT: there has been a change immediately after
where it was removed. Just re-add what was removed.
The test suite generator has been a Python script for a long time,
but tests/CMakeLists.txt still looked for Perl. The reference to
PYTHON_INTERP only worked due to a call to find_package(PythonInterp)
in the toplevel CMakeLists.txt, and cmake would not have printed the
expected error message if python was not available.
The test suite generator has been a Python script for a long time,
but tests/CMakeLists.txt still looked for Perl. The reference to
PYTHON_INTERP only worked due to a call to find_package(PythonInterp)
in the toplevel CMakeLists.txt, and cmake would not have printed the
expected error message if python was not available.
Add self-contained ASN.1 parsing tests, so that ASN.1 parsing is not
solely tested through X.509 and TLS.
The tests cover every function and almost complete line coverage in
asn1parse.c.
A few test cases containing negative and edge case INTEGER values are
deliberately deactivated because the historical library behavior is at
odds with official specifications, but changing the behavior might
break interoperability.
Other than that, these tests revealed a couple of minor bugs which
will be fixed in subsequent commits.
Conflict resolution:
* `scripts/config.pl`:
Take the exclusion of `MBEDTLS_PSA_CRYPTO_SE_C` from the API branch.
Take the removal of `MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C` (obsolete) from
the development branch.
* `tests/scripts/all.sh`:
Multiple instances of factoring a sequence of `config.pl` calls into
a mere `config.pl baremetal` in the development branch, and a change in
the composition of `baremetal` in the API branch. In each case, take the
version from development.
* `tests/suites/test_suite_psa_crypto_slot_management.function`:
A function became non-static in development and disappeared in the API
branch. Keep the version from the API branch. Functions need to be
non-static if they're defined but unused in some configurations,
which is not the case for any function in this file at the moment.
* `tests/suites/test_suite_psa_crypto.function`:
Consecutive changes in the two branches, reconciled.
When building Mbed Crypto when including it via CMake's
`add_subdirectory()`, the tests are also built by default. This means
all headers the tests need must be public, in order for the build of the
tests to have access to the headers.
Remove use of CMAKE_SOURCE_DIR in case mbedtls is built from within
another CMake project. Define MBEDTLS_DIR to ${CMAKE_CURRENT_SOURCE_DIR}
in the main CMakeLists.txt file and refer to that when defining target
include paths to enable mbedtls to be built as a sub project.
Fixes https://github.com/ARMmbed/mbedtls/issues/2609
Signed-off-by: Ashley Duncan <ashes.man@gmail.com>
Signed-off-by: Jaeden Amero <jaeden.amero@arm.com>
Merge Mbed TLS at f790a6cbee into Mbed Crypto.
Resolve conflicts by performing the following:
- Reject changes to README.md
- Don't add crypto as a submodule
- Remove test/ssl_cert_test from programs/Makefile
- Add cipher.nist_kw test to tests/CMakeLists.txt
- Reject removal of crypto-specific all.sh tests
- Reject update to SSL-specific portion of component_test_valgrind
in all.sh
- Reject addition of ssl-opt.sh testing to component_test_m32_o1 in
all.sh
* tls/development: (87 commits)
Call mbedtls_cipher_free() to reset a cipher context
Don't call mbedtls_cipher_setkey twice
Update crypto submodule
Minor fixes in get certificate policies oid test
Add certificate policy oid x509 extension
cpp_dummy_build: Add missing header psa_util.h
Clarify comment mangled by an earlier refactoring
Add an "out-of-box" component
Run ssl-opt.sh on 32-bit runtime
Don't use debug level 1 for informational messages
Skip uncritical unsupported extensions
Give credit to OSS-Fuzz for #2404
all.sh: remove component_test_new_ecdh_context
Remove crypto-only related components from all.sh
Remove ssl_cert_test sample app
Make CRT callback tests more robust
Rename constant in client2.c
Document and test flags in x509_verify
Fix style issues and a typo
Fix a rebase error
...
Resolve conflicts actions:
- Reject path changes to config.h
- Reject submodule-related changes in build scripts (Makefile,
CMakeLists.txt)
- Add oid test suite to list of tests in tests/CMakeLists.txt,
rejecting any test filtering related changes (which TLS uses to avoid
duplicating crypto tests)
- Add legacy ECDH test to all.sh without including
all.sh tests that depend on SSL
Ensure tests pass when the submodule is used by updating the list of
crypto tests to include test_suite_oid in both tests/CMakeLists.txt and
tests/Makefile.
* origin/pr/2531:
Add changeLog entry
Add certificate policy of type any policy id
Some tests use internal-only header files, which are stored in the
'library' folder, and therefore need the library folder passed in on the
include path. For non-submoudle builds, this is set globally in the
top-level CMakeLists.txt file. For submodule builds, this is set through
target includes to a path only meaningful when Mbed Crypto is built as a
submodule.
Now that we have ITS over files, we no longer need a direct backend
for key storage over files. Remove psa_crypto_storage_file and its
tests.
Switch MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C and MBEDTLS_PSA_ITS_FILE_C on
by default. This preserves functionality and test coverage in the
default configuration, but forgets any key previously stored using the
file backend.
Test the crypto implementation via tests from the Mbed Crypto submodule
instead of at the Mbed TLS top level.
The version test is the only test that is tested from both TLS and
Crypto, despite being entirely in libmbedcrypto. This is because the
test data is code-gen'd from the version updating script and the version
between Mbed TLS and Mbed Crypto don't necessarily always agree. The
test data must come from the top level module, as only the top level
module will have test data that matches the expected version.
Prepend ".crypto" to tests that came from the crypto submodule. This
allows, when this project is used as a submodule, for tests with names
the same between the parent and this project when used as a submodule to
both be built and run.
The version test suite is duplicated between Mbed TLS and Mbed Crypto.
Use TLS's copy and not Crypto's copy when Crypto is used as a submodule
of TLS.
The version test is the only test that is tested from both TLS and
Crypto, despite being entirely in libmbedcrypto. This is because the
test data is code-gen'd from the version updating script and the version
between Mbed TLS and Mbed Crypto don't necessarily always agree. The
test data must come from the top level module, as only the top level
module will have test data that matches the expected version.
For Makefiles, enable overriding where includes can come from in order
to enable the parent module to set the include path. This allows the
parent module to specify that its config.h should be used, even when the
submodule when built standalone would use a different config.h.
For CMake, always look in the parent's include folder and our own. List
the parent's include folder first, so that preference is given to parent
include files.
* origin/development: (113 commits)
Update query_config.c
Fix failure in SSLv3 per-version suites test
Adjust DES exclude lists in test scripts
Clarify 3DES changes in ChangeLog
Fix documentation for 3DES removal
Exclude 3DES tests in test scripts
Fix wording of ChangeLog and 3DES_REMOVE docs
Reduce priority of 3DES ciphersuites
Fix unused variable warning in ssl_parse_certificate_coordinate()
Update the crypto submodule to a78c958
Fix ChangeLog entry to correct release version
Fix typo in x509write test data
Add ChangeLog entry for unused bits in bitstrings
Improve docs for named bitstrings and their usage
Add tests for (named) bitstring to suite_asn1write
Add new function mbedtls_asn1_write_named_bitstring()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
...
Resolve conflicts by performing the following.
- Take the upstream Mbed TLS ChangeLog verbatim.
- Reject changes to Makefiles and CMake that are related to using Mbed
Crypto as a submodule. It doesn't make sense to use Mbed Crypto as a
submodule of itself.
- Reject README changes, as Mbed Crypto has its own, different README.
- Reject PSA-related changes to config.h. We don't want to disable the
availability of the PSA Crypto API by default in the Mbed Crypto
config.h.
- Don't inadvertently revert dead code removal in
mbedtls_cipher_write_tag() which was added in f2a7529403 ("Fix
double return statement in cipher.c")
- Where Mbed Crypto already had some MBEDTLS_USE_PSA_CRYPTO code (from
past companion PRs) take the latest version from Mbed TLS which
includes integration with MBEDTLS_CHECK_PARAMS.
- Update the version of the shared library files to match what's
currently present in Mbed TLS.
- Reject removal of testing with PSA from config full tests.
- Resolve conflicts in test tests/suites/helpers.function, where both
Mbed Crypto and Mbed TLS both added documentation for TEST_ASSERT.
Combine text from both documentation efforts.
- Reject adding a submodule of ourselves.
- Reject addition of submodule tests in all.sh.
- Reject addition of submodule to library path in
tests/scripts/run-test-suites.pl.
- Avoid using USE_CRYPTO_SUBMODULE=1 in
component_test_use_psa_crypto_full_cmake_asan() in all.sh.
Additional changes to temporarily enable running tests:
ssl_srv.c and test_suite_ecdh use mbedtls_ecp_group_load instead of
mbedtls_ecdh_setup
test_suite_ctr_drbg uses mbedtls_ctr_drbg_update instead of
mbedtls_ctr_drbg_update_ret
With the build option SKIP_TEST_SUITES=..., the specified test suites
are built, but skipped when running tests. Usage:
make check SKIP_TEST_SUITES=timing,gcm
or
cmake -D SKIP_TEST_SUITES=timing,gcm ...
The list can be separated by any of space, comma or semicolon, and each
element can be a regular expression in ERE syntax except that "." stands
for itself. Skipping "foo" skips not only "foo" itself but also
any "foo.bar", but does not skip "foobar".
Allow use of persistent keys, including configuring them, importing and
exporting them, and destroying them.
When getting a slot using psa_get_key_slot, there are 3 scenarios that
can occur if the keys lifetime is persistent:
1. Key type is PSA_KEY_TYPE_NONE, no persistent storage entry:
- The key slot is treated as a standard empty key slot
2. Key type is PSA_KEY_TYPE_NONE, persistent storage entry exists:
- Attempt to load the key from persistent storage
3. Key type is not PSA_KEY_TYPE_NONE:
- As checking persistent storage on every use of the key could
be expensive, the persistent key is assumed to be saved in
persistent storage, the in-memory key is continued to be used.
Add new functions, psa_load_persistent_key(),
psa_free_persistent_key_data(), and psa_save_persistent_key(), for
managing persistent keys. These functions load to or save from our
internal representation of key slots. Serialization is a concern of the
storage backend implementation and doesn't abstraction-leak into the
lifetime management code.
An initial implementation for files is provided. Additional storage
backends can implement this interface for other storage types.
* development: (182 commits)
Change the library version to 2.11.0
Fix version in ChangeLog for fix for #552
Add ChangeLog entry for clang version fix. Issue #1072
Compilation warning fixes on 32b platfrom with IAR
Revert "Turn on MBEDTLS_SSL_ASYNC_PRIVATE by default"
Fix for missing len var when XTS config'd and CTR not
ssl_server2: handle mbedtls_x509_dn_gets failure
Fix harmless use of uninitialized memory in ssl_parse_encrypted_pms
SSL async tests: add a few test cases for error in decrypt
Fix memory leak in ssl_server2 with SNI + async callback
SNI + SSL async callback: make all keys async
ssl_async_resume: free the operation context on error
ssl_server2: get op_name from context in ssl_async_resume as well
Clarify "as directed here" in SSL async callback documentation
SSL async callbacks documentation: clarify resource cleanup
Async callback: use mbedtls_pk_check_pair to compare keys
Rename mbedtls_ssl_async_{get,set}_data for clarity
Fix copypasta in the async callback documentation
SSL async callback: cert is not always from mbedtls_ssl_conf_own_cert
ssl_async_set_key: detect if ctx->slots overflows
...
AES-XEX is a building block for other cryptographic standards and not yet a
standard in and of itself. We'll just provide the standardized AES-XTS
algorithm, and not AES-XEX. The AES-XTS algorithm and interface provided
can be used to perform the AES-XEX algorithm when the length of the input
is a multiple of the AES block size.
XTS mode is fully known as "xor-encrypt-xor with ciphertext-stealing".
This is the generalization of the XEX mode.
This implementation is limited to an 8-bits (1 byte) boundary, which
doesn't seem to be what was thought considering some test vectors [1].
This commit comes with tests, extracted from [1], and benchmarks.
Although, benchmarks aren't really nice here, as they work with a buffer
of a multiple of 16 bytes, which isn't a challenge for XTS compared to
XEX.
[1] http://csrc.nist.gov/groups/STM/cavp/documents/aes/XTSTestVectors.zip
The test cases come from the XTS test vectors given by the CAVP initiative
from NIST (see [1]).
As mentioned in a previous commit, XEX is a simpler case of XTS.
Therefore, to construct the test_suite_aes.xex.data file, extraction of
the XEX-possible cases has been done on the given test vectors.
All of the extracted test vectors pass the tests on a Linux x86_64 machine.
[1] http://csrc.nist.gov/groups/STM/cavp/documents/aes/XTSTestVectors.zip
* development: (97 commits)
Updated version number to 2.10.0 for release
Add a disabled CMAC define in the no-entropy configuration
Adapt the ARIA test cases for new ECB function
Fix file permissions for ssl.h
Add ChangeLog entry for PR#1651
Fix MicroBlaze register typo.
Fix typo in doc and copy missing warning
Fix edit mistake in cipher_wrap.c
Update CTR doc for the 64-bit block cipher
Update CTR doc for other 128-bit block ciphers
Slightly tune ARIA CTR documentation
Remove double declaration of mbedtls_ssl_list_ciphersuites
Update CTR documentation
Use zeroize function from new platform_util
Move to new header style for ALT implementations
Add ifdef for selftest in header file
Fix typo in comments
Use more appropriate type for local variable
Remove useless parameter from function
Wipe sensitive info from the stack
...
While the old name is explicit and aligned with the RFC, it's also very long,
so with the mbedtls_ prefix prepended we get a 31-char prefix to each
identifier, which quickly conflicts with our 80-column policy.
The new name is shorter, it's what a lot of people use when speaking about
that construction anyway, and hopefully should not introduce confusion at
it seems unlikely that variants other than 20/1305 be standardised in the
foreseeable future.
This implementation is based off the description in RFC 7539.
The ChaCha20 code is also updated to provide a means of generating
keystream blocks with arbitrary counter values. This is used to
generated the one-time Poly1305 key in the AEAD construction.
generate add ctest test-suites, with the --verbose argument to be given
to the test suites.
The verbose output will be shown **only** if ctest is run with `-v` parameter
The verbose argument is to the test-suites, only when run through `ctest`
Changes use of mklink in Windows test builds, to create junctions instead of
directory symbolic links. This removes the need for an elevated command prompt
when running cmake to create the Visual Studio project files.
The main goal with these tests is to test the bug in question and
they are not meant to test the entire PKCS#1 v1.5 behaviour. To
achieve full test coverage, further test cases are needed.
For a start, they don't even compile with Visual Studio due to strcasecmp
being missing. Secondly, on Windows Perl scripts aren't executable and have
to be run using the Perl interpreter directly; thankfully CMake is able to
find cygwin Perl straight away without problems.
* development: (46 commits)
Fix url again
Fix small bug in base64_encode()
Fix depend that was checked but not documented
Fix dependency that was not checked
Minor gitginore fixes
Move some ignore patterns to subdirectories
Ignore CMake/MSVC-related build files.
Re-categorize changelog entry
Fix misattribution
Minor nits with stdout/stderr.
Add cmake compatibility targets
Add script for polarssl symlink creation
Fix more stdio inclusion issues
Add debug info for cert/suite selection
Fix possible portability issue
Fix bug in ssl_get_verify_result()
aescrypt2.c local char array not initial
Update Changelog
Fix mips64 bignum implementation
Fix usage string of ssl_client2
...
Conflicts:
include/polarssl/ssl.h
library/CMakeLists.txt
library/Makefile
programs/Makefile
programs/ssl/ssl_client2.c
programs/ssl/ssl_server2.c
visualc/VS2010/PolarSSL.sln
visualc/VS2010/mbedTLS.vcxproj
visualc/VS6/mbedtls.dsp
visualc/VS6/mbedtls.dsw
* commit 'd7e2483': (57 commits)
Skip signature_algorithms ext if PSK only
Fix bug in ssl_client2 reconnect option
Cosmetics in ssl_server2
Improve debugging message.
Fix net_usleep for durations greater than 1 second
Use pk_load_file() in X509
Create ticket keys only if enabled
Fix typo in #ifdef
Clarify documentation a bit
Fix comment on resumption
Update comment from draft to RFC
Use more #ifdef's on CLI_C and SRV_C in ssl_tls.c
Add recursion.pl to all.sh
Allow x509_crt_verify_child() in recursion.pl
Set a compile-time limit to X.509 chain length
Fix 3DES -> DES in all.sh (+ time estimates)
Add curves.pl to all.sh
Rework all.sh to use MSan instead of valgrind
Fix depends on individual curves in tests
Add script to test depends on individual curves
...
Conflicts:
CMakeLists.txt
programs/ssl/ssl_client2.c
* commit '8b9bcec':
Stop assuming chars are signed
Fix len miscalculation in buffer-based allocator
Fix NULL dereference in buffer-based allocator
Add test_suite_memory_buffer_alloc
Add memory_buffer_alloc_self_test()
Fix missing bound check
Add test for ctr_drbg_update() input sanitizing
Refactor for clearer correctness/security
Stop assuming chars are signed
Conflicts:
library/ssl_tls.c
