Merge Mbed TLS at f790a6cbee into Mbed Crypto.
Resolve conflicts by performing the following:
- Reject changes to README.md
- Don't add crypto as a submodule
- Remove test/ssl_cert_test from programs/Makefile
- Add cipher.nist_kw test to tests/CMakeLists.txt
- Reject removal of crypto-specific all.sh tests
- Reject update to SSL-specific portion of component_test_valgrind
in all.sh
- Reject addition of ssl-opt.sh testing to component_test_m32_o1 in
all.sh
* tls/development: (87 commits)
Call mbedtls_cipher_free() to reset a cipher context
Don't call mbedtls_cipher_setkey twice
Update crypto submodule
Minor fixes in get certificate policies oid test
Add certificate policy oid x509 extension
cpp_dummy_build: Add missing header psa_util.h
Clarify comment mangled by an earlier refactoring
Add an "out-of-box" component
Run ssl-opt.sh on 32-bit runtime
Don't use debug level 1 for informational messages
Skip uncritical unsupported extensions
Give credit to OSS-Fuzz for #2404
all.sh: remove component_test_new_ecdh_context
Remove crypto-only related components from all.sh
Remove ssl_cert_test sample app
Make CRT callback tests more robust
Rename constant in client2.c
Document and test flags in x509_verify
Fix style issues and a typo
Fix a rebase error
...
Running the generation script with "include_crypto" input parameter set to 1
makes the mbedtls sources being overshadowed by crypto sources.
In case of any duplicate sources, crypto ones take precedence.
There are a number of arguments being passed around, nearly all of
which are duplicated between the old and new versions. Moving these
into a separate class should hopefully make it simpler to follow
what is being done.
As before with wanting to compare revisions across different
repositories, the ability to select the crypto submodule from a
different repository is useful.
We may wish to compare ABI/API between Mbed TLS and Mbed Crypto,
which will cause issues as not all .so files are shared. Only
compare .so files which both libraries have.
As going forward we will have Crypto in a submodule, we will need to
be able to check ABI compatibility between versions using different
submodule versions. For TLS versions that support the submodule, we
will always build using the submodule.
If the Crypto submodule is used, libmbedcrypto.so is not in the main
library folder, but in crypto/library instead. Given this, the script
searches for *.so files and notes their path, in order to create the
dumps correctly.
By default abi-compliance-checker will check the entire ABI/API.
There are internal identifiers that we do not promise compatibility
for, so we want the ability to skip them when checking the ABI/API.
* origin/pr/2464:
Allow main() to lack a docstring.
Silence pylint
check-files.py: readability improvement in permission check
check-files.py: use class fields for class-wide constants
check-files.py: clean up class structure
abi_check.py: Document more methods
check-files.py: document some classes and methods
Fix pylint errors going uncaught
Call pylint3, not pylint
New, documented pylint configuration
Update the crypto submodule to the top of the Mbed Crypto development
branch. This brings in a version of Mbed Crypto that enables building
Mbed Crypto tests that depend on private headers, like
'psa_crypto_invasive.h'.
This also requires updating our config.h to include new configuration
options added to Mbed Crypto. MBEDTLS_PSA_ITS_FILE_C replaces
MBEDTLS_PSA_CRYPTO_STORAGE_FILE_C and MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C.
MBEDTLS_PSA_INJECT_ENTROPY replaces MBEDTLS_PSA_HAS_ITS_IO.
MBEDTLS_PSA_HAS_ITS_IO is not really useful since it doesn't actually
enable anything except the entropy seed file support, which only
requires the ITS interface and not a native implemetation. Remove it.
Now that we have ITS over files, we no longer need a direct backend
for key storage over files. Remove psa_crypto_storage_file and its
tests.
Switch MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C and MBEDTLS_PSA_ITS_FILE_C on
by default. This preserves functionality and test coverage in the
default configuration, but forgets any key previously stored using the
file backend.
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
* origin/development: (113 commits)
Update query_config.c
Fix failure in SSLv3 per-version suites test
Adjust DES exclude lists in test scripts
Clarify 3DES changes in ChangeLog
Fix documentation for 3DES removal
Exclude 3DES tests in test scripts
Fix wording of ChangeLog and 3DES_REMOVE docs
Reduce priority of 3DES ciphersuites
Fix unused variable warning in ssl_parse_certificate_coordinate()
Update the crypto submodule to a78c958
Fix ChangeLog entry to correct release version
Fix typo in x509write test data
Add ChangeLog entry for unused bits in bitstrings
Improve docs for named bitstrings and their usage
Add tests for (named) bitstring to suite_asn1write
Add new function mbedtls_asn1_write_named_bitstring()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
...
Additional work done as part of merge:
- Run ./tests/scripts/check-generated-files.sh and check in the
resulting changes to programs/ssl/query_config.c
Additional work done as part of merge:
- Run ./tests/scripts/check-generated-files.sh and check in the
resulting changes to programs/ssl/query_config.c
Declare the owner as psa_key_owner_id_t, of which an implementation
must be provided separately.
Make this a configuration option
MBEDTLS_PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER, to make the conditional
compilation flow easier to follow. Declare it in config.h to
pacify check_names.sh.
Support for a specific implementation of psa_key_owner_id_t in storage
backends will come in a subsequent commit.
Resolve conflicts by performing the following.
- Take the upstream Mbed TLS ChangeLog verbatim.
- Reject changes to Makefiles and CMake that are related to using Mbed
Crypto as a submodule. It doesn't make sense to use Mbed Crypto as a
submodule of itself.
- Reject README changes, as Mbed Crypto has its own, different README.
- Reject PSA-related changes to config.h. We don't want to disable the
availability of the PSA Crypto API by default in the Mbed Crypto
config.h.
- Don't inadvertently revert dead code removal in
mbedtls_cipher_write_tag() which was added in f2a7529403 ("Fix
double return statement in cipher.c")
- Where Mbed Crypto already had some MBEDTLS_USE_PSA_CRYPTO code (from
past companion PRs) take the latest version from Mbed TLS which
includes integration with MBEDTLS_CHECK_PARAMS.
- Update the version of the shared library files to match what's
currently present in Mbed TLS.
- Reject removal of testing with PSA from config full tests.
- Resolve conflicts in test tests/suites/helpers.function, where both
Mbed Crypto and Mbed TLS both added documentation for TEST_ASSERT.
Combine text from both documentation efforts.
- Reject adding a submodule of ourselves.
- Reject addition of submodule tests in all.sh.
- Reject addition of submodule to library path in
tests/scripts/run-test-suites.pl.
- Avoid using USE_CRYPTO_SUBMODULE=1 in
component_test_use_psa_crypto_full_cmake_asan() in all.sh.
test_psa_constant_names.py was originally written before the split of
crypto.h into crypto_values.h and more, so it now needs to read
crypto_values.h as well.
In both generate_psa_constants.py and test_psa_constant_names.py, read
crypto_extra.h as well. We don't currently define any value there, but
it's plausible that we will one day.
Some parts of the library, and crypto drivers, need to see key types,
algorithms, policies, etc. but not API functions. Move portable
integral types and macros to build and analyze values of these types
to a separate headers crypto_types.h and crypto_values.h.
No functional changes, code was only moved from crypto.h to the new headers.
Without a "--detach" option, git worktree will refuse to checkout a branch
that's already checked out. This makes the abi_check.py script not very
useful for checking the currently checked out branch, as git will error
that the branch is already checked out. Add the "--detach" option to check
out the new temporary worktree in detached head mode. This is acceptable
because we aren't planning on working on the branch and just want a
checkout to do ABI checking from.
Allow use of persistent keys, including configuring them, importing and
exporting them, and destroying them.
When getting a slot using psa_get_key_slot, there are 3 scenarios that
can occur if the keys lifetime is persistent:
1. Key type is PSA_KEY_TYPE_NONE, no persistent storage entry:
- The key slot is treated as a standard empty key slot
2. Key type is PSA_KEY_TYPE_NONE, persistent storage entry exists:
- Attempt to load the key from persistent storage
3. Key type is not PSA_KEY_TYPE_NONE:
- As checking persistent storage on every use of the key could
be expensive, the persistent key is assumed to be saved in
persistent storage, the in-memory key is continued to be used.
Add new functions, psa_load_persistent_key(),
psa_free_persistent_key_data(), and psa_save_persistent_key(), for
managing persistent keys. These functions load to or save from our
internal representation of key slots. Serialization is a concern of the
storage backend implementation and doesn't abstraction-leak into the
lifetime management code.
An initial implementation for files is provided. Additional storage
backends can implement this interface for other storage types.
The persistent key implementation will be split across multiple
files as it will eventually be implementing multiple storage
backends. As these internal functions will need to be callable by
other files, we will add the headers in the library folder. This
commit adds this include location to the necessary scripts.
For tests, the library is added as an include location as testing
on-target with Mbed OS is not possible with paths including ".."
Mbed TLS version 2.14.0
Resolved conflicts in include/mbedtls/config.h,
tests/scripts/check-files.py, and yotta/create-module.sh by removing yotta.
Resolved conflicts in tests/.jenkins/Jenkinsfile by continuing to run
mbedtls-psa job.
Even with a shallow clone of the repo where there are no tags available
to version with, don't error and instead show a unique abbreviated
commit hash as fallback.
OFB and CFB are streaming modes. XTS is a not a cipher mode but it
doesn't use a separate padding step. This leaves only CBC as a block
cipher mode that needs a padding step.
Since CBC is the only mode that uses a separate padding step, and is
likely to remain the only mode in the future, encode the padding mode
directly in the algorithm constant, rather than building up an
algorithm value from a chaining mode and a padding mode. This greatly
simplifies the interface as well as some parts of the implementation.
Create a tarball-based ditribution of the Mbed Crypto source code using the
code from Mbed TLS. Only export the files needed by Mbed Crypto to
implement the PSA Crypto API. Rename "mbedtls_*" and "MBEDTLS_*" symbols to
"mbedcrypto_*" and "MBEDCRYPTO_*".
The exported distribution can stand alone, without any Mbed TLS repository
needing to be present. "make test" will continue to work as before and will
run only the PSA Crypto test suite.
Print the symbolic name corresponding to a numerical value.
Supported types: status values, algorithms, elliptic curves,
key types, key usage masks.
The program is partly generated from parsing psa/crypto.h with a few
hard-coded assumptions. This isn't ideal but it works and requires
little machinery.
We failed check-names.sh due to using a define which wasn't described or
defined anywhere. Even though we won't realistically enable
MBEDTLS_PSA_CRYPTO_SPM via the configuration system (and will do it from
PSA Crypto SPM tooling instead), add a description of the configuration to
config.h as good practice. Exclude MBEDTLS_PSA_CRYPTO_SPM from the "full"
configuration as well.
* development: (182 commits)
Change the library version to 2.11.0
Fix version in ChangeLog for fix for #552
Add ChangeLog entry for clang version fix. Issue #1072
Compilation warning fixes on 32b platfrom with IAR
Revert "Turn on MBEDTLS_SSL_ASYNC_PRIVATE by default"
Fix for missing len var when XTS config'd and CTR not
ssl_server2: handle mbedtls_x509_dn_gets failure
Fix harmless use of uninitialized memory in ssl_parse_encrypted_pms
SSL async tests: add a few test cases for error in decrypt
Fix memory leak in ssl_server2 with SNI + async callback
SNI + SSL async callback: make all keys async
ssl_async_resume: free the operation context on error
ssl_server2: get op_name from context in ssl_async_resume as well
Clarify "as directed here" in SSL async callback documentation
SSL async callbacks documentation: clarify resource cleanup
Async callback: use mbedtls_pk_check_pair to compare keys
Rename mbedtls_ssl_async_{get,set}_data for clarity
Fix copypasta in the async callback documentation
SSL async callback: cert is not always from mbedtls_ssl_conf_own_cert
ssl_async_set_key: detect if ctx->slots overflows
...
* development: (97 commits)
Updated version number to 2.10.0 for release
Add a disabled CMAC define in the no-entropy configuration
Adapt the ARIA test cases for new ECB function
Fix file permissions for ssl.h
Add ChangeLog entry for PR#1651
Fix MicroBlaze register typo.
Fix typo in doc and copy missing warning
Fix edit mistake in cipher_wrap.c
Update CTR doc for the 64-bit block cipher
Update CTR doc for other 128-bit block ciphers
Slightly tune ARIA CTR documentation
Remove double declaration of mbedtls_ssl_list_ciphersuites
Update CTR documentation
Use zeroize function from new platform_util
Move to new header style for ALT implementations
Add ifdef for selftest in header file
Fix typo in comments
Use more appropriate type for local variable
Remove useless parameter from function
Wipe sensitive info from the stack
...
Motivation is similar to NO_UDBL_DIVISION.
The alternative implementation of 64-bit mult is straightforward and aims at
obvious correctness. Also, visual examination of the generate assembly show
that it's quite efficient with clang, armcc5 and arm-clang. However current
GCC generates fairly inefficient code for it.
I tried to rework the code in order to make GCC generate more efficient code.
Unfortunately the only way to do that is to get rid of 64-bit add and handle
the carry manually, but this causes other compilers to generate less efficient
code with branches, which is not acceptable from a side-channel point of view.
So let's keep the obvious code that works for most compilers and hope future
versions of GCC learn to manage registers in a sensible way in that context.
See https://bugs.launchpad.net/gcc-arm-embedded/+bug/1775263
While the old name is explicit and aligned with the RFC, it's also very long,
so with the mbedtls_ prefix prepended we get a 31-char prefix to each
identifier, which quickly conflicts with our 80-column policy.
The new name is shorter, it's what a lot of people use when speaking about
that construction anyway, and hopefully should not introduce confusion at
it seems unlikely that variants other than 20/1305 be standardised in the
foreseeable future.
This implementation is based off the description in RFC 7539.
The ChaCha20 code is also updated to provide a means of generating
keystream blocks with arbitrary counter values. This is used to
generated the one-time Poly1305 key in the AEAD construction.
* development: (504 commits)
Fix minor code style issues
Add the uodate to the soversion to the ChangeLog
Fix the ChangeLog for clarity, english and credit
Update version to 2.9.0
ecp: Fix binary compatibility with group ID
Changelog entry
Change accepted ciphersuite versions when parsing server hello
Remove preprocessor directives around platform_util.h include
Fix style for mbedtls_mpi_zeroize()
Improve mbedtls_platform_zeroize() docs
mbedtls_zeroize -> mbedtls_platform_zeroize in docs
Reword config.h docs for MBEDTLS_PLATFORM_ZEROIZE_ALT
Organize CMakeLists targets in alphabetical order
Organize output objs in alfabetical order in Makefile
Regenerate errors after ecp.h updates
Update ecp.h
Change variable bytes_written to header_bytes in record decompression
Update ecp.h
Update ecp.h
Update ecp.h
...
Add a function that deletes all the old Visual Studio prokect files
before generating the new ones. This ensure that project files for
delete example applications are removed from the repository.
Add mbedTLS.vcxproj to the VS2010 application template so that the next
time we auto-generate the application project files, the
LinkLibraryDependencies for mbedTLS.vcxproj are maintained.
Fixes#1347
Add MBEDTLS_ERR_XXX_HW_ACCEL_FAILED error codes for all cryptography
modules where the software implementation can be replaced by a hardware
implementation.
This does not include the individual message digest modules since they
currently have no way to return error codes.
This does include the higher-level md, cipher and pk modules since
alternative implementations and even algorithms can be plugged in at
runtime.
When printing an option's value, print a newline at the end.
When the requested option is missing, fail with status 1 (the usual
convention for "not found") rather than -1 (which has a
system-dependent effect).
scripts/config.pl would always rewrite config.h if it was reading it.
This commit changes it to not modify the file when only reading is
required, i.e. for the get command.
Also, die if writing config.h fails (e.g. disk full).
Between 2.5.0 and 2.6.0, "scripts/config.pl get MBEDTLS_XXX" was fixed
for config.h lines with a comment at the end, but that broke the case
of macros with an empty expansion. Support all cases.
Change the script generate_errors.pl to call the grep function in Perl
instead of calling the external tool grep directly as this causes
problems when ANSI escape sequences are included in the grep output
string.
The effect of `config.pl full` on 'negative' options such as
`NO_PLATFORM_ENTROPY` is usually inverted, but `MBEDTLS_RSA_NO_CRT` was not
included in the list of such options. This commit adds it.
Adds interim directories to the Visual Studio project files to avoid warning
MSB8028 in Visual Studio 2015, where shared directories of intermediate files
between project files generate the warning.
Fixes many typos, and errors in comments.
* Clarifies many comments
* Grammar correction in config.pl help text
* Removed comment about MBEDTLS_X509_EXT_NS_CERT_TYPE.
* Comment typo fix (Dont => Don't)
* Comment typo fix (assure => ensure)
* Comment typo fix (byes => bytes)
* Added citation for quoted standard
* Comment typo fix (one complement => 1's complement)
The is some debate about whether to prefer "one's complement", "ones'
complement", or "1's complement". The more recent RFCs related to TLS
(RFC 6347, RFC 4347, etc) use " 1's complement", so I followed that
convention.
* Added missing ")" in comment
* Comment alignment
* Incorrect comment after #endif
The library/net.c and its corresponding include/mbedtls/net.h file are
renamed to library/net_sockets.c and include/mbedtls/net_sockets.h
respectively. This is to avoid naming collisions in projects which also
have files with the common name 'net'.
Ideally, Makefile targets should be called irrespectively of their
execution order. In this sense, `clean` and `lib` should not be
passed both as target command line arguments in the same make
invocation. This is because out-of-order execution could cause a
failure. This change ensures that `clean` and `lib` are called in
separate executions of make in scripts/footprint.sh
When the output_env.sh script is executed, if any application wasn't found on
the path by hash, and error was being output, which as an error looked like
the script wasn't working.
This change redirects the output to stderr from hash to /dev/null to suppress
the error.
Also fixes a formatting error.
* Add script to print build environment info.
The new script is also included in:
- all.sh
- basic-build-test.sh
* Tidy up environment reporting script.
Changes include:
- making the echo calls portable
- removing unnecessary brackets
- using more efficient checks for the existance of commands
- correcting typos and copyright year
* Update references to output_env.sh
Add a switch that turns entropy collecting off entirely, but enables
mbed TLS to run in an entirely unsafe mode. Enables to test mbed TLS
on platforms that don't have their entropy sources integrated yet.
The script config.pl fails when setting a #define symbol if the symbol isn't
already in the configuration header. This adds an option '--force' to append
the symbol to the end of the file if it isn't already present.
Also clarified usage, and added copyright to the config.pl.
This re-introduces the apidoc with full config.h, but hopefully with the race
conditions and other issues that the previous implementation had.
Adapt doxygen test script to use that new script, and also check for errors
in addition to warnings while at it.
Otherwise we get warnings that some documentation items don't have
corresponding #define, and more importantly the corresponding snippets are not
included in the output.
For that we need a modified version of the "full" argument for config.pl.
Also, the new CMakeLists.txt target only works on Unix (which was already the
case of the Makefile target). Hopefully this is not an issue as people are
unlikely to need that target on Windows.
Just applying rename.pl with this file:
mbedtls_cipher_get_key_size mbedtls_cipher_get_key_bitlen
mbedtls_pk_get_size mbedtls_pk_get_bitlen
MBEDTLS_BLOWFISH_MIN_KEY MBEDTLS_BLOWFISH_MIN_KEY_BITS
MBEDTLS_BLOWFISH_MAX_KEY MBEDTLS_BLOWFISH_MAX_KEY_BITS
* mbedtls-1.3:
Use link-time garbage collection in memory.sh
scripts/memory.sh only work on Linux
Add missing 'const' on selftest data
Use only headers for doxygen (no doc in C files)
Add missing extern "C" guard in aesni.h
Fix compile error with renego disabled
Remove slow PKCS5 test
Stop checking key-cert match systematically
Make tests/*.sh runnable from anywhere
Update visual C files
* rasp/mem-leak:
Fix another potential memory leak found by find-mem-leak.cocci.
Add a rule for another type of memory leak to find-mem-leak.cocci.
Fix a potential memory leak found by find-mem-leak.cocci.
Add a semantic patch to find potential memory leaks.
Fix whitespace of 369e6c20.
Apply the semantic patch rm-malloc-cast.cocci.
Add a semantic patch to remove casts of malloc.
Conflicts:
programs/ssl/ssl_server2.c
* rich/platform:
Remove dependency on sscanf in lib x509
Fix extra guard in memory_buffer_alloc
rebase from development
implemented macro overriding for polarssl_* library functions
fix bug introduced by the addition of snprintf and assert macro which caused tests to fail without polarssl_platform_c defined
add initial symbols to config and checks to check_config to allow use of macros to define standard functions
reformat and arrange additions to config alphabetically
add missing checks to check_config
add macro definition of assert using polarssl_exit
modify library/memory_buffer_alloc.c, benchmark.c and the tests main code to use polarssl_exit
add POLARSSL_PLATFORM_EXIT_ALT
modify scripts/* and tests/* to use polarssl_snprintf
modify programs/*.c to use polarssl_snprintf
modify library/debug.c to use polarssl_snprintf
modify library/x509*.c to use polarssl_snprintf
modify library/net.c to use polarssl_snprintf
modify oid.c to use polarssl_snprintf
add platform_set_snprintf
Conflicts:
library/memory_buffer_alloc.c
programs/pkey/pk_sign.c
programs/pkey/pk_verify.c
programs/pkey/rsa_sign_pss.c
programs/pkey/rsa_verify_pss.c
programs/ssl/ssl_client2.c
programs/ssl/ssl_pthread_server.c
programs/test/benchmark.c
programs/test/ssl_cert_test.c
The SHA4 name was not clear with regards to the new SHA-3 standard. So
SHA2 and SHA4 have been renamed to better represent what they are:
SHA256 and SHA512 modules.
The error code POLARSSL_ERR_X509_PASSWORD_MISMATCH is now properly
returned in case of an encryption failure in the padding. The
POLARSSL_ERR_X509_PASSWORD_REQUIRED error code is only returned for PEM
formatted private keys as for DER formatted ones it is impossible to
distinguish if a DER blob is PKCS#8 encrypted or not.
(cherry picked from commit 1fd4321ba2016dfaff2b48c11f731fc9ccbd7ccf)
Conflicts:
include/polarssl/error.h
scripts/generate_errors.pl
PKCS#8 encrypted key file support has been added to x509parse_key() with
support for some PCKS#12 PBE functions (pbeWithSHAAnd128BitRC4,
pbeWithSHAAnd3-KeyTripleDES-CBC and pbeWithSHAAnd2-KeyTripleDES-CBC)
(cherry picked from commit cf6e95d9a81c7b22271beb58a09b5c756148e62a)
Conflicts:
scripts/generate_errors.pl
A new OID module has been created that contains the main OID searching
functionality based on type-dependent arrays. A base type is used to
contain the basic values (oid_descriptor_t) and that type is extended to
contain type specific information (like a pk_alg_t).
As a result the rsa sign and verify function prototypes have changed. They
now expect a md_type_t identifier instead of the removed RSA_SIG_XXX
defines.
All OID definitions have been moved to oid.h
All OID matching code is in the OID module.
The RSA PKCS#1 functions cleaned up as a result and adapted to use the
MD layer.
The SSL layer cleanup up as a result and adapted to use the MD layer.
The X509 parser cleaned up and matches OIDs in certificates with new
module and adapted to use the MD layer.
The X509 writer cleaned up and adapted to use the MD layer.
Apps and tests modified accordingly
- As a consequence all error code that were positive were changed. A lot of MALLOC_FAILED and FILE_IO_ERROR error codes added for different modules.
- Programs and tests were adapted accordingly
- The error codes have been remapped and combining error codes is now done with a PLUS instead of an OR as error codes used are negative.
- Descriptions to all error codes have been added.
- Generation script for error.c has been created to automatically generate error.c from the available error definitions in the headers.
