Every boolean (defined/undefined) symbol is a "variant toggle" in some
sense, even enabling a module with MBEDTLS_xxx_C. What matters is
whether the symbol influences some other part of the system in such a
way that we need to run tests separately with and without it being
defined.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
It's deprecated, but not otherwise counter-indicated for the full
config: it doesn't conflict with anything and enabling it doesn't make
testing harder (especially since it defaults off in compat.sh).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
MBEDTLS_DEPRECATED_REMOVED is turned off in full since we don't want
to turn off deprecated features. Also turn off
MBEDTLS_DEPRECATED_WARNING since we wouldn't want expected warnings:
we're aware that we're enabling deprecated modules.
Since MBEDTLS_DEPRECATED_WARNING is excluded from full, it doesn't
need to be excluded from baremetal explicitly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
It started out as be experimental, but it is now robust enough not to
break the rest, so there's no reason to leave it out.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Enable MBEDTLS_X509_ALLOW_EXTENSIONS_NON_V3 in the full config. There's
no reason to keep it out. We weren't testing it at all on the CI.
Add a missing dependency on !MBEDTLS_X509_ALLOW_EXTENSIONS_NON_V3 to
some test cases that either were testing that v3 extensions are only
accepted in v3 certificates, or where parsing returns a different
error when MBEDTLS_X509_ALLOW_EXTENSIONS_NON_V3 is enabled.
Add a few positive and negative test cases with
MBEDTLS_X509_ALLOW_EXTENSIONS_NON_V3 enabled.
Fix one test case with MBEDTLS_X509_ALLOW_EXTENSIONS_NON_V3 where the
intended output of mbedtls_x509_crt_info had changed in
890819a597 but the test case was missed
because it was never executed.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The intended logic around MBEDTLS_xxx_ALT is to exclude them from full
because they require the alternative implementation of one or more
library functions, except that MBEDTLS_PLATFORM_xxx_ALT are different:
they're alternative implementations of a platform function and they
have a built-in default, so they should be included in full. Document
this.
Fix a bug whereby MBEDTLS_PLATFORM_xxx_ALT didn't catch symbols where
xxx contains an underscore. As a consequence,
MBEDTLS_PLATFORM_GMTIME_R_ALT and MBEDTLS_PLATFORM_NV_SEED_ALT are now
enabled in the full config. Explicitly exclude
MBEDTLS_PLATFORM_SETUP_TEARDOWN_ALT because it behaves like the
non-platform ones, requiring an extra build-time dependency.
Explicitly exclude MBEDTLS_PLATFORM_NV_SEED_ALT from baremetal
because it requires MBEDTLS_ENTROPY_NV_SEED, and likewise explicitly
unset it from builds that unset MBEDTLS_ENTROPY_NV_SEED.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The presence of these markers in the original code was helpful to me in
figuring out that this portion of the code is auto-generated.
Therefore, I think those are useful and should be present.
Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
- Use switch case instead of loop to generate faster code
- Add #if defined to address compiler error
Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
Problem
-------
mbedtls_strerror is a utility function which converts an mbedTLS error code
into a human readable string. It requires the caller to allocate a buffer every
time an error code needs to be converted to a string. It is an overkill and a
waste of RAM for resource constrained microcontrollers - where the most common
use case is to use these strings for logging.
Solution
--------
The proposed commit adds two functions:
* const char * mbedtls_high_level_strerr( int error_code );
* const char * mbedtls_low_level_strerr( int error_code );
The above two functions convert the high level and low level parts of an mbedTLS
error code to human readable strings. They return a const pointer to an
unmodifiable string which is not supposed to be modified by the caller and only
to be used for logging purposes. The caller no longer needs to allocate a
buffer.
Backward Compatibility
----------------------
The proposed change is completely backward compatible as it does not change
the existing mbedtls_strerror function and ensures that it continues to behave
the same way.
Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
Don't mix Windows and Unix line endings, it's the worst of both worlds.
Update the Visual Studio templates and regenerate the generated files.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use CRLF consistently instead of cobbling a \r here and a \n there.
The generated files don't change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Instead of working with Markdown format, keep the classic Mbed TLS
ChangeLog format, with the classic category names. Keep the classic
file name as well. This way there's no risk of breaking third-party
scripts that may copy or even parse the changelog file.
Accordingly, expect ChangeLog/*.txt files instead of ChangeLog/*.md.
This commit completely rewrites the parsing and output code.
This commit systematically appends to the existing top version. A
subsequent commit will restore the capability of creating a new
version.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Document that git is needed.
Be clearer about the entry sort key being an entry sort key, not just
a merge order. Be clearer about what "merge order" means.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Changelog entry files were listed in reverse alphabetical order of the
file name, by happenstance. Now, changelog entry files are listed in
the order in which the changes were merged. More precisely: look for
the git commit where the entry file was created, and look where this
commit was merged into the current branch. List older merges first.
List never-merged commits in date order after all the merged ones.
List never-committed files in file timestamp order after all the
committed ones.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Conflicts:
* .github/issue_template.md, .github/pull_request_template.md:
The changes in Mbed Crypto are not relevant to Mbed TLS. Keep the
mbedtls versions.
Rather than sometimes returning an integer, sometimes a boolean and
sometimes implicitly returning None, always return 0 for success and 1
for failure.
No behavior change for the program as a whole, since the None/True/False
values were implicitly converted to the desired numerical value.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Don't use a function argument as a for loop variable. It worked (mostly) but
Pylint frowns on it (redefined-argument-from-local) and I think Pylint has a
point.
If the configuration file is not found, raise an exception mentioning the
search path rather than just its last element.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.
Conflicts:
* ChangeLog: add/add conflict. Resolve by using the usual section order.
Rename identifiers containing double-underscore (`__`) to avoid `__`.
The reason to avoid double-underscore is that all identifiers
containing double-underscore are reserved in C++. Rename all such
identifiers that appear in any public header, including ssl_internal.h
which is in principle private but in practice is installed with the
public headers.
This commit makes check-names.sh pass.
```
perl -i -pe 's/\bMBEDTLS_SSL__ECP_RESTARTABLE\b/MBEDTLS_SSL_ECP_RESTARTABLE_ENABLED/g; s/\bMBEDTLS_KEY_EXCHANGE_(_\w+)_(_\w+)\b/MBEDTLS_KEY_EXCHANGE${1}${2}/g' include/mbedtls/*.h library/*.c programs/*/*.c scripts/data_files/rename-1.3-2.0.txt tests/suites/*.function
```
Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.
The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:
* `.github/issue_template.md`: completely different. We may want to reconcile
them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
relevant except as a stopgap as mbed-crypto did not have its own product
versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
but the unremoval step restored an old version.
Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```
Resolve the remaining conflicts:
* `library/CMakeLists.txt`:
* Keep the TLS definition of `src_crypto`
* `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
`include` and `crypto/include` in `target_include_directories`, all with
version `2.21.0`.
* `programs/Makefile`:
* Reconcile the APPS lists (add/add from a differently-formatted common
ancestor): insert the `psa/*` from crypto into the tls list.
* Keep the `fuzz` target defined only in tls version.
* Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
stuff for the `clean` target.
* `scripts/config.py`:
* `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
* `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
version in crypto. Take the tls version.
* `component_test_malloc_0_null` and more: take
`component_test_malloc_0_null` from crypto (with `config.py` rather than
`config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
add the call to `ssl-opt.sh` from tls. Take the other components from
crypto.
With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.
There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.
Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
This reverts commit 9afb2e9921.
Conflicts:
* include/CMakeLists.txt
* "Make config.h available" comment: there has been a change
adjacent to where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
* compat.sh: there has been a change immediately before where it was
removed. Just re-add what was removed.
This reverts commit 8298d70bee.
Conflicts:
* library/Makefile: removal of SOEXT_X509 and SOEXT_TLS vs change of
value of SOEXT_CRYPTO. Keep all, with the new value of SOEXT_CRYPTO.
This reverts commit 1c66e48670.
Conflicts:
* include/mbedtls/check_config.h:
* MBEDTLS_SSL_PROTO_SSL3: there has been an addition (of
MBEDTLS_SHA512_NO_SHA384) at the place where it was removed. Re-add it
after (alphabetical order).
* MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
Re-add it after (alphabetical order).
* MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
after (alphabetical order).
* include/mbedtls/config.h:
* MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
Re-add it after (alphabetical order).
* MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
after (alphabetical order).
* library/version_features.c: re-generate by running
scripts/generate_features.pl.
* programs/test/query_config.c: re-generate by running
scripts/generate_query_config.pl.
* scripts/config.pl: this file has been replaced by config.py. Port
the reversed changes to config.py:
* Revert removing three symbols from the list of symbols to
exclude from full.
* Revert removing one symbol (MBEDTLS_NET_C) from the list of symbols
to exclude from baremetal.
* scripts/footprint.sh:
* Re-add the line to unset MBEDTLS_NET_C, but with config.py instead of
config.pl.
* tests/scripts/all.sh:
* component_test_no_platform: re-add the line to unset MBEDTLS_NET_C, but
with config.py instead of config.pl.
* component_build_arm_none_eabi_gcc,
component_build_arm_none_eabi_gcc_no_udbl_division,
component_build_arm_none_eabi_gcc_no_64bit_multiplication,
component_build_armcc: these components now use the baremetal
configuration, so they do not need to turn off MBEDTLS_NET_C explicitly.
This reverts commit bb1f701212.
* include/mbedtls/check_config.h:
* MBEDTLS_X509_RSASSA_PSS_SUPPORT: there has been an addition (of
MBEDTLS_SHA512_NO_SHA384) at the place where it was removed.
Re-add it before MBEDTLS_SHA512_NO_SHA384 to keep it grouped
with MBEDTLS_RSA_C.
Conflicts:
* scripts/config.pl: this file has been replaced by config.py. Port
the reversed changes to config.py:
* Revert removing three symbols from the list of symbols to
exclude from full.
This reverts commit 356acc82ad.
Conflicts:
* scripts/generate_errors.pl: a line adjacent to a changed line has
independently changed in the meantime. Just revert the change done
in the commit that's being reverted.
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.
This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
`pylint3 --version` will output to stderr the status of the config file it's
using. This can be "No config file found" or "Using config file" or nothing.
This means the pylint version may or may not be on the first line.
Therefore this commit changes the filters on the pylint3 version output to first
strip out the config line, and then to select only the pylint line.
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
Add the versions of Python, Perl, and Pylint to the version dump provided by
the output_env.sh script.
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
A number of clean-up improvements following review.
* removal of redundant `` quotes
* removal of non-portable echo "\n", in favour of additional echo commands
* change to use of uname to detemine if the platform is Linux or not
* revised formatting of output
* change to dpkg-query from dpkg to find installed libasan variants
Co-Authored-By: Gilles Peskine <gilles.peskine@arm.com>
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
This commit adds additional information to the output_env.sh script of:
* Linux distribution version (if available)
* GDB version (if available)
It also makes some information clearer:
* the type of OpenSSL/GNUTLS version (legacy/default/next)
* and whether certain versions are not installed, or not configured
And it simplifies the error messages for absent tools.
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
generate_visualc_files.pl has a list of directories that it pulls
headers from, so it knows what directories to put on the include path.
Make it inject the include path into the output files, rather than
hard-coding the include paths in template files.
A similar change (but with different code) was made in Mbed TLS in
commit b78cf2b261
"Adjust visual studio file generation to always use the crypto submodule".
No semantic change: this commit does not change the generated files.
There's no need to keep the directory lists separated until the last
minute.
No semantic change. The generated files change slightly because there
was one directory list where slashes were not changed to backslashes
like in the other five. This does not affect their semantics.
Other third-party components can now be added by just adding lines to
the definitions of @thirdparty_header_dirs and
@thirdparty_source_dirs.
No semantic change. The output does not change at all.
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
Files deleted by us: keep them deleted.
```
git rm $(git status -s | sed -n 's/^DU //p')
```
Individual files with conflicts:
* `README.md`: keep the crypto version.
* `doxygen/input/doc_mainpage.h`: keep the crypto version (with an obsolete Mbed Crypto version number).
* `include/mbedtls/error.h`:
* `ERROR`: similar additions made through parallel commits, with only whitespace differences. Align with the tls version.
* `library/CMakeLists.txt`: keep the crypto version.
* `library/Makefile`: keep the crypto version.
* `scripts/generate_errors.pl`: keep the crypto version (the relevant changes were made through parallel commits).
* `tests/scripts/check-test-cases.py`:
* `Results`: keep the crypto version, which has both the new argument to the constructor (added in crypto only) and the class docstring (added through parallel commits).
* `tests/suites/helpers.function`:
* `ARRAY_LENGTH`, `ASSERT_ALLOC`: additions in the same location. Keep both, in indifferent order.
* `tests/suites/target_test.function`:
* `receive_uint32`: keep the crypto version which has an additional bug fix. The tls changes made in tls are irrelevant after this bug fix.
* `visualc/VS2010/mbedTLS.vcxproj`: run `scripts/generate_visualc_files.pl`.
Review of non-conflicting changes:
* `all.sh`: 1 change.
* zlib test components: don't add them.
* `include/CMakeLists.txt`: 1 change.
* `target_include_directories`: doesn't work as is (different target name). Don't take the change.
* All other non-conflicting changes: take them.
All key types now have an encoding on 32 bits where the bottom 16 bits
are zero. Change to using 16 bits only.
Keep 32 bits for key types in storage, but move the significant
half-word from the top to the bottom.
Likewise, change EC curve and DH group families from 32 bits out of
which the top 8 and bottom 16 bits are zero, to 8 bits only.
Reorder psa_core_key_attributes_t to avoid padding.
Check that no line from any of the input files was lost.
This is not perfect for several reasons.
It doesn't check that the content goes to the desired location.
It doesn't check that sections are created as necessary.
It doesn't support whitespace normalization that the parsing code does.
But it's a good start.
Now that the Error module has error codes as well and is processed by
the generate_errors script like any other module, we don't need to
include the header manually.
One of the error codes was already reserved, this commit just makes it
explicit. The other one is a new error code for initializing return
values in the library: `MBEDTLS_ERR_ERROR_CORRUPTION_DETECTED` should
not be returned by the library. If it is returned, then it is surely a
bug in the library or somebody is tampering with the device.
Filter out non-ASCII characters in automatically processed headers.
Do this in a way that minimizes the code change: keep manipulating
strings, but strip off non-ASCII characters when reading lines, which
should only remove characters in comments that we don't parse anyway.
One of the error codes was already reserved, this commit just makes it
explicit. The other one is a new error code for initializing return
values in the library: `MBEDTLS_ERR_ERROR_CORRUPTION_DETECTED` should
not be returned by the library. If it is returned, then it is surely a
bug in the library or somebody is tampering with the device.
This is a variant toggle, not an extra feature, so it should be tested
separately.
We test most of the effect of MBEDTLS_ENTROPY_FORCE_SHA256 (namely,
using SHA-256 in the entropy module) when we test the library with the
SHA512 module disabled (which we do at least via depends-hashes.pl).
This commit removes testing of the MBEDTLS_ENTROPY_FORCE_SHA256 option
itself, which should be added separately.
Enabling MBEDTLS_MEMORY_BUFFER_ALLOC_C module together with
MBEDTLS_PLATFORM_MEMORY causes the library to use its own malloc
replacement. This makes memory management analyzers such as ASan
largely ineffective. We now test MBEDTLS_MEMORY_BUFFER_ALLOC_C
separately. Disable it in the "full" config.
This mirrors a change that was made in Mbed TLS on config.pl and had
not been ported to Mbed Crypto yet.
With this commit, config.py is aligned in Mbed Crypto and Mbed TLS.
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.
Add corresponding tests.
Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
