Commit graph

525 commits

Author SHA1 Message Date
Gilles Peskine
9515d76171
Merge pull request #3219 from aggarg/err_optimization
Remove error_description variable from strerr functions
2020-05-04 09:37:13 +02:00
Manuel Pégourié-Gonnard
0c103a8f40
Merge pull request #2853 from irwir/fix_vcxproj
Finalize fixing bugs in .vcxproj files
2020-04-28 13:03:13 +02:00
Gaurav Aggarwal
cabde25289 Align with coding style
return statements use parentheses to contain their value.

Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
2020-04-22 08:13:25 -07:00
Kenneth Soerensen
518d435e7b Fix GCC format-signedness warnings
Signed-off-by: Kenneth Soerensen <knnthsrnsn@gmail.com>
2020-04-22 16:01:48 +02:00
Gaurav Aggarwal
8663c7415a Remove error_description variable from strerr functions
This was suggested on this PR: https://github.com/ARMmbed/mbedtls/pull/3176

Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
2020-04-21 14:04:19 -07:00
Gilles Peskine
dc9c47da6c
Merge pull request #3176 from aggarg/development
Add support for const error description strings
2020-04-21 13:57:11 +02:00
Gaurav Aggarwal
6ea4fc7b53 Address review comments
1. The functions mbedtls_high_level_strerr and mbedtls_low_level_strerr
   accept any error code and extract the high-level and low-level parts
   respectively.
2. Documentation updates.

Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
2020-04-20 16:03:46 -07:00
Gilles Peskine
5a2710e9af
Merge pull request #3120 from gilles-peskine-arm/check-windows-files
Check Windows files for sanity as well
2020-04-20 13:59:27 +02:00
irwir
96b45a4370 Merge remote-tracking branch 'upstream/development' into fix_vcxproj
Signed-off-by: irwir <irwir@users.noreply.github.com>
2020-04-20 13:21:46 +03:00
irwir
febf8802d7 Merge remote-tracking branch 'upstream/development' into fix_vcxproj
Signed-off-by: irwir <irwir@users.noreply.github.com>
2020-04-20 13:20:06 +03:00
Gilles Peskine
5c34ee14d3
Merge pull request #3118 from gilles-peskine-arm/pylint-up-to-2.4-development
Pass Pylint up to 2.4
2020-04-20 09:47:40 +02:00
irwir
49d46a7473 Finalize fixing the issue #1430.
The previous PRs had correct template project, but somehow it was not merged.

Signed-off-by: irwir <irwir@users.noreply.github.com>
2020-04-19 16:25:54 +03:00
Gaurav Aggarwal
3d02db23e8 Add auto-generated code markers
The presence of these markers in the original code was helpful to me in
figuring out that this portion of the code is auto-generated.
Therefore, I think those are useful and should be present.

Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
2020-04-11 17:14:03 -07:00
Gilles Peskine
184c096e95 Pylint: abide by useless-object-inheritance warnings
Inheriting from object is a remainder of Python 2 habits and is just
clutter in Python 3.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gaurav Aggarwal
a4a2aa5169 Address review comments
- Use switch case instead of loop to generate faster code
- Add #if defined to address compiler error

Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
2020-04-09 11:58:24 -07:00
Gaurav Aggarwal
a9f64006ea Add support for const error description strings
Problem
-------
mbedtls_strerror is a utility function which converts an mbedTLS error code
into a human readable string. It requires the caller to allocate a buffer every
time an error code needs to be converted to a string. It is an overkill and a
waste of RAM for resource constrained microcontrollers - where the most common
use case is to use these strings for logging.

Solution
--------
The proposed commit adds two functions:

* const char * mbedtls_high_level_strerr( int error_code );
* const char * mbedtls_low_level_strerr( int error_code );

The above two functions convert the high level and low level parts of an mbedTLS
error code to human readable strings. They return a const pointer to an
unmodifiable string which is not supposed to be modified by the caller and only
to be used for logging purposes. The caller no longer needs to allocate a
buffer.

Backward Compatibility
----------------------
The proposed change is completely backward compatible as it does not change
the existing mbedtls_strerror function and ensures that it continues to behave
the same way.

Signed-off-by: Gaurav Aggarwal <aggarg@amazon.com>
2020-04-09 01:44:52 -07:00
Gilles Peskine
2ae678d238 Systematically use Windows line endings in Windows files
Don't mix Windows and Unix line endings, it's the worst of both worlds.

Update the Visual Studio templates and regenerate the generated files.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 13:42:24 +02:00
Gilles Peskine
0f3f9c3c73 Fix a stray CR inside a line
Use CRLF consistently instead of cobbling a \r here and a \n there.

The generated files don't change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 13:42:12 +02:00
Gilles Peskine
dba4de0a12 Clarify extract_top_version when creating a new section
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-30 11:37:26 +02:00
Gilles Peskine
b695d5e30a Add guidance on writing and maintaining changelog entries
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 20:14:19 +01:00
Gilles Peskine
4d977a4f40 Complain if there is junk before the first category title
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 20:14:19 +01:00
Gilles Peskine
e248e83f9f Start numbering lines at 1
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 20:14:19 +01:00
Gilles Peskine
eebf24f7a8 Create a new version section if the top version has a release date
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 20:14:17 +01:00
Gilles Peskine
6e97c43959 Switch to the classic Mbed TLS ChangeLog format
Instead of working with Markdown format, keep the classic Mbed TLS
ChangeLog format, with the classic category names. Keep the classic
file name as well. This way there's no risk of breaking third-party
scripts that may copy or even parse the changelog file.

Accordingly, expect ChangeLog/*.txt files instead of ChangeLog/*.md.

This commit completely rewrites the parsing and output code.

This commit systematically appends to the existing top version. A
subsequent commit will restore the capability of creating a new
version.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 20:13:45 +01:00
Gilles Peskine
c68c7c8864 Add a missing space in help text
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 19:09:18 +01:00
Gilles Peskine
ac0f0860f1 Reformatting to pacify pylint
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 10:56:45 +01:00
Gilles Peskine
42f384c186 Mbed Crypto isn't a separate project anymore
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 09:23:38 +01:00
Gilles Peskine
98a53aa399 Follow renames when looking for the creation of a file
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:50:11 +01:00
Gilles Peskine
13dc634282 Simplify the matching of the last line
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:49:44 +01:00
Gilles Peskine
7fa3eb7d44 Simplify sorting
Calling sort_key explicitly is marginally faster, but less readable.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:41:32 +01:00
Gilles Peskine
28af958ea4 Documentation improvements
Document that git is needed.

Be clearer about the entry sort key being an entry sort key, not just
a merge order. Be clearer about what "merge order" means.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:39:18 +01:00
Gilles Peskine
8f46bbf46f Sort changelog entries by their merge date
Changelog entry files were listed in reverse alphabetical order of the
file name, by happenstance. Now, changelog entry files are listed in
the order in which the changes were merged. More precisely: look for
the git commit where the entry file was created, and look where this
commit was merged into the current branch. List older merges first.
List never-merged commits in date order after all the merged ones.
List never-committed files in file timestamp order after all the
committed ones.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 11:58:00 +01:00
Gilles Peskine
27a1facd48 Factor out list_files_to_merge
No behavior changes.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 11:57:54 +01:00
Gilles Peskine
5d324b203a Merge remote-tracking branch 'upstream-crypto/development' into HEAD
Conflicts:
* .github/issue_template.md, .github/pull_request_template.md:
  The changes in Mbed Crypto are not relevant to Mbed TLS. Keep the
  mbedtls versions.
2020-03-25 15:47:43 +01:00
Gilles Peskine
8bad3ab45f
Merge pull request #300 from gilles-peskine-arm/assemble-changelog
Changelog assembly script
2020-03-25 15:17:04 +01:00
Gilles Peskine
e22a4dacf7 Explicit return value from main
Rather than sometimes returning an integer, sometimes a boolean and
sometimes implicitly returning None, always return 0 for success and 1
for failure.

No behavior change for the program as a whole, since the None/True/False
values were implicitly converted to the desired numerical value.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 19:36:17 +01:00
Gilles Peskine
ce674a90c5 Clearer code to search for config.h
Don't use a function argument as a for loop variable. It worked (mostly) but
Pylint frowns on it (redefined-argument-from-local) and I think Pylint has a
point.

If the configuration file is not found, raise an exception mentioning the
search path rather than just its last element.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 19:25:34 +01:00
Gilles Peskine
5e7d6fd240 Merge 'mbedtls/development' into merge-crypto-unremoved-20200304
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.

Conflicts:

* ChangeLog: add/add conflict. Resolve by using the usual section order.
2020-03-23 18:02:07 +01:00
Gilles Peskine
eccd888717 Rename identifiers containing double-underscore
Rename identifiers containing double-underscore (`__`) to avoid `__`.
The reason to avoid double-underscore is that all identifiers
containing double-underscore are reserved in C++. Rename all such
identifiers that appear in any public header, including ssl_internal.h
which is in principle private but in practice is installed with the
public headers.

This commit makes check-names.sh pass.

```
perl -i -pe 's/\bMBEDTLS_SSL__ECP_RESTARTABLE\b/MBEDTLS_SSL_ECP_RESTARTABLE_ENABLED/g; s/\bMBEDTLS_KEY_EXCHANGE_(_\w+)_(_\w+)\b/MBEDTLS_KEY_EXCHANGE${1}${2}/g' include/mbedtls/*.h library/*.c programs/*/*.c scripts/data_files/rename-1.3-2.0.txt tests/suites/*.function
```
2020-03-23 17:55:32 +01:00
Gilles Peskine
b99bd39b4e Merge mbed-crypto into mbedtls: the merge commit
Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.

The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:

* `.github/issue_template.md`: completely different. We may want to reconcile
  them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
  independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
  relevant except as a stopgap as mbed-crypto did not have its own product
  versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
  but the unremoval step restored an old version.

Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```

Resolve the remaining conflicts:

* `library/CMakeLists.txt`:
    * Keep the TLS definition of `src_crypto`
    * `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
      `include` and `crypto/include` in `target_include_directories`, all with
      version `2.21.0`.
* `programs/Makefile`:
    * Reconcile the APPS lists (add/add from a differently-formatted common
      ancestor): insert the `psa/*` from crypto into the tls list.
    * Keep the `fuzz` target defined only in tls version.
    * Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
      stuff for the `clean` target.
* `scripts/config.py`:
    * `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
    * `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
      version in crypto. Take the tls version.
    * `component_test_malloc_0_null` and more: take
      `component_test_malloc_0_null` from crypto (with `config.py` rather than
      `config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
      add the call to `ssl-opt.sh` from tls. Take the other components from
      crypto.

With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.

There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.

Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
2020-03-23 17:54:46 +01:00
Gilles Peskine
3b46cd3f15 Invoke config.py instead of config.pl in reverted content
perl -i -pe 's/\bconfig\.pl/config.py/g' $(git grep -l -Fw config.pl
-- . '#!tests/scripts/test_config_script.py')
2020-03-19 14:23:45 +01:00
Gilles Peskine
7dc97048d6 Revert "Remove tests that depend on TLS or X.509"
This reverts commit 9afb2e9921.

Conflicts:
* include/CMakeLists.txt
  * "Make config.h available" comment: there has been a change
    adjacent to where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
  * compat.sh: there has been a change immediately before where it was
    removed. Just re-add what was removed.
2020-03-19 14:17:54 +01:00
Gilles Peskine
722a7e6940 Revert "Only build libmbedcrypto"
This reverts commit 8298d70bee.

Conflicts:
* library/Makefile: removal of SOEXT_X509 and SOEXT_TLS vs change of
  value of SOEXT_CRYPTO. Keep all, with the new value of SOEXT_CRYPTO.
2020-03-19 14:07:55 +01:00
Gilles Peskine
4e1174967a Revert "config: Remove TLS and NET options"
This reverts commit 1c66e48670.

Conflicts:
* include/mbedtls/check_config.h:
    * MBEDTLS_SSL_PROTO_SSL3: there has been an addition (of
      MBEDTLS_SHA512_NO_SHA384) at the place where it was removed. Re-add it
      after (alphabetical order).
    * MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
      MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
      Re-add it after (alphabetical order).
    * MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
      MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
      after (alphabetical order).
* include/mbedtls/config.h:
    * MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
      MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
      Re-add it after (alphabetical order).
    * MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
      MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
      after (alphabetical order).
* library/version_features.c: re-generate by running
  scripts/generate_features.pl.
* programs/test/query_config.c: re-generate by running
  scripts/generate_query_config.pl.
* scripts/config.pl: this file has been replaced by config.py. Port
  the reversed changes to config.py:
    * Revert removing three symbols from the list of symbols to
      exclude from full.
    * Revert removing one symbol (MBEDTLS_NET_C) from the list of symbols
      to exclude from baremetal.
* scripts/footprint.sh:
    * Re-add the line to unset MBEDTLS_NET_C, but with config.py instead of
      config.pl.
* tests/scripts/all.sh:
    * component_test_no_platform: re-add the line to unset MBEDTLS_NET_C, but
      with config.py instead of config.pl.
    * component_build_arm_none_eabi_gcc,
    component_build_arm_none_eabi_gcc_no_udbl_division,
    component_build_arm_none_eabi_gcc_no_64bit_multiplication,
    component_build_armcc: these components now use the baremetal
    configuration, so they do not need to turn off MBEDTLS_NET_C explicitly.
2020-03-19 13:55:04 +01:00
Gilles Peskine
252e391cca Revert "config: Remove X.509 options"
This reverts commit bb1f701212.

* include/mbedtls/check_config.h:
    * MBEDTLS_X509_RSASSA_PSS_SUPPORT: there has been an addition (of
      MBEDTLS_SHA512_NO_SHA384) at the place where it was removed.
      Re-add it before MBEDTLS_SHA512_NO_SHA384 to keep it grouped
      with MBEDTLS_RSA_C.

Conflicts:
* scripts/config.pl: this file has been replaced by config.py. Port
  the reversed changes to config.py:
    * Revert removing three symbols from the list of symbols to
      exclude from full.
2020-03-19 13:53:18 +01:00
Gilles Peskine
458b8f2a59 Revert "Remove unused TLS, NET, and X.509 files"
This reverts commit a4308b29a4.
2020-03-19 13:36:33 +01:00
Gilles Peskine
314bc89b36 Revert "scripts: Remove dependency on TLS"
This reverts commit b58ff9541b.
2020-03-19 13:36:33 +01:00
Gilles Peskine
1bf45e1980 Revert "scripts: Remove dependency on X.509"
This reverts commit 43a450c858.
2020-03-19 13:36:33 +01:00
Gilles Peskine
1fcf721e11 Revert "scripts: Remove dependency on NET"
This reverts commit 356acc82ad.

Conflicts:
* scripts/generate_errors.pl: a line adjacent to a changed line has
  independently changed in the meantime. Just revert the change done
  in the commit that's being reverted.
2020-03-19 13:36:12 +01:00
Simon Butcher
e30d03e4f4 Change the use of pylint to optionally use pylint3
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.

This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-16 11:38:02 +00:00