Use case pattern matching instead of multiline split, given there is
only the well formatted PIDs to match on this should be safe.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
Improve the code structure in case we want to add other similar conditions
later. Document better what we're doing, and document why we're doing it.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
On machines with more modern kernels (>5.4 from testing so far) the
useage of -b seems to conflict with the usage of -p. Whilst the usage of
-b seems like a good idea to avoid blocks as we are tight looping on it,
the usage of -p seems to require the usage of stat() (specifically in
/proc) which -b forbids. All you get is a load of warnings
(suppressable by -w) but never a positive result, which means that all
servers are reported as "Failed to start". We are not keen on losing
-b, so instead parse the output of lsof (using -F to format it) to
check the if PIDs that it outputs match that we are looking for.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
Palliative for https://github.com/ARMmbed/mbedtls/issues/3377. If a test
case fails due to an unexpected resend, allow retrying, like in the case of
a client timeout.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This was causing some tests using the openssl s_client to not connect -
I suspect this was due to localhost (at least on my machine) resolving
to ::1 rather than 127.0.0.1. Note that the error seen would have been
that the session file specified with -sess_out did not get created.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
Missing wildcards meant that some servers were not identified as DTLS,
which lead to port checking on TCP rather than UDP, and thus mistakenly
cancelling tests as the server had not come up.
Signed-off-by: Paul Elliott <paul.elliott@arm.com>
This option only gated an ability to set a callback,
but was deemed unnecessary as it was yet another define to
remember when writing tests, or test configurations. Fixes#4653.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Those tests are so far only checking that ssl_client2/ssl_server2
recognize the arguments, nothing more.
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
Add functions and test cases to make sure
tls1.3 is available in openssl/gnutls
Change-Id: I797d15117a8de96614f392e6bb2ed16b6d71ba69
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
Base on version config, `handshack_{clinet,server}_step`
will call different step function. TLS1.3 features will
be gradully added base on it.
And a new test cases is added to make sure it reports
`feature is not available`.
Change-Id: I4f0e36cb610f5aa59f97910fb8204bfbf2825949
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
tls1.3 and tls1.2 can not be enabled at same
time before #4832 resolved.
And the test won't run into `handshake` stage, add
`skip_handshak_check` function to skip it.
Change-Id: I13f3b06b2f33b9c9beb8cac90f5fda41a4ed53f3
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
- Reword the comment on config.h to suggest that
`MAX_INTERMEDIATE_CA` may not exist in the config.
- Add a comment explaining why the tests are more restrictive
than necessary.
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
- Abstract out repetitive checks for IN and OUT content lens
- Remove unclear comment and redundant echo
- Add content length constraints in Renegotiation with fragment length test
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
- Replace last remaining dependency on config.py with query_config
- Replace hard exit with `requires_config_value_at_least` and
`requires_config_value_at_most` to maintain the same effect
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
- Replace calls to config.py for MAX_IN_LEN and MAX_OUT_LEN with
`get_config_value_or_default`
- Remove hard exit when MAX_IN/OUT_LEN < 4096, replace with
`requires_config_value_at_least`
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
Update maybe_requires_ciphersuite_enabled so that it will not skip the
ciphersuite requirement when running a test case where the test expects
a ciphersuite mismatch
Signed-off-by: Dave Rodgman <dave.rodgman@arm.com>
The default curve is now Curve25519, which doesn't support restartable ECC.
So run the restartable ECC tests with a curve that does support it. Use
secp256r1 which is required for these tests anyway for the server's
certificate.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Commit removes tests from ssl-opt.sh
that were specific to
MBEDTLS_SSL_TRUNCATED_HMAC extention.
Signed-off-by: Thomas Daubney <thomas.daubney@arm.com>
This commit removes the API
```
mbedtls_ssl_conf_ciphersuites_for_version()
```
which allows to configure lists of acceptable ciphersuites
for each supported version of SSL/TLS: SSL3, TLS 1.{0,1,2}.
With Mbed TLS 3.0, support for SSL3, TLS 1.0 and TLS 1.1
is dropped. Moreover, upcoming TLS 1.3 support has a different
notion of cipher suite and will require a different API.
This means that it's only for TLS 1.2 that we require
a ciphersuite configuration API, and
```
mbedtls_ssl_conf_ciphersuites()
```
can be used for that. The version-specific ciphersuite
configuration API `mbedtls_ssl_conf_ciphersuites_for_version()`,
in turn, is no longer needed.
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Our interoperability tests fail with a recent OpenSSL server. The
reason is that they force 1024-bit Diffie-Hellman parameters, which
recent OpenSSL (e.g. 1.1.1f on Ubuntu 20.04) reject:
```
140072814650688:error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small:../ssl/s3_lib.c:3782:
```
We've been passing custom DH parameters since
6195767554 because OpenSSL <=1.0.2a
requires it. This is only concerns the version we use as
OPENSSL_LEGACY. So only use custom DH parameters for that version. In
compat.sh, use it based on the observed version of $OPENSSL_CMD.
This way, ssl-opt.sh and compat.sh work (barring other issues) for all
our reference versions of OpenSSL as well as for a modern system OpenSSL.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
An SSL client can be configured to insist on a minimum size for the
Diffie-Hellman (DHM) parameters sent by the server. Add several test
cases where the server sends parameters with exactly the minimum
size (must be accepted) or parameters that are one bit too short (must
be rejected). Make sure that there are test cases both where the
boundary is byte-aligned and where it isn't.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
For calls to gnutls-serv and gnutls-cli where --priority is not
specified, explicitly add the default value: --priority=normal. This is
needed for some tests on Ubuntu 20.04 (gnutls 3.6.13).
For example:
./ssl-opt.sh -f "DTLS fragmenting: gnutls.*1.0"
requires this PR to work on Ubuntu 20.04
Signed-off-by: Dave Rodgman <dave.rodgman@arm.com>
We care about the exit code of our server, for example if it's
reporting a memory leak after having otherwise executed correctly.
We don't care about the exit code of the servers we're using for
interoperability testing (openssl s_server, gnutls-serv). We assume
that they're working correctly anyway, and they return 1 (gnutls-serv)
or die by the signal handle the signal (openssl) when killed by a
signal.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This used to be the case a long time ago but was accidentally broken.
Fix <github:nogrep> #4103 for ssl-opt.sh.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Force using IPv4 in the GNU_CLI SRTP tests, as introduced for
other tests in #1918.
Signed-off-by: Johan Pascal <johan.pascal@belledonne-communications.com>
Since `gnutls-cli` resolves `localhost` as an IPv6 address, and the server
is bound to IPv4 address, gnutl-cli fails to negotiate DTLS sessions.
Force the server to bind to IPv6 address, as a workaround.
Signed-off-by: Johan Pascal <johan.pascal@belledonne-communications.com>
1. Add DTLS-SRTP tests in `ssl-opts.sh`
2. Add logs for the tests to filter.
3. Add function to get the profile informations.
Signed-off-by: Johan Pascal <johan.pascal@belledonne-communications.com>
In a case exprssion, `|` separates patterns so it needs to be quoted.
Also `\` was not actually part of the set since it was quoting another
character.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
If `$FILTER` (`-f`) and `$EXCLUDE` (`-e`) are simple selections that
can be expressed as shell patterns, use a case statement instead of
calling grep to determine whether a test case should be executed.
Using a case statement significantly reduces the time it takes to
determine that a test case is excluded (but the improvement is small
compared to running the test).
This noticeably speeds up running a single test or a small number of
tests. Before:
```
tests/ssl-opt.sh -f Default 1.75s user 0.54s system 79% cpu 2.885 total
```
After:
```
tests/ssl-opt.sh -f Default 0.37s user 0.14s system 29% cpu 1.715 total
```
There is no perceptible difference when running a large number of tests.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Avoid using the external command grep for simple string-based checks.
Prefer a case statement. This improves performance.
The performance improvement is moderate but noticeable when skipping
most tests. When a test is run, the cost of the associated grep calls
is negligible. In this commit, I focused on the uses of grep that can
be easily replaced and that are executed a large number of times.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix `printf "$foo"` which treats the value of `foo` as a printf format
rather than a string.
I used the following command to find potentially problematic lines:
```
git ls-files '*.sh' | xargs egrep 'printf +("?[^"]*|[^ ]*)\$'
```
The remaining ones are false positives for this regexp.
The errors only had minor consequences: the output of `ssl-opt.sh`
contained lines like
```
Renegotiation: gnutls server strict, client-initiated .................. ./tests/ssl-opt.sh: 741: printf: %S: invalid directive
PASS
```
and in case of failure the GnuTLS command containing a substring like
`--priority=NORMAL:%SAFE_RENEGOTIATION` was not included in the log
file. With the current tests, there was no risk of a test failure
going undetected.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Showing a regexp to say that by default all tests are executed is not
particularly helpful.
If we ever add a default exclusion list or a default filter, we can
edit the documentation again.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
As a result, the copyright of contributors other than Arm is now
acknowledged, and the years of publishing are no longer tracked in the
source files.
Also remove the now-redundant lines declaring that the files are part of
MbedTLS.
This commit was generated using the following script:
# ========================
#!/bin/sh
# Find files
find '(' -path './.git' -o -path './3rdparty' ')' -prune -o -type f -print | xargs sed -bi '
# Replace copyright attribution line
s/Copyright.*Arm.*/Copyright The Mbed TLS Contributors/I
# Remove redundant declaration and the preceding line
$!N
/This file is part of Mbed TLS/Id
P
D
'
# ========================
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
Three tests were guarded by `MBEDTLS_KEY_EXCHANGE_ECJPAKE`,
not `MBEDTLS_KEY_EXCHANGE_ECJPAKE_ENABLED`, as it should be.
Curiously, the guard still functioned as intended, perhaps
because `MBEDTLS_KEY_EXCHANGE_ECJPAKE` is a prefix of
`MBEDTLS_KEY_EXCHANGE_ECJPAKE_ENABLED`.
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
The debug output for supported ciphersuites has been changed
from `deadbeef` to `0xdeadbeef` in a previous commit, but the
test script `ssl-opt.sh` grepping for lines in the debug log
to determine test success/failure hadn't been adjusted accordingly.
This commit fixes this.
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
For explicit proxy commands (included with `-p "$P_PXY <args>` in the test
case), it's the test's writer responsibility to handle IPv6; only fix the
proxy command when we're auto-adding it.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Test cases that force a specific ciphersuites are only executed if
this ciphersuite is enabled. But there are test cases (for RC4) whose
goal is to check that the ciphersuite is not used. These test cases
must run even if (or only if) the ciphersuite is disable, so add an
exception for these test cases.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
These fields might be shifted accordingly in `ssl_parse_record_header()`
when receiving a connection with CID, so they require a manual update
after calling the generic `mbedtls_ssl_reset_in_out_pointers()`.
This commit also adds a regression test which is run by all.sh.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
This commit was generated using the following script:
# ========================
#!/bin/sh
# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '
# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
/This file is part of/,$ {
/Copyright.*Arm/I! d
}
}
# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {
# Cut copyright declaration
/Copyright.*Arm/I {
h
N
d
}
# Paste copyright declaration
/"""/ {
x
/./ {
s/^/# / # Add #
x # Replace orignal buffer with Copyright declaration
p # Print original buffer, insert newline
i\
s/.*// # Clear original buffer
}
x
}
}
/Copyright.*Arm/I {
# Print copyright declaration
p
# Read the two lines immediately following the copyright declaration
N
N
# Insert Apache header if it is missing
/SPDX/! {
i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.
# Insert Mbed TLS declaration if it is missing
/This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
}
# Clear copyright declaration from buffer
D
}
'
# ========================
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
1. When `ssl_server2` export key functionality fails,
don't exit the server, but reset it, to have the
server recover for next connection.
2. Add text filters for `export keys functionality` test in ssl-opt.sh
to check for additional output, to verify if the export suceeded.
This was discovered in the `ssl-opt.sh` script, where the server exited,
before the test tried to kill the server priocess, resulting in a
`kill: No such process` message.
Fixes#2662
Signed-off-by: Ron Eldor <Ron.Eldor@arm.com>
This is a convenience for when we get log files from failed CI runs, or attach
them to bug reports, etc.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
A lot of DTLS test are timing-sensitive, especially those that contain
assertions about retransmission. Sometimes some DTLS test fails intermittently
on the CI with no clear apparent reason; we need more information in the log
to understand the cause of those failures.
Adding a proxy means we'll get timing information from the proxy logs.
An alternative would be to add timing information to the debug output of
ssl_server2 and ssl_client2. But that's more complex because getting
sub-second timing info is outside the scope of the C standard, and our current
timing module only provides a APi for sub-second intervals, not absolute time.
Using the proxy is easier as it's a single point that sees all messages, so
elapsed time is fine here, and it's already implemented in the proxy output.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This commit introduces two changes:
- Add in_msg and out_msg calculations for buffer upsizing. This was previously
considered as unnecessary, but renegotiation using certain ciphersuites needs
this.
- Improving the way out_msg and in_msg pointers are calculated, so that even
if no resizing is introduced, the pointers remain the same;
New tests added:
- various renegotiation schemes with a range of MFL's and ciphersuites;
- an ssl-opt.sh test exercising two things that were problematic: renegotiation
with TLS-ECDHE-ECDSA-WITH-AES-128-CCM-8 and a server MFL that's smaller
than the one negotiated by the client.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Since the server might want to have a different maximum fragment length
for the outgoing messages than the negotiated one - introduce a new way of
computing it. This commit also adds additional ssl-opt.sh tests ensuring
that the maximum fragment lengths are set as expected.
mbedtls_ssl_get_max_frag_len() is now a deprecated function,
being an alias to mbedtls_ssl_get_output_max_frag_len(). The behaviour
of this function is the same as before.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
There are currently 4 tests in ssl-opt.sh with either -C "resend" or -S
"resend", that is, asserting that no retransmission will occur. They sometimes
fail on loaded CI machines as one side doesn't send a message fast enough,
causing the other side to retransmit, causing the test to fail.
(For the "reconnect" test there was an other issue causing random failures,
fixed in a previous commit, but even after that fix the test would still
sometimes randomly fail, even if much more rarely.)
While it's a hard problem to fix in a general and perfect way, in practice the
probability of failures can be drastically reduced by making the timeout
values much larger.
For some tests, where retransmissions are actually expected, this would have
the negative effect of increasing the average running time of the test, as
each side would wait for longer before it starts retransmission, so we have a
trade-off between average running time and probability of spurious failures.
But for tests where retransmission is not expected, there is no such trade-off
as the expected running time of the test (assuming the code is correct most of
the time) is not impacted by the timeout value. So the only negative effect of
increasing the timeout value is on the worst-case running time on the test,
which is much less important, as test should only fail quite rarely.
This commit addresses the easy case of tests that don't expect retransmission
by increasing the value of their timeout range to 10s-20s. This value
corresponds to the value used for tests that assert `-S "autoreduction"` which
are in the same case and where the current value seems acceptable so far.
It also represents an increase, compared to the values before this commit, of
a factor 20 for the "reconnect" tests which were frequently observed to fail
in the CI, and of a factor 10 for the first two "DTLS proxy" tests, which were
observed to fail much less frequently, so hopefully the new values are enough
to reduce the probability of spurious failures to an acceptable level.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The server must check client reachability (we chose to do that by checking a
cookie) before destroying the existing association (RFC 6347 section 4.2.8).
Let's make sure we do, by having a proxy-in-the-middle inject a ClientHello -
the server should notice, but not destroy the connection.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
- "Default" should only be used for tests that actually use the defaults (ie,
not passing options on the command line, except maybe debug/dtls)
- All tests in the "Encrypt then MAC" group should start with that string as a
common prefix
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The ssl-opt.sh test cases using session resumption tend to fail occasionally
on the CI due to a race condition in how ssl_server2 and ssl_client2 handle
the reconnection cycle.
The server does the following in order:
- S1 send application data
- S2 send a close_notify alert
- S3 close the client socket
- S4 wait for a "new connection" (actually a new datagram)
- S5 start a handshake
The client does the following in order:
- C1 wait for and read application data from the server
- C2 send a close_notify alert
- C3 close the server socket
- C4 reset session data and re-open a server socket
- C5 start a handshake
If the client has been able to send the close_notify (C2) and if has been
delivered to the server before if closes the client socket (S3), when the
server reaches S4, the datagram that we start the new connection will be the
ClientHello and everything will be fine.
However if S3 wins the race and happens before the close_notify is delivered,
in S4 the close_notify is what will be seen as the first datagram in a new
connection, and then in S5 this will rightfully be rejected as not being a
valid ClientHello and the server will close the connection (and go wait for
another one). The client will then fail to read from the socket and exit
non-zero and the ssl-opt.sh harness will correctly report this as a failure.
In order to avoid this race condition in test using ssl_client2 and
ssl_server2, this commits introduces a new command-line option
skip_close_notify to ssl_client2 and uses it in all ssl-opt.sh tests that use
session resumption with DTLS and ssl_server2.
This works because ssl_server2 knows how many messages it expects in each
direction and in what order, and closes the connection after that rather than
relying on close_notify (which is also why there was a race in the first
place).
Tests that use another server (in practice there are two of them, using
OpenSSL as a server) wouldn't work with skip_close_notify, as the server won't
close the connection until the client sends a close_notify, but for the same
reason they don't need it (there is no race between receiving close_notify and
closing as the former is the cause of the later).
An alternative approach would be to make ssl_server2 keep the connection open
until it receives a close_notify. Unfortunately it creates problems for tests
where we simulate a lossy network, as the close_notify could be lost (and the
client can't retransmit it). We could modify udp_proxy with an option to never
drop alert messages, but when TLS 1.3 comes that would no longer work as the
type of messages will be encrypted.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The splitting of this test into two versions depending on whether SHA-1 was
allowed by the server was a mistake in
5d2511c4d4 - the test has nothing to do with
SHA-1 in the first place, as the server doesn't request a certificate from
the client so it doesn't matter if the server accepts SHA-1 or not.
While the whole script makes (often implicit) assumptions about the version of
GnuTLS used, generally speaking it should work out of the box with the version
packaged on our reference testing platform, which is Ubuntu 16.04 so far.
With the update from Jan 8 2020 (3.4.10-4ubuntu1.6), the patches for rejecting
SHA-1 in certificate signatures were backported, so we should avoid presenting
SHA-1 signed certificates to a GnuTLS peer in ssl-opt.sh.
* origin/pr/2843: (26 commits)
Make hyperlink a hyperlink in every markdown flavor
Update the crypto submodule to be the same as development
Document test case descriptions
Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
ssl-opt.sh: Fix some test case descriptions
Reject non-ASCII characters in test case descriptions
Process input files as binary
Factor description-checking code into a common function
Fix cosmetic error in warnings
Fix regex matching run_test calls in ssl-opt.sh
all.sh: run check-test-cases.py
Better information messages for quick checks
Fix configuration short name in key-exchanges.pl
Make test case descriptions unique
New test script check-test-cases.py
Document the test outcome file
Create infrastructure for architecture documents in Markdown
all.sh --outcome-file creates an outcome file
Set meaningful test configuration names when running tests
ssl-opt: remove semicolons from test case descriptions
...
Fix copypasta in some test cases with
MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES enabled.
Add unique suffix to the two
"DTLS fragmenting: proxy MTU: auto-reduction" test cases.
Don't use semicolons in test case descriptions. The test outcome file
is a semicolon-separated CSV file without quotes to keep things
simple, so fields in that file may not contain semicolons.
