dynarmic/docs/matchers.md

Matchers

Contents
Using Matchers
Built-in matchers
Writing custom matchers (old style)
Writing custom matchers (new style)

Matchers, as popularized by the Hamcrest framework are an alternative way to write assertions, useful for tests where you work with complex types or need to assert more complex properties. Matchers are easily composable and users can write their own and combine them with the Catch2-provided matchers seamlessly.

Using Matchers

Matchers are most commonly used in tandem with the REQUIRE_THAT or CHECK_THAT macros. The REQUIRE_THAT macro takes two arguments, the first one is the input (object/value) to test, the second argument is the matcher itself.

For example, to assert that a string ends with the "as a service" substring, you can write the following assertion

using Catch::Matchers::EndsWith;

REQUIRE_THAT( getSomeString(), EndsWith("as a service") );

Individual matchers can also be combined using the C++ logical operators, that is &&, ||, and !, like so:

using Catch::Matchers::EndsWith;
using Catch::Matchers::ContainsSubstring;

REQUIRE_THAT( getSomeString(),
              EndsWith("as a service") && ContainsSubstring("web scale"));

The example above asserts that the string returned from getSomeString both ends with the suffix "as a service" and contains the string "web scale" somewhere.

Both of the string matchers used in the examples above live in the catch_matchers_string.hpp header, so to compile the code above also requires #include <catch2/matchers/catch_matchers_string.hpp>.

IMPORTANT: The combining operators do not take ownership of the matcher objects being combined. This means that if you store combined matcher object, you have to ensure that the matchers being combined outlive its last use. What this means is that the following code leads to a use-after-free (UAF):

#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_string.hpp>

TEST_CASE("Bugs, bugs, bugs", "[Bug]"){
    std::string str = "Bugs as a service";

    auto match_expression = Catch::Matchers::EndsWith( "as a service" ) ||
        (Catch::Matchers::StartsWith( "Big data" ) && !Catch::Matchers::ContainsSubstring( "web scale" ) );
    REQUIRE_THAT(str, match_expression);
}

Built-in matchers

Every matcher provided by Catch2 is split into 2 parts, a factory function that lives in the Catch::Matchers namespace, and the actual matcher type that is in some deeper namespace and should not be used by the user. In the examples above, we used Catch::Matchers::Contains. This is the factory function for the Catch::Matchers::StdString::ContainsMatcher type that does the actual matching.

Out of the box, Catch2 provides the following matchers:

std::string matchers

Catch2 provides 5 different matchers that work with std::string,

• StartsWith(std::string str, CaseSensitive),
• EndsWith(std::string str, CaseSensitive),
• ContainsSubstring(std::string str, CaseSensitive),
• Equals(std::string str, CaseSensitive), and
• Matches(std::string str, CaseSensitive).

The first three should be fairly self-explanatory, they succeed if the argument starts with str, ends with str, or contains str somewhere inside it.

The Equals matcher matches a string if (and only if) the argument string is equal to str.

Finally, the Matches matcher performs an ECMAScript regex match using str against the argument string. It is important to know that the match is performed against the string as a whole, meaning that the regex "abc" will not match input string "abcd". To match "abcd", you need to use e.g. "abc.*" as your regex.

The second argument sets whether the matching should be case-sensitive or not. By default, it is case-sensitive.

std::string matchers live in catch2/matchers/catch_matchers_string.hpp

Vector matchers

Vector matchers have been deprecated in favour of the generic range matchers with the same functionality.

Catch2 provides 5 built-in matchers that work on std::vector.

These are

• Contains which checks whether a specified vector is present in the result
• VectorContains which checks whether a specified element is present in the result
• Equals which checks whether the result is exactly equal (order matters) to a specific vector
• UnorderedEquals which checks whether the result is equal to a specific vector under a permutation
• Approx which checks whether the result is "approx-equal" (order matters, but comparison is done via Approx) to a specific vector

Approx matcher was introduced in Catch2 2.7.2.

An example usage:

    std::vector<int> some_vec{ 1, 2, 3 };
    REQUIRE_THAT(some_vec, Catch::Matchers::UnorderedEquals(std::vector<int>{ 3, 2, 1 }));

This assertions will pass, because the elements given to the matchers are a permutation of the ones in some_vec.

vector matchers live in catch2/matchers/catch_matchers_vector.hpp

Floating point matchers

Catch2 provides 4 matchers that target floating point numbers. These are:

• WithinAbs(double target, double margin),
• WithinULP(FloatingPoint target, uint64_t maxUlpDiff), and
• WithinRel(FloatingPoint target, FloatingPoint eps).
• IsNaN()

WithinRel matcher was introduced in Catch2 2.10.0

IsNaN matcher was introduced in Catch2 3.3.2.

The first three serve to compare two floating pointe numbers. For more details about how they work, read the docs on comparing floating point numbers.

IsNaN then does exactly what it says on the tin. It matches the input if it is a NaN (Not a Number). The advantage of using it over just plain REQUIRE(std::isnan(x)), is that if the check fails, with REQUIRE you won't see the value of x, but with REQUIRE_THAT(x, IsNaN()), you will.

Miscellaneous matchers

Catch2 also provides some matchers and matcher utilities that do not quite fit into other categories.

The first one of them is the Predicate(Callable pred, std::string description) matcher. It creates a matcher object that calls pred for the provided argument. The description argument allows users to set what the resulting matcher should self-describe as if required.

Do note that you will need to explicitly specify the type of the argument, like in this example:

REQUIRE_THAT("Hello olleH",
             Predicate<std::string>(
                 [] (std::string const& str) -> bool { return str.front() == str.back(); },
                 "First and last character should be equal")
);

the predicate matcher lives in catch2/matchers/catch_matchers_predicate.hpp

The other miscellaneous matcher utility is exception matching.

Matching exceptions

Catch2 provides a utility macro for asserting that an expression throws exception of specific type, and that the exception has desired properties. The macro is REQUIRE_THROWS_MATCHES(expr, ExceptionType, Matcher).

REQUIRE_THROWS_MATCHES macro lives in catch2/matchers/catch_matchers.hpp

Catch2 currently provides two matchers for exceptions. These are:

• Message(std::string message).
• MessageMatches(Matcher matcher).

MessageMatches was introduced in Catch2 3.3.0

Message checks that the exception's message, as returned from what is exactly equal to message.

MessageMatches applies the provided matcher on the exception's message, as returned from what. This is useful in conjunctions with the std::string matchers (e.g. StartsWith)

Example use:

REQUIRE_THROWS_MATCHES(throwsDerivedException(),  DerivedException,  Message("DerivedException::what"));
REQUIRE_THROWS_MATCHES(throwsDerivedException(),  DerivedException,  MessageMatches(StartsWith("DerivedException")));

Note that DerivedException in the example above has to derive from std::exception for the example to work.

the exception message matcher lives in catch2/matchers/catch_matchers_exception.hpp

Generic range Matchers

Generic range matchers were introduced in Catch2 3.0.1

Catch2 also provides some matchers that use the new style matchers definitions to handle generic range-like types. These are:

• IsEmpty()
• SizeIs(size_t target_size)
• SizeIs(Matcher size_matcher)
• Contains(T&& target_element, Comparator = std::equal_to<>{})
• Contains(Matcher element_matcher)
• AllMatch(Matcher element_matcher)
• AnyMatch(Matcher element_matcher)
• NoneMatch(Matcher element_matcher)
• AllTrue(), AnyTrue(), NoneTrue()
• RangeEquals(TargetRangeLike&&, Comparator = std::equal_to<>{})
• UnorderedRangeEquals(TargetRangeLike&&, Comparator = std::equal_to<>{})

IsEmpty, SizeIs, Contains were introduced in Catch2 3.0.1

All/Any/NoneMatch were introduced in Catch2 3.0.1

All/Any/NoneTrue were introduced in Catch2 3.1.0

RangeEquals and UnorderedRangeEquals matchers were introduced in Catch2 3.3.0

IsEmpty should be self-explanatory. It successfully matches objects that are empty according to either std::empty, or ADL-found empty free function.

SizeIs checks range's size. If constructed with size_t arg, the matchers accepts ranges whose size is exactly equal to the arg. If constructed from another matcher, then the resulting matcher accepts ranges whose size is accepted by the provided matcher.

Contains accepts ranges that contain specific element. There are again two variants, one that accepts the desired element directly, in which case a range is accepted if any of its elements is equal to the target element. The other variant is constructed from a matcher, in which case a range is accepted if any of its elements is accepted by the provided matcher.

AllMatch, NoneMatch, and AnyMatch match ranges for which either all, none, or any of the contained elements matches the given matcher, respectively.

AllTrue, NoneTrue, and AnyTrue match ranges for which either all, none, or any of the contained elements are true, respectively. It works for ranges of bools and ranges of elements (explicitly) convertible to bool.

RangeEquals compares the range that the matcher is constructed with (the "target range") against the range to be tested, element-wise. The match succeeds if all elements from the two ranges compare equal (using operator== by default). The ranges do not need to be the same type, and the element types do not need to be the same, as long as they are comparable. (e.g. you may compare std::vector<int> to std::array<char>).

UnorderedRangeEquals is similar to RangeEquals, but the order does not matter. For example "1, 2, 3" would match "3, 2, 1", but not "1, 1, 2, 3" As with RangeEquals, UnorderedRangeEquals compares the individual elements using operator== by default.

Both RangeEquals and UnorderedRangeEquals optionally accept a predicate which can be used to compare the containers element-wise.

To check a container elementwise against a given matcher, use AllMatch.

Writing custom matchers (old style)

The old style of writing matchers has been introduced back in Catch Classic. To create an old-style matcher, you have to create your own type that derives from Catch::Matchers::MatcherBase<ArgT>, where ArgT is the type your matcher works for. Your type has to override two methods, bool match(ArgT const&) const, and std::string describe() const.

As the name suggests, match decides whether the provided argument is matched (accepted) by the matcher. describe then provides a human-oriented description of what the matcher does.

We also recommend that you create factory function, just like Catch2 does, but that is mostly useful for template argument deduction for templated matchers (assuming you do not have CTAD available).

To combine these into an example, let's say that you want to write a matcher that decides whether the provided argument is a number within certain range. We will call it IsBetweenMatcher<T>:

#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers.hpp>
// ...


template <typename T>
class IsBetweenMatcher : public Catch::Matchers::MatcherBase<T> {
    T m_begin, m_end;
public:
    IsBetweenMatcher(T begin, T end) : m_begin(begin), m_end(end) {}

    bool match(T const& in) const override {
        return in >= m_begin && in <= m_end;
    }

    std::string describe() const override {
        std::ostringstream ss;
        ss << "is between " << m_begin << " and " << m_end;
        return ss.str();
    }
};

template <typename T>
IsBetweenMatcher<T> IsBetween(T begin, T end) {
    return { begin, end };
}

// ...

TEST_CASE("Numbers are within range") {
    // infers `double` for the argument type of the matcher
    CHECK_THAT(3., IsBetween(1., 10.));
    // infers `int` for the argument type of the matcher
    CHECK_THAT(100, IsBetween(1, 10));
}

Obviously, the code above can be improved somewhat, for example you might want to static_assert over the fact that T is an arithmetic type... or generalize the matcher to cover any type for which the user can provide a comparison function object.

Note that while any matcher written using the old style can also be written using the new style, combining old style matchers should generally compile faster. Also note that you can combine old and new style matchers arbitrarily.

MatcherBase lives in catch2/matchers/catch_matchers.hpp

Writing custom matchers (new style)

New style matchers were introduced in Catch2 3.0.1

To create a new-style matcher, you have to create your own type that derives from Catch::Matchers::MatcherGenericBase. Your type has to also provide two methods, bool match( ... ) const and overridden std::string describe() const.

Unlike with old-style matchers, there are no requirements on how the match member function takes its argument. This means that the argument can be taken by value or by mutating reference, but also that the matcher's match member function can be templated.

This allows you to write more complex matcher, such as a matcher that can compare one range-like (something that responds to begin and end) object to another, like in the following example:

#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_templated.hpp>
// ...

template<typename Range>
struct EqualsRangeMatcher : Catch::Matchers::MatcherGenericBase {
    EqualsRangeMatcher(Range const& range):
        range{ range }
    {}

    template<typename OtherRange>
    bool match(OtherRange const& other) const {
        using std::begin; using std::end;

        return std::equal(begin(range), end(range), begin(other), end(other));
    }

    std::string describe() const override {
        return "Equals: " + Catch::rangeToString(range);
    }

private:
    Range const& range;
};

template<typename Range>
auto EqualsRange(const Range& range) -> EqualsRangeMatcher<Range> {
    return EqualsRangeMatcher<Range>{range};
}

TEST_CASE("Combining templated matchers", "[matchers][templated]") {
    std::array<int, 3> container{{ 1,2,3 }};

    std::array<int, 3> a{{ 1,2,3 }};
    std::vector<int> b{ 0,1,2 };
    std::list<int> c{ 4,5,6 };

    REQUIRE_THAT(container, EqualsRange(a) || EqualsRange(b) || EqualsRange(c));
}

Do note that while you can rewrite any matcher from the old style to a new style matcher, combining new style matchers is more expensive in terms of compilation time. Also note that you can combine old style and new style matchers arbitrarily.

MatcherGenericBase lives in catch2/matchers/catch_matchers_templated.hpp

---

Home