Arranging mocks using DSL

One of the biggest problems with unit tests is poor readability. Bad naming convention, long methods, hard to understand Arrangement and Assert parts are making unit tests one of the hardest code to read and refactor. In previous article,
Unit Tests as code specification
, I presented the way to increase readability of test method names and use them to create code specification. Now I would like to tackle the problem of unreadable test methods.

Most of unit tests methods start with test arrangement. It usually takes a form of setting up mocks and initialising local variables. It’s not uncommon to start the test with code similar to the one below:

_testService.Expect(i => i.Foo()).Throw(new WebException("The operation has timed out")).Repeat.Once();
_testService.Expect(i => i.Foo()).Return(9).Repeat.Once();

The example prepares _testService to throw a WebException the first time the method Foo is called and then return 9 when called again. There are few problems with this set-up. It requires careful reading and analysis of the code to understand the arrangement. If there are more set ups and you get confused about them, you will need to analyse it again. In the presented example, some information is concealed – the WebException represents a time out exception thrown by Google AdWords service, which is in opposition to CommunicationException thrown by web services such as Bing Ads.

The above example is fairly simple, but the situation gets worse when there is a need to set-up few mocks, or there are parameters to be passed into mocked method.

Lets see how above example could look when using DSL:

_testService
   .SimulateGoogleAdWords()
   .FailWithTimeOut().Once()
   .Then().Return(9);

We are using Domain Specific Language to say that test is for Google AdWords, where service throws time out when called first time and then returns 9 after retry. This version is easy to read and remember, carries the information that we are testing the case with Google AdWords, and it is easy to reference back to it.

The implementation of DSL is done using an extension methods on mocked object. Sample code associated with this article uses Rhino Mocks, but it can easily be converted to use other mocking frameworks.

Advertisements

Unit Tests as code specification

When asking people what is the purpose of writing unit tests we usually get following answer:

“To verify that the code actually does what it is supposed to do.”

Among other responses we will find that unit tests help to validate that changes are not breaking existing functionality (regression), or that practising TDD will guide the design. But are those the only purposes? Well, there is more. Because unit tests are executing our code, they can show how it is working. We can use them as a specification of the code. Well crafted tests, which have explaining names and are easy to read, create a live specification of the module, which is always up to date.

Whenever we need to analyse a class, whether because we are new to it or we are coming back, we can use reports from unit tests to get the understanding how the class is working and what is it’s contract.

To build a specification from unit tests, we need to keep them organised and apply proper naming convention.

Test class names

First of all, there is no need to keep all unit tests related to given class in one unit test class. We can, and actually should, have more than one unit test class per tested class. Good convention is to have a class per functionality.

Let’s consider an example of recoverable policy which retries service call on communication errors. Whenever the connection drops or times out we want to wait some time and retry the operation, using exponential back off. Here we have few functionalities which should be tested:

  • retrying on supported communication error
  • rethrowing unsupported exceptions
  • retrying logic

To organise our test suite let’s create a folder for all unit tests related to tested class. This will be name of tested class postfixed with word “Tests”. The postfix is important to avoid conflicts between the name of the class which we are going to tests and the name of the assembly containing tests. In out example it will be : CommunicationErrorRecoveryPolicyTests.

Unit tests classes are named after the functionality, such as: WhenOperationFailsWithCommunicationException, WhenFailsWithUnsupportedError or WhenHandlesError. The class name partially describes test Arrangement, i.e. what are the cases we are testing.

Test method names

The method name may optionally describe further test Arrangement. It also describes Assertion. Example method names for the WhenOperationFailsWithCommunicationException cases can be:

  • BecauseOfTimeoutThenRetires – which performs a test of the case when operation failed due to server time out
  • BecauseOfConnectionDropsThenRetires – for the case when remote server dropped connection

The naming convention is to elaborate more on Arrangement (BecauseOfTimeout) and then explain what is the result (ThenRetires). The first part can be skipped if all Arrangements are contained in class name. The second part is required and should not be omitted, same as method body should always have assertions.

Note, that with test method names there is no need to keep them concise as is the case with normal methods. They are not referenced anywhere and serve better its purpose when are expressive.

Generating specification

Once the naming convention is applied, we can use report from unit test runner to generate the specification. Below is a snapshot from ReSharper test runner:

Report from running Unit Tests can be used to generate code specification (example using ReSharper test runner).

Report from running Unit Tests can be used to generate code specification (example using ReSharper test runner).


I am grouping tests by Namespaces to create hierarchy which can be read like sentence:


CommunicationErrorRecoveryPolicyTests ->
   When operation fails with communication exception because of 502 then retries.
   When operation fails with communication exception because of time out then retries it.

To find out what exceptions are covered by the policy I will run associated tests (all contained in CommunicationErrorRecoveryPolicyTests folder) and look at generated report.

We can even go one step further, and autogenerate HTML documentation from unit test reports. This could be run by CI server after each build and saved among artefacts.

Accompanying source code

The accompanying source code can be found on github at: https://github.com/mariuszwojcik/SpecificationByExample. For examples related to this article check SpecificationByExample.Domain.UnitTests/SpecByExample.

Summary

Properly organising test suites adds another aspect to unit testing. Good, expressive naming convention allows for building a specification of the code. It also helps with future refactorings and unit tests maintenance. Using that simple method allows for increasing teams agility and productivity by helping developers quickly understand purpose of tests and workings of tested classes.

MSTest: ExpectedExceptionWithMessageAttribute

The unit testing system which comes with Visual Studio offers a way to assert whether a test has thrown specific exception. It is done by using ExpectedExceptionAttribute on test method:

[TestMethod]
[ExpectedException(typeof(InvalidOperationException))]
public void ThrowsInvalidOperationException()
{
    throw new InvalidOperationException();
}

The problem with this solution is that it checks only for the type of the exception, but there is no way to assert exception’s message (There is overloaded constructor which takes string as a second parameter, but this string is not a message to assert, but it is a message which will be displayed when assertion fails.). As exceptions are an important part of application’s domain, our unit tests should assert them and check whether messages are containing useful and valid information. Below example, which is very simplistic, shows how one exception type may reference to two different conditions:

        public void ProcessNameAndDescription(string name, string description)
        {
            if (name == null)
                throw new ArgumentNullException("name", "Parameter \"name\" may not be null.");

            if (description == null)
                throw new ArgumentNullException("description", "Parameter \"description\" may not be null.");

            // rest of the code
        }

In the above example, the ArgumentNullAttribute can be thrown because of two reasons: either the name or description parameter is null. The tests could be as follow:

        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void ThrowsArgumentNullExceptionWhenProcessingNullName()
        {
            ProcessNameAndDescription(null, "some description");
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void ThrowsArgumentNullExceptionWhenProcessingNullDescription()
        {
            ProcessNameAndDescription("some name", null);
        }

That works, although it is really easy to make a mistake and assert wrong condition (especially when you use copy pasting for such methods), and without asserting exception’s message there is no way to discover which parameter was wrong.

ExpectedExceptionWithMessageAttribute

Fortunately the unit testing framework can be easily extended and we can create our own attribute which will assert for exception and its message. All what has to be done is to create a new attribute class which inherits from ExpectedExceptionBaseAttribute and provides the implementation of Verify abstract method. When tested code throws an exception, this exception is passed to Verify method and can be checked whether it is what should be expected or not. Below is a source code for sample implementation:

    [AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
    public sealed class ExpectedExceptionWithMessageAttribute : ExpectedExceptionBaseAttribute
    {
        #region private members

        [DebuggerBrowsable(DebuggerBrowsableState.Never)]
        private readonly Type _exceptionType;

        [DebuggerBrowsable(DebuggerBrowsableState.Never)]
        private readonly string _expectedMessagePattern;

        #endregion

        #region public properties

        public Type ExceptionType
        {
            get { return this._exceptionType; }
        }

        public string ExpectedMessagePattern
        {
            get { return this._expectedMessagePattern; }
        }

        public bool AllowDerivedTypes
        {
            get;
            set;
        }

        #endregion

        #region ctor()

        public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage)
            : this(exceptionType, expectedMessage, String.Empty)
        {
        }

        public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage, string noExceptionMessage)
            : base(noExceptionMessage)
        {
            #region preconditions

            if (exceptionType == null)
                throw new ArgumentNullException("exceptionType", "Parameter \"exceptionType\" may not be null.");

            if (!typeof(Exception).IsAssignableFrom(exceptionType))
                throw new ArgumentException("The expected exception type must be System.Exception or a type derived from System.Exception.", "exceptionType");

            if (expectedMessage == null)
                throw new ArgumentNullException("expectedMessage", "Parameter \"expectedMessage\" may not be null.");

            #endregion

            this._exceptionType = exceptionType;
            this._expectedMessagePattern = expectedMessage;
        }

        #endregion

        protected override void Verify(Exception exception)
        {
            Type exceptionType = exception.GetType();

            if (this.AllowDerivedTypes)
            {
                if (!this.ExceptionType.IsAssignableFrom(exceptionType))
                {
                    base.RethrowIfAssertException(exception);
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw exception {2}, but exception {3} or a type derived from it was expected.\r\nException message: {4}.", exception);
                }
                else if (!Regex.IsMatch(exception.Message, this.ExpectedMessagePattern))
                {
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw expected exception {2} with message \"{4}\" but message with pattern \"{5}\" was expected.", exception);
                }
            }
            else
            {
                if (exceptionType != this.ExceptionType)
                {
                    base.RethrowIfAssertException(exception);
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw exception {2}, but exception {3} was expected.\r\nException message: {4}.", exception);
                }
                else if (!Regex.IsMatch(exception.Message, this.ExpectedMessagePattern))
                {
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw expected exception {2} with message \"{4}\" but message with pattern \"{5}\" was expected.", exception);
                }
            }
        }

        private void HandleInvalidExpectedExceptionOrMessage(string messageTemplate, Exception exceptionThrow)
        {
            throw new Exception(String.Format(messageTemplate,
                        base.TestContext.FullyQualifiedTestClassName,
                        base.TestContext.TestName,
                        exceptionThrow.GetType().FullName,
                        this.ExceptionType.FullName,
                        exceptionThrow.Message,
                        this.ExpectedMessagePattern));
        }
    }

Now, we can assert exception for a message pattern (using regular expressions) and our tests may be changed to following:

        [TestMethod]
        [ExpectedExceptionWithMessage(typeof(ArgumentNullException), "Parameter \"name\" may not be null.")]
        public void ThrowsArgumentNullExceptionWhenProcessingNullName()
        {
            ProcessNameAndDescription(null, "some description");
        }

        [TestMethod]
        [ExpectedExceptionWithMessage(typeof(ArgumentNullException), "Parameter \"description\".*")]
        public void ThrowsArgumentNullExceptionWhenProcessingNullDescription()
        {
            ProcessNameAndDescription("some name", null);
        }

Final note

One final note on asserting exceptions. I do not recommend using this approach for every exception and write your code in a way that only exception’s message distinguishes between different types of errors. The application should be designed in a way, that all domain specific exceptions are represented with their own exception types, such as: ClientAccountLockedOut, DeliveryAddressUnrecognised, etc. so we can assert for those exceptions and assert exception’s message to confirm that it contains valuable information (an Id of client’s account, or description why the address has not been recognised).

Create test objects using Factory Methods or Object Mother pattern

One of the nightmares in unit testing is creating instances of tested objects. While our tested object grows, very quickly the code repetition appears in the test fixture. It is very important to keep unit test code clean, so the changes to tested object can be easily introduced. It’s also important for unit test code to be easy to read. Very often you can find following code in the test fixture:

[TestMethod]
public void AddressIsEqualToItself()
{
    Address address = new Address("United Kingdom", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA");

    Assert.AreEqual(address, address);
}

[TestMethod]
public void AddressesAreEqualWhenAllFieldsAreEqual()
{
    Address address = new Address("United Kingdom", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA");

    Assert.AreEqual(new Address("United Kingdom", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA"), address);

    Assert.AreNotEqual(new Address("United States", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA"), address);
    Assert.AreNotEqual(new Address("United Kingdom", "Middlesex", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA"), address);
}

// more tests requiring instances of Address

The problem with this approach is, that if Address class will have to change the definition of constructor all those test will also require a change. This can be solved by using Factory Method pattern:

private Address CreateAddress(string country, string county, string city, string street, string houseName, string postCode)
{
    return new Address(country, county, city, street, houseName, postCode);
}

private Address CreateAddressInUK(string county = "Surrey", string city = "Guildford", string street = "St. George Av.", string houseName = "Elliot House", string postCode = "GU3 1DA")
{
    return this.CreateAddress("United Kingdom", county, city, street, houseName, postCode);
}

private Address CreateAddressInUS(string county = "Surrey", string city = "Guildford", string street = "St. George Av.", string houseName = "Elliot House", string postCode = "GU3 1DA")
{
    return this.CreateAddress("United States", county, city, street, houseName, postCode);
}

[TestMethod]
public void AddressesAreNotEqualWhenCountriesDiffer()
{
    Address address = this.CreateAddressInUK();

    Assert.AreNotEqual(this.CreateAddressInUS(), address);
}

In the above example there are three Factory Methods. One generic allowing to create any valid address, and two specialised to create addresses in the UK or US. The specialised methods are defined with optional parameters which simplifies tests even more.

To go even further with this approach, the Object Mother pattern can be applied by creating a class containing specialised factory methods, such as: CreateAddressInGuildfordUK, CreateAddressInLondonUK, CreateAddressInNY, etc.:

internal static class AddressMother
{
    public static Address CreateAddress(string country, string county, string city, string street, string houseName, string postCode)
    {
        return new Address(country, county, city, street, houseName, postCode);
    }

    public static Address CreateAddressInGuildfordUK(string street = "St. George Av.", string houseName = "Elliot House", string postCode = "GU3 1DA")
    {
        return CreateAddress("United Kingdom", "Surrey", "Guildford", street, houseName, postCode);
    }

    // ...
}

Using Object Mother class is very useful when there is more than one test which requires to create instances of the object, like the test for Customer class which aggregates an Address.

References: