MSTest: ExpectedExceptionWithMessageAttribute

The unit testing system which comes with Visual Studio offers a way to assert whether a test has thrown specific exception. It is done by using ExpectedExceptionAttribute on test method:

[TestMethod]
[ExpectedException(typeof(InvalidOperationException))]
public void ThrowsInvalidOperationException()
{
    throw new InvalidOperationException();
}

The problem with this solution is that it checks only for the type of the exception, but there is no way to assert exception’s message (There is overloaded constructor which takes string as a second parameter, but this string is not a message to assert, but it is a message which will be displayed when assertion fails.). As exceptions are an important part of application’s domain, our unit tests should assert them and check whether messages are containing useful and valid information. Below example, which is very simplistic, shows how one exception type may reference to two different conditions:

        public void ProcessNameAndDescription(string name, string description)
        {
            if (name == null)
                throw new ArgumentNullException("name", "Parameter \"name\" may not be null.");

            if (description == null)
                throw new ArgumentNullException("description", "Parameter \"description\" may not be null.");

            // rest of the code
        }

In the above example, the ArgumentNullAttribute can be thrown because of two reasons: either the name or description parameter is null. The tests could be as follow:

        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void ThrowsArgumentNullExceptionWhenProcessingNullName()
        {
            ProcessNameAndDescription(null, "some description");
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void ThrowsArgumentNullExceptionWhenProcessingNullDescription()
        {
            ProcessNameAndDescription("some name", null);
        }

That works, although it is really easy to make a mistake and assert wrong condition (especially when you use copy pasting for such methods), and without asserting exception’s message there is no way to discover which parameter was wrong.

ExpectedExceptionWithMessageAttribute

Fortunately the unit testing framework can be easily extended and we can create our own attribute which will assert for exception and its message. All what has to be done is to create a new attribute class which inherits from ExpectedExceptionBaseAttribute and provides the implementation of Verify abstract method. When tested code throws an exception, this exception is passed to Verify method and can be checked whether it is what should be expected or not. Below is a source code for sample implementation:

    [AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
    public sealed class ExpectedExceptionWithMessageAttribute : ExpectedExceptionBaseAttribute
    {
        #region private members

        [DebuggerBrowsable(DebuggerBrowsableState.Never)]
        private readonly Type _exceptionType;

        [DebuggerBrowsable(DebuggerBrowsableState.Never)]
        private readonly string _expectedMessagePattern;

        #endregion

        #region public properties

        public Type ExceptionType
        {
            get { return this._exceptionType; }
        }

        public string ExpectedMessagePattern
        {
            get { return this._expectedMessagePattern; }
        }

        public bool AllowDerivedTypes
        {
            get;
            set;
        }

        #endregion

        #region ctor()

        public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage)
            : this(exceptionType, expectedMessage, String.Empty)
        {
        }

        public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage, string noExceptionMessage)
            : base(noExceptionMessage)
        {
            #region preconditions

            if (exceptionType == null)
                throw new ArgumentNullException("exceptionType", "Parameter \"exceptionType\" may not be null.");

            if (!typeof(Exception).IsAssignableFrom(exceptionType))
                throw new ArgumentException("The expected exception type must be System.Exception or a type derived from System.Exception.", "exceptionType");

            if (expectedMessage == null)
                throw new ArgumentNullException("expectedMessage", "Parameter \"expectedMessage\" may not be null.");

            #endregion

            this._exceptionType = exceptionType;
            this._expectedMessagePattern = expectedMessage;
        }

        #endregion

        protected override void Verify(Exception exception)
        {
            Type exceptionType = exception.GetType();

            if (this.AllowDerivedTypes)
            {
                if (!this.ExceptionType.IsAssignableFrom(exceptionType))
                {
                    base.RethrowIfAssertException(exception);
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw exception {2}, but exception {3} or a type derived from it was expected.\r\nException message: {4}.", exception);
                }
                else if (!Regex.IsMatch(exception.Message, this.ExpectedMessagePattern))
                {
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw expected exception {2} with message \"{4}\" but message with pattern \"{5}\" was expected.", exception);
                }
            }
            else
            {
                if (exceptionType != this.ExceptionType)
                {
                    base.RethrowIfAssertException(exception);
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw exception {2}, but exception {3} was expected.\r\nException message: {4}.", exception);
                }
                else if (!Regex.IsMatch(exception.Message, this.ExpectedMessagePattern))
                {
                    this.HandleInvalidExpectedExceptionOrMessage("Test method {0}.{1} threw expected exception {2} with message \"{4}\" but message with pattern \"{5}\" was expected.", exception);
                }
            }
        }

        private void HandleInvalidExpectedExceptionOrMessage(string messageTemplate, Exception exceptionThrow)
        {
            throw new Exception(String.Format(messageTemplate,
                        base.TestContext.FullyQualifiedTestClassName,
                        base.TestContext.TestName,
                        exceptionThrow.GetType().FullName,
                        this.ExceptionType.FullName,
                        exceptionThrow.Message,
                        this.ExpectedMessagePattern));
        }
    }

Now, we can assert exception for a message pattern (using regular expressions) and our tests may be changed to following:

        [TestMethod]
        [ExpectedExceptionWithMessage(typeof(ArgumentNullException), "Parameter \"name\" may not be null.")]
        public void ThrowsArgumentNullExceptionWhenProcessingNullName()
        {
            ProcessNameAndDescription(null, "some description");
        }

        [TestMethod]
        [ExpectedExceptionWithMessage(typeof(ArgumentNullException), "Parameter \"description\".*")]
        public void ThrowsArgumentNullExceptionWhenProcessingNullDescription()
        {
            ProcessNameAndDescription("some name", null);
        }

Final note

One final note on asserting exceptions. I do not recommend using this approach for every exception and write your code in a way that only exception’s message distinguishes between different types of errors. The application should be designed in a way, that all domain specific exceptions are represented with their own exception types, such as: ClientAccountLockedOut, DeliveryAddressUnrecognised, etc. so we can assert for those exceptions and assert exception’s message to confirm that it contains valuable information (an Id of client’s account, or description why the address has not been recognised).

Advertisements

Create test objects using Builder pattern and methods chaining

In my previous post “Create test objects using Factory Methods or Object Mother pattern” I described how Factory Method and Object Mother patterns can be used to help with creating object instances. Presented solution allows to remove code duplication from unit tests making them easier to adopt changes in tested objects, but doesn’t help much with test code readability, which is very important factor in maintaining unit tests. Using sample Address class from the previous article, imagine following code:

            Address address = new Address("Poland", "Slask", "Zabrze", "Chrobry", "13", "46-230");

For someone who doesn’t know a lot about Poland’s geography and addressing system, it is really hard to recognise which parameter represents County, City or Street, and when it comes to numbers the situation is even worse. To solve the mystery meaning of parameters, reader must go to class definition or use help from tools such as IntelliSense. Maybe it doesn’t seem like much effort, but it breaks the reading process and de-concentrates reader. It is like reading a book in foreign language with many new words. The reader spends more time going through the dictionary than reading a book and in effect doesn’t remember much from what she read. Soon, the reader stops checking every unknown word in the dictionary and starts to assume its meaning from the context, although in many cases the assumption is wrong (Some of the novels I read when I was learning English I read several times, and every time it was like reading different book). To help the reader with understanding the unit test, it must be written in a way that there is no need to leave the page or assume any meaning, but code is kept conscience and free from duplication.

Now, look at following example:

            Address address = new AddressBuilder()
                .WithCountry("Poland")
                .WithCounty("Slask")
                .WithCity("Zabrze")
                .WithStreet("Chrobry")
                .WithHouseName("13")
                .WithPostCode("46-230")
              .Build();

This time, to create an instance of Address we used AddressBuilder which is an implementation of Builder pattern. We also used Method chaining technique which simplifies the code required to build an object. The above code is much easier to read and doesn’t require from reader the intensive knowledge of Address object, making reading unit tests written by or colleagues a real pleasure.

Below is the AddressBuilder class:

    internal sealed class AddressBuilder
    {
        #region private members

        private string _country;
        private string _county;
        private string _city;
        private string _street;
        private string _houseName;
        private string _postCode;

        #endregion

        public Address Build()
        {
            return new Address(this._country, this._county, this._city, this._street, this._houseName, this._postCode);
        }

        public AddressBuilder WithCountry(string country)
        {
            this._country = country;
            return this;
        }

        public AddressBuilder WithCounty(string county)
        {
            this._county = county;
            return this;
        }

        public AddressBuilder WithCity(string city)
        {
            this._city = city;
            return this;
        }

        public AddressBuilder WithStreet(string street)
        {
            this._street = street;
            return this;
        }

        public AddressBuilder WithHouseName(string houseName)
        {
            this._houseName = houseName;
            return this;
        }

        public AddressBuilder WithPostCode(string postCode)
        {
            this._postCode = postCode;
            return this;
        }

        public AddressBuilder InGuildfordUK()
        {
            this._country = "United Kingdom";
            this._county = "Surrey";
            this._city = "Guildford";
            this._street = "St. George Av.";
            this._houseName = "Elliot House";
            this._postCode = "GU3 1DA";
            return this;
        }

        public AddressBuilder InBrentfordUK()
        {
            this._country = "United Kingdom";
            this._county = "Middlesex";
            this._city = "Brentford";
            this._street = "Windmill Road";
            this._houseName = "5";
            this._postCode = "TW8 9NA";
            return this;
        }
    }

The two methods at the end: InGuildfordUK and InBrentfordUK make building sample addresses easier and can be used in tests, where the exact content is not as important as the fact that the addresses are different. The example can be a test of collection with addresses which doesn’t allow duplicated entries:

            // addresses is a collection which doesn't allow duplicated entries

            addresses.Add(new AddressBuilder().InBrentfordUK().Build());
            addresses.Add(new AddressBuilder().InGuildfordUK().Build());

By using descriptive names we are telling a reader that the former address is in Guildford in the United Kingdom, while the latter address is in Brentford, UK.

References:

Create test objects using Factory Methods or Object Mother pattern

One of the nightmares in unit testing is creating instances of tested objects. While our tested object grows, very quickly the code repetition appears in the test fixture. It is very important to keep unit test code clean, so the changes to tested object can be easily introduced. It’s also important for unit test code to be easy to read. Very often you can find following code in the test fixture:

[TestMethod]
public void AddressIsEqualToItself()
{
    Address address = new Address("United Kingdom", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA");

    Assert.AreEqual(address, address);
}

[TestMethod]
public void AddressesAreEqualWhenAllFieldsAreEqual()
{
    Address address = new Address("United Kingdom", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA");

    Assert.AreEqual(new Address("United Kingdom", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA"), address);

    Assert.AreNotEqual(new Address("United States", "Surrey", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA"), address);
    Assert.AreNotEqual(new Address("United Kingdom", "Middlesex", "Guildford", "St. George Av.", "Elliot House", "GU3 1DA"), address);
}

// more tests requiring instances of Address

The problem with this approach is, that if Address class will have to change the definition of constructor all those test will also require a change. This can be solved by using Factory Method pattern:

private Address CreateAddress(string country, string county, string city, string street, string houseName, string postCode)
{
    return new Address(country, county, city, street, houseName, postCode);
}

private Address CreateAddressInUK(string county = "Surrey", string city = "Guildford", string street = "St. George Av.", string houseName = "Elliot House", string postCode = "GU3 1DA")
{
    return this.CreateAddress("United Kingdom", county, city, street, houseName, postCode);
}

private Address CreateAddressInUS(string county = "Surrey", string city = "Guildford", string street = "St. George Av.", string houseName = "Elliot House", string postCode = "GU3 1DA")
{
    return this.CreateAddress("United States", county, city, street, houseName, postCode);
}

[TestMethod]
public void AddressesAreNotEqualWhenCountriesDiffer()
{
    Address address = this.CreateAddressInUK();

    Assert.AreNotEqual(this.CreateAddressInUS(), address);
}

In the above example there are three Factory Methods. One generic allowing to create any valid address, and two specialised to create addresses in the UK or US. The specialised methods are defined with optional parameters which simplifies tests even more.

To go even further with this approach, the Object Mother pattern can be applied by creating a class containing specialised factory methods, such as: CreateAddressInGuildfordUK, CreateAddressInLondonUK, CreateAddressInNY, etc.:

internal static class AddressMother
{
    public static Address CreateAddress(string country, string county, string city, string street, string houseName, string postCode)
    {
        return new Address(country, county, city, street, houseName, postCode);
    }

    public static Address CreateAddressInGuildfordUK(string street = "St. George Av.", string houseName = "Elliot House", string postCode = "GU3 1DA")
    {
        return CreateAddress("United Kingdom", "Surrey", "Guildford", street, houseName, postCode);
    }

    // ...
}

Using Object Mother class is very useful when there is more than one test which requires to create instances of the object, like the test for Customer class which aggregates an Address.

References: