Introducing NUnit.Specifications

On March 8, 2015, in Uncategorized, by derekgreer

 

I recently started working with a new team that uses NUnit as their testing framework.  While I think NUnit is a solid framework, I don’t think the default API and style lead to effective tests.

As an advocate of Test-Driven Development, I’ve always appreciated how context/specification-style frameworks such as Machine.Specifications (MSpec) allow for the expression of executable specifications which model how a system is expected to be used rather than the typical unit-test style of testing which tends to obscure the overall purpose of the system.

To facilitate a context/specification-style API, I created a base class which makes use of the hooks provided by the NUnit testing framework to emulate MSpec.  I’ve published this code under the project name NUnit.Specifications.

The following is an example NUnit test written using the ContextSpecification based class from NUnit.Specifications using the Should assertion library:

image01

One nice benefit of building on top of NUnit is the wide-spread tool support available.  Here is the test as seen through various test runners:

Resharper Test Runner:

image03

TestDriven.Net: (see notes below)

image04

NUnit Test Runner:

image00

NUnit Test Adaptor for Visual Studio:

image02

 

One caveat I discovered with the TestDriven.Net runner is it’s failure to recognize tests without the specification referencing types from the NUnit.Framework namespace (e.g. TestFixtureAttribute, CategoryAttribute, use of Assert, etc.).  That is to say, it didn’t seem to be enough that the spec inherited from a base type with NUnit attributes, but something in the derived class had to reference a type from the NUnit.Framework namespace for the test to be recognized.  Therefore, the TestDriven.Net results shown above were actually achieved by annotating the class with [Category(“component”)] explicitly.

 

Other Stuff

As a convenience, NUnit.Specifications also provides attributes for denoting categories of Unit, Component, Integration, Acceptance, and Subcutaneous as well as a Catch class (as provided by the MSpec library) for working with exceptions.

You can obtain the NUnit.Specifications from NuGet or grab the source from github.

Tagged with:  

Expected Objects Custom Comparisons

On November 17, 2013, in Uncategorized, by derekgreer

ExpectedObjects is a testing library I developed a few years ago to facilitate using the Expected Objects pattern within my specifications to avoid obscure tests.  You can find the original introduction to the library here.

As of version 1.1.0, the ExpectedObjects library has been updated to include a feature called Custom Comparisons.  The standard behavior of the library is to traverse a strategy chain (which is itself configurable) to determine which comparison strategy is to be used for each type of object encountered within the object graph.  The Custom Comparisons feature allows you to override this behavior for specific properties.

For example, let’s say we’re writing a end-to-end test which validates a Receipt class as follows:

public class Receipt

{
    public string Name { get; set; }
    public DateTime TransactionDate { get; set; }
    public string VerificationCode { get; set; }
}

 

Given the following class, the VerificationCode property would probably not be a value you could anticipate.  In such a case, while you can’t verify that the property has a specific value, you may care that it at least has some value.  This is where the Custom Comparisons feature can help.  We can verify that the actual Receipt received matches the expected receipt structure using the following expected object configuration:

var expected = new
{
	Name = "John Doe",
	DateTime = DateTime.Today,
	VerificationCode = Expect.NotNull()
}.ToExpectedObject();


var actual = new Receipt
{
	Name = "John Doe",
	DateTime = DateTime.Today,
	VerificationCode = "ABC123"
};



expected.ShouldMatch(actual);

In the event that the VerificationCode property is null, the library will raise an exception with the following message:

For Receipt.VerificationCode, expected a non-null value but found [null].

The ExpectedObjects library currently provides a static Expect class which  includes convenience methods to check for null, not null, and an Any<T> comparison for checking that an object is of a specific type (e.g. Expect.Any<Receipt>()).  To supply your own comparisons, simply implement the IComparsion interface which defines the custom comparison and the text to include within any exception messages raised (e.g. “For SomeType.SomeProperty, expected [text you supply here] but found “42”).

Tagged with:  

TDD Best Practices: Write Assertions First

On March 5, 2012, in Uncategorized, by derekgreer

When practicing Test-Driven Development, developers often organize their tests following a style first described by Bill Wake as Arrange, Act, Assert.  This division between the setup, the exercising the System Under Test, and the assertions are also reflected in the Context/Specification and Given/When/Then (GWT) styles of Behavior-Driven Development (with the GWT style often containing multiple “Act” steps).  This division is generally a Good Thing, as it guides the developer toward the authoring of well-organized and readable tests which provide a good template for modeling most specifications.  Unfortunately, developers are often lead to implement tests in the order of Arrange, Act, Assert which serves to undermine the purpose of practicing TDD: to allow the needs of the test to guide the design of the system.

When transforming a requirement into an executable specification, after establishing the logical class or object which will contain the test implementation, the first step should be to determine how the specification (i.e. the test) is to be validated.  That is to say, the first step is to write an assertion.

The goal of Test-Driven Development is to produce clean code that works.  By ‘clean’, we mean code that’s free of unnecessary behavior or abstractions.  The TDD techniques of first doing the simplest thing that could possibly work, refactoring to remove duplication, and using triangulation for identifying the need for generalizations are set forth to restrain developers from writing code that isn’t actually needed.  This leads to simpler, more maintainable code.  Starting with the setup implementation before determining how you’re going to verify the desired outcome is putting the cart before the horse.  How do you know a particular setup implementation will be needed to test the desired outcome before you’ve established what that is?  Perhaps you’ll be building upon the existing type you’re jumping ahead to setup, but perhaps it should be a new type altogether.  Figure out how you’re going to verify the desired outcome first and then determine what components make sense to fulfill the desired outcome.

To help illustrate this flow, let’s consider the following example.  Let’s say we’re writing an application which allows employee’s at a company to register for a training class.  When an employee registers for a class, they should be provided with a registration receipt.  Let’s start by establishing the shell of our specification:

public class when_registering_for_a_class
{
  Establish context;

  Because of;

  It should_return_a_registration_receipt;
}

Next, we need to determine how we want to verify that the logical condition “It should return a registration receipt” will be fulfilled.  Let’s assert that a variable named _registrationReceipt is not null:

public class when_registering_for_a_class
{
  Establish context;

  Because of;

  It should_return_a_registration_receipt = () => _registrationReceipt.ShouldNotBeNull();
}

It’s at this point some may have an objection to typing out an assertion without the aid of auto-completion (i.e. Intellisense).  Some are so dependent upon auto-completion that they’ll actually stop after typing _registrationReceipt, and go declare a variable just so their auto-completion will be there.  Resist that urge.  Assertions shouldn’t be difficult and this will actually force you to keep it simple.

Next, we need to decide how we are going to assign our variable, so we’ll move to our Because delegate.  At this point, we’ll also determine what component and usage API we’d like to use to retrieve our receipt:

public class when_registering_for_a_class
{
  Establish context;

  Because of = () => _registrationReceipt = _registrar.Register(EmployeeId, ClassId);

  It should_return_a_registration_receipt = () => _registrationReceipt.ShouldNotBeNull();
}

Next, we need to establish our context.  We’ll initialize the _registrar to a non-existent type named Registrar:

public class when_registering_for_a_class
{
  Establish context = () => { _registrar = new Registrar(); };

  Because of = () => _registrationReceipt = _registrar.Register(EmployeeId, ClassId);

  It should_return_a_registration_receipt = () => _registrationReceipt.ShouldNotBeNull();
}

At this point, we can use ReSharper to generate a field name _registrar, generate the Registrar class, and generate the Registrar.Register() method.  Once the Register() method is generated, we need to choose the type we’ll use for the return value and parameters and then replace the throw statement with a return of null so our assertion will fail for the right reason later.  Let’s make the receipt be a RegistrationReceipt type and use integers for our Ids:

public class when_registering_for_a_class
{
  static Registrar _registrar;

  Establish context = () => { _registrar = new Registrar(); };

  Because of = () => _registrationReceipt = _registrar.Register(EmployeeId, ClassId);

  It should_return_a_registration_receipt = () => _registrationReceipt.ShouldNotBeNull();
}

class Registrar
{
  public RegistrationReceipt Register(int employeeId, int classId)
  {
  }
}

Continuing, we generate the RegistrationReceipt type, _registrationReceipt field, and create some constants for our Ids:

public class when_registering_for_a_class
{
  const int EmployeeId = 1;
  const int ClassId = 2;
  static Registrar _registrar;
  static RegistrationReceipt _registrationReceipt;

  Establish context = () => { _registrar = new Registrar(); };
  
  Because of = () => _registrationReceipt = _registrar.Register(EmployeeId, ClassId);

  It should_return_a_registration_receipt = () => _registrationReceipt.ShouldNotBeNull();
}

class Registrar
{
  public RegistrationReceipt Register(int employeeId, int classId)
  {
    return null;
  }
}

class RegistrationReceipt
{
}

 

At this point everything compiles cleanly.  Running our test with the ReSharper test runner produces the following:

should return a registration receipt : Failed
Should be [not null] but is [null]

We’re now ready to make our test pass (which we can do by just returning a new RegistrationReceipt from our Register method), factor out any duplication, and move on to our next assertion or specification either to flesh out this design or to move on to our next feature.

By starting with our assertion first, moving to the execution of our System Under Test, and ending with our context setup, we’ve allowed our test to guide our design rather than allowing an existing design (coded or in our heads) to guide the implementation of our test.

In summary, organize your tests using Arrange, Act, Assert, but implement them in the order of Assert, Act, Arrange.

Tagged with:  

Part 3 of our series introduced Test-First Programming and briefly discussed its refinements: Test-Driven Development and Behavior-Driven Development.  This time, we’ll dive into a simple Test First example using the principles of Test-Driven Development along with some of the Behavior-Driven Development concepts introduced last time.

Before beginning our example, we’ll take some time to discuss the goals and approach of Test-Driven Development in a little more detail to establish the methodology we’ll be following.

 

Test-Driven Development Primer

So, what is Test-Driven Development?  The simple answer is that it’s a process of writing tests before writing code, but such a simple answer fails to convey the true spirit of Test-Driven Development.

Even if we’ve adopted a practice of writing tests first, we’re often tempted to start implementing all the functionality we think will be part of a particular feature in a way that appeals to our design sensibilities.  Many of us are conditioned to approach the implementation phase by asking such questions as “What will this feature need to fully work”, “What is the most flexible way of fulfilling this feature”, or perhaps “What future design needs will I have that I can begin leveraging for this feature”.  This line of thinking usually leads us down a road of implementing patterns and algorithms, creating infrastructure code, and configuring third-party libraries and frameworks we’ve already planned to use.  The end result of such efforts may be working code, perhaps even with high test coverage, but while this approach might be considered Test-Inspired Development, it wouldn’t be Test-Driven Development.  

The goal of Test-Driven Development isn’t to ensure we write tests by writing them first, but to produce working software that achieves a targeted set of requirements using simple, maintainable solutions.  To achieve this goal, TDD provides strategies for keeping code working, simple, relevant and free of duplication.

Keep it Working

To keep the code working, TDD encourages development in small steps under the protection and confidence of passing tests which properly validate the desired behavior of the system.  To help minimize the amount of time a test exists in a failing state, TDD sets forth two strategies for quickly getting a test to pass.  The first strategy, to be used when the solution is simple and straight forward, is to use the Obvious Implementation.  The second strategy, to be used when a solution isn’t clear or will take some time to complete, is to Fake It.  The Fake It strategy refers to a technique of satisfying the requirements of a test by merely returning hard-coded values.

Keep it Simple

To keep the code simple,  TDD sets forth several techniques to help guide the developer toward the simplest solution.  First, TDD encourages a willingness to take small steps.  Taking small steps, especially for complex and unclear tasks, helps develop an ability to identify simple solutions.  Second, TDD provides guidelines for when to add or change code.  By changing code only to satisfy a failing test, remove duplication, or to remove unused code, solutions are kept free of complexity.  Third, TDD encourages a willingness to just try the Simplest Thing That Could Possibly Work when the obvious implementation isn’t clear.  Fourth, TDD sets forth a technique called Triangulation (discussed later) for helping to identify needed generalization.

Keep it Relevant

To keep the code relevant, TDD encourages the removal of unused generalization.  While this also helps keep the code simple, simplicity doesn’t mean the code is actually needed.  Developers often create constructs to improve the flexibility and extensibility of an application or as an outlet for their creativity, but this frequently leads to overly complex solutions which are difficult to understand, difficult to maintain and which are often never used.  TDD helps avoid this by keeping us focused on just those things that are needed to satisfy our tests.

Keep it Free of Duplication

To keep the code free of duplication, TDD incorporates continuous refactoring into the development process.  After writing a failing test and making it pass as quickly as possible, TDD encourages the removal of duplication as the heuristic for the discovery and improvement of design.  In TDD, it is through the refactoring process, driven by the goal of removing duplication, that much of an application’s design emerges.

The TDD Process

To help facilitate these goals, Test-Driven Development prescribes a set of steps for us to follow:

  1. Write a test.

  2. Make it compile.

  3. Run it to see that it fails.

  4. Make it run.

  5. Remove duplication.

These steps are often simplified into a process referred to as Red/Green/Refactor.  First we write a failing test, then we make it pass, then we refactor.

With these steps and goals in mind, let’s dive into our example.

 

Requirements

For this example, we’ll be creating a simple Tic-tac-toe game that allows a single player to compete against the game.  Here are the requirements we’ll be working from:

When the player goes first it should put their mark in the selected position it should make the next move When the player gets three in a row it should announce the player as the winner When the game gets three in a row it should announce the game as the winner When the player attempts to select an occupied position it should tell the player the position is occupied When the player attempts to select an invalid position it should tell the player the position is invalid When the game goes first it should put an X in one of the available positions When the player can not win on the next turn it should try to get three in a row When the player can win on the next turn it should block the player

 

Creating the Specifications

While we’ll be striving to follow the Test-Driven Development process, we’ll also try to incorporate some of the Behavior-Driven Development concepts we learned from last time.   There are several frameworks which are designed specifically to facilitate  Behavior-Driven Development, but we’re going to use the Visual Studio Unit Testing Framework  to demonstrate how these concepts can be applied using traditional xUnit frameworks. While we could begin by creating some testing infrastructure code, let’s keep things simple for now and just use some of the improved naming concepts presented in our last article.

Our first step will be to create a new Visual Studio project. I’ll use the Test Project type and name the project TestFirstExample:

 

TestFirstExampleProjectNew

 

Next, let’s create a test class that reflects the context of our first requirement:

using Microsoft.VisualStudio.TestTools.UnitTesting; namespace TestFirstExample { [TestClass] public class WhenThePlayerGoesFirst { } }

 

Next, let’s create a method that expresses the first observable behavior for our context:

using Microsoft.VisualStudio.TestTools.UnitTesting; namespace TestFirstExample { [TestClass] public class WhenThePlayerGoesFirst { [TestMethod] public void ItShouldPutTheirChoiceInTheSelectedPosition() { } } }

 

Since we’ve chosen to follow a more Behavior-Driven Development approach to writing our tests, let’s take a second and reflect on how this reads. Thus far, we’ve used the same wording from our requirements, but the class and method identifiers seem a little difficult to read. Since we’re using code to represent sentences instead of entities and methods, let’s try changing our naming convention to use underscores for separating each word in our identifiers and see how it looks:

using Microsoft.VisualStudio.TestTools.UnitTesting; namespace TestFirstExample { [TestClass] public class When_the_player_goes_first { [TestMethod] public void it_should_put_their_choice_in_the_selected_position() { } } }

 

In C#, this breaks with our normal naming conventions for class and method names, but this seems a little easier to read in my opinion.

 

Naming Conventions

The use of underscores for separating the words within identifiers is a commonly used convention for writing BDD specifications, especially within languages that don’t support the use of spaces. When first encountered, some developers find this unsettling due to the fact that it often represents a deviation from the standard naming convention for the language being used.

For those who feel uncomfortable with this style due to it’s deviation from standard conventions, it may help to think about naming conventions in terms of modeling. Just as we use a domain model to represent a business domain, a data model to represent data persistence concerns, and a view model to represent a user interface, BDD specifications are a kind of model for representing our requirements. As such, they deserve to be viewed within their own context. In this case, specifications are all about modeling requirements, not real-world entities. As such, some have chosen to depart from standard naming conventions when modeling specifications.

Of course, this is a matter of preference. What’s important is to view the modeling of specifications as its own concern and allow this understanding to help guide our modeling efforts.

 

The first step of our test method should be to determine how we want to verify that a player’s choice results in an X being placed in the selected position. To do this, we’ll need a way of evaluating the state of the game. One approach we could take would be to simply ask the game to tell us what value is in a given position. Since a Tic-tac-toe game is comprised of a grid with nine positions, we can refer to each position by number, numbering them from left to right, top to bottom. Let’s assume we can call a method named GetPosition():

[TestMethod] public void it_should_put_their_choice_in_the_selected_position() { Assert.AreEqual('X', game.GetPosition(1)); }

 

Next, we need to create the instance of our Game variable and determine how we’re going to inform it about our choice. A common rule of Tic-tac-toe is to let ‘X’ go first, so we can keep things simple if we adopt this rule. The only input needed in that case is the position, so let’s call a method named ChoosePosition() and pass an integer value of 1 to indicate the top-left position on the board:

[TestMethod] public void it_should_put_their_choice_in_the_selected_position() { var game = new Game(); game.ChoosePosition(1); Assert.AreEqual('X', game.GetPosition(1)); }

 

Of course, this won’t compile since our Game class doesn’t exist yet. Let’s create our class and methods now:

public class Game { public void ChoosePosition(int position) { throw new NotImplementedException(); } public char GetPosition(int position) { throw new NotImplementedException(); } }

 

Everything should be ready to compile. Let’s run our test and see what happens:

 

Test method TestFirstExample.When_the_player_goes_first .it_should_put_their_choice_in_the_selected_position threw exception: System.NotImplementedException: The method or operation is not implemented.

 

As you probably expected, our test fails. Since we’re following the Red/Green/Refactor process, you might think we’re ready to work on getting the test to pass now. Not so fast! The purpose of writing a failing test isn’t failure, but validation. At this point, what we should be interested in is that the behavior of our code will be correctly validated once we write it. To help explain, let’s consider what would happen if we had mistakenly written the test as follows:

[TestClass] public class When_the_player_goes_first { [TestMethod] public void it_should_put_their_choice_in_the_selected_position() { var game = new Game(); game.ChoosePosition(1); Assert.AreNotEqual('X', game.GetPosition(1)); } }

 

Here, we’ve changed Assert.AreEqual() to Assert.AreNotEqual(). How do you think this will change the outcome of the test? If you guessed it would not change the outcome then you’re correct. Our test didn’t fail because of our assert, but rather because an exception halted the test before it even got to the assertion. That doesn’t tell us anything about the validity of how we’ve written our test. Merely getting a test to fail doesn’t serve a useful purpose. Our test needs to fail for the right reason.

In this case, we want our test to fail because the call to the GetPosition() method didn’t return an X. Let’s remove the calls which are causing exceptions to be thrown and change the GetPosition() method to return a value that forces the test to fail due to the assertion:

public class Game { public void ChoosePosition(int position) { } public char GetPosition(int position) { return '\0'; } }

 

Running our test again produces the following output:

 

When_the_player_goes_first Failed it_should_put_their_choice_in_the_selected_position Assert.AreEqual failed. Expected:<X>. Actual:<\0>.

 

We now see that our test is failing due to the assertion properly validating the behavior of our class. Our next step is getting our test to pass quickly. Let’s use the Fake It approach and simply return the expected value of ‘X’:

public class Game { public void ChoosePosition(int position) { } public char GetPosition(int position) { return 'X'; } }

 

 

 

Our first test passes! Our next step is to eliminate any unnecessary generalization and duplication. While we don’t have any generalization to remove, we do have duplication. At first, our duplication may be difficult to spot as we’re typically accustomed to thinking about it in terms of repeated code blocks, but duplication manifests itself in many forms. In this case we have a form of duplication which I like to think of as Mirroring. We’re simply reflecting back to the test what it is expecting to see. This form of duplication is often the result of using the Fake It approach to getting tests to pass quickly, but this isn’t a bad thing. On the contrary, starting with hard-coded values and generalizing based upon a need to remove duplication helps to ensure that we aren’t introducing generalization unnecessarily.

To remove our duplication, let’s keep track of the player’s choice in a layout field which we can set when the ChoosePosition() method is called and then retrieve when the GetPosition() method is called:

public class Game { char _layout; public void ChoosePosition(int position) { _layout = 'X'; } public char GetPosition(int position) { return _layout; } }

 

 

 

You might be wondering why I chose such a naïve solution here. Won’t we need something that can keep track of all the positions on the board to accommodate other scenarios? The answer is yes, but we don’t have tests to drive our need to accommodate other scenarios yet. By only using the simplest constructs needed by our existing tests, we help to ensure we aren’t writing more complex code than necessary.

Another question that could be posed is, if we’re concerned with eliminating waste, doesn’t this process itself end up causing waste? The answer is yes and no. It can be reasoned that doing simple things does lead to waste in small degrees, but the benefit of allowing our design to emerge from this process is that it helps to eliminate large areas of waste.

Another question some may be wondering is whether taking small steps is a rule we have to follow when doing TDD. The answer to that question is no, but it is a useful methodology that helps keep us honest. We can take larger steps when it makes sense to do so, but it’s important to watch that our steps aren’t getting too large. You’ll know your steps are getting too large when you or others start frequently discovering unnecessary code or simpler solutions. By using this approach as a general practice, it helps to condition our ability to identify simple solutions. In his book Test-Driven Development By Example, Kent Beck points out that TDD is not about taking teeny-tiny steps, it’s about being able to take teeny-tiny steps.

Let’s move on to our next requirement. We’ll start by creating a new test method:

[TestMethod] public void it_should_make_the_next_move() { }

 

Again, our first step is to determine how we want to verify the behavior of the requirement. Since this test would be satisfied by the game choosing any position, we can verify the expected behavior occurred by checking all the positions. A clean way to do this is to create the range we want to check using Enumerable.Range() and then use LINQ’s Any() extension method to check each position for the character ‘O’. The Any() method returns true as soon as one of the enumerated elements is satisfied by the predicate:

[TestMethod] public void it_should_make_the_next_move() { Assert.IsTrue(Enumerable.Range(1, 9) .Any(position => game.GetPosition(position) .Equals('O'))); }

 

Next, we need to establish the context for the test. To avoid duplication, let’s move the context setup from our first test into a setup method and modify our tests to use a class field:

[TestClass] public class When_the_player_goes_first { Game _game; [TestInitialize] public void establish_context() { _game = new Game(); _game.ChoosePosition(1); } [TestMethod] public void it_should_put_their_choice_in_the_selected_position() { Assert.AreEqual('X', _game.GetPosition(1)); } [TestMethod] public void it_should_make_the_next_move() { Assert.IsTrue(Enumerable.Range(1, 9) .Any(position => _game.GetPosition(position) .Equals('O'))); } }

 

Now we’re ready to run our test. Since we’ve modified things, let’s run both the tests just to make sure we didn’t break the first one:

 

When_the_player_goes_first Failed it_should_make_the_next_move Assert.IsTrue failed.

 

From the output, we see that we didn’t break our first test and our second test is failing for the right reason. Next, let’s make our new test pass. To do so, the GetPosition() method will need to return an ‘O’ for one of the nine times it’s called. Let’s use the Fake It approach again to get the test to pass quickly. We can change our _layout field to a two character array and hard-code the ChoosePosition() method to assign ‘O’ to the second position:

public class Game { char[] _layout = new char[2]; public void ChoosePosition(int position) { _layout[position - 1] = 'X'; _layout[1] = 'O'; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

Now that the tests pass, we can move on to factoring out any duplication we’ve introduced. This time, however, we aren’t checking a specific position that’s we’ve hard coded a value to satisfy. On the other hand, we know this solution isn’t going to be flexible enough based on our knowledge about the rules of the game. How should we proceed?

To introduce generalization in cases where duplication either doesn’t exist or isn’t readily apparent, we can use a strategy called Triangulation. Similar to the mathematical process of determining the third point of a triangle given the length and angles of the first two points, this process helps you locate a generalized solution based upon two tests which verify the results in intersecting ways.

To demonstrate, let’s create a second test which validates the behavior of our class in a slightly different way which the current implementation wouldn’t satisfy:

[TestClass] public class When_the_player_selects_the_second_position { [TestMethod] public void it_should_put_their_choice_in_the_second_position() { Game game = new Game(); game.ChoosePosition(2); Assert.AreEqual('X', game.GetPosition(2)); } }

 

 

When_the_player_selects_the_second_position Failed it_should_put_their_choice_in_the_second_position Assert.AreEqual failed. Expected:<X>. Actual:<O>.

 

Our test fails because the method is hard-coded to assign ‘O’ to the second position. In order to make this test pass, we’ll now need to change our code to a more general solution. To replace the hard-coded selection of the second position, we can use LINQ to locate the first unoccupied element of the array and use that position for our assignment:

public class Game { readonly char[] _layout = new char[2]; public void ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

Our new test now passes. Our next step would be to factor out any duplication we might have introduced to satisfy the new test, but it doesn’t look like we introduced any this time.

While the use of Triangulation provides a simple to understand and explicit driver for introducing a generalized solution, it’s use is a somewhat inelegant strategy. Once a generalized solution is introduced, intersecting tests become redundant from a specification perspective. They can be useful in cases when it isn’t clear why a generalized solution exists and you need to ensure the solution isn’t lost through a future refactoring, but the resulting extra tests lead to higher test maintenance costs and can lead to confusion when included in reports or used as documentation by other team members. When possible, prefer removing duplication over triangulation as a generalization strategy. If duplication isn’t present or is of a particularly difficult to spot nature and generalization won’t be addressed by a separate requirement then it may be beneficial to leave redundant tests. To help aid in clarifying the intent of the test, we can use a TestCategoryAttribute as follows:

[TestClass] public class When_the_player_selects_the_second_position { [TestCategory("Triangulation")] [TestMethod] public void it_should_put_their_choice_in_the_second_position() { var game = new Game(); game.ChoosePosition(2); Assert.AreEqual('X', game.GetPosition(2)); } }

 

This helps to identify the reason for the test to other team members in a standard way and provides a mechanism for creating reports which don’t include the redundant test cases when desired. From the command line, our tests can be run without including the triangulation tests using the following:

c:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /category:'!Triangulation' \
/testcontainer:TestFirstExample.dll

 

For frameworks which provide non-sealed attributes for identifying categories of tests, we can create our own strongly-typed attribute to minimize any additional syntax noise as shown in the following contrived example:

public class TriangulationAttribute : SomeFrameworkCategoryAttribute { public MyAttribute() : base("Triangulation") { } }

 

While perhaps not easily discerned, we might have avoided the use of Triangulation in the above example had we considered our Any() extension method to have been hiding a form of duplication. When we used the Any() method to look for the first occurrence of a position with the value ‘O’, this was logically the same as if we had written the following:

[TestMethod] public void it_should_make_the_next_move() { char[] positions = new char[2]; bool gamePositionFound = false; for(int i = 0; i < 2; i++) { positions[i] = _game.GetPosition(i + 1); } for (int i = 0; i < 9; i++) { if (positions[i].Equals('O')) { gamePositionFound = true; break; } } Assert.IsTrue(gamePositionFound); }

 

Breaking the logic down further, our for loop would have performed the following checks:

if(positions[0].Equals(‘O’)) { … } if(positions[1].Equals(‘O’)) { … }

 

Based on a more explicit expression of what our code was doing, it becomes easier to identify the duplication that was present.

Moving on, our next requirement concerns how we’ll determine when the player has won the game. Let’s create a new test class along with it’s single observation:

[TestClass] public class When_the_player_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_player_as_the_winner() { } }

 

We know that we want to end up with a message that reflects that the player won, so let’s start with this as our assertion:

[TestClass] public class When_the_player_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_player_as_the_winner() { Assert.AreEqual("Player wins.", message); } }

 

Next, we need to determine where our message will come from. Let’s assume our ChoosePosition() method returns a message after the player makes a move. We’ll call the ChoosePosition() method three times and declare our Game instance:

[TestClass] public class When_the_player_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_player_as_the_winner() { Game game = new Game(); game.ChoosePosition(1); game.ChoosePosition(2); string message = game.ChoosePosition(3); Assert.AreEqual("Player wins!", message); } }

 

To get this to compile, we need to change the ChoosePosition() method to return a string:

public string ChoosePosition(int position) { _layout[position - 1] = value; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; return string.Empty; }

 

We’re now ready to run our tests:

 

When_the_player_gets_three_in_a_row Failed it_should_announce_the_player_as_the_winner Test method TestFirstExample.When_the_player_gets_three_in_a_row .it_should_announce_the_player_as_the_winner threw exception: System.InvalidOperationException: Sequence contains no matching element

 

It looks like our test isn’t failing for the right reason yet. The problem is that our array isn’t large enough for our new test scenario. Let’s go ahead and take the larger step of increasing the layout to accommodate the full 9 positions of the game board and run our test again:

readonly char[] _layout = new char[9];

 

 

When_the_player_gets_three_in_a_row Failed it_should_announce_the_player_as_the_winner Assert.AreEqual failed. Expected:<Player wins!>. Actual:<>.

 

Now, let’s make it pass. We can just return the string “Player wins!” to get the test to pass quickly:

public class Game { readonly char[] _layout = new char[9]; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; return "Player wins!"; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

We now have the string “Player wins!” duplicated, so we’ll fix this by putting in some logic to test that the first three positions contain the value ‘X’:

public class Game { readonly char[] _layout = new char[9]; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; return string.Empty; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

We can now move on to our next requirement which concerns determining when the game wins. Here’s our test skeleton:

[TestClass] public class When_the_game_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_game_as_the_winner() { } }

 

Again, we’ll start by declaring our assertion and then fill in the context of the test. Since we know that the game chooses the first unoccupied space, we can leverage this by choosing positions which leave the first row open:

[TestClass] public class When_the_game_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_game_as_the_winner() { Game game = new Game(); game.ChoosePosition(4); game.ChoosePosition(6); string message = game.ChoosePosition(8); Assert.AreEqual("Game wins.", message); } }

 

 

When_the_game_gets_three_in_a_row Failed it_should_announce_the_game_as_the_winner Assert.AreEqual failed. Expected:<Game wins.>. Actual:<>.

 

To get the test to pass, we can use similar logic as before to check that the layout starts with “OOO”:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

 

 

Now, let’s refactor. In reviewing our code, we see that we have some similarities between the logic checking if the player wins and the logic checking if the game wins. We need a way of generalizing this comparison so that it works for both. One option would be to use regular expressions. If we have an array of winning patterns, we can convert the layout to a string and compare it to each one in a loop. Let’s figure up all the winning patterns real quick:

tic-tac-toe

It looks like we have eight different winning patterns. Flattening this out becomes:

X X X _ _ _ _ _ _, _ _ _ X X X _ _ _, _ _ _ _ _ _ X X X, X _ _ X _ _ X _ _, _ X _ _ X _ _ X _, _ _ X _ _ X _ _ X, X _ _ _ X _ _ _ X, _ _ X _ X _ X _ _

 

Let’s start by creating a new array reflecting the winning patterns for ‘X’:

public class Game { readonly char[] _layout = new char[9]; readonly string[] _winningXPatterns = new[] { "XXX......", "...XXX...", "......XXX", "X..X..X..", ".X..X..X.", "..X..X..X", "X...X...X", "..X.X.X..", }; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

Next, we need to create a string representation of our current layout:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; var layoutAsString = new string(_layout); if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

Next, let’s replace the previous comparison checking if the layout starts with “XXX” with a loop that checks each winning pattern to the current layout:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; var layoutAsString = new string(_layout); foreach (string pattern in _winningXPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Player wins!"; } if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

 

 

Everything is still working. Now, let’s make the same changes for the game comparison:

public class Game { readonly char[] _layout = new char[9]; readonly string[] _winningXPatterns = new[] { "XXX......", "...XXX...", "......XXX", "X..X..X..", ".X..X..X.", "..X..X..X", "X...X...X", "..X.X.X..", }; readonly string[] _winningOPatterns = new[] { "OOO......", "...OOO...", "......OOO", "O..O..O..", ".O..O..O.", "..O..O..O", "O...O...O", "..O.O.O..", }; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; var layoutAsString = new string(_layout); foreach (string pattern in _winningXPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Player wins!"; } foreach (string pattern in _winningOPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Game wins."; } return string.Empty; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

We’re still green. At this point we’ve actually introduced a little more duplication than we started with, but we’re making small steps that will eventually lead us to a solution. Next, let’s see if we can combine our two arrays to represent the winning position for both players. Since we’re using a regular expression, we can replace the characters with character sets like this:

readonly string[] _winningPatterns = new[] { "[XO][XO][XO]......", "...[XO][XO][XO]...", "......[XO][XO][XO]", "[XO]..[XO]..[XO]..", ".[XO]..[XO]..[XO].", "..[XO]..[XO]..[XO]", "[XO]...[XO]...[XO]", "..[XO].[XO].[XO]..", };

 

We’ll then need to change the comparisons to use the new array. Since we only want to compare one side at a time, we’ll also need to filter out the characters we don’t want:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; string layoutAsString = new string(_layout).Replace('O', '\0'); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Player wins!"; } layoutAsString = new string(_layout).Replace('X', '\0'); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Game wins."; } return string.Empty; }

 

 

 

Now, let’s take care of the looping duplication by creating a method that will perform the comparison based on the given side we’re interested in:

bool WinningPlayerIs(char player) { var layout = new string(_layout); if (player == 'X') layout = layout.Replace('O', '\0'); else layout = layout.Replace('X', '\0'); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layout, pattern)) return true; } return false; }

 

We can now replace our comparisons with a call to our new method:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (WinningPlayerIs('X')) return "Player wins!"; if (WinningPlayerIs('O')) return "Game wins."; return string.Empty; }

 

 

 

Now, let’s clean up our new WinningPlayerIs() method. We’re duplicating the call to Replace() based upon which character we’re wanting the layout for, so instead of this let’s use LINQ to create an array with the character we want filtered out:

bool WinningPlayerIs(char player) { var layout = new string(_layout.ToList() .Select(c => (c.Equals(player)) ? player : '\0') .ToArray()); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layout, pattern)) return true; } return false; }

 

 

 

That’s more concise, but it could stand to be more descriptive. Rather than adding a comment, let’s just wrap this in an intention-revealing method:

bool WinningPlayerIs(char player) { var layout = GetLayoutFor(player); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layout, pattern)) return true; } return false; } string GetLayoutFor(char player) { return new string(_layout.ToList() .Select(c => (c.Equals(player)) ? player : '\0') .ToArray()); }

 

We can also eliminate declaring multiple exit points and simplify the comparison by using LINQ’s Any() extension method:

bool WinningPlayerIs(char player) { var layout = GetLayoutFor(player); return _winningPatterns.Any(pattern => Regex.IsMatch(layout, pattern)); }

 

Let’s go ahead and in-line our call to GetLayoutFor(player):

bool WinningPlayerIs(char player) { return _winningPatterns .Any(pattern => Regex.IsMatch(GetLayoutFor(player), pattern)); }

 

 

 

Here’s what we have so far:

public class Game { readonly char[] _layout = new char[9]; readonly string[] _winningPatterns = new[] { "[XO][XO][XO]......", "...[XO][XO][XO]...", "......[XO][XO][XO]", "[XO]..[XO]..[XO]..", ".[XO]..[XO]..[XO].", "..[XO]..[XO]..[XO]", "[XO]...[XO]...[XO]", "..[XO].[XO].[XO]..", }; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (WinningPlayerIs('X')) return "Player wins!"; if (WinningPlayerIs('O')) return "Game wins."; return string.Empty; } bool WinningPlayerIs(char player) { return _winningPatterns .Any(pattern => Regex.IsMatch(GetLayoutFor(player), pattern)); } string GetLayoutFor(char player) { return new string(_layout.ToList() .Select(c => (c.Equals(player)) ? player : '\0') .ToArray()); } public char GetPosition(int position) { return _layout[position - 1]; } }

 

We’ll leave things here for now and continue our effort next time.

Tagged with:  

Effective Tests: Test First

On March 22, 2011, in Uncategorized, by derekgreer

In part 2 of our series, I presented a simple example of a traditional unit test.  In this article, we’ll discuss a completely different way to use unit tests: Test First Programming.

Test-First Programming

In the book Extreme Programming Explained: Embrace Change published in October 1999, Kent Beck introduced what was at the time a rather foreign concept to software development: Writing unit tests to drive software design.  Described as Test-First Programming, the technique involves first writing a test to verify a small increment of desired functionality and then writing code to make the test pass.  This process is then repeated until no additional functionality is desired.

Incremental Design

Another practice introduced alongside Test-First Programming was Incremental Design.  Incremental design is the process of evolving the design of an application as new functionality is developed and better design approaches are recognized.  Due to the fact that software development is an inherently empirical process,  following a defined model where the the software is designed up front generally leads to waste and rework.  Incremental design postpones design decisions until needed, introducing patterns and frameworks as a byproduct of refactoring as well as eliminating any flexibility that is unused.

Test Driven Development

Later, these ideas were  refined into a practice known as Test Driven Development.  Test-Driven Development can be described simply as the ideas of Test-First Programming coupled with Incremental Design.  In a later book entitled Test-Driven Development By Example published in November of 2002, Beck presents a refined process involving the writing of a failing test, taking the simplest steps possible to make the test pass, and removing any duplication introduced.  This process was described succinctly as Red/Green/Refactor, where red and green referred to the colors typically presented by test runners to indicate failing and passing tests respectively.

Behavior-Driven Development

Test Driven Development

 

While the practice of Test-Driven Development utilized unit tests primarily for driving application design, its outgrowth from traditional unit testing practices were still evident in its style,  language, and supporting frameworks. This presented an obstacle for some in understanding and communicating the concepts behind the practice.

In the course of teaching and practicing TDD, Dan North began encountering various strategies for designing tests in a more intent-revealing way which aided in his understanding and ability to convey the practice to others.  As a way to help distinguish the differences between what he saw as an exercise in defining behavior rather than testing, he began describing the practice of Test-Driven Development as Behavior Driven Development.  By shifting the language to the discussion about behavior, he found that many of the questions he frequently encountered in presenting TDD concepts became easier to answer.  North went on to introduce a framework incorporating this shift in thinking called JBehave.  North’s work on JBehave along with his promotion of the ideas behind Behavior-Driven Development served as the inspiration for the creation of many other frameworks including RSpec, Cucumber, NBehave, NSpec, SpecFlow, MSpec and many others.

Behavior-Driven Development Approaches

Two main approaches have emerged from BDD practices which I’ll categorize as the Plain-Text Story approach and the Executable Specification approach.

Plain-Text Stories

With the Plain-Text Story approach, software features are written as a series of plain-text scenarios which the framework associates with executable steps used to drive feature development.  This approach is typically associated with a style referred to Given/When/Then (GWT), named after the grammar set forth by Dan North.  This style is facilitated by popular BDD frameworks such as Cucumber and JBehave.

The following is an example story using Gherkin grammar, the language used by Cucumber and other Plain-Text Story BDD frameworks:

Feature: Addition
   In order to calculate totals
   As a user
   I want to calculate the sum of multiple numbers

Scenario: Add two numbers
   Given I have entered 10 into the calculator
   And I have entered 50 into the calculator
   When I press add
   Then the result should be 60 on the screen

 

Using Cucumber, the “Given” step in this scenario may then be associated to the following step definition in Ruby:

Given /I have entered (\d+) into the calculator/ do |n|
  calc = Calculator.new
  calc.push n.to_i
end

 

Using SpecFlow, the same “Given” step can also be associated to the following step definition in C#:

[Given("I have entered (.*) into the calculator")]
public void GivenIHaveEnteredSomethingIntoTheCalculator(int number)
{
    _calculator = new Calculator();
    _calculator.Enter(number);
}

Executable Specifications

In contrast to the use of plain-text stories, the Executable Specification approach uses source code as the medium for expressing specifications in the language of the business.  Executable Specifications can be written using existing unit-testing frameworks such as JUnit, NUnit, and MSTest, or are written in frameworks designed exclusively to facilitate the Behavior-Driven Development style such as RSpec and MSpec.

One style, referred to as Context/Specification, expresses software features in terms of specifications within a given usage context.  In this style, each unique way the software will be used is expressed as a context along with one or more expected outcomes.

While specialized frameworks have been created to support the Context/Specification style, this approach can also be facilitated using existing xUnit frameworks.  This is generally accomplished by creating a Context base class which uses the Template Method pattern to facilitate the desired language.

The following example demonstrates this technique using the NUnit framework:

    [TestFixture]
    public abstract class Context
    {
        [TestFixtureSetUp]
        public void setup()
        {
            establish_context();
            because_of();
        }

        [TestFixtureTearDown]
        public void teardown()
        {
            cleanup();
        }

        protected virtual void establish_context()
        {
        }

        protected virtual void because_of()
        {
        }

        protected virtual void cleanup()
        {
        }
    }

 

In addition to providing a Contact base class, the NUnit framework’s TestAttribute can also be sub-classed to aid in moving toward specification-oriented language:

 

public class ObservationAttribute : TestAttribute {}

 

By extending the Context base class, NUnit can then be used to write Context/Specification style BDD specifications:

 

public class when_adding_two_numbers : Context
{
    Calculator _calculator;

    protected override void establish_context()
    {
        _calculator = new Calculator();
        _calculator.Enter(2);
        _calculator.PressPlus();
        _calculator.Enter(2);
    }

    protected override void because_of()
    {
        _calculator.PressEnter();
    }

    [Observation]
    public void it_should_return_the_correct_sum()
    {
        Assert.AreEqual(_calculator.Result, 4); 
    }
}

Working, Maintainable Software That Matters

The ideas behind Test-First Programming continue to be refined by the software development community and have lead to still other variations in name and style, including Example Driven-Development, Specification-Driven Development, Need-Driven Development, and Story Test-Driven Development.  Regardless of its label or flavor, the refinements of Test-First Programming can be distilled down to the following goal: The creation of working, maintainable software that matters.  That is to say, the software should be functional, should be easily maintainable and should be relevant to needs of our customers.

Conclusion

This time we discussed Test-First Programming and traced its steps to today’s practice of Behavior-Driven Development.  Next time we’ll discuss Test-Driven Development in more detail and walk through a Test First example using the TDD methodology while incorporating some of the Behavior-Driven Development concepts presented here.

Tagged with: