Effective Tests: How Faking It Can Help You

On March 29, 2011, in Uncategorized, by derekgreer

In part 4 of our series, I presented a Test-Driven Development primer before beginning our exercise.  One of the techniques I’d like to discuss a little further before we continue is the TDD practice of using fake implementations as a strategy for getting a test to pass. 

While not discounting the benefits of using the Obvious Implementation first when a clear and fast implementation can be achieved, the recommendation to “Fake It (Until You Make It)” participates in several helpful strategies, each with their own unique benefits:

 

Going Green Fast

Faking it serves as one of the strategies for passing the test quickly. This has several benefits:

One, it provides rapid feedback that your test will pass when the expected behavior is met. This can be thought of as a sort of counterpart to "failing for the right reason".

Second, it has psychological benefits for some, which can aid in stress reduction through taking small steps, receiving positive feedback, and providing momentum.

Third, it facilitates a "safety net" which can be used to provide rapid feedback if you go off course during a refactoring effort.

 

Keeping Things Simple

Faking it serves as one of the strategies for writing maintainable software.

Ultimately, we want software that works through the simplest means possible. The "Fake It" strategy, coupled with Refactoring (i.e. eliminating duplication) or Triangulation (writing more tests to prove the need for further generalization), leads to an additive approach to arriving at a solution that accommodates the needs of the specifications in a maintainable way.  Faking It + Refactoring|Triangulation is a disciplined formula for achieving emergent design.

 

Finding Your Way

Faking it serves as a strategy for reducing mental blocks. 

As the ultimate manifestation of “Do the simplest thing that could possibly work", quickly seeing how the test can be made to pass tends to shine a bit of light on what the next step should be. Rather than sitting there wondering how to implement a particular solution, faking it and then turning your attention to the task of eliminating duplication or triangulating the behavior will push you in the right direction.

 

Identifying Gaps

Faking it serves as a strategy for revealing shortcomings in the existing specifications.

Seeing first hand how easy it is to make your tests pass can help highlight how an implementation might be modified in the future without breaking the existing specifications.  Part of the recommended strategy for keeping your code maintainable is to remove unused generalization.  Generalization  which eliminates duplication is needed, but your implementation may include generalization for which the driving need isn’t particularly clear.  Using a fake implementation can help uncover behavior you believe should be explicitly specified, but isn’t required by the current implementation.  Faking it can lead to such questions as: “If I can make it pass by doing anything that produces this value, what might prevent someone from altering what I’m thinking of doing to eliminate this duplication?

 

Conditioning

Lastly, faking it helps to condition you to seeing the simplest path first.  When you frequently jump to the complex, robust, flexible solution, you’ll tend to condition yourself to think that way when approaching problems.  When you frequently do simple things, you’ll tend to condition yourself to seeing the possible simplicity in the solution provided.

 

Conclusion

While we should feel free to use an Obvious Implementation when present, The Test-Driven Development strategy of “Fake It (Until You Make It)” can play a part in several overlapping strategies which help us to write working, maintainable software that matters.

Tagged with:  

Part 3 of our series introduced Test-First Programming and briefly discussed its refinements: Test-Driven Development and Behavior-Driven Development.  This time, we’ll dive into a simple Test First example using the principles of Test-Driven Development along with some of the Behavior-Driven Development concepts introduced last time.

Before beginning our example, we’ll take some time to discuss the goals and approach of Test-Driven Development in a little more detail to establish the methodology we’ll be following.

 

Test-Driven Development Primer

So, what is Test-Driven Development?  The simple answer is that it’s a process of writing tests before writing code, but such a simple answer fails to convey the true spirit of Test-Driven Development.

Even if we’ve adopted a practice of writing tests first, we’re often tempted to start implementing all the functionality we think will be part of a particular feature in a way that appeals to our design sensibilities.  Many of us are conditioned to approach the implementation phase by asking such questions as “What will this feature need to fully work”, “What is the most flexible way of fulfilling this feature”, or perhaps “What future design needs will I have that I can begin leveraging for this feature”.  This line of thinking usually leads us down a road of implementing patterns and algorithms, creating infrastructure code, and configuring third-party libraries and frameworks we’ve already planned to use.  The end result of such efforts may be working code, perhaps even with high test coverage, but while this approach might be considered Test-Inspired Development, it wouldn’t be Test-Driven Development.  

The goal of Test-Driven Development isn’t to ensure we write tests by writing them first, but to produce working software that achieves a targeted set of requirements using simple, maintainable solutions.  To achieve this goal, TDD provides strategies for keeping code working, simple, relevant and free of duplication.

Keep it Working

To keep the code working, TDD encourages development in small steps under the protection and confidence of passing tests which properly validate the desired behavior of the system.  To help minimize the amount of time a test exists in a failing state, TDD sets forth two strategies for quickly getting a test to pass.  The first strategy, to be used when the solution is simple and straight forward, is to use the Obvious Implementation.  The second strategy, to be used when a solution isn’t clear or will take some time to complete, is to Fake It.  The Fake It strategy refers to a technique of satisfying the requirements of a test by merely returning hard-coded values.

Keep it Simple

To keep the code simple,  TDD sets forth several techniques to help guide the developer toward the simplest solution.  First, TDD encourages a willingness to take small steps.  Taking small steps, especially for complex and unclear tasks, helps develop an ability to identify simple solutions.  Second, TDD provides guidelines for when to add or change code.  By changing code only to satisfy a failing test, remove duplication, or to remove unused code, solutions are kept free of complexity.  Third, TDD encourages a willingness to just try the Simplest Thing That Could Possibly Work when the obvious implementation isn’t clear.  Fourth, TDD sets forth a technique called Triangulation (discussed later) for helping to identify needed generalization.

Keep it Relevant

To keep the code relevant, TDD encourages the removal of unused generalization.  While this also helps keep the code simple, simplicity doesn’t mean the code is actually needed.  Developers often create constructs to improve the flexibility and extensibility of an application or as an outlet for their creativity, but this frequently leads to overly complex solutions which are difficult to understand, difficult to maintain and which are often never used.  TDD helps avoid this by keeping us focused on just those things that are needed to satisfy our tests.

Keep it Free of Duplication

To keep the code free of duplication, TDD incorporates continuous refactoring into the development process.  After writing a failing test and making it pass as quickly as possible, TDD encourages the removal of duplication as the heuristic for the discovery and improvement of design.  In TDD, it is through the refactoring process, driven by the goal of removing duplication, that much of an application’s design emerges.

The TDD Process

To help facilitate these goals, Test-Driven Development prescribes a set of steps for us to follow:

  1. Write a test.

  2. Make it compile.

  3. Run it to see that it fails.

  4. Make it run.

  5. Remove duplication.

These steps are often simplified into a process referred to as Red/Green/Refactor.  First we write a failing test, then we make it pass, then we refactor.

With these steps and goals in mind, let’s dive into our example.

 

Requirements

For this example, we’ll be creating a simple Tic-tac-toe game that allows a single player to compete against the game.  Here are the requirements we’ll be working from:

When the player goes first it should put their mark in the selected position it should make the next move When the player gets three in a row it should announce the player as the winner When the game gets three in a row it should announce the game as the winner When the player attempts to select an occupied position it should tell the player the position is occupied When the player attempts to select an invalid position it should tell the player the position is invalid When the game goes first it should put an X in one of the available positions When the player can not win on the next turn it should try to get three in a row When the player can win on the next turn it should block the player

 

Creating the Specifications

While we’ll be striving to follow the Test-Driven Development process, we’ll also try to incorporate some of the Behavior-Driven Development concepts we learned from last time.   There are several frameworks which are designed specifically to facilitate  Behavior-Driven Development, but we’re going to use the Visual Studio Unit Testing Framework  to demonstrate how these concepts can be applied using traditional xUnit frameworks. While we could begin by creating some testing infrastructure code, let’s keep things simple for now and just use some of the improved naming concepts presented in our last article.

Our first step will be to create a new Visual Studio project. I’ll use the Test Project type and name the project TestFirstExample:

 

TestFirstExampleProjectNew

 

Next, let’s create a test class that reflects the context of our first requirement:

using Microsoft.VisualStudio.TestTools.UnitTesting; namespace TestFirstExample { [TestClass] public class WhenThePlayerGoesFirst { } }

 

Next, let’s create a method that expresses the first observable behavior for our context:

using Microsoft.VisualStudio.TestTools.UnitTesting; namespace TestFirstExample { [TestClass] public class WhenThePlayerGoesFirst { [TestMethod] public void ItShouldPutTheirChoiceInTheSelectedPosition() { } } }

 

Since we’ve chosen to follow a more Behavior-Driven Development approach to writing our tests, let’s take a second and reflect on how this reads. Thus far, we’ve used the same wording from our requirements, but the class and method identifiers seem a little difficult to read. Since we’re using code to represent sentences instead of entities and methods, let’s try changing our naming convention to use underscores for separating each word in our identifiers and see how it looks:

using Microsoft.VisualStudio.TestTools.UnitTesting; namespace TestFirstExample { [TestClass] public class When_the_player_goes_first { [TestMethod] public void it_should_put_their_choice_in_the_selected_position() { } } }

 

In C#, this breaks with our normal naming conventions for class and method names, but this seems a little easier to read in my opinion.

 

Naming Conventions

The use of underscores for separating the words within identifiers is a commonly used convention for writing BDD specifications, especially within languages that don’t support the use of spaces. When first encountered, some developers find this unsettling due to the fact that it often represents a deviation from the standard naming convention for the language being used.

For those who feel uncomfortable with this style due to it’s deviation from standard conventions, it may help to think about naming conventions in terms of modeling. Just as we use a domain model to represent a business domain, a data model to represent data persistence concerns, and a view model to represent a user interface, BDD specifications are a kind of model for representing our requirements. As such, they deserve to be viewed within their own context. In this case, specifications are all about modeling requirements, not real-world entities. As such, some have chosen to depart from standard naming conventions when modeling specifications.

Of course, this is a matter of preference. What’s important is to view the modeling of specifications as its own concern and allow this understanding to help guide our modeling efforts.

 

The first step of our test method should be to determine how we want to verify that a player’s choice results in an X being placed in the selected position. To do this, we’ll need a way of evaluating the state of the game. One approach we could take would be to simply ask the game to tell us what value is in a given position. Since a Tic-tac-toe game is comprised of a grid with nine positions, we can refer to each position by number, numbering them from left to right, top to bottom. Let’s assume we can call a method named GetPosition():

[TestMethod] public void it_should_put_their_choice_in_the_selected_position() { Assert.AreEqual('X', game.GetPosition(1)); }

 

Next, we need to create the instance of our Game variable and determine how we’re going to inform it about our choice. A common rule of Tic-tac-toe is to let ‘X’ go first, so we can keep things simple if we adopt this rule. The only input needed in that case is the position, so let’s call a method named ChoosePosition() and pass an integer value of 1 to indicate the top-left position on the board:

[TestMethod] public void it_should_put_their_choice_in_the_selected_position() { var game = new Game(); game.ChoosePosition(1); Assert.AreEqual('X', game.GetPosition(1)); }

 

Of course, this won’t compile since our Game class doesn’t exist yet. Let’s create our class and methods now:

public class Game { public void ChoosePosition(int position) { throw new NotImplementedException(); } public char GetPosition(int position) { throw new NotImplementedException(); } }

 

Everything should be ready to compile. Let’s run our test and see what happens:

 

Test method TestFirstExample.When_the_player_goes_first .it_should_put_their_choice_in_the_selected_position threw exception: System.NotImplementedException: The method or operation is not implemented.

 

As you probably expected, our test fails. Since we’re following the Red/Green/Refactor process, you might think we’re ready to work on getting the test to pass now. Not so fast! The purpose of writing a failing test isn’t failure, but validation. At this point, what we should be interested in is that the behavior of our code will be correctly validated once we write it. To help explain, let’s consider what would happen if we had mistakenly written the test as follows:

[TestClass] public class When_the_player_goes_first { [TestMethod] public void it_should_put_their_choice_in_the_selected_position() { var game = new Game(); game.ChoosePosition(1); Assert.AreNotEqual('X', game.GetPosition(1)); } }

 

Here, we’ve changed Assert.AreEqual() to Assert.AreNotEqual(). How do you think this will change the outcome of the test? If you guessed it would not change the outcome then you’re correct. Our test didn’t fail because of our assert, but rather because an exception halted the test before it even got to the assertion. That doesn’t tell us anything about the validity of how we’ve written our test. Merely getting a test to fail doesn’t serve a useful purpose. Our test needs to fail for the right reason.

In this case, we want our test to fail because the call to the GetPosition() method didn’t return an X. Let’s remove the calls which are causing exceptions to be thrown and change the GetPosition() method to return a value that forces the test to fail due to the assertion:

public class Game { public void ChoosePosition(int position) { } public char GetPosition(int position) { return '\0'; } }

 

Running our test again produces the following output:

 

When_the_player_goes_first Failed it_should_put_their_choice_in_the_selected_position Assert.AreEqual failed. Expected:<X>. Actual:<\0>.

 

We now see that our test is failing due to the assertion properly validating the behavior of our class. Our next step is getting our test to pass quickly. Let’s use the Fake It approach and simply return the expected value of ‘X’:

public class Game { public void ChoosePosition(int position) { } public char GetPosition(int position) { return 'X'; } }

 

 

 

Our first test passes! Our next step is to eliminate any unnecessary generalization and duplication. While we don’t have any generalization to remove, we do have duplication. At first, our duplication may be difficult to spot as we’re typically accustomed to thinking about it in terms of repeated code blocks, but duplication manifests itself in many forms. In this case we have a form of duplication which I like to think of as Mirroring. We’re simply reflecting back to the test what it is expecting to see. This form of duplication is often the result of using the Fake It approach to getting tests to pass quickly, but this isn’t a bad thing. On the contrary, starting with hard-coded values and generalizing based upon a need to remove duplication helps to ensure that we aren’t introducing generalization unnecessarily.

To remove our duplication, let’s keep track of the player’s choice in a layout field which we can set when the ChoosePosition() method is called and then retrieve when the GetPosition() method is called:

public class Game { char _layout; public void ChoosePosition(int position) { _layout = 'X'; } public char GetPosition(int position) { return _layout; } }

 

 

 

You might be wondering why I chose such a naïve solution here. Won’t we need something that can keep track of all the positions on the board to accommodate other scenarios? The answer is yes, but we don’t have tests to drive our need to accommodate other scenarios yet. By only using the simplest constructs needed by our existing tests, we help to ensure we aren’t writing more complex code than necessary.

Another question that could be posed is, if we’re concerned with eliminating waste, doesn’t this process itself end up causing waste? The answer is yes and no. It can be reasoned that doing simple things does lead to waste in small degrees, but the benefit of allowing our design to emerge from this process is that it helps to eliminate large areas of waste.

Another question some may be wondering is whether taking small steps is a rule we have to follow when doing TDD. The answer to that question is no, but it is a useful methodology that helps keep us honest. We can take larger steps when it makes sense to do so, but it’s important to watch that our steps aren’t getting too large. You’ll know your steps are getting too large when you or others start frequently discovering unnecessary code or simpler solutions. By using this approach as a general practice, it helps to condition our ability to identify simple solutions. In his book Test-Driven Development By Example, Kent Beck points out that TDD is not about taking teeny-tiny steps, it’s about being able to take teeny-tiny steps.

Let’s move on to our next requirement. We’ll start by creating a new test method:

[TestMethod] public void it_should_make_the_next_move() { }

 

Again, our first step is to determine how we want to verify the behavior of the requirement. Since this test would be satisfied by the game choosing any position, we can verify the expected behavior occurred by checking all the positions. A clean way to do this is to create the range we want to check using Enumerable.Range() and then use LINQ’s Any() extension method to check each position for the character ‘O’. The Any() method returns true as soon as one of the enumerated elements is satisfied by the predicate:

[TestMethod] public void it_should_make_the_next_move() { Assert.IsTrue(Enumerable.Range(1, 9) .Any(position => game.GetPosition(position) .Equals('O'))); }

 

Next, we need to establish the context for the test. To avoid duplication, let’s move the context setup from our first test into a setup method and modify our tests to use a class field:

[TestClass] public class When_the_player_goes_first { Game _game; [TestInitialize] public void establish_context() { _game = new Game(); _game.ChoosePosition(1); } [TestMethod] public void it_should_put_their_choice_in_the_selected_position() { Assert.AreEqual('X', _game.GetPosition(1)); } [TestMethod] public void it_should_make_the_next_move() { Assert.IsTrue(Enumerable.Range(1, 9) .Any(position => _game.GetPosition(position) .Equals('O'))); } }

 

Now we’re ready to run our test. Since we’ve modified things, let’s run both the tests just to make sure we didn’t break the first one:

 

When_the_player_goes_first Failed it_should_make_the_next_move Assert.IsTrue failed.

 

From the output, we see that we didn’t break our first test and our second test is failing for the right reason. Next, let’s make our new test pass. To do so, the GetPosition() method will need to return an ‘O’ for one of the nine times it’s called. Let’s use the Fake It approach again to get the test to pass quickly. We can change our _layout field to a two character array and hard-code the ChoosePosition() method to assign ‘O’ to the second position:

public class Game { char[] _layout = new char[2]; public void ChoosePosition(int position) { _layout[position - 1] = 'X'; _layout[1] = 'O'; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

Now that the tests pass, we can move on to factoring out any duplication we’ve introduced. This time, however, we aren’t checking a specific position that’s we’ve hard coded a value to satisfy. On the other hand, we know this solution isn’t going to be flexible enough based on our knowledge about the rules of the game. How should we proceed?

To introduce generalization in cases where duplication either doesn’t exist or isn’t readily apparent, we can use a strategy called Triangulation. Similar to the mathematical process of determining the third point of a triangle given the length and angles of the first two points, this process helps you locate a generalized solution based upon two tests which verify the results in intersecting ways.

To demonstrate, let’s create a second test which validates the behavior of our class in a slightly different way which the current implementation wouldn’t satisfy:

[TestClass] public class When_the_player_selects_the_second_position { [TestMethod] public void it_should_put_their_choice_in_the_second_position() { Game game = new Game(); game.ChoosePosition(2); Assert.AreEqual('X', game.GetPosition(2)); } }

 

 

When_the_player_selects_the_second_position Failed it_should_put_their_choice_in_the_second_position Assert.AreEqual failed. Expected:<X>. Actual:<O>.

 

Our test fails because the method is hard-coded to assign ‘O’ to the second position. In order to make this test pass, we’ll now need to change our code to a more general solution. To replace the hard-coded selection of the second position, we can use LINQ to locate the first unoccupied element of the array and use that position for our assignment:

public class Game { readonly char[] _layout = new char[2]; public void ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

Our new test now passes. Our next step would be to factor out any duplication we might have introduced to satisfy the new test, but it doesn’t look like we introduced any this time.

While the use of Triangulation provides a simple to understand and explicit driver for introducing a generalized solution, it’s use is a somewhat inelegant strategy. Once a generalized solution is introduced, intersecting tests become redundant from a specification perspective. They can be useful in cases when it isn’t clear why a generalized solution exists and you need to ensure the solution isn’t lost through a future refactoring, but the resulting extra tests lead to higher test maintenance costs and can lead to confusion when included in reports or used as documentation by other team members. When possible, prefer removing duplication over triangulation as a generalization strategy. If duplication isn’t present or is of a particularly difficult to spot nature and generalization won’t be addressed by a separate requirement then it may be beneficial to leave redundant tests. To help aid in clarifying the intent of the test, we can use a TestCategoryAttribute as follows:

[TestClass] public class When_the_player_selects_the_second_position { [TestCategory("Triangulation")] [TestMethod] public void it_should_put_their_choice_in_the_second_position() { var game = new Game(); game.ChoosePosition(2); Assert.AreEqual('X', game.GetPosition(2)); } }

 

This helps to identify the reason for the test to other team members in a standard way and provides a mechanism for creating reports which don’t include the redundant test cases when desired. From the command line, our tests can be run without including the triangulation tests using the following:

c:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /category:'!Triangulation' \
/testcontainer:TestFirstExample.dll

 

For frameworks which provide non-sealed attributes for identifying categories of tests, we can create our own strongly-typed attribute to minimize any additional syntax noise as shown in the following contrived example:

public class TriangulationAttribute : SomeFrameworkCategoryAttribute { public MyAttribute() : base("Triangulation") { } }

 

While perhaps not easily discerned, we might have avoided the use of Triangulation in the above example had we considered our Any() extension method to have been hiding a form of duplication. When we used the Any() method to look for the first occurrence of a position with the value ‘O’, this was logically the same as if we had written the following:

[TestMethod] public void it_should_make_the_next_move() { char[] positions = new char[2]; bool gamePositionFound = false; for(int i = 0; i < 2; i++) { positions[i] = _game.GetPosition(i + 1); } for (int i = 0; i < 9; i++) { if (positions[i].Equals('O')) { gamePositionFound = true; break; } } Assert.IsTrue(gamePositionFound); }

 

Breaking the logic down further, our for loop would have performed the following checks:

if(positions[0].Equals(‘O’)) { … } if(positions[1].Equals(‘O’)) { … }

 

Based on a more explicit expression of what our code was doing, it becomes easier to identify the duplication that was present.

Moving on, our next requirement concerns how we’ll determine when the player has won the game. Let’s create a new test class along with it’s single observation:

[TestClass] public class When_the_player_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_player_as_the_winner() { } }

 

We know that we want to end up with a message that reflects that the player won, so let’s start with this as our assertion:

[TestClass] public class When_the_player_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_player_as_the_winner() { Assert.AreEqual("Player wins.", message); } }

 

Next, we need to determine where our message will come from. Let’s assume our ChoosePosition() method returns a message after the player makes a move. We’ll call the ChoosePosition() method three times and declare our Game instance:

[TestClass] public class When_the_player_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_player_as_the_winner() { Game game = new Game(); game.ChoosePosition(1); game.ChoosePosition(2); string message = game.ChoosePosition(3); Assert.AreEqual("Player wins!", message); } }

 

To get this to compile, we need to change the ChoosePosition() method to return a string:

public string ChoosePosition(int position) { _layout[position - 1] = value; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; return string.Empty; }

 

We’re now ready to run our tests:

 

When_the_player_gets_three_in_a_row Failed it_should_announce_the_player_as_the_winner Test method TestFirstExample.When_the_player_gets_three_in_a_row .it_should_announce_the_player_as_the_winner threw exception: System.InvalidOperationException: Sequence contains no matching element

 

It looks like our test isn’t failing for the right reason yet. The problem is that our array isn’t large enough for our new test scenario. Let’s go ahead and take the larger step of increasing the layout to accommodate the full 9 positions of the game board and run our test again:

readonly char[] _layout = new char[9];

 

 

When_the_player_gets_three_in_a_row Failed it_should_announce_the_player_as_the_winner Assert.AreEqual failed. Expected:<Player wins!>. Actual:<>.

 

Now, let’s make it pass. We can just return the string “Player wins!” to get the test to pass quickly:

public class Game { readonly char[] _layout = new char[9]; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; return "Player wins!"; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

We now have the string “Player wins!” duplicated, so we’ll fix this by putting in some logic to test that the first three positions contain the value ‘X’:

public class Game { readonly char[] _layout = new char[9]; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; return string.Empty; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

We can now move on to our next requirement which concerns determining when the game wins. Here’s our test skeleton:

[TestClass] public class When_the_game_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_game_as_the_winner() { } }

 

Again, we’ll start by declaring our assertion and then fill in the context of the test. Since we know that the game chooses the first unoccupied space, we can leverage this by choosing positions which leave the first row open:

[TestClass] public class When_the_game_gets_three_in_a_row { [TestMethod] public void it_should_announce_the_game_as_the_winner() { Game game = new Game(); game.ChoosePosition(4); game.ChoosePosition(6); string message = game.ChoosePosition(8); Assert.AreEqual("Game wins.", message); } }

 

 

When_the_game_gets_three_in_a_row Failed it_should_announce_the_game_as_the_winner Assert.AreEqual failed. Expected:<Game wins.>. Actual:<>.

 

To get the test to pass, we can use similar logic as before to check that the layout starts with “OOO”:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

 

 

Now, let’s refactor. In reviewing our code, we see that we have some similarities between the logic checking if the player wins and the logic checking if the game wins. We need a way of generalizing this comparison so that it works for both. One option would be to use regular expressions. If we have an array of winning patterns, we can convert the layout to a string and compare it to each one in a loop. Let’s figure up all the winning patterns real quick:

tic-tac-toe

It looks like we have eight different winning patterns. Flattening this out becomes:

X X X _ _ _ _ _ _, _ _ _ X X X _ _ _, _ _ _ _ _ _ X X X, X _ _ X _ _ X _ _, _ X _ _ X _ _ X _, _ _ X _ _ X _ _ X, X _ _ _ X _ _ _ X, _ _ X _ X _ X _ _

 

Let’s start by creating a new array reflecting the winning patterns for ‘X’:

public class Game { readonly char[] _layout = new char[9]; readonly string[] _winningXPatterns = new[] { "XXX......", "...XXX...", "......XXX", "X..X..X..", ".X..X..X.", "..X..X..X", "X...X...X", "..X.X.X..", }; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

Next, we need to create a string representation of our current layout:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; var layoutAsString = new string(_layout); if (new string(_layout.ToArray()).StartsWith("XXX")) return "Player wins!"; if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

Next, let’s replace the previous comparison checking if the layout starts with “XXX” with a loop that checks each winning pattern to the current layout:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; var layoutAsString = new string(_layout); foreach (string pattern in _winningXPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Player wins!"; } if (new string(_layout.ToArray()).StartsWith("OOO")) return "Game wins."; return string.Empty; }

 

 

 

Everything is still working. Now, let’s make the same changes for the game comparison:

public class Game { readonly char[] _layout = new char[9]; readonly string[] _winningXPatterns = new[] { "XXX......", "...XXX...", "......XXX", "X..X..X..", ".X..X..X.", "..X..X..X", "X...X...X", "..X.X.X..", }; readonly string[] _winningOPatterns = new[] { "OOO......", "...OOO...", "......OOO", "O..O..O..", ".O..O..O.", "..O..O..O", "O...O...O", "..O.O.O..", }; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; var layoutAsString = new string(_layout); foreach (string pattern in _winningXPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Player wins!"; } foreach (string pattern in _winningOPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Game wins."; } return string.Empty; } public char GetPosition(int position) { return _layout[position - 1]; } }

 

 

 

We’re still green. At this point we’ve actually introduced a little more duplication than we started with, but we’re making small steps that will eventually lead us to a solution. Next, let’s see if we can combine our two arrays to represent the winning position for both players. Since we’re using a regular expression, we can replace the characters with character sets like this:

readonly string[] _winningPatterns = new[] { "[XO][XO][XO]......", "...[XO][XO][XO]...", "......[XO][XO][XO]", "[XO]..[XO]..[XO]..", ".[XO]..[XO]..[XO].", "..[XO]..[XO]..[XO]", "[XO]...[XO]...[XO]", "..[XO].[XO].[XO]..", };

 

We’ll then need to change the comparisons to use the new array. Since we only want to compare one side at a time, we’ll also need to filter out the characters we don’t want:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; string layoutAsString = new string(_layout).Replace('O', '\0'); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Player wins!"; } layoutAsString = new string(_layout).Replace('X', '\0'); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layoutAsString, pattern)) return "Game wins."; } return string.Empty; }

 

 

 

Now, let’s take care of the looping duplication by creating a method that will perform the comparison based on the given side we’re interested in:

bool WinningPlayerIs(char player) { var layout = new string(_layout); if (player == 'X') layout = layout.Replace('O', '\0'); else layout = layout.Replace('X', '\0'); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layout, pattern)) return true; } return false; }

 

We can now replace our comparisons with a call to our new method:

public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (WinningPlayerIs('X')) return "Player wins!"; if (WinningPlayerIs('O')) return "Game wins."; return string.Empty; }

 

 

 

Now, let’s clean up our new WinningPlayerIs() method. We’re duplicating the call to Replace() based upon which character we’re wanting the layout for, so instead of this let’s use LINQ to create an array with the character we want filtered out:

bool WinningPlayerIs(char player) { var layout = new string(_layout.ToList() .Select(c => (c.Equals(player)) ? player : '\0') .ToArray()); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layout, pattern)) return true; } return false; }

 

 

 

That’s more concise, but it could stand to be more descriptive. Rather than adding a comment, let’s just wrap this in an intention-revealing method:

bool WinningPlayerIs(char player) { var layout = GetLayoutFor(player); foreach (string pattern in _winningPatterns) { if (Regex.IsMatch(layout, pattern)) return true; } return false; } string GetLayoutFor(char player) { return new string(_layout.ToList() .Select(c => (c.Equals(player)) ? player : '\0') .ToArray()); }

 

We can also eliminate declaring multiple exit points and simplify the comparison by using LINQ’s Any() extension method:

bool WinningPlayerIs(char player) { var layout = GetLayoutFor(player); return _winningPatterns.Any(pattern => Regex.IsMatch(layout, pattern)); }

 

Let’s go ahead and in-line our call to GetLayoutFor(player):

bool WinningPlayerIs(char player) { return _winningPatterns .Any(pattern => Regex.IsMatch(GetLayoutFor(player), pattern)); }

 

 

 

Here’s what we have so far:

public class Game { readonly char[] _layout = new char[9]; readonly string[] _winningPatterns = new[] { "[XO][XO][XO]......", "...[XO][XO][XO]...", "......[XO][XO][XO]", "[XO]..[XO]..[XO]..", ".[XO]..[XO]..[XO].", "..[XO]..[XO]..[XO]", "[XO]...[XO]...[XO]", "..[XO].[XO].[XO]..", }; public string ChoosePosition(int position) { _layout[position - 1] = 'X'; int firstUnoccupied = Enumerable.Range(0, _layout.Length) .First(p => _layout[p].Equals('\0')); _layout[firstUnoccupied] = 'O'; if (WinningPlayerIs('X')) return "Player wins!"; if (WinningPlayerIs('O')) return "Game wins."; return string.Empty; } bool WinningPlayerIs(char player) { return _winningPatterns .Any(pattern => Regex.IsMatch(GetLayoutFor(player), pattern)); } string GetLayoutFor(char player) { return new string(_layout.ToList() .Select(c => (c.Equals(player)) ? player : '\0') .ToArray()); } public char GetPosition(int position) { return _layout[position - 1]; } }

 

We’ll leave things here for now and continue our effort next time.

Tagged with:  

Effective Tests: Test First

On March 22, 2011, in Uncategorized, by derekgreer

In part 2 of our series, I presented a simple example of a traditional unit test.  In this article, we’ll discuss a completely different way to use unit tests: Test First Programming.

Test-First Programming

In the book Extreme Programming Explained: Embrace Change published in October 1999, Kent Beck introduced what was at the time a rather foreign concept to software development: Writing unit tests to drive software design.  Described as Test-First Programming, the technique involves first writing a test to verify a small increment of desired functionality and then writing code to make the test pass.  This process is then repeated until no additional functionality is desired.

Incremental Design

Another practice introduced alongside Test-First Programming was Incremental Design.  Incremental design is the process of evolving the design of an application as new functionality is developed and better design approaches are recognized.  Due to the fact that software development is an inherently empirical process,  following a defined model where the the software is designed up front generally leads to waste and rework.  Incremental design postpones design decisions until needed, introducing patterns and frameworks as a byproduct of refactoring as well as eliminating any flexibility that is unused.

Test Driven Development

Later, these ideas were  refined into a practice known as Test Driven Development.  Test-Driven Development can be described simply as the ideas of Test-First Programming coupled with Incremental Design.  In a later book entitled Test-Driven Development By Example published in November of 2002, Beck presents a refined process involving the writing of a failing test, taking the simplest steps possible to make the test pass, and removing any duplication introduced.  This process was described succinctly as Red/Green/Refactor, where red and green referred to the colors typically presented by test runners to indicate failing and passing tests respectively.

Behavior-Driven Development

Test Driven Development

 

While the practice of Test-Driven Development utilized unit tests primarily for driving application design, its outgrowth from traditional unit testing practices were still evident in its style,  language, and supporting frameworks. This presented an obstacle for some in understanding and communicating the concepts behind the practice.

In the course of teaching and practicing TDD, Dan North began encountering various strategies for designing tests in a more intent-revealing way which aided in his understanding and ability to convey the practice to others.  As a way to help distinguish the differences between what he saw as an exercise in defining behavior rather than testing, he began describing the practice of Test-Driven Development as Behavior Driven Development.  By shifting the language to the discussion about behavior, he found that many of the questions he frequently encountered in presenting TDD concepts became easier to answer.  North went on to introduce a framework incorporating this shift in thinking called JBehave.  North’s work on JBehave along with his promotion of the ideas behind Behavior-Driven Development served as the inspiration for the creation of many other frameworks including RSpec, Cucumber, NBehave, NSpec, SpecFlow, MSpec and many others.

Behavior-Driven Development Approaches

Two main approaches have emerged from BDD practices which I’ll categorize as the Plain-Text Story approach and the Executable Specification approach.

Plain-Text Stories

With the Plain-Text Story approach, software features are written as a series of plain-text scenarios which the framework associates with executable steps used to drive feature development.  This approach is typically associated with a style referred to Given/When/Then (GWT), named after the grammar set forth by Dan North.  This style is facilitated by popular BDD frameworks such as Cucumber and JBehave.

The following is an example story using Gherkin grammar, the language used by Cucumber and other Plain-Text Story BDD frameworks:

Feature: Addition
   In order to calculate totals
   As a user
   I want to calculate the sum of multiple numbers

Scenario: Add two numbers
   Given I have entered 10 into the calculator
   And I have entered 50 into the calculator
   When I press add
   Then the result should be 60 on the screen

 

Using Cucumber, the “Given” step in this scenario may then be associated to the following step definition in Ruby:

Given /I have entered (\d+) into the calculator/ do |n|
  calc = Calculator.new
  calc.push n.to_i
end

 

Using SpecFlow, the same “Given” step can also be associated to the following step definition in C#:

[Given("I have entered (.*) into the calculator")]
public void GivenIHaveEnteredSomethingIntoTheCalculator(int number)
{
    _calculator = new Calculator();
    _calculator.Enter(number);
}

Executable Specifications

In contrast to the use of plain-text stories, the Executable Specification approach uses source code as the medium for expressing specifications in the language of the business.  Executable Specifications can be written using existing unit-testing frameworks such as JUnit, NUnit, and MSTest, or are written in frameworks designed exclusively to facilitate the Behavior-Driven Development style such as RSpec and MSpec.

One style, referred to as Context/Specification, expresses software features in terms of specifications within a given usage context.  In this style, each unique way the software will be used is expressed as a context along with one or more expected outcomes.

While specialized frameworks have been created to support the Context/Specification style, this approach can also be facilitated using existing xUnit frameworks.  This is generally accomplished by creating a Context base class which uses the Template Method pattern to facilitate the desired language.

The following example demonstrates this technique using the NUnit framework:

    [TestFixture]
    public abstract class Context
    {
        [TestFixtureSetUp]
        public void setup()
        {
            establish_context();
            because_of();
        }

        [TestFixtureTearDown]
        public void teardown()
        {
            cleanup();
        }

        protected virtual void establish_context()
        {
        }

        protected virtual void because_of()
        {
        }

        protected virtual void cleanup()
        {
        }
    }

 

In addition to providing a Contact base class, the NUnit framework’s TestAttribute can also be sub-classed to aid in moving toward specification-oriented language:

 

public class ObservationAttribute : TestAttribute {}

 

By extending the Context base class, NUnit can then be used to write Context/Specification style BDD specifications:

 

public class when_adding_two_numbers : Context
{
    Calculator _calculator;

    protected override void establish_context()
    {
        _calculator = new Calculator();
        _calculator.Enter(2);
        _calculator.PressPlus();
        _calculator.Enter(2);
    }

    protected override void because_of()
    {
        _calculator.PressEnter();
    }

    [Observation]
    public void it_should_return_the_correct_sum()
    {
        Assert.AreEqual(_calculator.Result, 4); 
    }
}

Working, Maintainable Software That Matters

The ideas behind Test-First Programming continue to be refined by the software development community and have lead to still other variations in name and style, including Example Driven-Development, Specification-Driven Development, Need-Driven Development, and Story Test-Driven Development.  Regardless of its label or flavor, the refinements of Test-First Programming can be distilled down to the following goal: The creation of working, maintainable software that matters.  That is to say, the software should be functional, should be easily maintainable and should be relevant to needs of our customers.

Conclusion

This time we discussed Test-First Programming and traced its steps to today’s practice of Behavior-Driven Development.  Next time we’ll discuss Test-Driven Development in more detail and walk through a Test First example using the TDD methodology while incorporating some of the Behavior-Driven Development concepts presented here.

Tagged with:  

Effective Tests: A Unit Test Example

On March 14, 2011, in Uncategorized, by derekgreer

In part 1 of our series, I introduced some common types of automated tests: Unit Tests, Integration Tests, and Acceptance Tests.  In this article, we’ll take a look at a traditional approach to unit testing and discuss a few practices surrounding good test communication.

First Things First

Before getting started, we obviously need something to test, so let’s create a Calculator class.

   public class Calculator
    {
        bool _isDirty;

        string _operation;
        decimal _state;

        public decimal Display { get; private set; }

        public void Enter(decimal number)
        {
            _state = number;
            _isDirty = true;
        }

        public void PressPlus()
        {
            _operation = "+";
            if (_isDirty) Calculate();
        }

        public void PressEquals()
        {
            if (_isDirty) Calculate();
        }

        void Calculate()
        {
            switch (_operation)
            {
                case "+":
                    Display += _state;
                    break;
            }

            _isDirty = false;
        }
    }

To keep things simple, our Calculator class just adds decimals, but that gives us enough to work with for our discussion.

Anatomy of a Unit Test

Now, let’s create a unit test to validate that our Calculator class behaves properly.  I’ll use NUnit for this example:

[TestFixture]
public class CalculatorTests
{
    [Test]
    public void TestPressEquals()
    {
        Calculator calculator = new Calculator();
        calculator.Enter(2m);
        calculator.PressPlus();
        calculator.Enter(2m);
        calculator.PressEquals();
        Assert.AreEqual(4m, calculator.Display);
    }
}

So, what’s going on here?  Starting at the top, we have a class with a TestFixture attribute.  This tells NUnit that this class contains unit tests.  We’ve named our test “CalculatorTests” to denote that this class contains the tests for our Calculator class.  Pretty descriptive, eh?

Next, we’ve defined a method with a Test attribute.  This tells NUnit that this method should be considered a test when it scans our types for tests to execute.  We’ve named our method TestPressEquals() to denote that we’re testing the behavior of the PressEquals method. 

Note

The use of the prefix “Test” is a common convention which originated from a requirement within earlier versions of JUnit.  The first version of NUnit was a direct port of JUnit and therefore shared this requirement until it was rewritten to take advantage of .Net attributes.

Next, we’ve defined an instance of the Calculator class we’re going to be testing.  In unit testing parlance, this is generally called the System Under Test, or the SUT.

Next, we make several calls to the calculator object to instruct it to add two numbers and then call the PressEquals() method.

Our final call is to the NUnit Assert.AreEqual() method.  This method tests the expected value against the actual value and raises an exception if the values are not equal.

If we run the test at this point, we should see our test pass.

 

Hooray!

 

Improving Communication

While our test certainly does the job of validating the Calculator’s behavior, there are a few aspects of our test that we could improve upon.  To start, let’s consider what would be displayed should our test actually ever fail.  To force it to fail, I’ll comment out the line which increments the Display property.  Running the test produces the following:

 

CalculatorTests.TestPressEquals : Failed 
Expected: 4m 
But was:  0m

 

Ok, so CalculatorTests.TestEquals failed because "expected 4m, but was 0m".  Given that we’ve just written this test, this may not seem like much of an issue.  The context of how our Calculator class works as well as how our test verifies it behavior is still fresh in our minds.  But what if this fails six months from now?  Worse, what if it fails and is discovered by someone who wasn’t the author of this test?  Without looking at the test code, the most that could be derived from this message is that the Calculator.PressEquals method failed in some way.  Let’s see if we can improve upon this.

What would be great is if we could have the test runner print out a descriptive message describing what didn’t work.  As it happens, NUnit provides an overload to the TestEquals method which allows us to supply our own message.  Let’s change our test to include a custom message:

        [Test]
        public void TestPressEquals2()
        {
            var calculator = new Calculator();
            calculator.Enter(2);
            calculator.PressPlus();
            calculator.Enter(2);
            calculator.PressEquals();
            Assert.AreEqual(4, calculator.Display,
                "When adding 2 + 2, expected 4 but found {0}.", calculator.Display);
        }

When we run the test again, we get the following:

 

CalculatorTests.TestPressEquals : Failed 
When adding 2 + 2, expected 4 but found 0. 
Expected: 4m 
But was:  0m

 

By supplying  a custom message, our test now provides some context around why the test failed.  As an added benefit, our custom message also adds a bit of description to the purpose of our test when reading the source code.

Unfortunately, these changes have introduced a bit of duplication into our test code.  We now have the expected result, the actual result, and the values being added together repeated throughout out test.  Also, it seems like a good idea to have the assert message automatically reflect any changes that might be made to the values used by the test.  Let’s take a stab at reducing the duplication:

        [Test]
        public void TestPressEquals2()
        {
            var value1 = 2m;
            var value2 = 2m;
            var calculator = new Calculator();
            calculator.Enter(value1);
            calculator.PressPlus();
            calculator.Enter(value2);
            calculator.PressEquals();
            decimal expected = 4m;
            decimal actual = calculator.Display;
            Assert.AreEqual(expected, actual, 
                "When adding {0} + {1}, expected {2} but found {3}.", value1, value2, expected, actual);
        }

Now we’ve eliminated the duplication, but the test doesn’t seem as easy to follow.  Also, our custom message no longer communicates the purpose of the test clearly.  If we revisit this test later, we might as well work through the logic of the test than work through the logic of what this assertion  is going to produce.  Let’s stick to our principles on keeping our code free of duplication for now, but we’ll revisit this topic later in our series.

Aside from there being nothing about our test that allows us to easily pick up on what’s occurring, our code is also looking a little crowded.  We could easily address this by spacing things out a bit and perhaps adding a few comments.

One popular style of organization is a pattern first described by Bill Wake as Arrange, Act, Assert.  With this organization, the test is separated into three sections: one which Arranges the objects to be used as part of the test, one which Acts upon the objects, and one which Asserts that the expected outcomes have occurred.  Since our test mostly follows this sequence already, let’s just space the sections out a bit and add some comments denoting the Arrange, Act and Assert sections:

        [Test]
        public void TestPressEquals()
        {
            // Arrange
            decimal value1 = 2m;
            decimal value2 = 2m;
            decimal expected = 4m;
            var calculator = new Calculator();

            // Act
            calculator.Enter(value1);
            calculator.PressPlus();
            calculator.Enter(value2);
            calculator.PressEquals();
            decimal actual = calculator.Display;

            // Assert
            Assert.AreEqual(expected, actual,
                            "When adding {0} + {1}, expected {2} but found {3}.", value1, value2, expected, actual);
        }

That organizes the flow of the test a bit better, but we’re still missing the clarity that initial custom message was providing.  We could communicate this through a comment as well, but another approach would be to just improve upon the actual test name.

One common naming convention is to create a name reflecting the name of the method being tested, the scenario being tested and the expected behavior.  Let’s change our test method name to reflect this convention:

        [Test]
        public void PressEquals_AddingTwoPlusTwo_ReturnsFour()
        {
            // Arrange
            decimal value1 = 2m;
            decimal value2 = 2m;
            decimal expected = 4m;
            var calculator = new Calculator();

            // Act
            calculator.Enter(value1);
            calculator.PressPlus();
            calculator.Enter(value2);
            calculator.PressEquals();
            decimal actual = calculator.Display;

            // Assert
            Assert.AreEqual(expected, actual,
                            "When adding {0} + {1}, expected {2} but found {3}.", value1, value2, expected, actual);
        }

 

After we’ve adjusted to reading this naming convention, we can now derive the same information that was communicated in that first message.  Someone get a banner printed up, we’re declaring victory!

Conclusion

In this article, we’ve taken a look at a basic approach to writing unit tests and introduced a few techniques for improving unit test communication.  While our resulting test correctly validates the behavior of our class, communicates what it does and provides a descriptive error message when it fails, there’s still room for improvement.  In our next article, we’ll discuss some alternative approaches that will facilitate these goals as well as help us write working, maintainable software that matters.

Tagged with:  

Effective Tests: Introduction

On March 7, 2011, in Uncategorized, by derekgreer

This is part 1 of a series discussing  topics surrounding effective automated testing.  Automated testing can be instrumental to both the short and long-term success of a project.  Unfortunately, it is too often overlooked due to either a lack of knowledge of how to incorporate test into an existing process, or a lack of recognition of the deficiencies within an existing process.  As with any new pursuit, learning how to use automated testing effectively within a development process can take time.  The goal of this series is to help those new to the practice of automated testing by gradually introducing concepts which aid in the creation of working, maintainable software that matters.

This introduction will start things off by discussing some of the fundamental types of automated tests.

 

Unit Testing

Unit tests are perhaps the most widely recognized form of test automation.  A Unit Test is a process which validates behavior within an application in isolation from any of its dependencies.   Unit tests are typically written in the same language as the software being tested, take the form of a method or class designed for a particular testing framework (such as JUnit, NUnit, or MSTests) and are generally designed to validate the behavior of individual methods or functions within an application.  The behavior of a component being tested can be isolated from the behavior of its dependencies using substitute dependencies known as Test Doubles.  By testing each component’s behavior in isolation, failing tests can be used to more readily identify which components are causing a regression in behavior.

The primary goal of traditional unit testing is verification.  By establishing a suite of unit tests, the software can be tested each time modifications are made to ensure the software still behaves as expected.  To ensure that existing tests are always run when modifications are introduced, the tests can also be run at regular intervals or triggered as part of a check-in policy using a process known as Continuous Integration.

 

Integration Testing

While unit tests are useful for verifying that the encapsulated behavior of a component works as expected within a known context, they often fall short of anticipating how the component will interact with other components used by the system.  Tests which verify that components behave as expected with all, or a subset of their real dependencies are often categorized as Integration Tests.  Of particular interest are interactions with third-party libraries and external resources such as file systems, databases, and network services.  This is due to the fact that the behavior of such dependencies may not be fully known or controlled by the consuming development team, or may change in unexpected ways when new versions are introduced.

Integration tests often require more setup and/or reliance upon communication with external processes and are therefore usually slower than unit test suites.  Due to this fact, separate strategies are often used to ensure regular feedback of integration tests.  In some cases, slow-running tests can be mitigated by the use of “almost-real” substitutes such as in memory file systems and databases which are known to adequately represent the functionality expected by the real dependencies.

While the term “integration test” is often applied to any test verifying the behavior of collaborating components, it can be useful for test organization and the development of testing strategies to draw a distinction between tests which verify integration with disparate systems and those that verify that a collection of classes correctly provide a logical service.  In the book xUnit Test Patterns: Refactoring Test Code, Gerard Meszaros refers to tests for such collaborating classes as Component Tests.  While both test the interaction of multiple components, Component Tests ask the question “Does this logical service perform the expected behavior?” while Integration Tests asks the question “Do these components work together?”.

 

Acceptance Testing

Acceptance tests, or end-to-end tests, verify that particular use cases of the system work as a whole.  Such tests are characterized by a focus on how the system will be used by the customer by exercising as much of the real system as possible.  While finer-grained tests such as unit and component tests can help ensure the functional integrity of the individual components, acceptance testing ensures that the components function correctly together.

Although the purpose of acceptance testing is to verify the system works as a whole, it may still be necessary in some cases to substitute portions of the system where full end-to-end testing isn’t  cost-effective or practical.  For instance, some external services may not provide integration testing environments or may place limits on its use.  In other cases, the user interface technology used may not lend itself to test automation in which case tests may be written against a layer just below the user interface layer.  This is referred to as Subcutaneous Testing.

 

Classifications

The terms unit test, integration test and acceptance test classify tests in terms of their utility.  Another way to classify tests are in terms of their audience.  In Extreme Programming (XP), tests are broken down into the categories of Customer Tests and Programmer Tests.  Customer Tests are synonymous with User Acceptance Tests and are focused on the external quality of the system.  Programmer Tests are similar to Unit Tests in that they are written by programmers and are generally focused on the internal quality of the system, but they are less prescriptive about their level of test isolation.  Programmer Tests will be discussed in more detail later in our series.

 

Conclusion

This article presents only a brief introduction to some of the classifications of automated tests.  We’ll continue to explore these and others throughout the series.  Next time, we’ll take a look a traditional approach to writing unit tests.

Tagged with: