Proving Traits

"A play there is, my lord, some ten words long,
Which is as ‘ brief ’ as I have known a play.
But by ten words, my lord, it is too long"
-- Shakespeare, Midsummer Night's Dream

Act V, Scene I, Lines 61-63

Martin Heidegger in his paper, The Origin of the Work of Art, gives us three different ways of defining thingness.

1. bearer of traits and properties
2. sense perceptions
3. formed stuff

Traits is an interesting term in computer science, it is a collection of behaviors and their implementation which can be used to extend an object.  Most languages do not directly support the creation of stand-alone traits but traits are the essence of what makes up a programs value.

I am going to make a very bold statement here, I am going to say that BDD (Behavior Driven Development) is really about proving the traits of software.  Using the definition of traits given above we see that a trait is a collection of behaviors and how they have been implemented.  This sounds a lot like what we are proving out when we do BDD.

In BDD what are we doing?  We are proving that behavior X as it is implemented by code S does the following when given state G with stimulus W yielding state T.  In other words we are showing Given the following conditions, When we do the following against X in this Source code, Then the following result will occur.  This sounds a lot like proving the traits of the software to me.

"Words, words, words."

-- Shakespeare, Hamlet

Act II, Scene II, Line 193

Let us look at a example.

	namespace FizzBuzz
	{
	public class FizzBuzzer
	{
	public string Translate(int value)
	{
	var result = string.Empty;
	if (value % 3 == 0) result += "Fizz";
	if (value % 5 == 0) result += "Buzz";
	return string.IsNullOrEmpty(result) ? value.ToString() : result;
	}
	}
	}

view raw FizzBuzzer.cs hosted with ❤ by GitHub

	Feature: FizzBuzzerSpec
	I want to be able to translate numbers to their FizzBuzz value
	In order to be able to communicate with the FizzBuzzons
	As a Starfleet Captain
	Scenario: 2 must translate to the string 2
	Given a FizzBuzzer
	And the value of '2'
	When translate is invoked
	Then the result should be '2'
	Scenario: 3 must translate to the string Fizz
	Given a FizzBuzzer
	And the value of '3'
	When translate is invoked
	Then the result should be 'Fizz'
	Scenario: 5 must translate to the string Buzz
	Given a FizzBuzzer
	And the value of '5'
	When translate is invoked
	Then the result should be 'Buzz'
	Scenario: 15 must translate to the string FizzBuzz
	Given a FizzBuzzer
	And the value of '15'
	When translate is invoked
	Then the result should be 'FizzBuzz'
	Scenario Outline: FizzBuzz translation
	Given a FizzBuzzer
	And the value of '<value>'
	When translate is invoked
	Then the result should be '<expected value>'
	Examples:
	| value | expected value |
	| 2 | 2 |
	| 4 | 4 |
	| 3 | Fizz |
	| 9 | Fizz |
	| 5 | Buzz |
	| 25 | Buzz |
	| 15 | FizzBuzz |
	| 30 | FizzBuzz |

view raw FizzBuzzerSpec.feature hosted with ❤ by GitHub

	using FizzBuzz;
	using TechTalk.SpecFlow;
	using NUnit.Framework;
	namespace FizzBuzzSpec
	{
	[Binding]
	public class FizzBuzzerSpecSteps
	{
	[Given(@"a FizzBuzzer")]
	public void GivenAFizzBuzzer()
	{
	ScenarioContext.Current.Add("FizzBuzzer", new FizzBuzzer());
	}
	[Given(@"the value of '(.*)'")]
	public void GivenTheValueOf(int value)
	{
	ScenarioContext.Current.Add("value", value);
	}
	[When(@"translate is invoked")]
	public void WhenTranslateIsInvoked()
	{
	var fizzbuzzer = ScenarioContext.Current.Get<FizzBuzzer>("FizzBuzzer");
	var value = ScenarioContext.Current.Get<int>("value");
	ScenarioContext.Current.Add("actual", fizzbuzzer.Translate(value));
	}
	[Then(@"the result should be '(.*)'")]
	public void ThenTheResultShouldBe(string expected)
	{
	var actual = ScenarioContext.Current.Get<string>("actual");
	Assert.That(expected, Is.EqualTo(actual));
	}
	}
	}

view raw FizzBuzzerSpecSteps.cs hosted with ❤ by GitHub

In the example above we see that a Starfleet Captain needs to be able to communicate with the FizzBuzzons and thus needs to have a translator.

She is given the following C# source code to preform the needed behavior of translating.

        public string Translate(int value)

        {

            var result = string.Empty;

 

            if (value % 3 == 0) result += "Fizz";

            if (value % 5 == 0) result += "Buzz";

 

            return string.IsNullOrEmpty(result) ? value.ToString() : result;

        }

To prove to her that it works the engineers and communication specialist came up with the following specifications listing what happens in a Given state When the an input value is translated Then the value will have a certain output.

Scenario: 2 must translate to the string 2

 Given a FizzBuzzer

 And the value of '2'

 When translate is invoked

 Then the result should be '2'

 

Scenario: 3 must translate to the string Fizz

 Given a FizzBuzzer

 And the value of '3'

 When translate is invoked

 Then the result should be 'Fizz'

 

Scenario: 5 must translate to the string Buzz

 Given a FizzBuzzer

 And the value of '5'

 When translate is invoked

 Then the result should be 'Buzz'

 

Scenario: 15 must translate to the string FizzBuzz

 Given a FizzBuzzer

 And the value of '15'

 When translate is invoked

 Then the result should be 'FizzBuzz'

By testing around these specifications we can prove to our captain that the translator does hold the desired traits.

Does this slight different definition around BDD change anything?  No, but I hope it does widen the thought around it.

"If we shadows have offended
Think but this, and all is mended:
That you have but slumbered here
While these visions did appear."

-- Shakespeare, Midsummer Night's Dream

Act V, Scene I, Lines 413-416

Specs in Scala and C#

"That I have uttered. Bring me to the test,
And I the matter will re-word, which madness"

-- Shakespeare, Hamlet

Act III, Scene IV, Lines 143-144

Prolog

Places, places, the play is about to begin.

Tonight we shall look at a tale of two specification frameworks, Scala's spec2 and .Net's SpecFlow.

Act I - spec2

Say we have the following basic FizzBuzz functionality in Scala.

object FizzBuzzer {

class FizzBuzzExtractor[T](f: T => Boolean) {

def unapply(x: T) = f(x)

}



val FizzBuzz = new FizzBuzzExtractor[Int](_ % 15 == 0)

val Fizz = new FizzBuzzExtractor[Int](_ % 3 == 0)

val Buzz = new FizzBuzzExtractor[Int](_ % 5 == 0) 



def eval(value: Int) = value match {

case FizzBuzz() => "FizzBuzz"

case Fizz()     => "Fizz"

case Buzz()     => "Buzz"

case _          => value.toString

} 

}

How could we go about testing this functionality?  We could do so simple unit testing, but we have business people that really want to make sure that we get this right and they will not be able to follow our unit tests without a bit of help on our part.

This is when a specification framework using a Cucumber or Gherkin style of syntax comes into play.  As DHH pointed out in "Is TDD Dead" part IV, Cucumber is really a pain to use and as such we really need to think about the cost of using this for our testing.

In this made up case we are going to say that the business is ready and willing to fully learn and use the the Given / When / Then style and thinking that Cucumber will force upon them.  In reality I have worked with business partners in which this was the case, others that thought this was the case and then stop doing it, and still others that wanted nothing to do with this "IT thing".  If you are going to do testing in this way, know what you are getting yourself into.

Given all that, let us put together our tests working closely with our business partners.

Note, I am new to Scala, as such I need to look at this gist form Kingsley Davies in order to do the kata.

	name := "Fizz Buzz"
	version := "1.0"
	scalaVersion := "2.11.1"
	libraryDependencies ++= Seq(
	"org.specs2" %% "specs2" % "2.3.12" % "test"
	)
	scalacOptions in Test ++= Seq("-Yrangepos")
	resolvers ++= Seq("snapshots", "releases").map(Resolver.sonatypeRepo)

view raw build.sbt hosted with ❤ by GitHub

	package com.kata.spec2
	import org.specs2._
	import matcher.DataTables
	object FizzBuzzer {
	class FizzBuzzExtractor[T](f: T => Boolean) {
	def unapply(x: T) = f(x)
	}
	val FizzBuzz = new FizzBuzzExtractor[Int](_ % 15 == 0)
	val Fizz = new FizzBuzzExtractor[Int](_ % 3 == 0)
	val Buzz = new FizzBuzzExtractor[Int](_ % 5 == 0)
	def eval(value: Int) = value match {
	case FizzBuzz() => "FizzBuzz"
	case Fizz() => "Fizz"
	case Buzz() => "Buzz"
	case _ => value.toString
	}
	}
	class FizzBuzzSpec extends Specification with DataTables { def is = s2"""
	the following truth $table gives inputs and expected results for FizzBuzz
	"""
	def table =
	"input" | "result" |>
	2 ! "2" |
	3 ! "Fizz" |
	5 ! "Buzz" |
	15 ! "FizzBuzz" |
	33 ! "Fizz" |
	55 ! "Buzz" |
	150 ! "FizzBuzz" |
	151 ! "151" | { (a, b) => { FizzBuzzer.eval(a) must_== b } }
	}

view raw FizzBuzzSpec2.scala hosted with ❤ by GitHub

We see that for this kind of functionality the team as a whole felt that a truth table would be the best way to verify functionality, as such we set up the following truth table in spec2.

def table =



"input" | "result"     |>

2      !   "2"        |  

3      !   "Fizz"     |  

5      !   "Buzz"     | 

15     !   "FizzBuzz" |  

33     !   "Fizz"     |  

55     !   "Buzz"     |

150    !   "FizzBuzz" |

151    !   "151"      | 

We go ahead and run the input and expected results through our Scala function using the following.

{ (a, b) => { FizzBuzzer.eval(a) must_== b } }

In this case, in our lambda we are binding a to our input and b to our expected result.  Then we go a head and assert using the must assertion that value which comes out of our FizzBuzzer does indeed equal our excepted result.



The nice thing about setting up a continuous testing environment with sbt using ~test is that you can implement the functionality in a BDD style.



As we see in the example above if we have a failing condition in our truth table we are notified which row in the table is not true (in this case 151).  We are then able to write enough code to pass the failing row and move on.

Act II - SpecFlow

Say we have the following basic FizzBuzz functionality in C#.

public class FizzBuzzer

{

public string Translate(int value)

{

var result = string.Empty;

if (value%3 == 0) result += "Fizz";

if (value%5 == 0) result += "Buzz";



return string.IsNullOrEmpty(result) ? value.ToString() : result;

}

}

Like in Act I our business partners wants to get involved in the verification and validation of this functionality.  As such they are willing and ready to learn the Cucumber syntax and thinking so that the whole team can be on the same page.

Given the nature of the functionality we are developing the team feels that a truth table will be the best way to test it.  As such the following truth table is come up with as a team.

 Examples: 

 | value | expected |

 | 1     | 1        |

 | 2     | 2        |

 | 3     | Fizz     |

 | 4     | 4        |

 | 5     | Buzz     |

 | 6     | Fizz     |

 | 7     | 7        |

 | 8     | 8        |

 | 9     | Fizz     |

 | 10    | Buzz     |

 | 11    | 11       |

 | 12    | Fizz     |

 | 13    | 13       |

 | 14    | 14       |

 | 15    | FizzBuzz |

 | 30    | FizzBuzz |

 | 45    | FizzBuzz |

In it we have the input and the expected output of the functionality.  We use SpecFlow to come up with the following.

	Feature: FizzBuzz
	As a Software Craftsman
	I want to do the FizzBuzz kata
	So that I can get better at my craft
	Scenario Outline: Validate the functionality of FizzBuzz using the following
	Given the follow input <value>
	When FizzBuzz is invoked with the input <value>
	Then the actual result will equal the <expected>
	Examples:
	| value | expected |
	| 1 | 1 |
	| 2 | 2 |
	| 3 | Fizz |
	| 4 | 4 |
	| 5 | Buzz |
	| 6 | Fizz |
	| 7 | 7 |
	| 8 | 8 |
	| 9 | Fizz |
	| 10 | Buzz |
	| 11 | 11 |
	| 12 | Fizz |
	| 13 | 13 |
	| 14 | 14 |
	| 15 | FizzBuzz |
	| 30 | FizzBuzz |
	| 45 | FizzBuzz |

view raw FizzBuzz.feature hosted with ❤ by GitHub

	using NUnit.Framework;
	using TechTalk.SpecFlow;
	namespace FizzBuzzSpec
	{
	[Binding]
	public class FizzBuzzSteps
	{
	[Given(@"the follow input (.*)")]
	public void GivenTheFollowInput(string value)
	{
	ScenarioContext.Current.Add("target", new FizzBuzzer());
	}
	[When(@"FizzBuzz is invoked with the input (.*)")]
	public void WhenFizzBuzzIsInvokedWithTheInput(int value)
	{
	var target = ScenarioContext.Current.Get<FizzBuzzer>("target");
	var actual = target.Translate(value);
	ScenarioContext.Current.Add("value", actual);
	}
	[Then(@"the actual result will equal the (.*)")]
	public void ThenTheActualResultWillEqualThe(string expected)
	{
	var actual = ScenarioContext.Current.Get<string>("value");
	Assert.That(actual, Is.EqualTo(expected));
	}
	}
	public class FizzBuzzer
	{
	public string Translate(int value)
	{
	var result = string.Empty;
	if (value%3 == 0) result += "Fizz";
	if (value%5 == 0) result += "Buzz";
	return string.IsNullOrEmpty(result) ? value.ToString() : result;
	}
	}
	}

view raw FizzBuzzSpec.cs hosted with ❤ by GitHub

In our given we using the following line to place in the input into memory to be used in the when step.

ScenarioContext.Current.Add("target", new FizzBuzzer());

In our when we are then able to read back out the input from the given step and pass it to our FizzBuzz functionality giving use the actual result which we place into memory to be used in the then step.

var target = ScenarioContext.Current.Get<FizzBuzzer>("target");



var actual = target.Translate(value);



ScenarioContext.Current.Add("value", actual);

Last in our then step we take the actual result and compare it to the expected result, using NUnit to assert that are in fact equal to each other.

var actual = ScenarioContext.Current.Get<string>("value");



Assert.That(actual, Is.EqualTo(expected));

The nice thing about SpecFlow is that you generate the C# coded spec directly from the Cucumber feature file.  The other really nice thing is that it works with NCrunch for continuous testing!

We see below that if we have row not passing in our truth table we get a detail error message stated what is the following row and why.



Once we write just enough code to get back to green we can continue along refactoring and write our next failing testing repeating the red / green / refactor loop.

Act III - Now what

We see that if we, the development team and our business partners, are willing to pay the cost associated with Cucumber we can as a team get everyone on the same page with the same language and understanding of requirements.  Now what?  If you are looking for more information about BDD and SpecFlow check out a colleague of mine, Richard Yu's presentation called "50 Shades of BDD" (he does not know that I am adding him to this blog post, so I hope he does not mind).

Leap Year in the Key of C#, F#, Haskell, and Scala

"Shall be their father's bail, and bane to those"
-- Shakespeare, Henry VI Part 2
Act V, Scene I, Line 120

Leap Year

Leap year, one of the time based banes of a programmers' existence.

The rules around if a year is a leap year are simple enough.

if year is divisible by 400 then

   is_leap_year

else if year is divisible by 100 then

   not_leap_year

else if year is divisible by 4 then

   is_leap_year

else

   not_leap_year

Most languages do have a built in check for Leap Years, but the point of the kata is to implement your own in order to learn something.  With that in mind let's look at few ways to solve this problem and see what we can learn.

C#

Note, if we want to do it the most efficient way possible we could just use the built in DateTime.IsLeapYear, but the point of the kata is to implement the algorithm yourself in order to learn something.

	using NUnit.Framework;
	namespace LeapYear
	{
	public class LeapYearChecker
	{
	public bool IsLeapYear(int year)
	{
	if (year % 400 == 0) return true;
	if (year % 100 == 0) return false;
	if (year % 4 == 0) return true;
	return false;
	// or simply
	//return DateTime.IsLeapYear(year);
	}
	}
	[TestFixture]
	public class LeapYearCheckerTest
	{
	private LeapYearChecker leapYearChecker;
	[TestFixtureSetUp]
	public void BeforeEachTest()
	{
	leapYearChecker = new LeapYearChecker();
	}
	[TestCase(2000, Result = true)]
	[TestCase(4, Result = true)]
	[TestCase(1, Result = false)]
	[TestCase(64, Result = true)]
	[TestCase(1900, Result = false)]
	[TestCase(100, Result = false)]
	public bool IsGivenYearALeapYear(int year)
	{
	return leapYearChecker.IsLeapYear(year);
	}
	}
	}

view raw LeapYearChecker.cs hosted with ❤ by GitHub

Here is an example using C# with NUnit for the testing.

We see that we can just implement the algorithm as a series of conditional statements.  We also see that with NUnit we can use the TestCase attribute which allows us to just give the input along with the expected result.  This kind of testing I feel leads to more readable test cases which focus on the data and not the steps needed.  Having a test case which focuses on the data is the right approach in my opinion for testing a pure calculation based function like this.

Haskell

Note, this looks a lot like the code on Rosetta Code for the TDD Haskell implementation of Leap Year, because it is the same code that I contributed there for that entry.

	import Test.HUnit
	isLeapYear::Int->Bool
	isLeapYear y
	| mod y 400 == 0 = True
	| mod y 100 == 0 = False
	| mod y 4 == 0 = True
	| otherwise = False
	tests = TestList[TestCase $ assertEqual "4 is a leap year" True $ isLeapYear 4
	,TestCase $ assertEqual "1 is not a leap year" False $ isLeapYear 1
	,TestCase $ assertEqual "64 is a leap year" True $ isLeapYear 64
	,TestCase $ assertEqual "2000 is a leap year" True $ isLeapYear 2000
	,TestCase $ assertEqual "1900 is not a leap year" False $ isLeapYear 1900]

view raw leap_year.hs hosted with ❤ by GitHub

This Haskell solution uses pattern matching.  If you are new to pattern matching you can think of it as a switch statement.

With this style of pattern matching in Haskell we would get an error message if we call the function without a value which was covered, but the otherwise keyword at the end covers everything else so getting an exception would not happen.  Comparing this to the C# solution with a series of conditional statements the Haskell caller knows that every input is covered by the function whilst the C# call will have to hope that all input cases are covered.

Scala

Note, this is my third Scala kata ever, as such I need to look at this gist form Kingsley Davies in order to do the kata.

	name := "LeapYear"
	version := "1.0"
	scalaVersion := "2.11.1"
	libraryDependencies += "org.scalatest" %% "scalatest" % "2.1.7" % "test"

view raw build.sbt hosted with ❤ by GitHub

	package com.bigcompany.utils.test
	import org.scalatest._
	object LeapYear {
	def isLeapYear(year: Int) = (year % 4, year % 100, year % 400) match {
	case (_, _, 0) => true
	case (_, 0, _) => false
	case (0, _, _) => true
	case _ => false
	}
	}
	class LeapYearSpec extends FunSpec {
	describe("Validate Leap Year calculation") {
	it("should return true for year 4") {
	assert(LeapYear.isLeapYear(4) == true)
	}
	it("should return true for year 1") {
	assert(LeapYear.isLeapYear(1) == false)
	}
	it("should return true for year 64") {
	assert(LeapYear.isLeapYear(64) == true)
	}
	it("should return true for year 2000") {
	assert(LeapYear.isLeapYear(2000) == true)
	}
	it("should return true for year 1900") {
	assert(LeapYear.isLeapYear(1900) == false)
	}
	}
	}

view raw LeapYear.scala hosted with ❤ by GitHub

With the Scala solution we are again using pattern matching, but this time we are doing all of the testing in a tuple which we are matching against (note, we could have done this in the Haskell and below in the F# solution).  Using this style of pattern matching we are matching against a pattern but in this case I feel that the algorithm is hidden, as such for this function I do not think this is an improvement over the Haskell solution above (note, I did this on purpose to try out this approach with this solution to see what it would look like).

F#

	namespace LeapYearChecker
	open Xunit
	open FsUnit.Xunit
	module ``leap year kata`` =
	let (|DivisibleBy|_|) by x = if x % by = 0 then Some DivisibleBy else None
	let isLeapYear (year) =
	match year with
	| DivisibleBy 400 -> true
	| DivisibleBy 100 -> false
	| DivisibleBy 4 -> true
	| _ -> false
	[<Fact>]
	let ``Given 4 it will return true`` () =
	isLeapYear 4 |> should be True
	[<Fact>]
	let ``Given 1 it will return false`` () =
	isLeapYear 1 |> should be False
	[<Fact>]
	let ``Given 64 it will return true`` () =
	isLeapYear 64 |> should be True
	[<Fact>]
	let ``Given 2000 it will return true`` () =
	isLeapYear 2000 |> should be True
	[<Fact>]
	let ``Given 1900 it will return false`` () =
	isLeapYear 1900 |> should be False

view raw LeapYearChecker.fs hosted with ❤ by GitHub

This last solution using F# uses Active Pattern Matching.  Personally I find the Active Pattern Matching with a Maybe Monad very readable.  We can see that the algorithm states if the year is divisible by 400 we have leap year, but if it is divisible by 100 we do not.

Note, you can do this pattern in Scala and Haskell.

Conclusion

We see above 4 somewhat different ways to find if a year is a Leap Year.  All these approaches used unit testing frameworks along with TDD.  None of the solutions are the way to do it, but each look at the problem from a different point-of-view.

Advance Unit Testing with NUnit OR How to do Property Based Testing in C# Without Using F#

"You do advance your cunning more and more."
-- Shakespeare, A Midsummer's Night Dream
Act III, Scene II, Line 128

Intro

Sir you got F# in my C#.  I do not wish to add F# to my Solution just to be able to test my code.  Is there a way to get similar functionality to FsCheck without using F#?

Glad you asked.  I believe NUnit can assist us here.

FizzBuzz with NUnit

	using System;
	using System.Collections.Generic;
	using System.Globalization;
	using System.Linq;
	using System.Text;
	using System.Threading.Tasks;
	using NUnit.Framework;
	using NUnit.Framework.Constraints;
	namespace NUnit
	{
	[TestFixture]
	public class CharacterizationTests
	{
	public string FizzBuzz(int value)
	{
	if(value < 0) throw new ArgumentException("Value must be positive.");
	var ret = string.Empty;
	if (value%3 == 0) ret += "Fizz";
	if (value%5 == 0) ret += "Buzz";
	return string.IsNullOrEmpty(ret) ? value.ToString() : ret;
	}
	[Test, ExpectedException(typeof (ArgumentException))]
	public void Negative_Values_Throws_An_ArgumentException()
	{
	FizzBuzz(-1);
	}
	[Test, Sequential]
	public void In_Order_Test_Fizz_Buzz(
	[Values( 1, 2, 3, 4, 5, 15)] int value,
	[Values("1", "2", "Fizz", "4", "Buzz", "FizzBuzz")] string expected)
	{
	Assert.That(FizzBuzz(value), Is.EqualTo(expected));
	}
	[Test]
	public void Generate_A_Range_Of_Fizz_Data(
	[Range(3, 300, 3)] int value)
	{
	var removeBuzz = (value % 5 == 0) ? 3 : value;
	Assert.That(FizzBuzz(removeBuzz), Is.EqualTo("Fizz"));
	}
	[Test, ExpectedException(typeof (ArgumentException))]
	public void Generate_A_Range_Of_Invali_Fizz_Data(
	[Range(-1000, -1, 1)] int value)
	{
	FizzBuzz(value);
	Assert.Fail("Should have thrown an exception");
	}
	// Random does not work with NCrunch unless NUnit is set to UseStaticAnalysis see also: http://forum.ncrunch.net/yaf_postst433_NUnit--Random--supported.aspx
	[Test]
	public void Generate_Buzz_Data(
	[Random(1, 10000, 100)] int value)
	{
	var removeFizz = (value % 3 == 0) ? 5 : value * 5;
	Assert.That(FizzBuzz(removeFizz), Is.EqualTo("Buzz"));
	}
	[TestCase( 0, Result = "FizzBuzz")]
	[TestCase( 1, Result = "1")]
	[TestCase( 2, Result = "2")]
	[TestCase( 3, Result = "Fizz")]
	[TestCase( 4, Result = "4")]
	[TestCase( 5, Result = "Buzz")]
	[TestCase( 6, Result = "Fizz")]
	[TestCase(10, Result = "Buzz")]
	[TestCase(15, Result = "FizzBuzz")]
	[TestCase(45, Result = "FizzBuzz")]
	[TestCase(-1, ExpectedException = typeof(ArgumentException))]
	public string FizzBuzz_Test_Cases_With_Expected_Results(int value)
	{
	return FizzBuzz(value);
	}
	[TestCase( 0, "FizzBuzz")]
	[TestCase( 1, "1")]
	[TestCase( 2, "2")]
	[TestCase( 3, "Fizz")]
	[TestCase( 4, "4")]
	[TestCase( 5, "Buzz")]
	[TestCase( 6, "Fizz")]
	[TestCase(10, "Buzz")]
	[TestCase(15, "FizzBuzz")]
	[TestCase(45, "FizzBuzz")]
	[TestCase(-1, "error", ExpectedException = typeof(ArgumentException))]
	public void FizzBuzz_Test_Cases(int value, string expected)
	{
	Assert.That(FizzBuzz(value), Is.EqualTo(expected));
	}
	[Datapoints] public int[] Values = new[] {-1, 0, 2, 3, 4, 5, 9, 15, 25, 45};
	[Theory]
	public void Numbers_Divisible_By_15_Will_Return_FizzBuzz(int value)
	{
	Assume.That(value % 15 == 0);
	var actual = FizzBuzz(value);
	Assert.That(actual, Is.Not.Null);
	Assert.That(actual, Is.EqualTo("FizzBuzz"));
	}
	[Test]
	public void Values_Not_Divisible_By_3_Or_5_Return_ToString_Value()
	{
	Assert.That(FizzBuzz(7), new EqualConstraint("7"));
	var sixteen = 16;
	Assert.That(FizzBuzz(sixteen), new EqualConstraint(sixteen.ToString(CultureInfo.InvariantCulture)));
	}
	public static object[] FizzBuzzTestData =
	{
	new object[] { 1, "1"},
	new object[] { 2, "2"},
	new object[] { 3, "Fizz"},
	new object[] { 9, "Fizz"},
	new object[] { 5, "Buzz"},
	new object[] {10, "Buzz"},
	new object[] { 0, "FizzBuzz"},
	new object[] {15, "FizzBuzz"}
	};
	[Test, TestCaseSource("FizzBuzzTestData")]
	public void FizzBuzz_Test_Data(int value, string expected)
	{
	Assert.That(FizzBuzz(value), Is.EqualTo(expected));
	}
	public class FizzBuzzTestCaseDataFactory
	{
	public static IEnumerable<TestCaseData> TestCaseData
	{
	get
	{
	yield return new TestCaseData(1).Returns("1");
	yield return new TestCaseData(2).Returns("2");
	yield return new TestCaseData(3).Returns("Fizz");
	yield return new TestCaseData(33).Returns("Fizz");
	yield return new TestCaseData(5).Returns("Buzz");
	yield return new TestCaseData(55).Returns("Buzz");
	yield return new TestCaseData(15).Returns("FizzBuzz");
	yield return new TestCaseData(165).Returns("FizzBuzz");
	yield return new TestCaseData(-1).Throws(typeof (ArgumentException));
	yield return new TestCaseData(-11).Throws(typeof (ArgumentException));
	}
	}
	}
	[Test, TestCaseSource(typeof(FizzBuzzTestCaseDataFactory), "TestCaseData")]
	public string Data_Factory_Test_Case(int value)
	{
	return FizzBuzz(value);
	}
	[Test,
	ExpectedException(typeof (ArgumentException), ExpectedMessage = "Value must be positive.",
	MatchType = MessageMatch.Exact)]
	public void Given_A_Negative_1_It_Will_Throw_An_ArgumentException_With_The_Message_Of_ValueMustBePostive()
	{
	FizzBuzz(-1);
	Assert.Fail("Should have thrown an execpetion");
	}
	}
	}

view raw NunitCharacterizationTestsUsingFizzBuzz.cs hosted with ❤ by GitHub

First thing we see is a very uninteresting version of FizzBuzz.

public string FizzBuzz(int value)

{

if(value < 0) throw new ArgumentException("Value must be positive.");



var ret = string.Empty;

if (value%3 == 0) ret += "Fizz";

if (value%5 == 0) ret += "Buzz";

return string.IsNullOrEmpty(ret) ? value.ToString() : ret;

}

Nothing really interesting going on here other than using a variable to preserve state so that we do not have to check for 15 or have more than one return statement.

Excepted Exception

        [Test, ExpectedException(typeof (ArgumentException))]

public void Negative_Values_Throws_An_ArgumentException()

{

FizzBuzz(-1);

}

We see that we can declare that an exception will be thrown and thus have test coverage for our exception cases.

If we want to make sure that every detail of our exception matches what we think it should be NUnit offers a more verbose check too.

        [Test,

         ExpectedException(typeof (ArgumentException), ExpectedMessage = "Value must be positive.",

             MatchType = MessageMatch.Exact)]

public void Given_A_Negative_1_It_Will_Throw_An_ArgumentException_With_The_Message_Of_ValueMustBePostive()

{

FizzBuzz(-1);

Assert.Fail("Should have thrown an execpetion");

}

Ranges of Values in One Test Case

        [Test]

public void Generate_A_Range_Of_Fizz_Data(

            [Range(3, 300, 3)] int value)

{

var removeBuzz = (value % 5 == 0) ? 3 : value;



Assert.That(FizzBuzz(removeBuzz), Is.EqualTo("Fizz"));

} 

If we want to we can define a range of values to check.  With the example above we see that we are checking the values: 3, 6, 9, ..., 300 are not divisible by 5 (thus, 15, 30, 45, ... will not be checked).  This data will be used to verify that Fizz is returned for each of these test cases.  We can do a similar thing with Excepted Exceptions.

        [Test, ExpectedException(typeof (ArgumentException))]

public void Generate_A_Range_Of_Invali_Fizz_Data(

            [Range(-1000, -1, 1)] int value)

{

FizzBuzz(value);

Assert.Fail("Should have thrown an exception");

}

Test Case

We can set up a test case using the TestCase attribute.

        [TestCase( 0, "FizzBuzz")]

        [TestCase( 1, "1")]

        [TestCase( 2, "2")]

        [TestCase( 3, "Fizz")]

        [TestCase( 4, "4")]

        [TestCase( 5, "Buzz")]

        [TestCase( 6, "Fizz")]

        [TestCase(10, "Buzz")]

        [TestCase(15, "FizzBuzz")]

        [TestCase(45, "FizzBuzz")]

        [TestCase(-1, "error", ExpectedException = typeof(ArgumentException))]

public void FizzBuzz_Test_Cases(int value, string expected)

{

Assert.That(FizzBuzz(value), Is.EqualTo(expected));

}

This allows us to reuse the boilerplate test case setup while allowing us to provide the test data.  In this case we are providing both the value to test and the expected result.

We can be more explicit with the result and use the Result property of the TestCase attribute.

        [TestCase( 0, Result = "FizzBuzz")]

        [TestCase( 1, Result = "1")]

        [TestCase( 2, Result = "2")]

        [TestCase( 3, Result = "Fizz")]

        [TestCase( 4, Result = "4")]

        [TestCase( 5, Result = "Buzz")]

        [TestCase( 6, Result = "Fizz")]

        [TestCase(10, Result = "Buzz")]

        [TestCase(15, Result = "FizzBuzz")]

        [TestCase(45, Result = "FizzBuzz")]

        [TestCase(-1, ExpectedException = typeof(ArgumentException))]

public string FizzBuzz_Test_Cases_With_Expected_Results(int value)

{

return FizzBuzz(value);

}

Note, when testing this way you do not call Assert but instead return the value (note also the return type of the test function is a string in this case and not a void).  NUnit will assert the result of method for you!

Generating Test Data

Property Based Testing is a very a powerful idea which decouples the behavior which you are testing from the generating of test data.  I believe an example would be good about now.

// Random does not work with NCrunch unless NUnit is set to UseStaticAnalysis

        [Test]

public void Generate_Buzz_Data(

            [Random(1, 10000, 100)] int value)

{

var removeFizz = (value % 3 == 0) ? 5 : value * 5;



Assert.That(FizzBuzz(removeFizz), Is.EqualTo("Buzz"));

}

We see above the use of the Random attribute.  Random will generate a value between 1 and 1000 (the first two arguments that we pass it), in this case, this will be done 100 times!  (100 is the third argument that we pass it.)  We see also that for this test we want to verify the Buzz functionality, so we reshape the data in such a way that we always get a value which should produce Buzz.  This kind of testing is a great way to find edge cases.

We see a comment above the test case which means something is going wrong.  In this case the comment is telling use that when we use Random with NCrunch we need to change the Configuration to use UseStaticAnalysis for NUnit.  You can read all about it here.  If you do not change this setting you'll get the following error message.

"This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."

NUnit offers other ways to generate data, one of which is the Datapoints / Theory combo.

        [Datapoints] public int[] Values = new[] {-1, 0, 2, 3, 4, 5, 9, 15, 25, 45};



        [Theory]

public void Numbers_Divisible_By_15_Will_Return_FizzBuzz(int value)

{

Assume.That(value % 15 == 0);



var actual = FizzBuzz(value);



Assert.That(actual, Is.Not.Null);

Assert.That(actual, Is.EqualTo("FizzBuzz"));

}

The way that Theory works is that it will use the values from the field marked as Datapoints.  In the case above we are restricting the values just to what is divisible by 15, thus we are testing the FizzBuzz functionality.

We can also set up an array of arrays which contain test case values using the TestCaseSource attribute.

public static object[] FizzBuzzTestData =

{

new object[] { 1, "1"},

new object[] { 2, "2"},

new object[] { 3, "Fizz"},

new object[] { 9, "Fizz"},

new object[] { 5, "Buzz"},

new object[] {10, "Buzz"},

new object[] { 0, "FizzBuzz"},

new object[] {15, "FizzBuzz"}

};



        [Test, TestCaseSource("FizzBuzzTestData")]

public void FizzBuzz_Test_Data(int value, string expected)

{

Assert.That(FizzBuzz(value), Is.EqualTo(expected));

}

We see with the example above that we are defining both the value and excepted result which are passed into the test.

We can take this a step forward and define an actual test generator class.

public class FizzBuzzTestCaseDataFactory

{

public static IEnumerable<TestCaseData> TestCaseData

{

get

{

yield return new TestCaseData(1).Returns("1");

yield return new TestCaseData(2).Returns("2");

yield return new TestCaseData(3).Returns("Fizz");

yield return new TestCaseData(33).Returns("Fizz");

yield return new TestCaseData(5).Returns("Buzz");

yield return new TestCaseData(55).Returns("Buzz");

yield return new TestCaseData(15).Returns("FizzBuzz");

yield return new TestCaseData(165).Returns("FizzBuzz");

yield return new TestCaseData(-1).Throws(typeof (ArgumentException));

yield return new TestCaseData(-11).Throws(typeof (ArgumentException));

}

}

}



        [Test, TestCaseSource(typeof(FizzBuzzTestCaseDataFactory), "TestCaseData")]

public string Data_Factory_Test_Case(int value)

{

return FizzBuzz(value);

}

We see that by using the TestCaseSource attribute and telling it the typeof the test generator class and the name of the method for generating test case data, NUnit will call the method and verify our functionality for us!

We also see that the TestCaseData class allows us to specify the results.  In my opinion this allows for very high levels of readability.

I know what you might be thinking, this test data generating is fine but why use this over a for loop?  Well the for loop would test the same functionality, but it would not show up as different test cases to the test runner (unless you do so real hacking), while the NUnit test data generators would.  With the NUnit test data generators, if one of the values fail the test case you'll see the offending value instead of just seeing that the test case with a for loop broke.

This is what the values for Generate_Buzz_Data (the test using the Random attribute) actually look like to the test runner.

Generate_Buzz_Data

NUnit.CharacterizationTests.Generate_Buzz_Data(5224):

NUnit.CharacterizationTests.Generate_Buzz_Data(8147):

NUnit.CharacterizationTests.Generate_Buzz_Data(8619):

...

As you can see it would be very easy to see why a value would make a test fail using this, the same could not be said for the loop.

Look Mom, No Quickcheck

There you have it advance unit testing with NUnit.  Use NUnit's different test data generators we were able to do Property Based Testing without using quickcheck.

I do want to make a quick call out to Luke Wickstead's excellent posts on NUnit.  Reading this post allowed me to figure out how the TestCaseSource really worked.