Sunday, June 29, 2014

Proving Traits

"A play there is, my lord, some ten words long,
Which is as ‘ brief ’ as I have known a play.
But by ten words, my lord, it is too long
-- Shakespeare, Midsummer Night's Dream

Martin Heidegger in his paper, The Origin of the Work of Art, gives us three different ways of defining thingness.

  1. bearer of traits and properties
  2. sense perceptions
  3. formed stuff

Traits is an interesting term in computer science, it is a collection of behaviors and their implementation which can be used to extend an object.  Most languages do not directly support the creation of stand-alone traits but traits are the essence of what makes up a programs value.

I am going to make a very bold statement here, I am going to say that BDD (Behavior Driven Development) is really about proving the traits of software.  Using the definition of traits given above we see that a trait is a collection of behaviors and how they have been implemented.  This sounds a lot like what we are proving out when we do BDD.

In BDD what are we doing?  We are proving that behavior X as it is implemented by code S does the following when given state G with stimulus W yielding state T.  In other words we are showing Given the following conditions, When we do the following against X in this Source code, Then the following result will occur.  This sounds a lot like proving the traits of the software to me.

"Words, words, words."
-- Shakespeare, Hamlet

Let us look at a example.

In the example above we see that a Starfleet Captain needs to be able to communicate with the FizzBuzzons and thus needs to have a translator.

She is given the following C# source code to preform the needed behavior of translating.

public string Translate(int value)
var result = string.Empty;
if (value % 3 == 0) result += "Fizz";
if (value % 5 == 0) result += "Buzz";
return string.IsNullOrEmpty(result) ? value.ToString() : result;

To prove to her that it works the engineers and communication specialist came up with the following specifications listing what happens in a Given state When the an input value is translated Then the value will have a certain output.

Scenario: 2 must translate to the string 2
Given a FizzBuzzer
And the value of '2'
When translate is invoked
Then the result should be '2'
Scenario: 3 must translate to the string Fizz
Given a FizzBuzzer
And the value of '3'
When translate is invoked
Then the result should be 'Fizz'
Scenario: 5 must translate to the string Buzz
Given a FizzBuzzer
And the value of '5'
When translate is invoked
Then the result should be 'Buzz'
Scenario: 15 must translate to the string FizzBuzz
Given a FizzBuzzer
And the value of '15'
When translate is invoked
Then the result should be 'FizzBuzz'

By testing around these specifications we can prove to our captain that the translator does hold the desired traits.

Does this slight different definition around BDD change anything?  No, but I hope it does widen the thought around it.

"If we shadows have offended
Think but this, and all is mended:
That you have but slumbered here
While these visions did appear.
-- Shakespeare, Midsummer Night's Dream

Saturday, June 21, 2014

Specs in Scala and C#

"That I have uttered. Bring me to the test,
And I the matter will re-word, which madness
-- Shakespeare, Hamlet
Act III, Scene IV, Lines 143-144


Places, places, the play is about to begin.

Tonight we shall look at a tale of two specification frameworks, Scala's spec2 and .Net's SpecFlow.

Act I - spec2

Say we have the following basic FizzBuzz functionality in Scala.

object FizzBuzzer {
class FizzBuzzExtractor[T](f: T => Boolean) {
def unapply(x: T) = f(x)
val FizzBuzz = new FizzBuzzExtractor[Int](_ % 15 == 0)
val Fizz = new FizzBuzzExtractor[Int](_ % 3 == 0)
val Buzz = new FizzBuzzExtractor[Int](_ % 5 == 0)
def eval(value: Int) = value match {
case FizzBuzz() => "FizzBuzz"
case Fizz() => "Fizz"
case Buzz() => "Buzz"
case _ => value.toString

How could we go about testing this functionality?  We could do so simple unit testing, but we have business people that really want to make sure that we get this right and they will not be able to follow our unit tests without a bit of help on our part.

This is when a specification framework using a Cucumber or Gherkin style of syntax comes into play.  As DHH pointed out in "Is TDD Dead" part IV, Cucumber is really a pain to use and as such we really need to think about the cost of using this for our testing.

In this made up case we are going to say that the business is ready and willing to fully learn and use the the Given / When / Then style and thinking that Cucumber will force upon them.  In reality I have worked with business partners in which this was the case, others that thought this was the case and then stop doing it, and still others that wanted nothing to do with this "IT thing".  If you are going to do testing in this way, know what you are getting yourself into.

Given all that, let us put together our tests working closely with our business partners.

Note, I am new to Scala, as such I need to look at this gist form Kingsley Davies in order to do the kata.

We see that for this kind of functionality the team as a whole felt that a truth table would be the best way to verify functionality, as such we set up the following truth table in spec2.

def table =
"input" | "result" |>
2 ! "2" |
3 ! "Fizz" |
5 ! "Buzz" |
15 ! "FizzBuzz" |
33 ! "Fizz" |
55 ! "Buzz" |
150 ! "FizzBuzz" |
151 ! "151" |

We go ahead and run the input and expected results through our Scala function using the following.

{ (a, b) => { FizzBuzzer.eval(a) must_== b } }

In this case, in our lambda we are binding a to our input and b to our expected result.  Then we go a head and assert using the must assertion that value which comes out of our FizzBuzzer does indeed equal our excepted result.

The nice thing about setting up a continuous testing environment with sbt using ~test is that you can implement the functionality in a BDD style.

As we see in the example above if we have a failing condition in our truth table we are notified which row in the table is not true (in this case 151).  We are then able to write enough code to pass the failing row and move on.

Act II - SpecFlow

Say we have the following basic FizzBuzz functionality in C#.

public class FizzBuzzer
public string Translate(int value)
var result = string.Empty;
if (value%3 == 0) result += "Fizz";
if (value%5 == 0) result += "Buzz";
return string.IsNullOrEmpty(result) ? value.ToString() : result;

Like in Act I our business partners wants to get involved in the verification and validation of this functionality.  As such they are willing and ready to learn the Cucumber syntax and thinking so that the whole team can be on the same page.

Given the nature of the functionality we are developing the team feels that a truth table will be the best way to test it.  As such the following truth table is come up with as a team.
| value | expected |
| 1 | 1 |
| 2 | 2 |
| 3 | Fizz |
| 4 | 4 |
| 5 | Buzz |
| 6 | Fizz |
| 7 | 7 |
| 8 | 8 |
| 9 | Fizz |
| 10 | Buzz |
| 11 | 11 |
| 12 | Fizz |
| 13 | 13 |
| 14 | 14 |
| 15 | FizzBuzz |
| 30 | FizzBuzz |
| 45 | FizzBuzz |

In it we have the input and the expected output of the functionality.  We use SpecFlow to come up with the following.

In our given we using the following line to place in the input into memory to be used in the when step.

ScenarioContext.Current.Add("target", new FizzBuzzer());

In our when we are then able to read back out the input from the given step and pass it to our FizzBuzz functionality giving use the actual result which we place into memory to be used in the then step.

var target = ScenarioContext.Current.Get<FizzBuzzer>("target");
var actual = target.Translate(value);
ScenarioContext.Current.Add("value", actual);

Last in our then step we take the actual result and compare it to the expected result, using NUnit to assert that are in fact equal to each other.

var actual = ScenarioContext.Current.Get<string>("value");
Assert.That(actual, Is.EqualTo(expected));

The nice thing about SpecFlow is that you generate the C# coded spec directly from the Cucumber feature file.  The other really nice thing is that it works with NCrunch for continuous testing!

We see below that if we have row not passing in our truth table we get a detail error message stated what is the following row and why.

Once we write just enough code to get back to green we can continue along refactoring and write our next failing testing repeating the red / green / refactor loop.

Act III - Now what

We see that if we, the development team and our business partners, are willing to pay the cost associated with Cucumber we can as a team get everyone on the same page with the same language and understanding of requirements.  Now what?  If you are looking for more information about BDD and SpecFlow check out a colleague of mine, Richard Yu's presentation called "50 Shades of BDD" (he does not know that I am adding him to this blog post, so I hope he does not mind).

Saturday, June 7, 2014

Leap Year in the Key of C#, F#, Haskell, and Scala

"Shall be their father's bail, and bane to those"
-- Shakespeare, Henry VI Part 2
Act V, Scene I, Line 120

Leap Year

Leap year, one of the time based banes of a programmers' existence.

The rules around if a year is a leap year are simple enough.

if year is divisible by 400 then
else if year is divisible by 100 then
else if year is divisible by 4 then

Most languages do have a built in check for Leap Years, but the point of the kata is to implement your own in order to learn something.  With that in mind let's look at few ways to solve this problem and see what we can learn.


Note, if we want to do it the most efficient way possible we could just use the built in DateTime.IsLeapYear, but the point of the kata is to implement the algorithm yourself in order to learn something.

Here is an example using C# with NUnit for the testing.

We see that we can just implement the algorithm as a series of conditional statements.  We also see that with NUnit we can use the TestCase attribute which allows us to just give the input along with the expected result.  This kind of testing I feel leads to more readable test cases which focus on the data and not the steps needed.  Having a test case which focuses on the data is the right approach in my opinion for testing a pure calculation based function like this.


Note, this looks a lot like the code on Rosetta Code for the TDD Haskell implementation of Leap Year, because it is the same code that I contributed there for that entry.

This Haskell solution uses pattern matching.  If you are new to pattern matching you can think of it as a switch statement.

With this style of pattern matching in Haskell we would get an error message if we call the function without a value which was covered, but the otherwise keyword at the end covers everything else so getting an exception would not happen.  Comparing this to the C# solution with a series of conditional statements the Haskell caller knows that every input is covered by the function whilst the C# call will have to hope that all input cases are covered.


Note, this is my third Scala kata ever, as such I need to look at this gist form Kingsley Davies in order to do the kata.

With the Scala solution we are again using pattern matching, but this time we are doing all of the testing in a tuple which we are matching against (note, we could have done this in the Haskell and below in the F# solution).  Using this style of pattern matching we are matching against a pattern but in this case I feel that the algorithm is hidden, as such for this function I do not think this is an improvement over the Haskell solution above (note, I did this on purpose to try out this approach with this solution to see what it would look like).


This last solution using F# uses Active Pattern Matching.  Personally I find the Active Pattern Matching with a Maybe Monad very readable.  We can see that the algorithm states if the year is divisible by 400 we have leap year, but if it is divisible by 100 we do not.

Note, you can do this pattern in Scala and Haskell.


We see above 4 somewhat different ways to find if a year is a Leap Year.  All these approaches used unit testing frameworks along with TDD.  None of the solutions are the way to do it, but each look at the problem from a different point-of-view.

Sunday, June 1, 2014

Advance Unit Testing with NUnit OR How to do Property Based Testing in C# Without Using F#

"You do advance your cunning more and more."
-- Shakespeare, A Midsummer's Night Dream
Act III, Scene II, Line 128


Sir you got F# in my C#.  I do not wish to add F# to my Solution just to be able to test my code.  Is there a way to get similar functionality to FsCheck without using F#?

Glad you asked.  I believe NUnit can assist us here.

FizzBuzz with NUnit

First thing we see is a very uninteresting version of FizzBuzz.

public string FizzBuzz(int value)
if(value < 0) throw new ArgumentException("Value must be positive.");
var ret = string.Empty;
if (value%3 == 0) ret += "Fizz";
if (value%5 == 0) ret += "Buzz";
return string.IsNullOrEmpty(ret) ? value.ToString() : ret;

Nothing really interesting going on here other than using a variable to preserve state so that we do not have to check for 15 or have more than one return statement.

Excepted Exception

[Test, ExpectedException(typeof (ArgumentException))]
public void Negative_Values_Throws_An_ArgumentException()

We see that we can declare that an exception will be thrown and thus have test coverage for our exception cases.

If we want to make sure that every detail of our exception matches what we think it should be NUnit offers a more verbose check too.

ExpectedException(typeof (ArgumentException), ExpectedMessage = "Value must be positive.",
MatchType = MessageMatch.Exact)]
public void Given_A_Negative_1_It_Will_Throw_An_ArgumentException_With_The_Message_Of_ValueMustBePostive()
Assert.Fail("Should have thrown an execpetion");

Ranges of Values in One Test Case

public void Generate_A_Range_Of_Fizz_Data(
[Range(3, 300, 3)] int value)
var removeBuzz = (value % 5 == 0) ? 3 : value;
Assert.That(FizzBuzz(removeBuzz), Is.EqualTo("Fizz"));

If we want to we can define a range of values to check.  With the example above we see that we are checking the values: 3, 6, 9, ..., 300 are not divisible by 5 (thus, 15, 30, 45, ... will not be checked).  This data will be used to verify that Fizz is returned for each of these test cases.  We can do a similar thing with Excepted Exceptions.

[Test, ExpectedException(typeof (ArgumentException))]
public void Generate_A_Range_Of_Invali_Fizz_Data(
[Range(-1000, -1, 1)] int value)
Assert.Fail("Should have thrown an exception");

Test Case

We can set up a test case using the TestCase attribute.

[TestCase( 0, "FizzBuzz")]
[TestCase( 1, "1")]
[TestCase( 2, "2")]
[TestCase( 3, "Fizz")]
[TestCase( 4, "4")]
[TestCase( 5, "Buzz")]
[TestCase( 6, "Fizz")]
[TestCase(10, "Buzz")]
[TestCase(15, "FizzBuzz")]
[TestCase(45, "FizzBuzz")]
[TestCase(-1, "error", ExpectedException = typeof(ArgumentException))]
public void FizzBuzz_Test_Cases(int value, string expected)
Assert.That(FizzBuzz(value), Is.EqualTo(expected));

This allows us to reuse the boilerplate test case setup while allowing us to provide the test data.  In this case we are providing both the value to test and the expected result.

We can be more explicit with the result and use the Result property of the TestCase attribute.

[TestCase( 0, Result = "FizzBuzz")]
[TestCase( 1, Result = "1")]
[TestCase( 2, Result = "2")]
[TestCase( 3, Result = "Fizz")]
[TestCase( 4, Result = "4")]
[TestCase( 5, Result = "Buzz")]
[TestCase( 6, Result = "Fizz")]
[TestCase(10, Result = "Buzz")]
[TestCase(15, Result = "FizzBuzz")]
[TestCase(45, Result = "FizzBuzz")]
[TestCase(-1, ExpectedException = typeof(ArgumentException))]
public string FizzBuzz_Test_Cases_With_Expected_Results(int value)
return FizzBuzz(value);

Note, when testing this way you do not call Assert but instead return the value (note also the return type of the test function is a string in this case and not a void).  NUnit will assert the result of method for you!

Generating Test Data

Property Based Testing is a very a powerful idea which decouples the behavior which you are testing from the generating of test data.  I believe an example would be good about now.

// Random does not work with NCrunch unless NUnit is set to UseStaticAnalysis
public void Generate_Buzz_Data(
[Random(1, 10000, 100)] int value)
var removeFizz = (value % 3 == 0) ? 5 : value * 5;
Assert.That(FizzBuzz(removeFizz), Is.EqualTo("Buzz"));

We see above the use of the Random attribute.  Random will generate a value between 1 and 1000 (the first two arguments that we pass it), in this case, this will be done 100 times!  (100 is the third argument that we pass it.)  We see also that for this test we want to verify the Buzz functionality, so we reshape the data in such a way that we always get a value which should produce Buzz.  This kind of testing is a great way to find edge cases.

We see a comment above the test case which means something is going wrong.  In this case the comment is telling use that when we use Random with NCrunch we need to change the Configuration to use UseStaticAnalysis for NUnit.  You can read all about it here.  If you do not change this setting you'll get the following error message.

"This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."

NUnit offers other ways to generate data, one of which is the Datapoints / Theory combo.

[Datapoints] public int[] Values = new[] {-1, 0, 2, 3, 4, 5, 9, 15, 25, 45};
public void Numbers_Divisible_By_15_Will_Return_FizzBuzz(int value)
Assume.That(value % 15 == 0);
var actual = FizzBuzz(value);
Assert.That(actual, Is.Not.Null);
Assert.That(actual, Is.EqualTo("FizzBuzz"));

The way that Theory works is that it will use the values from the field marked as Datapoints.  In the case above we are restricting the values just to what is divisible by 15, thus we are testing the FizzBuzz functionality.

We can also set up an array of arrays which contain test case values using the TestCaseSource attribute.

public static object[] FizzBuzzTestData =
new object[] { 1, "1"},
new object[] { 2, "2"},
new object[] { 3, "Fizz"},
new object[] { 9, "Fizz"},
new object[] { 5, "Buzz"},
new object[] {10, "Buzz"},
new object[] { 0, "FizzBuzz"},
new object[] {15, "FizzBuzz"}
[Test, TestCaseSource("FizzBuzzTestData")]
public void FizzBuzz_Test_Data(int value, string expected)
Assert.That(FizzBuzz(value), Is.EqualTo(expected));

We see with the example above that we are defining both the value and excepted result which are passed into the test.

We can take this a step forward and define an actual test generator class.

public class FizzBuzzTestCaseDataFactory
public static IEnumerable<TestCaseData> TestCaseData
yield return new TestCaseData(1).Returns("1");
yield return new TestCaseData(2).Returns("2");
yield return new TestCaseData(3).Returns("Fizz");
yield return new TestCaseData(33).Returns("Fizz");
yield return new TestCaseData(5).Returns("Buzz");
yield return new TestCaseData(55).Returns("Buzz");
yield return new TestCaseData(15).Returns("FizzBuzz");
yield return new TestCaseData(165).Returns("FizzBuzz");
yield return new TestCaseData(-1).Throws(typeof (ArgumentException));
yield return new TestCaseData(-11).Throws(typeof (ArgumentException));
[Test, TestCaseSource(typeof(FizzBuzzTestCaseDataFactory), "TestCaseData")]
public string Data_Factory_Test_Case(int value)
return FizzBuzz(value);

We see that by using the TestCaseSource attribute and telling it the typeof the test generator class and the name of the method for generating test case data, NUnit will call the method and verify our functionality for us!

We also see that the TestCaseData class allows us to specify the results.  In my opinion this allows for very high levels of readability.

I know what you might be thinking, this test data generating is fine but why use this over a for loop?  Well the for loop would test the same functionality, but it would not show up as different test cases to the test runner (unless you do so real hacking), while the NUnit test data generators would.  With the NUnit test data generators, if one of the values fail the test case you'll see the offending value instead of just seeing that the test case with a for loop broke.

This is what the values for Generate_Buzz_Data (the test using the Random attribute) actually look like to the test runner.






As you can see it would be very easy to see why a value would make a test fail using this, the same could not be said for the loop.

Look Mom, No Quickcheck

There you have it advance unit testing with NUnit.  Use NUnit's different test data generators we were able to do Property Based Testing without using quickcheck.

I do want to make a quick call out to Luke Wickstead's excellent posts on NUnit.  Reading this post allowed me to figure out how the TestCaseSource really worked.