Saturday, July 5, 2014

I Have No Freaking Clue What I Am Doing... OR Saving Throws with Property Testing

"What majesty should be, what duty is,
Why day is day, night night, and time is time,
Were nothing but to waste night, day, and time.
Therefore, since brevity is the soul of wit,
-- Shakespeare, Hamlet
Act II, Scene II, Lines 87-90

I'll admit it, sometimes I have no freaking clue what I am doing.  Much like the internet dog meme, I feel completely out of my league.  You have to start somewhere and no clue is often the first stop on the journey to mastery.

"A journey of a thousand leagues started with what was under one footstep."
-- Tao Te Ching verse 64, translated by Jan J. L. Duyvendak

Property Testing allows one to run their code under test through it's paces.  Let us take a look at using Property Testing against FizzBuzz and see what we can learn.

We see that we have a fairly simple implementation of FizzBuzz.

public string Translate(int value)
var result = string.Empty;
if (value % 3 == 0) result += "Fizz";
if (value % 5 == 0) result += "Buzz";
return string.IsNullOrEmpty(result) ? value.ToString() : result;

For our first test case we use the Test Case property of NUnit to show what the results of the Translate on the FizzBuzzer would be for different inputs.  Note, I am showing the result here after many rounds of Red, Green, Refactor.  I cannot predict the future and as such I had two different test methods for 2 and 3 and ended up refactoring them to the "one" you see below, but I digress.

[TestCase(2, Result = "2")]
[TestCase(3, Result = "Fizz")]
[TestCase(5, Result = "Buzz")]
[TestCase(15, Result = "FizzBuzz")]
public string Given_Value_It_Must_Return_The_Given_Result(int value)
return _fizzBuzzer.Translate(value);

This is great, but what happens for other values?  How do we know if this really works?

public void Given_A_Number_Not_Divisible_By_3_Or_5_It_Must_Return_That_Number()
var value =
from number in Any.OfType<int>()
where number%3 != 0 && number%5 != 0
select number;
Spec.For(value, v => _fizzBuzzer.Translate(v).Equals(v.ToString()))

We use the Property Test above using fscheck to show that any number not divisible by 3 or 5 will return the ToString value of the number.  We do not have to check 15 since 3 * 5 =15 and therefore it is covered by the Fundamental Theory of Arithmetic.

private static Gen<int> DivisibleBy(int divisor)
var divisibleBy =
from number in Any.OfType<int>()
where number % divisor == 0
select number;
return divisibleBy;
[TestCase(3, "Fizz")]
[TestCase(5, "Buzz")]
[TestCase(15, "FizzBuzz")]
public void Given_A_Number_Divisible_By_Divisor_It_Must_Contain_Expected(
int divisor, string expected)
Spec.For(DivisibleBy(divisor), d => _fizzBuzzer.Translate(d).Contains(expected))

Next we test that every number divisible by 3 contains the string "Fizz", likewise we do the same with 5 and "Buzz" and 15 with "FizzBuzz".  We check that they contain the string, so that if we get a value like 45, which is 3 * 15, for our divisible by 3 value we do not have a failing test because we got "FizzBuzz" back instead of just "Fizz", this is a very important thing to think about with Property Testing.  Note also, this is another case were refactoring played a big part in the final result.  I did not start off with a DivisibleBy function; no I found that the generate code for the 3 and 5 looked a lot a liked, so I combined them into the function you now see.

public void Given_A_Number_It_Must_Return_Fizz_Buzz_FizzBuzz_Or_A_Number()
Spec.ForAny<int>(x => true)
.Classify(x => _fizzBuzzer.Translate(x).Equals("Fizz"), "Fizz")
.Classify(x => _fizzBuzzer.Translate(x).Equals("Buzz"), "Buzz")
.Classify(x => _fizzBuzzer.Translate(x).Equals("FizzBuzz"), "FizzBuzz")
.Classify(x => Regex.IsMatch(_fizzBuzzer.Translate(x), @"\d+"), "number")

To show that the generator was in fact covering all four different possibilities I set up a "test" which was using fscheck's classification to show the break down of the different values which were coming out of Translate.

public void Given_A_Number_Divisible_By_3_And_5_It_Must_Contain_Both_Fizz_And_Buzz()
Spec.For(DivisibleBy(3*5), d => string.IsNullOrEmpty(_fizzBuzzer.Translate(d)) == false)
.And(d => _fizzBuzzer.Translate(d).Contains("Fizz"))
.And(d => _fizzBuzzer.Translate(d).Contains("Buzz"))

Last I set up a test case to show that if a value was divisible by both 3 and 5 it will contain both "Fizz" and "Buzz".  Now this test case was not needed since it was covered above, but I wanted to show how the And works.

There you have it FizzBuzz using Property Testing.  I found the examples in fscheck's GitHub repo very helpful.

Note, I used QuickCheckThrownOnFailure to cause the unit test to actually fail the test runner when the property is not true for some value.  This is very important to do if you are using something like NCrunch to run your tests.  If you have a property which fails for some value and do not use QuickCheckThrownOnFailure on your tests it will "pass" from the runner's point of view, but the result will give the value which falsifies it, this was not what I wanted so I had fscheck throw an exception when the property was falsified.

Sunday, June 29, 2014

Proving Traits

"A play there is, my lord, some ten words long,
Which is as ‘ brief ’ as I have known a play.
But by ten words, my lord, it is too long
-- Shakespeare, Midsummer Night's Dream

Martin Heidegger in his paper, The Origin of the Work of Art, gives us three different ways of defining thingness.

  1. bearer of traits and properties
  2. sense perceptions
  3. formed stuff

Traits is an interesting term in computer science, it is a collection of behaviors and their implementation which can be used to extend an object.  Most languages do not directly support the creation of stand-alone traits but traits are the essence of what makes up a programs value.

I am going to make a very bold statement here, I am going to say that BDD (Behavior Driven Development) is really about proving the traits of software.  Using the definition of traits given above we see that a trait is a collection of behaviors and how they have been implemented.  This sounds a lot like what we are proving out when we do BDD.

In BDD what are we doing?  We are proving that behavior X as it is implemented by code S does the following when given state G with stimulus W yielding state T.  In other words we are showing Given the following conditions, When we do the following against X in this Source code, Then the following result will occur.  This sounds a lot like proving the traits of the software to me.

"Words, words, words."
-- Shakespeare, Hamlet

Let us look at a example.

In the example above we see that a Starfleet Captain needs to be able to communicate with the FizzBuzzons and thus needs to have a translator.

She is given the following C# source code to preform the needed behavior of translating.

public string Translate(int value)
var result = string.Empty;
if (value % 3 == 0) result += "Fizz";
if (value % 5 == 0) result += "Buzz";
return string.IsNullOrEmpty(result) ? value.ToString() : result;

To prove to her that it works the engineers and communication specialist came up with the following specifications listing what happens in a Given state When the an input value is translated Then the value will have a certain output.

Scenario: 2 must translate to the string 2
Given a FizzBuzzer
And the value of '2'
When translate is invoked
Then the result should be '2'
Scenario: 3 must translate to the string Fizz
Given a FizzBuzzer
And the value of '3'
When translate is invoked
Then the result should be 'Fizz'
Scenario: 5 must translate to the string Buzz
Given a FizzBuzzer
And the value of '5'
When translate is invoked
Then the result should be 'Buzz'
Scenario: 15 must translate to the string FizzBuzz
Given a FizzBuzzer
And the value of '15'
When translate is invoked
Then the result should be 'FizzBuzz'

By testing around these specifications we can prove to our captain that the translator does hold the desired traits.

Does this slight different definition around BDD change anything?  No, but I hope it does widen the thought around it.

"If we shadows have offended
Think but this, and all is mended:
That you have but slumbered here
While these visions did appear.
-- Shakespeare, Midsummer Night's Dream

Saturday, June 21, 2014

Specs in Scala and C#

"That I have uttered. Bring me to the test,
And I the matter will re-word, which madness
-- Shakespeare, Hamlet
Act III, Scene IV, Lines 143-144


Places, places, the play is about to begin.

Tonight we shall look at a tale of two specification frameworks, Scala's spec2 and .Net's SpecFlow.

Act I - spec2

Say we have the following basic FizzBuzz functionality in Scala.

object FizzBuzzer {
class FizzBuzzExtractor[T](f: T => Boolean) {
def unapply(x: T) = f(x)
val FizzBuzz = new FizzBuzzExtractor[Int](_ % 15 == 0)
val Fizz = new FizzBuzzExtractor[Int](_ % 3 == 0)
val Buzz = new FizzBuzzExtractor[Int](_ % 5 == 0)
def eval(value: Int) = value match {
case FizzBuzz() => "FizzBuzz"
case Fizz() => "Fizz"
case Buzz() => "Buzz"
case _ => value.toString

How could we go about testing this functionality?  We could do so simple unit testing, but we have business people that really want to make sure that we get this right and they will not be able to follow our unit tests without a bit of help on our part.

This is when a specification framework using a Cucumber or Gherkin style of syntax comes into play.  As DHH pointed out in "Is TDD Dead" part IV, Cucumber is really a pain to use and as such we really need to think about the cost of using this for our testing.

In this made up case we are going to say that the business is ready and willing to fully learn and use the the Given / When / Then style and thinking that Cucumber will force upon them.  In reality I have worked with business partners in which this was the case, others that thought this was the case and then stop doing it, and still others that wanted nothing to do with this "IT thing".  If you are going to do testing in this way, know what you are getting yourself into.

Given all that, let us put together our tests working closely with our business partners.

Note, I am new to Scala, as such I need to look at this gist form Kingsley Davies in order to do the kata.

We see that for this kind of functionality the team as a whole felt that a truth table would be the best way to verify functionality, as such we set up the following truth table in spec2.

def table =
"input" | "result" |>
2 ! "2" |
3 ! "Fizz" |
5 ! "Buzz" |
15 ! "FizzBuzz" |
33 ! "Fizz" |
55 ! "Buzz" |
150 ! "FizzBuzz" |
151 ! "151" |

We go ahead and run the input and expected results through our Scala function using the following.

{ (a, b) => { FizzBuzzer.eval(a) must_== b } }

In this case, in our lambda we are binding a to our input and b to our expected result.  Then we go a head and assert using the must assertion that value which comes out of our FizzBuzzer does indeed equal our excepted result.

The nice thing about setting up a continuous testing environment with sbt using ~test is that you can implement the functionality in a BDD style.

As we see in the example above if we have a failing condition in our truth table we are notified which row in the table is not true (in this case 151).  We are then able to write enough code to pass the failing row and move on.

Act II - SpecFlow

Say we have the following basic FizzBuzz functionality in C#.

public class FizzBuzzer
public string Translate(int value)
var result = string.Empty;
if (value%3 == 0) result += "Fizz";
if (value%5 == 0) result += "Buzz";
return string.IsNullOrEmpty(result) ? value.ToString() : result;

Like in Act I our business partners wants to get involved in the verification and validation of this functionality.  As such they are willing and ready to learn the Cucumber syntax and thinking so that the whole team can be on the same page.

Given the nature of the functionality we are developing the team feels that a truth table will be the best way to test it.  As such the following truth table is come up with as a team.
| value | expected |
| 1 | 1 |
| 2 | 2 |
| 3 | Fizz |
| 4 | 4 |
| 5 | Buzz |
| 6 | Fizz |
| 7 | 7 |
| 8 | 8 |
| 9 | Fizz |
| 10 | Buzz |
| 11 | 11 |
| 12 | Fizz |
| 13 | 13 |
| 14 | 14 |
| 15 | FizzBuzz |
| 30 | FizzBuzz |
| 45 | FizzBuzz |

In it we have the input and the expected output of the functionality.  We use SpecFlow to come up with the following.

In our given we using the following line to place in the input into memory to be used in the when step.

ScenarioContext.Current.Add("target", new FizzBuzzer());

In our when we are then able to read back out the input from the given step and pass it to our FizzBuzz functionality giving use the actual result which we place into memory to be used in the then step.

var target = ScenarioContext.Current.Get<FizzBuzzer>("target");
var actual = target.Translate(value);
ScenarioContext.Current.Add("value", actual);

Last in our then step we take the actual result and compare it to the expected result, using NUnit to assert that are in fact equal to each other.

var actual = ScenarioContext.Current.Get<string>("value");
Assert.That(actual, Is.EqualTo(expected));

The nice thing about SpecFlow is that you generate the C# coded spec directly from the Cucumber feature file.  The other really nice thing is that it works with NCrunch for continuous testing!

We see below that if we have row not passing in our truth table we get a detail error message stated what is the following row and why.

Once we write just enough code to get back to green we can continue along refactoring and write our next failing testing repeating the red / green / refactor loop.

Act III - Now what

We see that if we, the development team and our business partners, are willing to pay the cost associated with Cucumber we can as a team get everyone on the same page with the same language and understanding of requirements.  Now what?  If you are looking for more information about BDD and SpecFlow check out a colleague of mine, Richard Yu's presentation called "50 Shades of BDD" (he does not know that I am adding him to this blog post, so I hope he does not mind).