-- Shakespeare, A Midsummer's Night Dream
Act III, Scene II, Line 128
Intro
Sir you got F# in my C#. I do not wish to add F# to my Solution just to be able to test my code. Is there a way to get similar functionality to FsCheck without using F#?
Glad you asked. I believe NUnit can assist us here.
FizzBuzz with NUnit
First thing we see is a very uninteresting version of FizzBuzz.
public string FizzBuzz(int value){if(value < 0) throw new ArgumentException("Value must be positive.");var ret = string.Empty;if (value%3 == 0) ret += "Fizz";if (value%5 == 0) ret += "Buzz";return string.IsNullOrEmpty(ret) ? value.ToString() : ret;}
Nothing really interesting going on here other than using a variable to preserve state so that we do not have to check for 15 or have more than one return statement.
Excepted Exception
[Test, ExpectedException(typeof (ArgumentException))]public void Negative_Values_Throws_An_ArgumentException(){FizzBuzz(-1);}
We see that we can declare that an exception will be thrown and thus have test coverage for our exception cases.
If we want to make sure that every detail of our exception matches what we think it should be NUnit offers a more verbose check too.
[Test,ExpectedException(typeof (ArgumentException), ExpectedMessage = "Value must be positive.",MatchType = MessageMatch.Exact)]public void Given_A_Negative_1_It_Will_Throw_An_ArgumentException_With_The_Message_Of_ValueMustBePostive(){FizzBuzz(-1);Assert.Fail("Should have thrown an execpetion");}
Ranges of Values in One Test Case
[Test]public void Generate_A_Range_Of_Fizz_Data([Range(3, 300, 3)] int value){var removeBuzz = (value % 5 == 0) ? 3 : value;Assert.That(FizzBuzz(removeBuzz), Is.EqualTo("Fizz"));}
If we want to we can define a range of values to check. With the example above we see that we are checking the values: 3, 6, 9, ..., 300 are not divisible by 5 (thus, 15, 30, 45, ... will not be checked). This data will be used to verify that Fizz is returned for each of these test cases. We can do a similar thing with Excepted Exceptions.
[Test, ExpectedException(typeof (ArgumentException))]public void Generate_A_Range_Of_Invali_Fizz_Data([Range(-1000, -1, 1)] int value){FizzBuzz(value);Assert.Fail("Should have thrown an exception");}
Test Case
We can set up a test case using the TestCase attribute.
[TestCase( 0, "FizzBuzz")][TestCase( 1, "1")][TestCase( 2, "2")][TestCase( 3, "Fizz")][TestCase( 4, "4")][TestCase( 5, "Buzz")][TestCase( 6, "Fizz")][TestCase(10, "Buzz")][TestCase(15, "FizzBuzz")][TestCase(45, "FizzBuzz")][TestCase(-1, "error", ExpectedException = typeof(ArgumentException))]public void FizzBuzz_Test_Cases(int value, string expected){Assert.That(FizzBuzz(value), Is.EqualTo(expected));}
This allows us to reuse the boilerplate test case setup while allowing us to provide the test data. In this case we are providing both the value to test and the expected result.
We can be more explicit with the result and use the Result property of the TestCase attribute.
[TestCase( 0, Result = "FizzBuzz")][TestCase( 1, Result = "1")][TestCase( 2, Result = "2")][TestCase( 3, Result = "Fizz")][TestCase( 4, Result = "4")][TestCase( 5, Result = "Buzz")][TestCase( 6, Result = "Fizz")][TestCase(10, Result = "Buzz")][TestCase(15, Result = "FizzBuzz")][TestCase(45, Result = "FizzBuzz")][TestCase(-1, ExpectedException = typeof(ArgumentException))]public string FizzBuzz_Test_Cases_With_Expected_Results(int value){return FizzBuzz(value);}
Note, when testing this way you do not call Assert but instead return the value (note also the return type of the test function is a string in this case and not a void). NUnit will assert the result of method for you!
Generating Test Data
Property Based Testing is a very a powerful idea which decouples the behavior which you are testing from the generating of test data. I believe an example would be good about now.
// Random does not work with NCrunch unless NUnit is set to UseStaticAnalysis[Test]public void Generate_Buzz_Data([Random(1, 10000, 100)] int value){var removeFizz = (value % 3 == 0) ? 5 : value * 5;Assert.That(FizzBuzz(removeFizz), Is.EqualTo("Buzz"));}
We see above the use of the Random attribute. Random will generate a value between 1 and 1000 (the first two arguments that we pass it), in this case, this will be done 100 times! (100 is the third argument that we pass it.) We see also that for this test we want to verify the Buzz functionality, so we reshape the data in such a way that we always get a value which should produce Buzz. This kind of testing is a great way to find edge cases.
We see a comment above the test case which means something is going wrong. In this case the comment is telling use that when we use Random with NCrunch we need to change the Configuration to use UseStaticAnalysis for NUnit. You can read all about it here. If you do not change this setting you'll get the following error message.
"This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."
NUnit offers other ways to generate data, one of which is the Datapoints / Theory combo.
[Datapoints] public int[] Values = new[] {-1, 0, 2, 3, 4, 5, 9, 15, 25, 45};[Theory]public void Numbers_Divisible_By_15_Will_Return_FizzBuzz(int value){Assume.That(value % 15 == 0);var actual = FizzBuzz(value);Assert.That(actual, Is.Not.Null);Assert.That(actual, Is.EqualTo("FizzBuzz"));}
The way that Theory works is that it will use the values from the field marked as Datapoints. In the case above we are restricting the values just to what is divisible by 15, thus we are testing the FizzBuzz functionality.
We can also set up an array of arrays which contain test case values using the TestCaseSource attribute.
public static object[] FizzBuzzTestData ={new object[] { 1, "1"},new object[] { 2, "2"},new object[] { 3, "Fizz"},new object[] { 9, "Fizz"},new object[] { 5, "Buzz"},new object[] {10, "Buzz"},new object[] { 0, "FizzBuzz"},new object[] {15, "FizzBuzz"}};[Test, TestCaseSource("FizzBuzzTestData")]public void FizzBuzz_Test_Data(int value, string expected){Assert.That(FizzBuzz(value), Is.EqualTo(expected));}
We see with the example above that we are defining both the value and excepted result which are passed into the test.
We can take this a step forward and define an actual test generator class.
public class FizzBuzzTestCaseDataFactory{public static IEnumerable<TestCaseData> TestCaseData{get{yield return new TestCaseData(1).Returns("1");yield return new TestCaseData(2).Returns("2");yield return new TestCaseData(3).Returns("Fizz");yield return new TestCaseData(33).Returns("Fizz");yield return new TestCaseData(5).Returns("Buzz");yield return new TestCaseData(55).Returns("Buzz");yield return new TestCaseData(15).Returns("FizzBuzz");yield return new TestCaseData(165).Returns("FizzBuzz");yield return new TestCaseData(-1).Throws(typeof (ArgumentException));yield return new TestCaseData(-11).Throws(typeof (ArgumentException));}}}[Test, TestCaseSource(typeof(FizzBuzzTestCaseDataFactory), "TestCaseData")]public string Data_Factory_Test_Case(int value){return FizzBuzz(value);}
We see that by using the TestCaseSource attribute and telling it the typeof the test generator class and the name of the method for generating test case data, NUnit will call the method and verify our functionality for us!
We also see that the TestCaseData class allows us to specify the results. In my opinion this allows for very high levels of readability.
I know what you might be thinking, this test data generating is fine but why use this over a for loop? Well the for loop would test the same functionality, but it would not show up as different test cases to the test runner (unless you do so real hacking), while the NUnit test data generators would. With the NUnit test data generators, if one of the values fail the test case you'll see the offending value instead of just seeing that the test case with a for loop broke.
This is what the values for Generate_Buzz_Data (the test using the Random attribute) actually look like to the test runner.
Generate_Buzz_Data
NUnit.CharacterizationTests.Generate_Buzz_Data(5224):
NUnit.CharacterizationTests.Generate_Buzz_Data(8147):
NUnit.CharacterizationTests.Generate_Buzz_Data(8619):
...
Look Mom, No Quickcheck
There you have it advance unit testing with NUnit. Use NUnit's different test data generators we were able to do Property Based Testing without using quickcheck.
I do want to make a quick call out to Luke Wickstead's excellent posts on NUnit. Reading this post allowed me to figure out how the TestCaseSource really worked.
