-- Shakespeare, A Midsummer's Night Dream
Act III, Scene II, Line 128
Sir you got F# in my C#. I do not wish to add F# to my Solution just to be able to test my code. Is there a way to get similar functionality to FsCheck without using F#?
Glad you asked. I believe NUnit can assist us here.
FizzBuzz with NUnit
First thing we see is a very uninteresting version of FizzBuzz.
Nothing really interesting going on here other than using a variable to preserve state so that we do not have to check for 15 or have more than one return statement.
We see that we can declare that an exception will be thrown and thus have test coverage for our exception cases.
If we want to make sure that every detail of our exception matches what we think it should be NUnit offers a more verbose check too.
Ranges of Values in One Test Case
If we want to we can define a range of values to check. With the example above we see that we are checking the values: 3, 6, 9, ..., 300 are not divisible by 5 (thus, 15, 30, 45, ... will not be checked). This data will be used to verify that Fizz is returned for each of these test cases. We can do a similar thing with Excepted Exceptions.
We can set up a test case using the TestCase attribute.
This allows us to reuse the boilerplate test case setup while allowing us to provide the test data. In this case we are providing both the value to test and the expected result.
We can be more explicit with the result and use the Result property of the TestCase attribute.
Note, when testing this way you do not call Assert but instead return the value (note also the return type of the test function is a string in this case and not a void). NUnit will assert the result of method for you!
Generating Test Data
Property Based Testing is a very a powerful idea which decouples the behavior which you are testing from the generating of test data. I believe an example would be good about now.
We see above the use of the Random attribute. Random will generate a value between 1 and 1000 (the first two arguments that we pass it), in this case, this will be done 100 times! (100 is the third argument that we pass it.) We see also that for this test we want to verify the Buzz functionality, so we reshape the data in such a way that we always get a value which should produce Buzz. This kind of testing is a great way to find edge cases.
We see a comment above the test case which means something is going wrong. In this case the comment is telling use that when we use Random with NCrunch we need to change the Configuration to use UseStaticAnalysis for NUnit. You can read all about it here. If you do not change this setting you'll get the following error message.
"This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."
NUnit offers other ways to generate data, one of which is the Datapoints / Theory combo.
The way that Theory works is that it will use the values from the field marked as Datapoints. In the case above we are restricting the values just to what is divisible by 15, thus we are testing the FizzBuzz functionality.
We can also set up an array of arrays which contain test case values using the TestCaseSource attribute.
We see with the example above that we are defining both the value and excepted result which are passed into the test.
We can take this a step forward and define an actual test generator class.
We see that by using the TestCaseSource attribute and telling it the typeof the test generator class and the name of the method for generating test case data, NUnit will call the method and verify our functionality for us!
We also see that the TestCaseData class allows us to specify the results. In my opinion this allows for very high levels of readability.
I know what you might be thinking, this test data generating is fine but why use this over a for loop? Well the for loop would test the same functionality, but it would not show up as different test cases to the test runner (unless you do so real hacking), while the NUnit test data generators would. With the NUnit test data generators, if one of the values fail the test case you'll see the offending value instead of just seeing that the test case with a for loop broke.
This is what the values for Generate_Buzz_Data (the test using the Random attribute) actually look like to the test runner.
Look Mom, No Quickcheck
There you have it advance unit testing with NUnit. Use NUnit's different test data generators we were able to do Property Based Testing without using quickcheck.
I do want to make a quick call out to Luke Wickstead's excellent posts on NUnit. Reading this post allowed me to figure out how the TestCaseSource really worked.