--- title: Property based testing | Codurance description: Our team of dedicated software craftspeople provides consultancy, software development, and training services to clients seeking high quality development processes and software solutions. --- [Publications | Codurance ](https://www.codurance.com/publications) # [Property based testing | Codurance](https://www.codurance.com/publications/2015/11/25/property-based-testing-fscheck-resharper-ncrunch-visual-studio-2013) Written by [Alessandro Di Gioia](https://www.codurance.com/publications/author/alessandro-di-gioia) | 25 Nov 2015 ### Background Craftsmanship in software strives for high, predictable, repeatable and affordable quality. This is the reason why we care about design, code reviews and of course testing. Test Driven Development helps us create a flexible implementation that meets the expectations of our clients in terms of features, performance and quality. As professionals we have to master as many tools as possible to reach our goal of steadily providing value. ### Property based testing The above is the reason why I am exploring Property Based Testing. I like to think about it as *"Let the machine do the heavy lifting for you"* or as [John Huges](http://vimeo.com/68383317) says *"Don't write tests, generate them"*. I am not claiming that you should stop writing unit tests. I find that when driving the design [Outside-in](https://www.youtube.com/watch?v=XHnuMjah6ps) hence declaring the relations between my components, unit tests are the tool to use. Sometimes when I want to discover deeper properties of my domain, I am faced with features that have more edge cases than my brain could handle. In such cases, generating a wide array of inputs to test my program is appealing. This is where I think property testing becomes a viable tool. If you are asking "How can I set up a development environment in .NET that would allow me to experiment and work with this tool?" the following tutorial is for you. ### Development environment details - Windows 10 Home Edition - Visual Studio 2013 version 12 Update 5 - Resharper version 8.2.3 - NCrunch version 1.48.0.5 - FsCheck version 2.2.2 - FsCheck.NUnit version 2.2.2 - NUnit version 2.6.4 ### Steps 1. Open Visual Studio and create an F# Project 2. Open the Nuget Package Manager following *TOOLS > Nuget Package Manager > Package Manager Console* from the main menu and type 1. `Install-Package FsCheck` 2. `Install-Package FsCheck.Nunit` 3. Comment out the content in **FsCheckAddin.fs** file to allow [NCrunch](http://www.ncrunch.net/) to run the tests ``` //[] //type FsCheckAddin() = // interface IAddin with // override x.Install host = // let tcBuilder = new FsCheckTestCaseBuilder() // host.GetExtensionPoint("TestCaseBuilders").Install(tcBuilder) // true ``` 4. In your test file add the following ``` module Properties = open NUnit.Framework open FsCheck ``` 5. Add this test to enable [NCrunch](http://www.ncrunch.net/) ``` //Needed to enable NCrunch [][] let ignoreMe() = () ``` 6. Create a static class whose methods are the properties you want to test. For this example I'll test an incorrect implementation of a list - **the reverse of a list is always equal to the original**. ``` type ListProperties = // Note: should fail static member reverseIsAsTheOriginal (xs:int list) = List.rev xs = xs ``` 7. Add the following test to verify the property defined in the `ListProperties` type ``` [] let verifyAll () = Check.QuickThrowOnFailureAll() ``` 8. Enable [NCrunch](http://www.ncrunch.net/) following *NCRUNCH > Enable NCrunch* from the main menu and let the engine catch the error 9. If everything is in place you should see the visual error for the feedback 10. Run All Tests in the Resharper Runner to check the output 11. You should read a message similar to ``` --- Checking ListProperties --- System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation. ----> System.Exception : ListProperties.ReverseOfReverseIsAsTheOriginal-Falsifiable, after 6 tests (3 shrinks) (StdGen (1165808905,296086422)): Original: [-1; 2; -2] Shrunk: [1; 0] ``` 12. This shows that the property was falsifiable (as expected) but let's try to gather more informations to address the issue 13. Change the test so that it provides a verbose output when failing ``` [] let verifyAll () = Check.VerboseThrowOnFailureAll() ``` 14. Re-run the tests through Resharper Runner and read the output ``` --- Checking ListProperties --- 0: [-2] 1: [-2; -2] 2: [] 3: [1; 1] 4: [-2] 5: [-3; 1; 5; -3; 6; 2; 6; -2] shrink: [1; 5; -3; 6; 2; 6; -2] shrink: [5; -3; 6; 2; 6; -2] shrink: [-3; 6; 2; 6; -2] shrink: [6; 2; 6; -2] shrink: [2; 6; -2] shrink: [6; -2] shrink: [6; 2] shrink: [6; 0] shrink: [3; 0] shrink: [2; 0] shrink: [1; 0] System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation. ----> System.Exception : ListProperties.ReverseOfReverseIsAsTheOriginal-Falsifiable, after 6 tests (11 shrinks) (StdGen (1681135586,296086423)): Original: [-3; 1; 5; -3; 6; 2; 6; -2] Shrunk: [1; 0] ``` 15. The output shows the "shrinking process" from the sixth input (identified by 5:) original value `[-3; 1; 5; -3; 6; 2; 6; -2]` to the "shrunk" (reduced) input `[1; 0]` that still falsifies (makes it fail) the property 16. Let's correct the property as **the reverse of the reverse of a list is equal to the original** ``` type ListProperties = static member ReverseOfReverseIsAsTheOriginal (xs:int list) = List.rev (List.rev xs) = xs ``` 17. Re-run the tests that should be all green by now and read the output. You'll notice it's very verbose as expected and shows all the inputs used to verify the property. ### Conclusion You are now ready to experiment with [FsCheck](http://fscheck.github.io/FsCheck/index.html)! [Let me know](https://twitter.com/Parajao) what is your experience with property based testing in F#. ### Resources: - [FsCheck](http://fscheck.github.io/FsCheck/index.html) - [NCrunch](http://www.ncrunch.net/) - [QuickCheck](http://www.eecs.northwestern.edu/~robby/courses/395-495-2009-fall/quick.pdf) - [Testing FSharp](http://www.packtpub.com/application-development/testing-f) Chapter 11 [View full post](https://www.codurance.com/publications/2015/11/25/property-based-testing-fscheck-resharper-ncrunch-visual-studio-2013) ```json { "@context" : "http://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Alessandro Di Gioia" }, "dateModified" : "2021-12-02T22:51:13.430Z", "datePublished" : "2015-11-25T00:00:00Z", "headline" : "Property based testing", "image" : { "@type" : "ImageObject", "height" : 370, "url" : "https://f.hubspotusercontent10.net/hubfs/3042464/Imported_Blog_Media/2015-12-01-Faran-crane-truck-1.jpg", "width" : 640 }, "mainEntityOfPage" : "https://www.codurance.com/publications/2015/11/25/property-based-testing-fscheck-resharper-ncrunch-visual-studio-2013", "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "height" : 60, "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png", "width" : 60 }, "name" : "Publications" } } ```