Using xUnit

A key goal of the architecture of the functional domain project is that it is unit testable. xUnit.net seems to be the default test tool of choice for .NET Core, so I decided to try it out.

xUnit seems to be an evolution of earlier testing frameworks (see this for a comparison), in particular adding data driven testing.

Trial by xUnit

Unfortunately the documentation for xUnit is poor – while the homepage links to basic attribute use, there is no obvious link to documentation on theories and the various forms of data they take – so using data driven testing requires assembling information from widespread Google searching (which this post may add to) or downloading and figuring out the source code (which I just didn’t feel like this time!). Furthermore at one point I receieved a ‘NotSupportedException’ without a line number or anything else indicating what wasn’t supported, eventually discovering that exception referred to the MemberData attribute.

I’m going to engage in a small rant now. This lack of attention to documentation is a disappointing but fairly expected consequence of being open-source. It’s something that was repeatedly infuriating about working with Node.js, and frankly the opposite of what I expect from something that exists in the eco-bubble of Microsoft, an enterprise framework provider. I don’t know the whole governance structure of the .NET Foundation, but to me .NET and Microsoft go together, so perhaps Microsoft could lend a few technical writers to the cause. And to be clear, no disrespect to the developers – their greatest value is in progressing the platform not spending weeks documenting it. End rant.

Using ClassData

Finally I was able to get the xUnit ClassData attribute working in F#, and from this construct a nice base class so data can be easily created as sub-classes.

open System
open System.Collections
open System.Collections.Generic
open Xunit

[<AbstractClass>]
type BaseTestData() =
    abstract member data: seq<obj[]>
    interface IEnumerable<obj[]> with 
        member this.GetEnumerator() : IEnumerator<obj[]> = this.data.GetEnumerator()
        member this.GetEnumerator() : IEnumerator = this.data.GetEnumerator() :> IEnumerator

type MyTestData() =
    inherit Util.BaseTestData()
    override this.data = Seq.ofList [[| box 4 |]; [| box 5 |]]

type SomeTestClass() =
    [<Theory>]
    [<ClassData(typeof<MyTestData>)>]
    let IsOdd value = Assert.True(value % 2 = 1)

To complete the first test, a little additional type coercion was needed to avoid errors like “Object of type ‘…Railway+Result`2+Failure[System.Object,AssetManagementApi.Commands.Template+TemplateCommandError]’ cannot be converted to type ‘…Railway+Result`2[Microsoft.FSharp.Core.Unit,AssetManagementApi.Commands.Template+TemplateCommandError]’. Note the extra (pointless) conversion at line 7 which is the same type as the argument to the test method.

type TemplateValidationTestData() =
        inherit Util.BaseTestData()
        override this.data = 
            Seq.ofList [
                [|
                    ({ Id = System.Guid.NewGuid(); Name = "abc"; Fields = []; MaintenanceProgramId = System.Guid.Empty } : DomainTypes.Template) :> obj
                    (Railway.Failure (InvalidTemplate "Template may not have an empty list")) :> Railway.Result :> obj
                |] 
            ]
type TemplateDomainOperationsTests() =
    [<Theory>]
    [<ClassData(typeof<MyTestData>)>]
    let ``Template validation`` (input: DomainTypes.Template) (expected: Railway.Result<unit,TemplateCommandError>) =

        let cmd = TemplateCommand.Create(input)
        let result : Railway.Result = TemplateCommandHandler.Execute cmd mockRepo
        Assert.Equal(result, expected)
        ()

Overall this is pretty ugly. It’s incredibly far from ideal that the test data essentially gets its type destroyed because ClassData deals in object arrays. An alternative would be to use InlineData with basic types to populate data but that is very limiting as we can’t use an empty list or discriminated union type as data.

The end result is that I’ll live with theories and ClassData for now, but certainly keep my eye on something better, and something more suited to F#. FsUnit may be better, but currently doesn’t support CoreCLR (although there is a PR open since Jun 1 with changes for that).

Gotcha

One weird behavior I ran into was that a let binding kept returning null during a test, so I put a couple of logging statements in and discovered the root of the module is never run. This makes sense given the test runner is instantiating specific classes – but it is a tricky little trap. Solving the problem is as simple as turning the let (line 1) into a function.

let basicTemplate = { ... }

Logger.info (sprintf "1: %A" basicTemplate) // never called!

type TemplateDomainOperationsTests() =
    [<Fact>]
    let ``Test method description`` =
        Logger.info (sprintf "2: %A" basicTemplate) // returns 

One thought on “Using xUnit

  1. Pingback: FsUnit | winter limelight

Comments are closed.