Tidier Controllers with Request Filters

All of my ASP.NET Core controllers are starting to follow a certain pattern (as follows) and in the spirit of good DRY code, I’d like to define this in a single place.

try
    Logger.info ("TemplateController.Get(" + id.ToString() + ")")
    // Actual logic
with
| _ as ex -> Logger.error (sprintf "%O" ex); this.BadRequest() :> IActionResult

Furthermore, I’ve also noted that ASP.NET Core will happily provide me with an invalid model, which means I then have to do an extra null check. It’d be nice to handle all these scenarios transparently for all controllers.

ASP.NET Core provides a solution for this via filters, which exposes interfaces that can be implemented to intercept requests during the processing pipeline.

Exception Filter

An exception filter can be used to handle exceptions. An exception filter class must implement IExceptionFilter which includes a single method, OnException (context: ExceptionContext).

Unfortunately the setup for an exception filter goes in the Startup.ConfigureServices while the global logger factory is available from Startup.Configure. So using the application logger factory requires an un-F# hack as follows:

type Startup(env: IHostingEnvironment) =
    let mutable _loggerFactory : ILoggerFactory option = None

    member this.ConfigureServices(services: IServiceCollection) =
        let mvc = services.AddMvcCore()
        mvc.AddMvcOptions(fun mvcOptions -> mvcOptions.Filters.Add(new GlobalExceptionFilter(_loggerFactory.Value)))

    member this.Configure (app: IApplicationBuilder, loggerFactory: ILoggerFactory) =
        _loggerFactory <- Some(loggerFactory)

In my case I’m happy using my global logger, so my GlobalExceptionFilter won’t take any arguments. And here it is:

type GlobalExceptionFilter() =
    interface IExceptionFilter with
        member this.OnException (context: ExceptionContext) =
            Logger.error (sprintf "%O" context.Exception)

Action Filter

The action filter needs to do two things:
1. Log calls
2. Prevent invalid model state from reaching the actions.

Action Filters implement either the IActionFilter or IAsyncActionFilter interface and their execution surrounds the execution of action methods. Action filters are ideal for any logic that needs to see the results of model binding, or modify the controller or inputs to an action method. Additionally, action filters can view and directly modify the result of an action method.
https://docs.microsoft.com/en-us/aspnet/core/mvc/controllers/filters#action-filters

The aim here is to check and log the state before the action method is called so only the before-action method, OnActionExecuting, needs to be implemented. This implementation checks if the model state is valid, and if not logs the error and terminates the request with a 400 error without the actual action being executed. Where the model is valid, it logs the parameters.

type GeneralActionFilter() = 
    interface IActionFilter with

        member this.OnActionExecuting (context: ActionExecutingContext) =
            if not context.ModelState.IsValid then
                let errors = 
                    context.ModelState.Values 
                    |> Seq.collect (fun (value: ModelStateEntry) -> value.Errors)
                    |> Seq.map (fun (modelError: ModelError) -> sprintf "%s" modelError.Exception.Message)
                    |> String.concat "\n\t  "

                Logger.error (sprintf "Called %s. Error: Invalid model state\n\tException messages: \n\t  %s" context.ActionDescriptor.DisplayName errors)
                context.Result <- new BadRequestObjectResult(context.ModelState)
            else
                let args = [ for kvp in context.ActionArguments -> sprintf "%s %A" kvp.Key kvp.Value ] |> String.concat "\n\t"
                Logger.info (sprintf "Called %s with: \n\t%s" context.ActionDescriptor.DisplayName args)

        member this.OnActionExecuted (context: ActionExecutedContext) = ()

The end result is much cleaner controller methods:

    [<HttpGet>]
    member this.Get(id: System.Guid) : IActionResult =
        match GetTemplate id (new TemplateRepository()) with
        | Some template -> this.Json(template) :> IActionResult
        | None -> this.NotFound() :> IActionResult

    [<HttpPut>]
    member this.Create([<FromBody>]template: Template) : IActionResult =       
        CreateTemplate template (new TemplateCommandHandler())
        let url = new UrlActionContext (Controller = "Template", Action = "Get", Values = new RouteValueDictionary(dict [("id", box template.Id)]))
        this.Created((this.Url.Action url), "") :> IActionResult

Filters are added to the MVC pipeline in the Startup.ConfigureServices method:

member this.ConfigureServices(services: IServiceCollection) =
        let mvc = services.AddMvcCore()
        mvc.AddMvcOptions(fun mvcOptions -> mvcOptions.Filters.Add(new Api.Filters.GlobalExceptionFilter()))
        mvc.AddMvcOptions(fun mvcOptions -> mvcOptions.Filters.Add(new Api.Filters.GeneralActionFilter()))
        mvc.AddJsonFormatters() |> ignore

p.s.

As I noted in an earlier post I don’t have a functioning debugger or intellisense, and I’ve come to appreciate just how much time having a debugger and a navigable watch window saves – being able to scan through fields to find something appropriate is much faster and easier than trawling through documentation. An additional challenge is that the docs don’t include inherited members, so you have to open those separately to find all inherited members.

Geonet

Full credit to the team who runs the Geonet website. For those who don’t know, it’s the website of the New Zealand crown research institute that handles all things seismic. Like many New Zealanders I’ve been to the website a lot since the recent Kaikoura Earthquake and I’ve never once experienced performance problems.

What I find particularly amazing is the scale they must be capable of handling. I imagine the normal website load is fairly light, perhaps in the order of a few hits per second. But following a major event like the earthquake, half of New Zealand, and I imagine a fair few others, are all over the website. I’ve got friends and family who talk about refreshing it every few minutes to see if that “really was an earthquake”. I’m guessing they’ve been handling tens or hundreds of thousands of hits per second in the last few weeks, maybe 4 orders of magnitude greater than usual.

I was interested to know what powers this (I’d hate to think my tax dollars were being wasted on resources that were mostly underutilized 🙂 ). Their blog indicates that they’re using Fastly which provides, in Fastly’s words, a “real-time CDN”. I imagine this event was a pretty good test of Fastly’s capability and Geonet’s integration, and from what I’ve seen, they performed very nicely.
Well done Geonet and Fastly.

Web Services Round Trip

My objective today was to have a web client save some data and retrieve it again. The data was to be stored in an F# record, representing a domain object, and be serialized and deserialized at the web service boundary.

The first issue was with JSON serialization. While a number of blogs indicated a need to change the serializerSettings ContractResolver, this does not appear to be necessary in ASP.NET Core.

This may be because of the change from app.UseMvc() to app.UseMvcCore() which is a lighter method that allows the developer to decide what MVC features are required. In my case this meant adding in app.AddJsonFormatters() so the services would serialize JSON.
One issue I keep running into is not knowing what namespace or module methods found in internet samples are in. To that end I’ve included the whole Program.fs here.

open System
open System.IO
open Microsoft.AspNetCore.Hosting
open Microsoft.AspNetCore.Builder
open Microsoft.AspNetCore.Http
open Microsoft.AspNetCore.Mvc.Formatters.Json
open Microsoft.AspNetCore.Diagnostics
open Microsoft.Extensions.DependencyInjection

type Startup(env: IHostingEnvironment) =

    member this.ConfigureServices(services: IServiceCollection) =
        let mvc = services.AddMvcCore()
        mvc.AddJsonFormatters() |> ignore

    member this.Configure (app: IApplicationBuilder) =
        app.UseDeveloperExceptionPage() |> ignore
        app.UseMvc() |> ignore

[<EntryPoint>]
let main argv = 
    printfn "Starting"
    Logger.info "Startup"
    let host = WebHostBuilder().UseKestrel().UseContentRoot(Directory.GetCurrentDirectory()).UseStartup<Startup>().Build()
    host.Run()
    0 //exit code

In contrast, the CLIMutable attribute is required on an F# record in order for it to be serializable. Without it the serialization of objects fails because MVC requires a default (empty) constructor and public properties to perform serialization. Using CLIMutable causes the record to be compiled with these.

[<CLIMutable>]
type Template = {
    Id: System.Guid
    Name: string
    Fields: FieldDefinition list
    MaintenanceProgramId: System.Guid
}

The template service (still called ‘First’ here) exposes a GET and a PUT, and passes the request onto domain layer operations (I will put together a post on the full architectural implementation when I have all the layers) to process and store it in-memory.

[<Route("api/first")>]
type FirstController() =
    inherit Controller()

    [<HttpGet>]
    member this.Get(id: System.Guid) =
        DomainOperations.GetTemplate id (new TemplateRepository())

    [<HttpPut>]
    member this.Create([<FromBody>]template: Template) =
        DomainOperations.CreateTemplate template (new TemplateCommandHandler())

Using Postman I was able to PUT the following to http://localhost:5000/first/api

{
  "id":"{{id}}",
  "name":"{{templateName}}",
  "fields":[],
  "maintenanceProgramId":"00000000-0000-0000-0000-000000000000"
}

and was returned a similar object from GET http://localhost:5000/first/api?id={{id}}
The full source as it was at this stage is available on my github.

A partial application gotcha

I ran into my first F# gotcha today.
Here was the code in one file:

let template = DomainOperations.GetTemplate id

and in the other:

let GetTemplate (id: System.Guid) (templateRepository: ITemplateRepository) =
    templateRepository.FindById id

When I compiled this there were no errors or warnings, but when it ran GetTemplate wasn’t called.

Then the headslap moment – I’d left out an argument – and because of the beauty of type inference and partial application in F# the compiler was completely happy with what I’d written. It simply assumed that I meant to create a partially applied function that takes a repository as an argument, i.e. template: ITemplateRepository -> Template

Domain Model

This post explores creating a domain model in F#. It continues the series functional domain project which targets creating a basic asset management API.

Domain Requirements

These are the requirements (I’ve made up) for our asset management system.

  • An asset is a thing that the organization owns that has some capital value
  • An asset may contain another asset (e.g. a park contains a playground that contains a slide)
  • An asset is based on a template to ensure similar assets store the same information
  • A template contains custom fields which have a type (text, numeric, date)
  • An asset stores the values for the fields in its template
  • An asset may have a maintenance program. This specifies what work is carried out at certain intervals. For these requirements, an asset may only have a single maintenance program.
  • Any maintenance carried out must be recorded.

First Model

The following is a first pass at a model expressing these requirements.

module DomainTypes

/// MaintenanceProgram must be immutable in data store so that maintenance records referencing it have accurate information.
type MaintenanceProgram = {
    Id: System.Guid
    Summary: string
    Period: System.TimeSpan
    Details: string
}

type MaintenanceRecord = {
    Id: System.Guid
    // One of MaintenanceProgramId and Summary must be set
    MaintenanceProgramId: System.Guid option
    Summary: string option
    DateComplete: System.DateTime
    Details: string
    Cost: decimal
}

type FieldValue = 
    | StringField of string
    | DateField of System.DateTime
    | NumericField of float

type FieldDefinition = {
    Id: System.Guid
    Name: string
    Field: FieldValue
}

type Field = {
    Definition: FieldDefinition
    Value: FieldValue
}

type Template = {
    Id: System.Guid
    Name: string
    Fields: FieldDefinition list
    // I considered making MaintenanceProgram compositional, but that
    // might be limiting for organizations that want a simple set
    // of scheduled maintenance periods without specifics on the type of maintenance
    MaintenanceProgramId: System.Guid
}

type Asset = {
    Id: System.Guid
    Name: string
    Commissioned: System.DateTime
    Cost: decimal
    Fields: Field list
    TemplateId: System.Guid 
    Subassets: Asset list
}

F# records and discriminated unions provide a very natural way to express a domain model. In this model the majority of information is stored as records, with discriminated unions used to indicate and control the type of value stored in a field.

This model also makes the difference between composition and reference relationships clear. Composition implies the target instance will die with the source instance, whereas in a reference relationship the lifetime of the target is independent of the source. The Asset type illustrates this: the field values make no sense if there is no asset so they are compositional, and therefore we include the Field type within the Fields property of the Asset; whereas the template will be used by many assets so it is referenced by id.

Interestingly (at least to me), one could argue a sub-asset, i.e. a component of a system, could be reused in another asset when its current parent asset is decommissioned. This model is forbidding this by including subassets in the Asset rather than referencing them.

Refining Intent

At this point I wanted to enforce my constraint on the MaintenanceRecord, and via google, ran into the excellent series on F# for fun and profit, Designing with types

Firstly, several fields represent the same concept, but there is no way of identifying that. If the field type is replaced by a single case union type then that relationship becomes clearer. For instance:

type MaintenanceSummary = MaintenanceSummary of string

type MaintenanceProgram = {
    Summary: MaintenanceSummary
    ...
}

type MaintenanceRecord = {
    Summary = MaintenanceSummary option
    ...
}

Next, as noted in comments, we want to ensure one of MaintenanceProgramId and Summary is set in MaintenanceRecord. We can use a discriminated union to enforce this by replacing

type MaintenanceRecord = {
    MaintenanceProgramId: System.Guid option
    Summary: MaintenanceSummary option
    ...
}

with

type MaintenanceRecordSummary = 
    | MaintenanceProgramId of System.Guid
    | Summary of MaintenanceSummary

type MaintenanceRecord = {
    Summary: MaintenanceRecordSummary
    ....
}

This is as far as I’m going to take it for now as my overall knowledge of F# is still pretty limited, but I expect that as I implement the surrounding layers I’ll better understand the purpose of more of the advice within designing with types likely implement them.

Considering Architecture

I’m getting through Expert F#, and was starting to consider how a domain model might be implemented. But I think before that it is important to consider where that model fits into the solution, which requires us to think about architecture.

I must say that my disorganized thoughts were put into a much clearer perspective by the F# for fun and profit post Organizing modules in a project and you will likely find that a more practical and clearer explanation than I’m about to make.

Architecture in general

Any solution beyond a few hundred lines of code needs an architecture to organize the code. An architecture provides hard rules and soft guidelines on how different parts of a solution should interact, with the primary goal of making the solution maintainable. A maintainable solution is one that is understandable, and therefore easier to work on and more secure; is slower to accumulate debt; and is more testable and therefore of higher quality.

An architecture creates organization by separating unrelated concerns, controlling message paths, and hiding information. The goal of this project is to create an API for an asset management solution so we are implementing a web service, and the most common organization for a web service architecture is to split the solution into layers. The previous post, ASP.NET Core with F#, briefly considered an implementation for the top layer – the web interface. Beyond that we must also consider:

  • domain – the business rules and (for an API) the core business value;
  • persistence – how data is stored and retrieved;
  • infrastructure – cross-cutting concerns such as user context, configuration, logging.

In the most naive implementation, this creates the following architecture, where each layer depends on the one above:

archstack1

Domain Layer

I believe an architecture should be designed from the domain outwards because the domain is where the core business value resides. The domain comprises groups of data, which I will refer to as data objects, and domain operations which create or act upon the data. In an OO domain model the relevant operations are stored with the relevant data objects, although this tends to get messy.

Coming from C#, F# adds a new constraint: that types must be declared before they are used. This is not as different as it first seems since a domain focused solution (even in an OO stack) should define its data and behavior separate from its implementation to allow for unit tests (not to mention all the other benefits of clear architectural layers). In C# this means defining domain dependencies as interfaces, then injecting the concrete implementations once they are defined.

As F# is strongly typed, in order to have unit tests we must be able to provide at least two different objects (the concrete and a mock) that share a type. The end result is identical to C#: we must create interfaces to express behavior.

So the domain operations depend on:
1. data objects;
2. interfaces defining operations/services that must be provided. Some of those behaviors will be other domain operations and for the sake of consistency it might be desirable to include all the domain operation interfaces in this dependency.

Other Layers

In defining operations/services interfaces, the domain layer has essentially created the persistence and infrastructure interfaces.

With the domain nicely isolated, the remainder of the architecture must now provide what it needs. We expect that the concrete classes for these dependencies will be largely in the persistence and a little in the infrastructure. This results in a network as follows, with the addition of a startup layer which creates key context classes, such as the dependency injector.

archnetwork2

At this point nothing here is F# specific – and that is a good thing! A good architecture should allow implementation decisions to be made as late as practical.

The end result is we now have a place for our domain data objects and operations, and we can begin to define those.

ASP.NET Core with F#

While I’m making my way through the tome that is Expert F# 4.0 I’m itching to create some actual code so I thought I’d try making a basic web API using ASP.NET Core.

.NET Core seems to be evolving quite rapidly so it’s difficult to know what is currently considered the ‘right’ way of doing this. I’ve used the command line tools from powershell to create a new project in an empty folder:

dotnet new -l F#

I then created a Core Web Application project in C#, choosing the Web Api template, to use as the basis for porting to F#. After porting the necessary sections across to F# I ended up with a project as follows:

aspnetcore3project

project.json

{
    "version": "1.0.0-*",
    "buildOptions": {
        "debugType": "portable",
        "emitEntryPoint": true,
        "compilerName": "fsc",
        "compile": {
            "includeFiles": [
                "Controllers/FirstController.fs",
                "Program.fs"
            ]
        },
        "define": []
    },
    "tools": {
        "dotnet-compile-fsc": "1.0.0-preview2-*"
    },
    "frameworks": {
        "netcoreapp1.0": {
            "dependencies": {
                "Microsoft.NETCore.App": {
                    "type": "platform",
                    "version": "1.0.1"
                },
                "Microsoft.FSharp.Core.netcore": "1.0.0-alpha-161102",
                "Microsoft.AspNetCore.Mvc": "1.0.1",
                "Microsoft.AspNetCore.Routing": "1.0.1",
                "Microsoft.AspNetCore.Server.Kestrel": "1.0.1",
                "Microsoft.Extensions.Options.ConfigurationExtensions": "1.0.0"
            }
        }
    }
}

Program.fs

open System
open System.IO
open Microsoft.AspNetCore.Hosting
open Microsoft.AspNetCore.Builder
open Microsoft.AspNetCore.Http
open Microsoft.Extensions.DependencyInjection

type Startup(env: IHostingEnvironment) =

    member this.ConfigureServices(services: IServiceCollection) =
        services.AddMvc() |> ignore

    member this.Configure (app: IApplicationBuilder) =
        app.UseMvc() |> ignore

[<EntryPoint>]
let main argv = 
    printfn "Starting"
    let host = WebHostBuilder().UseKestrel().UseContentRoot(Directory.GetCurrentDirectory()).UseStartup<Startup>().Build()
    host.Run()
    0 //exit code

FirstController.fs

namespace aspnetcore3.Controllers

open System
open System.Collections.Generic
open Microsoft.AspNetCore.Mvc

[<Route("api/first")>]
type FirstController() =
    inherit Controller()

    [<HttpGet>]
    member this.Get() =
        ["x"; "z"]

And with that, browsing to http://localhost:5000/api/first gave me the intended result of returning [“x”,”z”].

I had some problems with F# in Visual Studio 2015 Update 3, and it’s not clear to me whether these are general problems with F# and .NET Core or something I haven’t set up right. The issues I had were:

  • Intellisense doesn’t work with the AspNetCore libraries, so I forever have squiggly red error lines under my open Microsoft.AspNetCore.Mvc statements
  • The compiler doesn’t seem to know when a file has changed, so I was always forcing the build using rebuild or by directly using the command line (\projectdir> dotnet build)
  • I couldn’t get the debugger to work. I was always left with “No symbols have been loaded for this document” and there is a .pdb file in the bin folder. [Edit: F# for CoreCLR status currently indicates Enable CoreCLR debugging of F# is not started, so that may be the problem.