.NET Core Secrets

Securing sensitive configuration information is one of those things that we know as developers is important, but so often gets deferred for more pressing commercial concerns, usually because of our confidence in the security infrastructure of our environments e.g. firewalls, VPNs. However in the field I’m heading into, security of personal information is important and expected to be audited, so with that commercial consideration in mind today I decided to tackle the challenge of securing configuration.

Secret Manager

There is a lot of chatter around the .NET Core Secret Manager but there appears to be two problems with it. Firstly, it is not a trusted store: “The Secret Manager tool does not encrypt the stored secrets and should not be treated as a trusted store. It is for development purposes only. The keys and values are stored in a JSON configuration file in the user profile directory.”. Secondly, and more significantly I believe, it is user specific. That means that each user has their own credentials.

When I set up a development environment for a team I want it to be as uniform as possible for the whole team. A uniform environment makes it easier for team members to help each other, and makes it easier to script tools for automation. And many development resources will be shared, such as an AWS test instance.

Finally, this doesn’t help with production. For production the above website suggests using environment variables. Such variables are almost certainly stored in plaintext somewhere – in my case in Elastic Beanstalk configurations. Storing in plain-text is insecure and if nothing else is going to be a black mark on a security audit.

Extending .NET Core Configuration

What I want is sensitive information to be stored in an encrypted file where the encrypted file and the key are stored separately i.e. at least one of those is not checked into the source repository. I also still want to have different configurations available for different environments. It is also important that the file is relatively easy to modify.

What I propose is a custom configuration provider that is inserted into the ConfigurationBuilder which processes the other settings file when it is instantiated. The concept is outlined in this extension method:

public static IConfigurationBuilder AddEncryptedAndJsonFiles(this IConfigurationBuilder builder, string fileName, string basePath, bool optional, bool reloadOnChange = false)
    string jsonFilePath = builder.GetFileProvider().GetFileInfo(fileName).PhysicalPath;
    var encryptedConfiguration = new EncryptedConfigurationSource(jsonFilePath, basePath);

    return builder
        .AddJsonFile(fileName, optional, reloadOnChange)

UpdateStoredSettings() will look through the appsettings file for keys starting with SENSITIVE_name. It will then add the name and corresponding value to the encrypted file and remove it from the appsettings file. The ConfigurationProvider returned by the IConfigurationSource.Build method will read the encrypted file and return a data dictionary of keys and values. The location of the key file will be set in the appsettings and read by both the source method and provider.

The extension method above will allow a simple replacement of AddJsonFile with AddEncryptedAndJsonFiles leaving Startup like this:

var builder = new ConfigurationBuilder()
    .AddEncryptedAndJsonFiles("appsettings.json", configBasePath, optional: true, reloadOnChange: true)
    .AddEncryptedAndJsonFiles($"appsettings.{env.EnvironmentName}.json", configBasePath, optional: true)
Configuration = builder.Build();


The implementation requires three classes as is standard for configuration providers:

  • a ConfigurationProvider which writes the properties into the dictionary used by consumers of configuration;
  • an IConfigurationSource which is the factory for ConfigurationProvider and where I opted to put the pre-processing method; and
  • an extension method for convenience.

The implementation uses AES for encryption. I considered deliberately using a slower method, but had trouble finding documentation and examples specific to .NET Core for symmetric encryption (as opposed to password hashing which is where those algorithms tend to be used).

Unlike the appsettings.json, the encrypted settings are stored in a single flat object, with the key being that used for configuration lookups e.g. configuration["parent:child"]. If a matching setting is found then it will overwrite the old one, allowing settings to be repeatedly updated.

One delightful problem I had was that the default IFileProvider refused to resolve paths above the base path, making it impossible to use for a relative path pointing outside the repository. As a result I had to pass in the base path, which feels like something of a hack.

A gist with the full source code can be found here

Perils of AddDbContext

The following snippet is the suggested approach to injecting an Entity Framework context in ASP.NET Core, taken from the docs page on Dependency Injection in ASP.NET Core:

services.AddDbContext<SqlContext>(options => 

This means that whenever you have a constructor that includes SqlContext, the runtime will provide an instance of SqlContext without the developer having to type new anywhere.

Dependency injection in ASP.NET Core comes in three flavours: Transient, meaning a new object (e.g. SqlContext) is called each time a constructor requires the dependency; Scoped, meaning the same object is used for all dependencies of that type during the current web request; and Singleton, where only one instance of the object is ever created. The default for AddDbContext is Scoped meaning that during a web request all classes will be accessing the same SqlContext.

When Entity Framework fetches a record from the database it caches it so if the application requests it again it returns the results from its cache. If someone else updated the database during the request, their change won’t be seen by the first caller. For short and stateless web requests this seems like a reasonable optimization. While technically the data you’re seeing is inconsistent, it was valid a few milliseconds ago, so any permissions that might have been revoked were valid then, and if you try and write anything then optimistic concurrency will alert you to a problem.


However care must be taken as it is very easy to end up getting cached values when that wasn’t intended.

If the dependent object is a singleton, then it will receive a context instance when the object is first created, and hold onto that context until the server recycles. During that time anything it reads will be cached in the context and any changes to the database ignored. This is particularly problematic where there are balanced web servers where the singletons on two different servers will be unaware of the changes the other made.

This approach is also quite opaque. It is not obvious from the code, compared to say a using statement, the lifetime of the context and therefore what side-effects might occur among the many classes in a given web request using this context. This is true of any object with scoped lifetime and is why I tend to avoid using scoped.

While not really a problem with its usage, that we injecting a concrete class rather than an interface doesn’t really follow the basic pattern of dependency injection. It is not terribly difficult to create an interface for the context, but as noted earlier, it is not the standard being documented. I assume the reason for this is that the testability problem with injecting a concrete context has been removed by the introduction of DbContextOptionsBuilder.UseInMemoryDatabase().

A Solution

Frankly I prefer more control and transparency so I’ve returned to the time-honored tradition of having a data factory so that my classes can create a database context when they want to. To do this the factory uses the IoC container to create contexts on demand.

public interface ISqlContextFactory { SqlContext NewContext { get; }}

public class SqlContextFactory : ISqlContextFactory
    private Func<SqlContext> _getSqlContext;
    public SqlContextFactory(Func<SqlContext> getSqlContext)
        _getSqlContext = getSqlContext;
    public SqlContext NewContext { get { return _getSqlContext(); } }

This could be done by passing the IoC container into the factory, but I chose to pass the factory a function which generates the contexts, leaving options configuration in Startup.

Edit 16-March-2017: The original code source caused problems with the mysql connector similar to those discussed on Connection reuse here. The following code has been updated so the options are set for every request, rather than just once at startup, and this appears to have fixed the issue.

services.AddSingleton<ISqlContextFactory, SqlContextFactory>(provider => {

    string mysqlConnStr = Configuration.GetConnectionString("Mysql");

    return new Data.SqlContextFactory(() => {
        var options = new DbContextOptionsBuilder<Data.SqlContext>();
        return new Data.SqlContext(options.Options);

By removing the AddDbContext we have broken migrations and fixing this requires a little unsavory use of statics. The dotnet ef command line seems to run the Startup class but does not use the dependency injector, so instead I created a little class just for the migrations to use, and set the static connection string in Startup.ConfigureServices() so that the migration class doesn’t have to repeat the configuration loading code.

using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;

/// This is ONLY for use by dotnet command line tool
public class SqlContextMigrationsTarget : IDbContextFactory<SqlContext>
    public static string ConnectionString;
    private DbContextOptionsBuilder<SqlContext> _builder = new DbContextOptionsBuilder<SqlContext>();

    public SqlContextMigrationsTarget() {}

    public SqlContext Create(DbContextFactoryOptions options)
        return new SqlContext(_builder.Options);

// and in Startup.ConfigureServices()
SqlContextMigrationsTarget.ConnectionString = Configuration.GetConnectionString("Mysql");

Tidier Controllers with Request Filters

All of my ASP.NET Core controllers are starting to follow a certain pattern (as follows) and in the spirit of good DRY code, I’d like to define this in a single place.

    Logger.info ("TemplateController.Get(" + id.ToString() + ")")
    // Actual logic
| _ as ex -> Logger.error (sprintf "%O" ex); this.BadRequest() :> IActionResult

Furthermore, I’ve also noted that ASP.NET Core will happily provide me with an invalid model, which means I then have to do an extra null check. It’d be nice to handle all these scenarios transparently for all controllers.

ASP.NET Core provides a solution for this via filters, which exposes interfaces that can be implemented to intercept requests during the processing pipeline.

Exception Filter

An exception filter can be used to handle exceptions. An exception filter class must implement IExceptionFilter which includes a single method, OnException (context: ExceptionContext).

Unfortunately the setup for an exception filter goes in the Startup.ConfigureServices while the global logger factory is available from Startup.Configure. So using the application logger factory requires an un-F# hack as follows:

type Startup(env: IHostingEnvironment) =
    let mutable _loggerFactory : ILoggerFactory option = None

    member this.ConfigureServices(services: IServiceCollection) =
        let mvc = services.AddMvcCore()
        mvc.AddMvcOptions(fun mvcOptions -> mvcOptions.Filters.Add(new GlobalExceptionFilter(_loggerFactory.Value)))

    member this.Configure (app: IApplicationBuilder, loggerFactory: ILoggerFactory) =
        _loggerFactory <- Some(loggerFactory)

In my case I’m happy using my global logger, so my GlobalExceptionFilter won’t take any arguments. And here it is:

type GlobalExceptionFilter() =
    interface IExceptionFilter with
        member this.OnException (context: ExceptionContext) =
            Logger.error (sprintf "%O" context.Exception)

Action Filter

The action filter needs to do two things:
1. Log calls
2. Prevent invalid model state from reaching the actions.

Action Filters implement either the IActionFilter or IAsyncActionFilter interface and their execution surrounds the execution of action methods. Action filters are ideal for any logic that needs to see the results of model binding, or modify the controller or inputs to an action method. Additionally, action filters can view and directly modify the result of an action method.

The aim here is to check and log the state before the action method is called so only the before-action method, OnActionExecuting, needs to be implemented. This implementation checks if the model state is valid, and if not logs the error and terminates the request with a 400 error without the actual action being executed. Where the model is valid, it logs the parameters.

type GeneralActionFilter() = 
    interface IActionFilter with

        member this.OnActionExecuting (context: ActionExecutingContext) =
            if not context.ModelState.IsValid then
                let errors = 
                    |> Seq.collect (fun (value: ModelStateEntry) -> value.Errors)
                    |> Seq.map (fun (modelError: ModelError) -> sprintf "%s" modelError.Exception.Message)
                    |> String.concat "\n\t  "

                Logger.error (sprintf "Called %s. Error: Invalid model state\n\tException messages: \n\t  %s" context.ActionDescriptor.DisplayName errors)
                context.Result <- new BadRequestObjectResult(context.ModelState)
                let args = [ for kvp in context.ActionArguments -> sprintf "%s %A" kvp.Key kvp.Value ] |> String.concat "\n\t"
                Logger.info (sprintf "Called %s with: \n\t%s" context.ActionDescriptor.DisplayName args)

        member this.OnActionExecuted (context: ActionExecutedContext) = ()

The end result is much cleaner controller methods:

    member this.Get(id: System.Guid) : IActionResult =
        match GetTemplate id (new TemplateRepository()) with
        | Some template -> this.Json(template) :> IActionResult
        | None -> this.NotFound() :> IActionResult

    member this.Create([<FromBody>]template: Template) : IActionResult =       
        CreateTemplate template (new TemplateCommandHandler())
        let url = new UrlActionContext (Controller = "Template", Action = "Get", Values = new RouteValueDictionary(dict [("id", box template.Id)]))
        this.Created((this.Url.Action url), "") :> IActionResult

Filters are added to the MVC pipeline in the Startup.ConfigureServices method:

member this.ConfigureServices(services: IServiceCollection) =
        let mvc = services.AddMvcCore()
        mvc.AddMvcOptions(fun mvcOptions -> mvcOptions.Filters.Add(new Api.Filters.GlobalExceptionFilter()))
        mvc.AddMvcOptions(fun mvcOptions -> mvcOptions.Filters.Add(new Api.Filters.GeneralActionFilter()))
        mvc.AddJsonFormatters() |> ignore


As I noted in an earlier post I don’t have a functioning debugger or intellisense, and I’ve come to appreciate just how much time having a debugger and a navigable watch window saves – being able to scan through fields to find something appropriate is much faster and easier than trawling through documentation. An additional challenge is that the docs don’t include inherited members, so you have to open those separately to find all inherited members.