Scaling an IdentityServer4 service

I think I have worked this out. To resolve my issue, I have done two things:

1. Create my own X509 certificate and shared this certificate between each of my IdentityServer's. There are lots of examples of how to create valid certificates on the net; I just used

   services.AddIdentityServer(...).AddSigningCredential(new X509Certificate2(bytes, "password")
   

   in my startup class.

2. Dug into the MVC framework code and worked out that I needed to implement a Key storage provider in order to share state between different instances of the MVC part of Identity Server which serves up the login page.

It turns out that there is a Redis backed KSP available from NuGet, which means that I just need to spin up a private redis instance in my Kube cluster (which isn't accessible outside of my cluster) to share decryption secrets.

/* Note: Use an IP, or resolve from DNS prior to adding redis based key store as direct DNS resolution doesn't work for this inside a K8s cluster, though it works quite happily in a Windows environment. */  
var redis = ConnectionMultiplexer.Connect("1.2.3.4:6379");
services.AddDataProtection()
        .PersistKeysToRedis(redis, "DataProtection-Keys");

I can now scale my identity service to 3 instances and have a Kube service acting as a facade over all the available instances. I can watch the logs as Kubernetes round-robin's requests between the identity service, and my authentication happens just as I expect.

Thanks to those who commented on the question prior to this post.

For those using Kubernetes, it's possible to use the File System Key Storage Provider

public void ConfigureServices(IServiceCollection services)
{
    services.AddDataProtection()
        .PersistKeysToFileSystem(new DirectoryInfo(@"/app/key-storage"));
}

where the directory '/app/key-storage' is mapped to an nfs backed persistent volume.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-key-storage
spec:
  selector:
    matchLabels:
      type: nfs-pv
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Mi

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv
  labels:
    type: nfs-pv
spec:
  storageClassName: manual
  capacity:
    storage: 10Mi
  accessModes:
    - ReadWriteMany
  nfs:
    server: <server>
    path: /<path>
  persistentVolumeReclaimPolicy: Delete

And in the IDP Deployment

template:
  spec:
    containers:
      - name: <name>
        volumeMounts:
          - name: key-storage
            mountPath: /app/key-storage
            readOnly: false
    volumes:
      - name: key-storage
        persistentVolumeClaim:
        claimName: pvc-key-storage

And you need the signing cert. This can added as a secret, and then the IDP Deployment can use another volume to mount the secret (not shown).

apiVersion: v1
kind: Secret
metadata:
  name: cert-secret
  labels:
    app: <app-label>
type: Opaque
data:
  signingcert.pfx: <base64 cert value>