Can Windows nanoserver run inside Linux docker?

Containers provide process isolation on a shared OS, rather than OS isolation on shared hardware that you get with VMs. Since the OS is shared, the host OS needs to be able to run the desired binaries. You will see this in the architecture of the docker engine and the architecture of the image you want to run, they must be compatible:

$ docker system info --format '{{.OSType}} {{.Architecture}}'
linux x86_64

$ docker image inspect busybox --format '{{.Os}} {{.Architecture}}'
linux amd64

If you try to run an architecture that isn't compatible with your host, you will get an error because the binary format isn't recognized by the kernel:

$ docker image pull --platform arm64 busybox:latest
latest: Pulling from library/busybox
acafde7ce2e7: Pull complete
Digest: sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812
Status: Downloaded newer image for busybox:latest

$ docker run -it --rm busybox:latest echo hello
standard_init_linux.go:190: exec user process caused "no such file or directory"

Desktop versions of Docker, and the Windows Server version of Docker, include a Linux VM under the covers to run Linux containers (Linux containers are the dominant container environment, so Docker implements this using Linuxkit, to simplify the developer workflow). For Windows, there is a switch in the engine to use the Linux VM or run native Windows containers.

However, Docker does not have an embedded VM for Windows to run their binaries on a Linux host (since Windows is not open source and would require licensing), so the only way to run Windows native containers is on a Windows host.

short and simply: no

Containers are about Kernel/OS virtualisation.

It's a group of processes you run distincly in a container hosts that shares the kernel. If that container host is a Linux deamon it cannot share the windows kernel, which the nanoserver image needs