I put Windows 11 in a Kubernetes pod on purpose
There is no good reason to run Windows 11 inside a Kubernetes pod. I did it anyway. The reasons were bad ones. I had a node with hardware acceleration sitting idle. I'd seen a container image floating around that claimed it could install Windows from scratch inside a Linux container. And I'd just finished standing up the cluster, so the impulse to test it with the weirdest workload I could think of was hard to ignore.
The result works. It is also, in every way that matters, a worse Windows VM than the one you'd get from running VirtualBox or Parallels on the same machine. The lessons are about that gap.
#What the thing actually is
The image is dockurr/windows. It's a containerized installer that, on first boot, downloads a Windows ISO, spins up KVM (the Linux kernel's built-in virtualization), and installs Windows into a virtual disk. The container runs the VM. The pod hosting the container exposes a VNC port, an RDP port, and an SSH port forwarded to the guest.
The pod's spec is shorter than you'd expect:
spec:
containers:
- name: windows
image: dockurr/windows:latest
env:
- name: VERSION
value: "11"
- name: USER_PORTS
value: "22"
- name: RAM_SIZE
value: "8G"
- name: CPU_CORES
value: "4"
ports:
- containerPort: 8006
- containerPort: 3389
- containerPort: 2222
securityContext:
capabilities:
add: ["NET_ADMIN"]
privileged: true
volumeDevices:
- name: kvm
devicePath: /dev/kvm
volumes:
- name: kvm
hostPath:
path: /dev/kvm
Mount the host's /dev/kvm device into the container so the container can use it. Give the container enough Linux capabilities to set up its own internal networking. Forward a few ports. Pick which Windows version to install, in this case 11.
That securityContext: privileged: true line is the thing I want to flag right away. Privileged means the container is, for most security purposes, the host. It can mount filesystems, manipulate kernel namespaces, do most of the things root on the bare machine can do. In a production environment I'd refuse to run this. In a homelab cluster I run by myself, on hardware I own, for an experiment I'm doing because I'm curious, the math is different. But it's the kind of thing you should know you're agreeing to before you apply the YAML.
#The first failure was the port-forwarding env var
The first time I tried this, SSH on port 2222 didn't work. Not "connection refused," not "auth failed." It hung, then timed out. RDP and VNC worked fine. SSH didn't.
The clue was in the dockurr image's docs, buried in a paragraph I hadn't read carefully. The image has a HOST_PORTS environment variable that I'd set to 22. The docs imply this means "forward host port 22 into the VM."
It does not mean that. HOST_PORTS means "these host ports should NOT be forwarded to the VM." It's an exclusion list. I had been telling the image to exclude port 22 from forwarding, then wondering why port 22 wasn't forwarded.
The variable I actually wanted was USER_PORTS, which is an inclusion list. Setting USER_PORTS=22 forwards host port 22 into the VM. Once I swapped them, SSH worked on the next pod restart.
This is the single most common shape of homelab time-loss, in my experience: a config knob that's named exactly opposite of what it does. Open source projects do this constantly because the naming usually made sense when someone wrote the code, and nobody renames things later for fear of breaking other people's setups. You learn to suspect the names you trust the most, which is a skill that does not translate cleanly to environments where the names are usually correct.
#The second failure was DNS, again
Once SSH worked, the next surprise was that the Windows VM couldn't reach the internet on first boot. The Windows installer needs to phone home for license activation and to download some optional components, and all those calls were timing out.
If you've read the post about my DNS search-domain trap, you know where this is going. The pod inherited the host's /etc/resolv.conf, which included a search domain pointing at a wildcard A record on my domain. The Windows VM inside the pod inherited the pod's DNS via DHCP from the pod's internal network stack. Windows's resolver appended the search domain to every hostname it tried, hit the wildcard, got back my reverse proxy's IP, and gave up.
The fix was the same fix I'd applied to six other pods: dnsPolicy: None and a dnsConfig block that explicitly listed cluster nameservers and a sane search list. The pod re-rolled. The VM rebooted. Windows finished its setup.
#What it's actually like to use
The VM works. SSH in, get a shell. RDP in, get a desktop. VNC in, watch it boot. I can run Windows-specific tooling that I sometimes need for testing, without having to find a spare physical machine or boot a separate Mac partition.
It is also, to be clear, a worse experience than any of the alternatives. The disk performance is mediocre because there are two filesystem abstractions stacked between the guest and the physical disk (Windows on the virtual disk, KVM on the container's filesystem, the container on the node's actual filesystem). The graphics are software-rendered, so anything visual is sluggish. Networking is fine for SSH and for most TCP services, but anything latency-sensitive feels off.
If I had a real reason to run Windows daily, I'd run it on its own hardware or in a proper hypervisor. What I have instead is a workload I can spin up on demand, run for a few hours, snapshot, and tear down without any of it touching real hardware. For the use case of "I need a throwaway Windows machine for an afternoon," it's actually pretty close to ideal.
#The deeper question
The thing I keep going back to is: why does this work at all? You have a container, which is a Linux process with some namespaces around it. Inside that container, you have a virtual machine, which is a CPU emulator implemented in software. Inside the VM, you have Windows, which is not a Linux thing. Each layer is doing real work, and each layer is making assumptions about what's underneath it that mostly happen to be true.
There's no architectural reason this should be reliable. The fact that it is, most of the time, is a quiet testimony to how much engineering work has gone into making virtualization and containers play nicely with each other. KVM is part of the Linux kernel. Containers are part of the Linux kernel. They share enough internals that the gap between "I have a container that needs to run a VM" and "I have a process that's running a VM" turns out to be smaller than it looks.
Could I run this in production? No. Would I trust it for anything important? Also no. Was it worth doing? Yeah. I now know what's between Kubernetes and bare metal in a way I wouldn't have if I'd only ever run normal pods. And I have a Windows VM I can stand up in three minutes when I need one, which is mostly never, but sometimes turns out to be exactly what I need.
The next experiment is whether I can run an actual desktop environment inside one of these pods with GPU passthrough on the AMD card sitting in my other node. Almost certainly that's another bad idea worth doing.