I have running a kubernetes cluster of 4 raspberry pi’s running Ubuntu Server 22.04 (upgraded from 20.04). When I first installed them Ubuntu 20.04 was the only supported 64-bit os at the time. There are working fine.
I am just curious if there is an advantage of running Raspberry OS Lite 64-bit compared to Ubuntu Server, which may justify a switch ?
The only reason you would want to run Raspberry Pi OS would be the support for all the weird addons, like hats and sensors and cameras and such. If you only use the Pis as mini-servers, then Ubuntu should be fine. Of course, not my first choice, but use whatever you prefer and fits your needs. For kubernetes, it would probably be a downgrade to switch to raspbian.
Are you running the Pis for real services or testing in your infrastructure? Or are you just learning k8s? If the later, you can probably be better served by running k8s on a single RPi, by leveraging LXD. You already run Ubuntu, you can just make 4 LXD containers, enable Nested Containers and install k8s in the containers. For learning, it should be more than enough and you can repurpose your other Pis. Or sell them, you probably will get more money on them than you actually paid for them, because everyone is looking to buy them and will pay upwards of $200.
I use kubernetes for both. Learning experience, therefore I have installed the vanilla k8s (in stead of the microk8s). I have running some containers for home use on it. Of course I could run those without any issues on my Synology NAS docker app, but I like to my education to do real world examples.
K8s configuration is hardware agnostic, so configuring k8s on 4 RPis or on 4 LXD containers is the same thing, really. 
But it does have that nice feel to it when you run things on bare metal, doesn’t it? The reason why I prefer the method of putting k8s in LXD is because k8s has a hard limit of somewhere between 150 and 200 containers / pods running on a single worker node. Even if the worker node has 64 cores and 2 TB of RAM. So by leveraging LXD, you can remove that limit and make k8s limitless. It is a really good combination. Someone who runs a pokemon go community server for all kinds of documentation, I think run this configuration. They only have 1 big monster server, but they could not run their k8s without converting all the worker nodes into LXD containers.
It does increase the overhead a bit. Resource wise, it is negligible, but you do have to maintain the OS of the containers now, just like you would update the OS on each individual RPi, so you would update the LXD container OS. Not that big of a deal though.
Well, good luck moving forward!
Thanks for the comment. Interesting using LXD containers for k8s. I will have a look at it… Btw I have documented my kubernetes configuration on my blog https://www.debontonline.com/p/kubernetes.html if youŕe interested.
Cool, but I think your documentation could use a “next” button after each article, as at least a link to the next page at the last line, instead of having to go back to the “index” to go to the next page. Unless that is actually a thing with JS enabled (I use noscript) that I can’t see.