Hi! I´m using Proxmox for some time now and I really like it. I use it for some virtual machines but mostly for lxc containers which are always my first try. I worked with docker a few times but have not gone deeper with it because lxc-containers are my preference till now.
Now I´m wondering in which situations you prefer lxc or docker and how you run docker in your lab? For me it should run on my proxmox server, but directly on the host? inside VM? as privileged LXC?
Also I encountered some problem trying to run docker in lxc on zfs which seems a litte special…
I´m excited to hear your opinions and stories…
I think this slide from Mark’s presentation on Container Camp 2016 explains a lot in a single image, and makes a lot of sense to me:
Video link: Why we need a different container purely for apps - Mark Shuttleworth (Canonical) - YouTube
To sum it up short:
- LXC/LXD are “machine containers” with a persistent filesystem that works like a VM
- Docker are “process containers” with an overlay filesystem over a static image (with options for persistent storage)
- Snaps are “application containers” that directly extends functionality of the underlying host
I mostly use Docker for all the different web services on my home network (UniFi, Grafana/Prometheus, Bookstack, Nextcloud, Syncthing, Transmission, Proxy Manager, Iperf3 etc.), since Docker is especially well suited for isolating data and exposing ports for different services.
If I would ever need “VM” like functionality (where I needed to install custom packages that would persist in the container), I would take a look at LXC/LXD.
I have mixed feelings about snaps, and I think that depends entirely on your usecase. For those apps that are officially distributed as snaps, they might be worth it (for instance LXD or Multipass, which are both official Canonical snaps).
I know you didn’t ask about snaps, but I mentioned it anyway - sorry! 
Hope that helps a bit. 
Thank you very much for your answer. Snaps are indeed something to consider also in this scenario, so very good to mention them.
I think there is no right or wrong and the decision which to choose also depends on many factors. MariaDB for example: if I just need an instance for some dev purpose like running tests or to satisfy the need of an application for some tables and rows, docker is probably the easiest way, especially if the application is dockerized as well. But if I have a multi-gigabyte database with special settings to the datafile storage a lxc-container or even a VM may be a better choice where I can tune the underlying filesystem.
But then there are cases where I´m quite puzzled which option to choose. For example if I would like to run my own nextcloud. docker, lxc and vm are all valid options. I do not see an anvantage for a VM over a lxc container (but maybe there are) and a lxc container with all dependencies and an additional data volume “feels” better to me than a bunch of docker containers. But maybe this is because of my limited knowledge/experience…
This is what I would like to hear from other people out here… See what your opinions are in those situations, what would you coose and why.
I also don’t think there are any definitive answers - much comes down to personal preference.
However, to add a bit to the image I posted, I also think that resource utilization comes into play. In my opinion as a general rule, VMs/containers go from “most resource demanding” on the left (KVM) to “least resource demanding” on the right (snaps).
What I mean by resource usage in the above is a combination of both CPU, RAM and disk space usage - and only a general rule of thumb.
I personally would always prefer the least resource demanding option, if that option includes all the possibilities I would expect. I think that’s the reason I user Docker, because for the apps I want to run, snaps does NOT provide a suitable solution, hence Docker is the next best from a resource perspective.
I could run all my service in LXD also, but that would:
- Require more resources
- Be more difficult for me to manage
So I mainly stick with Docker as long as there are suitable and safe Docker images for the apps I require. And if I have an application that demands more than Docker can achieve, then I would look into LXD.
Absolutely, resource usage is the reason why I use VMs only as a last resort 
May I ask you if you use proxmox, xcp or something else?
Sure - my situation is a bit different than you, since I don’t use either Proxmox or XCP.
I run Docker containers (and potentially LXD and KVM should I choose) directly on Ubuntu server 20.04.
So my situation is a bit different, but I think the general discussion about which technology to use is equally valid, no matter which platform is used as the base system. 
Thank you very much.
Proxmox (which is currently my prefered host) is based on Debian 10 and uses KVM and LXC for VMs/Containers. But it does not directly support docker containers, so there is a bias hidden… docker is not as comfortable to manage in this scenario than the other ones 
That makes sense. And for me it may be the opposite, since I already have Portainer and other parts of the ecosystem set up for managing Docker containers.
So for me the bias may be towards Docker, since I already use that and are comfortable with it.
I think those are also very valid deciding factors - the key here is “whatever works for you”. Cheers! 
