Hello, I have a few HP Servers that use iLO4 for their OOBM.
What I have found is that for anything past Ubuntu 18, the GUI becomes so slow over iLO4 that it’s unusable.
All versions of Windows are fine and Ubuntu 18_LTS is fine, but as soon as you try and do a remote install of late versions it starts.
Any suggestions of what the problem could be from the given info?
Welcome to the forum!
Sounds like hardware acceleration not being present, which would indicate wayland. Out of curiosity, why are you installing ubuntu desktop on HP servers? You should probably be using Ubuntu server instead, with the subiquity TUI installer, which shouldn’t have problems with the acceleration.
After the install is done, you can test it by logging out, selecting the environment in the login screen and changing from Ubuntu to Ubuntu on Xorg.
Yes, sometimes I install a GUI albeit the minimal version on top of my server installs ~ get over it. The problem at hand is finding out the reason iLO4 works on Ubuntu 18 but not the later versions. What changed in that zone of releases that could affect the remote display?
I don’t care what other people really use, was just curious. 
The server option has most of the server stuff, like the openssh-server already there, the desktop doesn’t (although it’s just an apt update and apt install away, I’m sure you already have your installation workflow set up).
Wayland became default, but I’m not sure if wayland is used for the live install. Easiest thing to do would be to try the Live Environment, see if it lags, then try to log out and see if there is any option to log back in, but with Ubuntu on Xorg instead, if you are sent to GDM when you log out. The setting should be on the GDM’s gear icon. That would give you an indication if Wayland is at fault.
I’m not suspecting wayland itself necessarily though. It’a an IPMI, and it’s also maybe an older server that you are trying to install on, it’s possible that the video drivers got nuked in a newer kernel release, leaving you with no video acceleration, which would make both wayland and xorg to be affected by lag in the new release.
One thing you can do in the live environment on the newer ubuntu versions would be to check /var/log/messages and dmesg and try to spot any errors about video acceleration, video memory, drivers, software rendering or anything else.