You might find yourself locked out of the server if you attempt to connect multiple times… it has happened to me when moving to a separate machine and separate username, such that the path to the ssh key was different and the attempt to log in was blocked.
# Will show you the number of jails
sudo fail2ban-client status
# For ssh is most likely set as sshd
# This command will show you the total banned IP addresses
sudo fail2ban-client status sshd
# Use this command to unban a particular IP if you get locked out of your system
sudo fail2ban-client set sshd unban ip xx.xx.xx.xx
You can create a configuration where root is not allowed to login directly via SSH, but is allowed to login at the console (assuming you have physical access to the host). That might be an acceptable backdoor.