Openstack instance creation fails on attempt to attach a volume

I am following the Openstack administrators series. I started out with devstack but was told by folks to use kolla-ansible as my base which I did. I am using the wallaby version all-in-one in a virtual machine.

I successfully installed it and got public and private networks working something I struggled with in devstack. I also got through the Terraform video. But unlike devstack, cinder was not automatically enabled in kolla-ansible so I could not do the storage or Openshift videos.

I followed the steps for creating an LVM partition and used lvm as my cinder backend in kolla and then enabled cinder in the globals.yml file. I am now able to now volumes but I cannot create instances anymore. I get the following error when I attempt to create a new instance:

Error: Build of instance d37d1277-683e-41d5-8013-2dce63c3d655 aborted: Unable to update the attachment. (HTTP 500)

The instance is in an error state but the volume is created! I have looked around for a solution to this problem and have found nothing. Can anyone help me with what a possible fix for this is? I am new to Openstack and only have a basic knowledge of docker. Any help would be greatly appreciated. I am really enjoying the series and want to be able to complete it!

Welcome to the forum!

I’m unable to help, unfortunately, I lack both knowledge of OpenStack and have never ran it. I find that unless you are actually running a tenant-hosting cloud infrastructure, then it’s not worth the effort to set up, nor the huge amounts of resources it needs to run, compared to other solutions. It is a powerful software though.

Useless unproductive rant.

In my previous workplace, the previous admin has set up OpenNebula. It is an openstack alternative and direct competitor. It’s much easy to set up, involving just some hosts, libvirtd and installing the opennebula sunstone, onestor, a few other packages and a database. We had sqlite, but I highly suggest using postgres.

Maybe Jay can, or someone who has dealt with OpenStack can help you here, but I don’t know how much of the active LLTV Forum members have. Even on the L1 Forum, most people only dealt with VMWare, XCP-ng, oVirt, OpenShift, portainer, virt-manager and most of us, with Proxmox (the last which is almost plug and play, really easy to install and manage).

I wish you good luck though.


By the way, http 500 means internal server error. Meaning that the error is coming from whatever server you are trying to access. Supposing this is the OpenStack management GUI, then there’s definitely a config somewhere that’s bugged. Keep in mind yaml files do not like mixing and matching spaces and tabs. I suggest only using spaces in the yaml. If you use vim, make sure that you uncheck the auto-tab thing (set nocindent nosmartindent noautoindent).

Thanks for responding and welcoming me to the forum!!

To be clear, I am just going through Jay’s tutorial series on administrating Openstack. I don’t know that I would actually use it but want to know more about it. My all-in-one kolla-ansible implementation is actually running on a Ubuntu 20.04 VM hosted by Proxmox!

I am trying to get volumes/cinder to work because they are needed by Openshift which is one of the applications Jay explores running on Openstack in his series.

I don’t know if I would use Openstack, but I want to try it out. Hence my need to get cinder working. I have been pretty careful about editing the yaml files so I don’t think the problem is there unless I have not included a configuration statement that cinder using LVM needs. Appreciate the tip about vim!

Because I am new to Openstack and Docker, I am not even sure where to look for additional details on the failure. It is weird I can create a volume and creating a volume occurs during instance creation but the “attachment” of the volume to the instance fails.

1 Like

I actually ran into this some time back, and I do remember it being a pain to figure out. Normally, I would try and reproduce this in order to figure out a fix, but unfortunately I don’t have a spare server to set this up on as of yet. I am hoping to get a test-lab set up as soon as possible, but I don’t think it will be done for at least another month :frowning:

Thanks Jay. At least I know I am not the only one who has experienced this!

I am running kolla-anisble all-in-one on a Proxmox Ubuntu 20.04 VM because I did not have a spare bare metal server either.

In doing a tail -f on nova-compute.log part of the error message is this:

Blockquote

2023-04-21 22:18:38.339 7 INFO nova.compute.manager [req-8216d19d-f7a4-43a8-829e-05382657d3d2 d70b2d518b7440959bf8dddfb703a639 e1aed1f219ff486893bb14aa2f0df128 - default default] [instance: 2ad8c961-9c97-4608-94c5-2baf3abf38bb] Attaching volume b86fc978-6484-4c5d-a593-0819f18c973d to /dev/vdb
2023-04-21 22:18:38.477 7 WARNING os_brick.initiator.connectors.nvmeof [req-8216d19d-f7a4-43a8-829e-05382657d3d2 d70b2d518b7440959bf8dddfb703a639 e1aed1f219ff486893bb14aa2f0df128 - default default] Process execution error in _get_host_uuid: Unexpected error while running command.
Command: blkid overlay -s UUID -o value
Exit code: 2

Blockquote

My LVM partition is based on /dev/sdb. Not sure where the /dev/vdb is coming from unless that is internal to Openstack virtualization.

I think I might have come upon a solution to this problem as I can now make instance that have volumes attached to them.

I made LVM my backend for Cinder in the globals.yml file.

Ran:

ansible-kolla prechecks
ansible-kolla pull
ansible-kolla deploy

I had not been running ansible-kolla pull which apparently pulls new cinder images per the documentation. I made two other changes which I need to verify had nothing to do with the problem being resolved. More on that tomorrow when I do that verification.

1 Like

Has that work-around been working for you? I looked at your log output and it doesn’t look familiar, so I’m not 100% sure if the problem I ran into was the same as yours. But I do remember at the time, coming to the conclusion that some sort of required documentation element was clearly missing. (This is yet another reason why I like creating tutorial videos, because they sometimes end up becoming the documentation people needed in the first place).

1 Like

The work-around has worked out. I need to go over your video on storage again, but I am able to now create instances where volumes are automatically attached and create volumes and attach them to an instance afterward. Thanks for checking back in with me!

I do exactly what you say, I use your how to videos as part of the documentation of build out of a tool or service. Then I start either using it in production or play around with it in my homelab.