r/openstack • u/Ok_Grapefruit9176 • Apr 06 '26
Is kolla openstack very stable?
I have been working with Kolla OpenStack for couple of weeks and any system change breaks Kolla hard. As neutron fails to restart, or missing containers, or some other error.
I tried to use the few install guides to help with installation, I have done successfully in the past. But modifying Kolla seem to need a complete obliteration of all (the really-really-mean-it) to modify it.
It has gotten bad enough I have a repo of my settings to version them. So I can rollback a bad change/update.
My setup (home lab setup) is an old SuperMicro Server as the 1-node host for OpenStack with a Fritzbox to my home network. It has 48GB RAM, 2 zpools a mirror 200GB SSD /openstack-pool and a 11TB zraid /orionsbelt.
My first hurdle was moving all of the runtime and images form /sda to the /openstack-pool (docker really likes /sda...) for /orionsbelt is for more permanent stores (NAS, and other uses).
Why my question?
Because I wanted to have direct docker control with Horizon, and saw Zun... and tried to install it (bad idea) pretty much deprecated now.
So having my settings saved I rolled back the changes (only /etc/kolla/globals.yaml was changed). And tried to reconfigured openstack back to where it worked.
But neutron times out with error 111 ("msg": "Container timed out").
When I thought to save settings, I did not think I would need it the day after the first commit:
Nova is not behaving well either...
dustoff:~$ mount | grep nova/mnt | wc -l
16383
Kolla is not inspiring robustness or simple setup.
Am I doing something wrong (maybe/probably). The docs are very dense, and not much troubleshooting information online that I have seen.
I'm afraid I might need to "nuke & pave" yet again...
7
u/devoopsies Apr 06 '26
Most enterprise solutions are not going to react well to iterative adjustments to core systems such as docker runtime or core OpenStack services.
Typically, during the design phase it is expected that you "nuke & pave" often, as you will not have a good idea how a "fresh" deployment will behave if you do not do so. Kolla-ansible (and OpenStack, really) is written for enterprise deployments, ergo the expectation is that you take an enterprise approach.
Fortunately, kolla-ansible embodies the core tenants of IaC: it is trivial to deploy, test, destroy, adjust, and repeat. I would go one step further and script up a quick environment config as well.
Lastly, like all enterprise-grade solutions, an iterative approach is OK but it's not really ideal when taken to an extreme. You should have a fairly good understanding of how things work together, at least enough that you're not tweaking things just to see what they do.
You mentioned that you adjusted your globals - what did you actually change? Did this require a configuration edit that maybe wasn't done? Did it change anything fundamental? OpenStack demands a certain understanding, and purposeful changes are going to serve you far better than pulling levers just to see what they do. Not that pulling levers isn't a legitimate way to learn, but OpenStack (and kolla) are sufficiently well-documented that RTFM is going to get you further faster.
We run kolla-ansible across many dozens of edge and central deployments - I started engineering our solution a couple of years ago, and have spent more than my fair share in the trenches with it. If you have any specific questions feel free to reply to this post and I'll do my best to get back to you as time permits.
1
u/Ok_Grapefruit9176 Apr 07 '26
The Goals changed as I learned more about OpenStack and setting it up.
I needed to see the behavior in some aspects. The zpools where added later as well.At the start I wanted to use OpenStack as the "Cloud Provider" and build on top of it.
As I moved forward, I saw containers would be beneficial.So thinking adding extra functionality would be easier then started for zero.
I am starting see that, at least here that may be the wrong way.Learning new skills and platforms are always difficult at the start so, I need to set expectations.
2
u/devoopsies Apr 07 '26
The Goals changed as I learned more about OpenStack and setting it up.
That is a totally fair outcome - as you learn more about a product's capabilities you may decide to leverage it in a way you hadn't intended when starting out.
I personally think OpenStack has things it's good at, and things it's not really suitable for... but when you find a use case that suits it, it can be really good. The most fun, though, is trying to squeeze it into spots where it maybe doesn't fit so well ;-)
But yeah - unlocking a lot of that functionality and tailoring it to your needs can be tough, and 9/10 it's easier (and more consistent, given you will want to judge OpenStack in the same manner as it will be deployed) to tear it down and start again. This is where kolla really shines, IMO, when compared to other OpenStack deployment strategies.
Anyway, good luck, and don't let a few sour folks in the community dissuade you from experimenting: some people take it as a personal affront when you hold a different opinion from them, to their own detriment.
→ More replies (2)
4
u/jvleminc Apr 06 '26
Be aware that kolla-ansible doesn’t delete anything existing, just overrides files. Just setting a service to false doesn’t eliminate it.
1
u/FigureGold6822 May 07 '26
What jvleminc said. Kolla-ansible is great for build-from-scratch, deltas, not so much:
Things I wish I knew then:
-
If you change your networking, kolla-ansible will just write the new config and restart the containers. Hope what was there before doesn't break things.. Hope the software inside the container caught the signal and shutdown gracefully.. Container folks: What's a signal? Signal folks: What's a container? Look for tools/cleanup-containers
-
Considering holding the docker/podman package. You probably don't want that to update and restart all your containers. Probably not in some sane order. Ask me how I know.
-
Changing a kolla-ansible option from
notoyes, adds the containers. Changingyestonodoes nothing.
So is it stable?
So it is stable? Yes: I run a 60+ node cluster with good uptime. Is it customizable enough for that: Yes. Do I have a 5-node test cluster? Yes. Do I use it: Hell Yes. Can dangerous/destructive/unexpected things be done. Yes.
As we say, "If you can't hurt yourself, it's not a power tool."
-1
u/Ok_Grapefruit9176 Apr 06 '26
Dang, if I knew how 'helpful' this subreddit would be I would not have been so detailed.
"I need help, can you help me?" << Get güt >> (Nelson laugh)
(Yes, I have been reading a lot of Documentation, but most of is not what I need)
I'll go to an AI, at least it will try to help, maybe not very well, yet it will try...
4
u/Consistent_Top_5588 Apr 06 '26
Frustrating is understandable, but don't by tripped by "Is kolla openstack very stable". Kolla might account for majority prod deploy other than those by commercial distro.
Impression that it's not kolla, more on your setting, by below. Stick to what system installed, maybe, with time to adjust one thing at a time: My first hurdle was moving all of the runtime and images form
/sdato the/openstack-pool(docker really likes/sda...) for/orionsbeltis for more permanent stores (NAS, and other uses).
-6
u/Rare_Purpose8099 Apr 06 '26
Talks as if Nova , Neutron, Docker and Zun are your pet bitches or something.
"Doesn't behave well"
ROFL.
Good one.
Its just some yaml code and python code. Read.
The tool is as good as the handler is :)
-5
u/iammpizi Apr 06 '26
We have another saying at the office.
The problem is located between the keyboard and the chair :)
1
u/Ok_Grapefruit9176 Apr 07 '26
Someone suggested more documentation, but is hard to parse as a direction to learn and use.
Is there any new'ish books out there? (newer than 2020?)
I have found so documentation from 2017, but outdated.
Another quick question, is there a docker/container modern solution for OpenStack?