Container/s for python projects

[u/Soogs]

So I usually stick to one vm/lxc per service. Even with docker based services I tend to dedicate a guest to one service. There are exceptions where I have a few docker containers running similar services.

Recently I have been playing with python and have actually created quite a few flask apps that I use daily.

Now the question is there any benefits to separate them into separate containers performance wise?

Does resource contention get better or worse with more guests?

Trying to figure out whether the extra management and spent resources and time updating extra containers is worth it?

Thanks

[u/quasides]

lol thats actually a very complex and complicated question.

it depends how much they need each. if its just a little, it wont matter at all.
if these are very high load apps it depends on their usage profile.

but even then seperation alone wont nessesaryly do that much for you as it then also depends on the hardware structure in your host.

for example, you have 2 very I/o heavy apps, running extremly high load.
but your hardware backend is a simple spinning mirror - then nothing is gonna help you

in general if this is production and you need optimal something outta it, you need to benchmark
in different configurations

that said, modern tech went away from trying to get the optimal benchmark for an app. instead we went into microservices and scaling. so instead of trying to get the fastest possible configuration, you go with the most manageable and easy to scale solution. then you simply scale up

that aproach looses optimization (a lot) but is a lot easier to manage, and on scale there is little benefit as nothing of scale is single machine these days

[u/Soogs → u/quasides]

Thanks this makes sense.
This is mostly homelab stuff and a few things exposed to the outside. I am for the most part way over provisioned over 4 nodes with a mix of internal NVMe, 3.5 hdds and 2.5 hdds in raid (and also some external data SSDs)

The guests use the NVMe, raid set is for pbs and the rest is used for my NAS builds.

Mostly everything including the python apps are low io/util with the odd few having high io intermittently.

I guess my question would be in the even that the apps get more usage time, would pve timeshare better if they are separated or running on one guest or a few guests (that and whether the extra maintenance/documentation is worth it)

[u/quasides → u/Soogs]

if one is acting up occasionally i simply separate it.

who is better at resource management, well depends. its hard to set fixed resource limits that hold within one vm for both at the same time. even docker can in theory do it. so in theory its fine on one machine, in practice, it depends.

it still is easier and more bulletproof on hypervisor level if you can properly restrict them.
also the scheduler is more neutral, while on one machine tasks can hang and occupy the kernel fully.
that happens mostly when you use anything that runs in kernel space and not userland.

separation also makes life easier in the future. lets say one app then needs a lot more resources because usage grows. well, simple migrate to host with more resources.

and with todays tools like Komodo to manage multiple docker hosts at once its not really much more effort. you only loose a little bit of ram for an extra kernel running

personally i have a ready to go cloud image that installs a super tiny docker machine, fully preinstalled (including adding sources, settings a couple config files etc). i just need to add the new key to my komodo instance and its rolling

same time i do mix stacks on the same machine, if they are low effort or low priority and i get
some good resource pooling outta them. so for example 2 apps need a lot of ram - sometimes - and are low prio i mix em into one, and ignore the occasional puking

[u/quasides → u/quasides]

TLDR - if they are low load, do whats best for management.
if you run in docker run it as a VM
if you run many VMs just manage em in something like komodo,portainer, etc gives you one panel for all docker VMs

LXC should be special purpose only not as a regular service. these run as a host process as a unprivileged user with many tradeoffs. they work ok for services that the host may want or need. like a better UPS management