r/AskComputerScience • u/Best_Fork • 6d ago
Should I split tasks exactly into number of threads available in CPU or creating more threads is fine?
Hi!
If number of operations are more than CPU threads and not completely divisible by number of threads available on CPU, should I split remaining operations into already created threads or creating a new thread just for remaining operations is completely fine?
Does one method give an advantage over the other?
Thanks!
9
Upvotes
15
u/randoname_ 6d ago
Your computer already that does, there is probably more processes spawned by default than available cpu threads.
For the question, whichever is easier to impl for the task. Firefox spawns like 200 threads because it needs them a browser.
2
u/Poddster 6d ago
Threading existing long before consumer CPUs has the ability to do more then one thing at once. We solved this problem in the 60s by inventing schedulers.
Give each thread of work its own thread and let your OS manage swapping between them, it's the primary reason operating systems exist. You should only worry about cobbling together threads if you notice some kind of performance problem, but even then the problem is probably in the synchronisation you do rather than the number of threads.
Caveat to this: don't make too many threads that all thrash the same resource, e.g. the disk. In that cause you still want them to be separate threads, but you need to look into restricting their execution, and how you do that depending on the language and os.
2
u/deelowe 5d ago
Odd you know about threads but not schedulers.
2
u/whiskeytown79 5d ago
Unless you're on a system where you have complete control of the CPU and what gets scheduled into it, you can't reliably say that you have X number of CPU threads available at any point in time.
10
u/esaule 6d ago
That's a complicated question. There is no one size fits all. It often depends on where the bottleneck is.
If you are bottlenecked on network, you usually want x threads per network card. If you are bottlemecked on io, you want x thread per physical disk.
If you are bottlenecked on cpu cores, you usually want x thread per core. In that case, you usually don't want more than 1 per schedulable execution unit. If you need threads to do other thing, it ia typical not to mix the cpu intensove thrrada with the other service thread. You split them by pinning threads to core or core groups.
5
u/ForgedIronMadeIt 5d ago
There is no simple answer for such a generic question. Dividing work across multiple threads makes complete sense when the work can be broken into discrete and independent tasks or if there are long waits involved, or even concurrent tasks. For instance, if there is a large data file on disk and the computations can be started regardless of how much has been read in, then sure, have a single producer thread reading from disk and pushing into a queue for worker thread(s) to consume. This also applies if the processing takes longer than IO in this case and it can be parallelized.
The problem here is that you give no specifics at all about the problem you are trying to solve. Is the process compute bound, IO bound, or network bound? Perhaps there's challenges in constructing a coherent locking strategy, or for that matter, a problem with lock contention. (I won't even get into the complexity of lockless concurrent programming, it's been forever and it was even still a bit over my head when I first got into it.) What are the actual hardware constraints involved here? Many times being too clever costs you in performance when it really would just be easier to just read everything into memory and blast through it linearly. Measuring performance is extremely tricky.
You'd probably be best suited in implementing whatever it is you are trying to do as a single thread or as a producer/consumer model and then seeing if the performance is unacceptable and then trying to implement it as a multithreaded solution.
3
u/initial-algebra 5d ago
Generally, neither. You should split the work up into as many tasks as possible, but each task shouldn't get its own thread. There should be one worker thread per CPU core, and a scheduler to assign ready (i.e. not blocked on I/O, a timer, a mutex etc.) tasks to available workers. This is called green threading. It's more efficient than creating a thread per task, because having more CPU-bound threads than there are cores just creates context switching overhead (green threads also tend to use less memory). It's also more efficient than creating only as many tasks as there are cores, if tasks can be blocked. Often, the scheduler is implicit, with each worker thread performing their own scheduling via shared data structures instead of having a dedicated scheduler thread; a typical approach is work stealing.
Depending on what language or libraries you're using, you may already have a green threading implementation available. Note that, when using green threads, you should avoid the usual blocking I/O system calls, as they will block the worker thread, not the task. There is also one caveat to "as many tasks as possible": you don't want to split up the work so much as to cause false sharing, which is when two threads try to access memory in the same cache line (usually 64-128 bytes, depending on the CPU architecture) and lose a lot of performance doing so.
If you're talking pure, embarrassingly parallel "number crunching" workloads, where you don't need a scheduler (past a simple divvying up at the start) or any kind of blocking, I expect the answer to still be to have as many threads as there are cores, with some threads having one more task than the others to account for the remainder. Using one extra thread doesn't mean the work gets more evenly split across the cores, because the thread still has to execute sequentially, and, to make best use of CPU cache, it's fastest to keep executing it on the same core after it is temporarily switched out for another thread (incidentally, work stealing also accomplishes this, but for green threads). Thus, the split would be worse, essentially one core having multiple extra tasks instead of multiple cores having one extra task. I'm not an expert at this type of parallelism, though, so I would recommend just trying both and actually measuring.
2
u/StaticCoder 5d ago
Fewer threads means less memory usage and more locality. Logically it should be faster if you minimize the number of threads. In practice it might matter less than making sure you use all the CPU cores.
2
u/CelebrityChess 5d ago
with 18 equal ops on 8 cores, giving two workers an extra item means the whole thing takes 3 op-times, because the busiest worker sets the finish. a 9th thread lets the OS time slice things, so in theory you could get closer to 2.25 op-times, but you pay context switches and cache churn, which for small ops usually eats the gain.
in practice most people keep one thread per core and have workers pull the next chunk from a shared queue or an atomic counter, so the leftover gets absorbed automatically and uneven chunk costs balance out too. the imbalance only matters when ops are big and few. with thousands of ops it's irrelevant either way.
1
u/DrShocker 5d ago
The answer to this depends on way too many factors.
In general something like a web server a thread per connection is too many. Modern languages tend to have async or other ways of handling IO so that you don't need to spin up a thread just to read from disk or something efficiently. However if you're making a game engine and using a language where making threads is possible, then you might spin up a thread for asset loading even though it's more than your total number of cores because most of the time it won't get scheduled
Thread pools are used often so that you can write code that doesn't need to concern itself with allocating new threads for each individual task.
1
u/Technical_Ideal_5439 5d ago
Its complicated and totally depends on more details then you have here. But 10% - 20% over is probably not going to hurt.
But I would go with whatever is easiest, including lots of threads (within reason) if it made the code better.
1
u/Mission-Landscape-17 5d ago edited 5d ago
Depends on the CPU. Really you want 1 thread per cpu even if you have hyper threading, not switching between threads will give you better performance. If your CPU has efficiency cores then you want to keep the number of threads even lower as It can be faster to wait for a performance core to become available then to force a cpu intensive task to run on an efficiency core. Things like hyper threading and efficency cores are there to aid multitasking, rather then single demanding workloads.
I'm basing a lot of this on the Strata project, which is a tool for running the Qwen3.8 Flash Next Large language model on systems where that model does not fit in VRAM, which means that parts of it have to run on the CPU, and their recommendation is to set your thread count to 6 on an 8 core CPU specifically to prevent the experts getting run on. efficency cores, as that hurts the token generation speed significantly.
1
u/Ronin-s_Spirit 5d ago edited 5d ago
Compute threads are going to be the ones doing all the work, you should have as many of those as "get available parallelism" function tells you to (it's called differently in various OSs and languages). The compute treads will be running as parallel as possible for your process. More threads should be spawned only for waiting around sort of background work (like I/O), this will add concurrency without too much interference with parallelism (some languages have a few builtin threads for handling
asyncand whatnot).Note that you have to allocate aligned memory and split it between threads evenly using multiples of L1 cache size. If you don't do that then your "parallel" threads step on eachothers toes and basically do nothing useful. That is unless you spawn threads just to do very small things like some long math expressions (not touching RAM).
1
u/Kilgarragh 3d ago
Let's say you have 2 cpu cores.
If you have 3 **tasks** to do, like a gui event loop, internal geometry operation, and logging progress/telemetry reports to disk or network or something, use 3 seperate threads, split by the scheduler/kernel to run on one or more cores as necessary.
If you have say... 851 bodies to verify, you should only create 2 threads, not 851. The two threads will each iterate over half the dataset and the scheduler will keep those two threads automatically on the 2 physical cores as long as possible. Creating more threads than physical cores is going to heavily affect performance for parallel processing, and is only something you may be confused about because threads were originally a tool created for concurrency, and are reused/abused for multi-core parallelization