Hi, I'm hoping someone who is more knowledgeable on multi-threading could share their thoughts here.
I was wondering what the implications were for having a set of threads dedicated to a particular task, versus a larger pool of general purpose threads that are called upon for all/any tasks?
To give an example, say I have a program with a master audio and master graphics thread, both with which I can get better performance by using sub-threads to complete their subsequent processing. What would be the implications of using a dedicated pool of worker threads just for the audio, and a separate pool just for graphics, versus a general worker pool with a combined total number of threads, for both?
I've been imagining many scenarios where there are different implications in terms of architecture, platform, cache usage, and performance requirements, but I'm only really guessing. Some solid guidance would be much appreciated.
Dedicated worker threads versus a general purpose thread pool?
Edited by
Mārtiņš Možeiko
on
I think it all depends on exact use case and there is no generic answer to whether use one or another approach.
I could see that there two different reasons why to use threads. One is to have something always running constantly in background and it should never be interrupted. Example would be audio mixer - you don't want your pathfinding or whatever work to interrupt it. Second reason is to have small work like AI, pathfinding or similar work run in background and give you result. For this you usually don't care much how and when it runs (as long as it runs eventually).
So that would mean that there are some work which you want to run in dedicated thread. And others to run in threadpool. That's the most generic answer I can come up for your situation. Of course once you get into more details then there are lot more choices to make. Like threads with I/O work (disk/network) etc.. Which could require a bit different abstraction. Usually you want to free thread for some other work if it is now waiting on I/O operation to complete. Node.js, for example, does all I/O work in async manner - it requires you to pass callback to functions that can potentially block, and this callback is invoked when operation resumes (file read, file open, etc..)
Here's a presentation that maybe you'll find useful - its about design of multi-threading in Destiny engine architecture: pdf, video.
I could see that there two different reasons why to use threads. One is to have something always running constantly in background and it should never be interrupted. Example would be audio mixer - you don't want your pathfinding or whatever work to interrupt it. Second reason is to have small work like AI, pathfinding or similar work run in background and give you result. For this you usually don't care much how and when it runs (as long as it runs eventually).
So that would mean that there are some work which you want to run in dedicated thread. And others to run in threadpool. That's the most generic answer I can come up for your situation. Of course once you get into more details then there are lot more choices to make. Like threads with I/O work (disk/network) etc.. Which could require a bit different abstraction. Usually you want to free thread for some other work if it is now waiting on I/O operation to complete. Node.js, for example, does all I/O work in async manner - it requires you to pass callback to functions that can potentially block, and this callback is invoked when operation resumes (file read, file open, etc..)
Here's a presentation that maybe you'll find useful - its about design of multi-threading in Destiny engine architecture: pdf, video.
Dedicated worker threads versus a general purpose thread pool?
Hi Mārtiņš, thank you for the detailed reply! I think I should have provided a more concrete example relating to my situation.
I'm referring to the first type of thread usage that you mention - those for real time processing that must be completed within a specific time requirement. The application I'm working on has a multi-instrument, multi voice, multi channel audio chain, so the processing requirements for the audio are quite high compared with the graphics, where most of the processing can be off-loaded to the GPU.
So, I'm processing my audio and graphics in the following way:
Relating to the above example, would it be better to have a dedicated audio thread pool with say 4 threads for audio processing jobs, and then a separate dedicated graphics pool of 2 threads for tasks such as physics calculations etc, object position updates etc? Or would it be better to have a general purpose shared pool of all 6 threads that either the graphics or the audio uses when tasks are required to be processed?
I can hypothesize several advantages and disadvantages for each solution:
Shared Pool:
- Fluctuations in processing requirements can be appropriately handled. I.e, more threads can be thrown at intense audio sections, and vice versa for graphics.
- Easier to respond to different hardware configurations? Especially for platforms that support a wide range of physical cores.
- Context switching probably really bad for CPU cache usage?
Separate Pools:
- Better i-cache and d-cache usage?
- Easier to manage? Easier to profile sub thread usage.
- Harder to configure based on platform/physical available cores?
- Ideally, the audio and graphics processing requirements should be relatively consistent anyway. Unpredictable fluctuations could lead to frame drops and audio stuttering.
I'm only really guessing here, however. The only way for me to really know is to try both configurations, but I wanted to see if someone had some experience with a situation like this.
Thank you for the links too, there's some good reading and watching there, I'm sure they will provide a great deal of insight.
I'm referring to the first type of thread usage that you mention - those for real time processing that must be completed within a specific time requirement. The application I'm working on has a multi-instrument, multi voice, multi channel audio chain, so the processing requirements for the audio are quite high compared with the graphics, where most of the processing can be off-loaded to the GPU.
So, I'm processing my audio and graphics in the following way:
Relating to the above example, would it be better to have a dedicated audio thread pool with say 4 threads for audio processing jobs, and then a separate dedicated graphics pool of 2 threads for tasks such as physics calculations etc, object position updates etc? Or would it be better to have a general purpose shared pool of all 6 threads that either the graphics or the audio uses when tasks are required to be processed?
I can hypothesize several advantages and disadvantages for each solution:
Shared Pool:
- Fluctuations in processing requirements can be appropriately handled. I.e, more threads can be thrown at intense audio sections, and vice versa for graphics.
- Easier to respond to different hardware configurations? Especially for platforms that support a wide range of physical cores.
- Context switching probably really bad for CPU cache usage?
Separate Pools:
- Better i-cache and d-cache usage?
- Easier to manage? Easier to profile sub thread usage.
- Harder to configure based on platform/physical available cores?
- Ideally, the audio and graphics processing requirements should be relatively consistent anyway. Unpredictable fluctuations could lead to frame drops and audio stuttering.
I'm only really guessing here, however. The only way for me to really know is to try both configurations, but I wanted to see if someone had some experience with a situation like this.
Thank you for the links too, there's some good reading and watching there, I'm sure they will provide a great deal of insight.