That’s precisely what I was talking about. I basically said it’s better to split an application by type of parallelism (CPU bound vs I/O bound) than to mix them.
An I/O heavy service benefits from having lots of available threads mapped to a smaller number of CPU cores, whereas a calculation heavy service benefits from pinning threads to cores to limit context switching. So scaling each will be quite different.
If you separate them into separate processes (one for I/O and one for compute), it’s much easier to scale them separately (more machines and whatnot). If I combine them, I’d need to continually balance how cores are split between concerns, and I wouldn’t have as much control over the types of cores (I/O is happy with lots of generic cores, whereas compute would benefit from specialized instructions).
So that’s my practical application of the “separate thread pools” idea, splitting thread pools at the process boundary is usually useful as an application grows in complexity. This increases latency, but it enables other types of tuning.
That’s precisely what I was talking about. I basically said it’s better to split an application by type of parallelism (CPU bound vs I/O bound) than to mix them.
An I/O heavy service benefits from having lots of available threads mapped to a smaller number of CPU cores, whereas a calculation heavy service benefits from pinning threads to cores to limit context switching. So scaling each will be quite different.
If you separate them into separate processes (one for I/O and one for compute), it’s much easier to scale them separately (more machines and whatnot). If I combine them, I’d need to continually balance how cores are split between concerns, and I wouldn’t have as much control over the types of cores (I/O is happy with lots of generic cores, whereas compute would benefit from specialized instructions).
So that’s my practical application of the “separate thread pools” idea, splitting thread pools at the process boundary is usually useful as an application grows in complexity. This increases latency, but it enables other types of tuning.