On 5/28/06, Francis Cianfrocca <garbagecat10 / gmail.com> wrote:
> Windows: one is sorely tempted to ask why you're considering Windows for a
> seriously scalable application, but let's finesse it by noting that
> aggregate cost does become a significant factor with scale. So if you have
> to use Windows, you're probably trying to meet a political requirement
> rather than a technical one ;-).

Windows threads are pretty scalable in a SMP system. Was better than
Linuxthreads in my experience.

> More to the point, you don't really want to be forking a lot of processes to
> run a cooperative multiprocess application. Rather, you want the processes
> to be long-running. This does amortize their startup cost (which is large
> even on Unix), but far more importantly it gives you an opportunity to avoid
> context-switch overhead, which can be extremely expensive on modern
> hardware. If your workpile consists of long-running tasks (which it probably
> doesn't), then you don't have to work too hard to get long-running
> processes. Otherwise you need an event-driven sytem to keep them busy (and
> pinned to their respective processors).

When the tasks are runtime generated, we found thread pool is useful.

> Shared memory: no. Don't do that. Use IPC or network communications. No,
> don't do that either. Use a proper event-passing library that wraps all of
> that up for you, so your remote-operation activations look like simple
> function calls. Remember, you'll want to run your multiprocesses on multiple
> machines before you know it. (Avoid distributed objects if possible, because
> for one thing they force you to couple client and server processes, and for
> another you really don't want the management hassles if your network is
> asynchronous.)
>

Shared memory is very convenient for certain workloads. I developed
applications with various programming models like dataflow, event
driven, threading, message passing, and found shared memory
programming frequently outperformed others in both programmability and
performance.

-xiaofeng