HPC for workstations is wide open opportunity

Jul 23, 2008 at 3:21 PM
MPAPI and some other "small" MPI-ish message passing implementations work fine on multi-way workstations as well as multi-way servers.

I think it is interesting to discuss that Microsoft's MS-MPI is being designed to work on servers specifically Micrsoft Server 2008.  However Microsoft for their own reasons are not attempting to support MS-MPI on the workstation.

Well, workstations only have, what, 4 CPUs you say?  Sure, but there are a number of workstation applications that can benefit from the local parallel programming model.  There is no going back, workstations are going to have 2, 4, 8, more CPUs and this is the case today already.  And as well, applications could also participate in distributed parallel operations with some amount classical cluster servers.

There is an opportunity now due to Microsoft abandoning message passing on the workstation, to make "the best" MPI for the Windows workstations.

MPAPI might fit that need well, or maybe some adjustments could be made to improve this opportunity.  There are now several "small" message passing frameworks which are suitable for maximizing workstations performance, MPAPI being one of them.  MPI.NET, PureMPI.NET are possible alternatives.  

Which MPI framework will take up the workstation opportunity that Microsoft has chosen to abandon?

MPAPI does not have collective operations, but maybe this is perfectly OK on a workstation which won't have hundreds or thousands of nodes.  You can just program your message passing using singular operations since there are only a handful of processors on a local computer or a small network.

Message passing suits massively parallel applications, but message passing also supports "slightly parallel" applications also.  And there are some of these applications.

I think there are going to be a number of applications designed to work on workstations that could benefit from message passing parallelism. 

Real time camera feeds for example is a situation that an application I am developing could definitely benefit from a dedicated processor for receiving the camera's data.  If I had two cameras my single-CPU workstation would be unable to keep up.  And add the load of analyzing the photos in real time, and saving the interesting photos to disk devices, and displaying the photos on the GUI, I definitely have a need for multiple processors on my workstation and message passing is key to making this work in my system.  Microsoft is unwilling to let people use MS-MPI on non-servers so I have to use something else on my workstation to develop parallel code which is based on the message passing paradigm.  MPAPI is filling that bill today. 

MS-MPI and MPAPI have different API designs so I am in a bit of trouble in my example application, if I choose for some reason to integrate the two systems. 

It might be very nice to have a smooth or transparent message passing API for workstations and servers when you want to scale.  It's too complicated and messy to use 2 different message passing APIs in the same application.

Does anyone agree with me that there is an opportunity here for supporting multi-way workstation developers with a message passing framework to fill in the hole while Microsoft has chosen to forfeit the workstation message passing and instead, chase increased sales of Microsoft Server 2008?

Jul 27, 2008 at 10:29 AM
Edited Jul 27, 2008 at 10:30 AM
I most certainly agree with you that there is a great oppotunity here to fill out the gap on workstations. Message passing has long been regarded as either esoteric, not worth the trouble, or something for massive parallel systems (like supercomputers). But as you say the number of cores on workstations will increase, and furthermore there are a number of systems that must run on a (compared to a supercomputer) small number of nodes. And the developer community is finally starting to realize that their software is not scalable, and that multithreaded programming should be employed more to utilize the new multicore processors. This is where frameworks like MPAPI, and programming languages like Erlang and F#, fits like a glove.

Your idea of making a standardized interface for these small-scale parallel frameworks is great. However, I don't think anyone could agree on such an interface (or programming model, whichever fits). This would demand that the different projects work more closely together - something I can't see happening right now. But hopefully my pessimism is not warranted.

I am well aware that MPAPI does not have features like collective operations or cluster partitioning, but the intention was to create a framework which did not have a steep and long learning curve to figure out. But that is not to say that these features should not be implemented in the future.

The problem with MPAPI right now is that I am the only one doing work in it - in my free time. That is not a compliant, by the way, but just a statement that MPAPI is not the only thing I am working on. I would love to give it more attention, but right now (for the next couple of months that is) it is not possible. A larger user base would certainly also induce me to do more work on it. Not that I am not grateful to those who already use it - not at all! Even a single user would make it worth the trouble, but such frameworks are complicated and hard work is needed.
Jul 28, 2008 at 10:18 PM
I didn't mean to make anyone feel bad that they can't spend time on something or other; rather, my intention was to make people feel good by enjoying the possibilities that an opportunity might be presenting.  The glass is half-full my friend!