Meseret Gebre

Different Types of Parallelism in HPDC part1

// March 19th, 2009 // HPDC

There are many types of parallelism in HPDC, but in general the term is typically used to describe the idea of performing two or more tasks at the same time using different computational devices. These computational devices can be:

1. A different computer thats networked or interconnected
2. Another microprocessor on the same computer
3. Another core on a microprocessor
4. A seprate ALU or part of ALU on a processor

This idea of parallelism is extended and we have different types of parallelism. Imagine a spectrum if you will. In this spectrum, one extreme end will be implicit parallelism and the other extreme is explicit parallelism.

Implicit parallelism in its basic form is parallelism that is automatically or semi-automatically realized. Meaning the microprocessor and the compiler work together to extract parallelism. Note, that the scope of what you gain are limited to a single processor or single machine. Explicit parallelism is the other extreme, so parallelism here is realized manually. The programmer is responsible for developing the program to run in parallel. Typically, for anything worthwhile, would require considerable programming efforts. However, the gain be scoped to multiple machines or supercomputing clusters!

You might be asking, why is explicit parallelism so difficult to code? If you are not asking yourself this question, pretend you just did! It’s tough because most of the time the serial running program that you created is hard to break up into multiple small chunks of code. Basically, parallelism strives to effectively utilize concurrency in code to reduce the overall computational time required to complete processing.

By concurrency, I am talking about the lack of dependency or relationship between two tasks. Identifying concurrency is what makes explicit parallelism so hard. If you understood everything to this point, then you have the basics of parallelism and the challenges. Part2 of this article will discuss the different types of parallelism, the stuff between our imaginary spectrum we described earlier.