The Message Passing Interface (MPI) first appeared as a standard in 1994 for performing distributed-memory parallel computing. Since then, it has become the dominant model for high-performance computing, and it is used widely in research, academia, and industry.
The functionality of MPI is extremely rich, offering the programmer with the ability to perform: point-to-point communication, collective communication, one-sided communication, parallel I/O, and even dynamic process management. These terms probably sound quite strange to a beginner, but by the end of all of the tutorials, the terminology will be common place.
Before starting the tutorials, familiarize yourself with the basic concepts below. These are all related to MPI, and many of these concepts are referred to throughout the tutorials.
The Message Passing Model
The message passing model is a model of parallel programming in which processes can only share data by messages. MPI adheres to this model. If one process wishes to transfer data to another, it must initiate a message and explicitly send data to that process. The other process will also have to explicitly receive the other message (except in the case of one-sided communication, but we will get to that later).
Forcing communication to happen in this way offers several advantages for parallel programs. For example, the message passing model is portable across a wide range of architectures. An MPI program can run across computers that are spread across the globe and connected by the internet, or it can execute on tightly-coupled clusters. An MPI program can even run on the cores of a shared-memory processor and pass messages through the shared memory. All of these details are abstracted by the interface. The debugging of these programs is often easier too, since one does not need to worry about processes overwriting the address space of another.
MPI’s Design for the Message Passing Model
MPI has a couple classic concepts that encourage clear parallel program design using the message passing model. The first is the notion of a communicator. A communicator defines a group of processes that have the ability to communicate with one another. In this group of processes, each is assigned a unique rank, and they explicitly communicate with one another by their ranks.
The foundation of communication is built upon the simple send and receive operations. A process may send a message to another process by providing the rank of the process and a unique tag to identify the message. The receiver can then post a receive for a message with a given tag (or it may not even care about the tag), and then handle the data accordingly. Communications such as this which involve one sender and receiver are known as point-to-point communications.
There are many cases where processes may need to communicate with everyone else. For example, when a master process needs to broadcast information to all of its worker processes. In this case, it would be cumbersome to write code that does all of the sends and receives. In fact, it would often not use the network in an optimal manner. MPI can handle a wide variety of these types of collective communications that involve all processes.
Mixtures of point-to-point and collective communications can be used to create highly complex parallel programs. In fact, this functionality is so powerful that it is not even necessary to start describing the advanced mechanisms of MPI. We will save that until a later lesson. For now, you should work on installing MPI on your machine. If you already have MPI installed, great! You can head over to the MPI Hello World lesson.
HAS MPITUTORIAL.COM HELPED YOU?
Donate a small sum of 5 dollars today to help me out with my hosting costs. Thank you!
No related posts.
