Our objective in this thesis is to perform an analysis of the IBM SP's most widely used communication systems. Specifically we determine the degree to which the IBM MPI and the MPICH-G packages utilize the SP2 communication hardware effectively. We determine the relative effectiveness of IBM MPI and MPICH-G. We also determine how both these MPI packages compare with a lightweight communications system such as the active message infrastructure on the SP. We benchmark these packages using standard benchmarking techniques. Our results show that IBM MPI outperforms MPICH-G on a standalone parallel machine by more than 100% for message sizes in the range 4KB and 400KB based on the UDP/IP protocols. We show that the peak bandwidth of 110MB/s for the SP's high performance switch cannot be achieved by any of the packages due to software overhead. MPI achieves a maximum bandwidth of 45MB/s for a message size of 100MB whereas MPICH-G attains 35MB/s for the same size. We also show that active messages perform best of the three.
Based on these results we discuss an implementation of MPI in accordance with the MPI standard, which makes use of active messages. We show how this implementation can perform better than the current IBM MPI package. We also discuss various issues that are involved in such an implementation.