Effects of multithreading on data and workload distribution for distributed-memory multiprocessors
- 23 December 2002
- proceedings article
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 116-122
- https://doi.org/10.1109/ipps.1996.508047
Abstract
While data and workload distribution can be tailored to fit a particular problem to a particular distributed-memory architecture, it is often difficult to do so for various practical issues. This report presents our study on multithreading for distributed-memory multiprocessors. Specifically, we investigate the effects of multithreading on data distribution and workload distribution with variable thread granularity. Various types of workload distribution strategies are defined along thread granularity. Three types of data distribution strategies are investigated, including row-wise cyclic, k-way partial-row cyclic, and blocked distribution. We have implemented all of these on the 80-processor EM-4 distributed-memory multiprocessor using highly sequential Gaussian Elimination with Partial Pivoting and highly parallel Matrix Multiplication. Experimental results indicated that multithreading can offset the loss that is due to the mismatch of data distribution to workload distribution for even sequential and irregular problems while giving high absolute performance.Keywords
This publication has 8 references indexed in Scilit:
- The MIT Alewife machine: architecture and performancePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- The EM-X parallel computer: architecture and basic performancePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Multithreaded processor architecturesIEEE Spectrum, 1995
- Reducing communication by honoring multiple alignmentsPublished by Association for Computing Machinery (ACM) ,1995
- StarT-NG: Delivering seamless parallel computingPublished by Springer Nature ,1995
- SP2 system architectureIBM Systems Journal, 1995
- TAM - A Compiler Controlled Threaded Abstract MachineJournal of Parallel and Distributed Computing, 1993
- The Tera computer systemPublished by Association for Computing Machinery (ACM) ,1990