Heterogeneous Supercomputing: Problems and Issues

While promising feasible solutions to several short­ comings of homogeneous parallel computing, Hetero­ geneous SuperComputing (HSC) poses new problems to be solved. In this paper, we outline the issues and problems arising in using heterogeneous environments for parallel solutions to various applications. Prelim­ inary solutions to a set of problems in heterogeneous supercomputing are presented. AB an example appli­ cation, implementation of image understanding algo­ rithms in a heterogeneous environment is studied.

[1]  Mary Mehrnoosh Eshaghian-Wilner.  Cluster-M parallel programming model , 1992, Proceedings Sixth International Parallel Processing Symposium.

[2]  James C. Browne,et al.  CODE: a unified approach to parallel programming , 1989, IEEE Software.

[3]  Eugene Wong,et al.  Multibase: integrating heterogeneous distributed database systems , 1981, AFIPS '81.

[4]  Catherine E. Houstis,et al.  Module Allocation of Real-Time Applications to Distributed Systems , 1987, IEEE Trans. Software Eng..

[5]  Howard Jay Siegel,et al.  Instruction execution trade-offs for SIMD vs. MIMD vs. mixed mode parallelism , 1991, [1991] Proceedings. The Fifth International Parallel Processing Symposium.

[6]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[7]  Howard Jay Siegel,et al.  PASM: A Partitionable SIMD/MIMD System for Image Processing and Pattern Recognition , 1981, IEEE Transactions on Computers.

[8]  H. T. Kung,et al.  The design of nectar: a network backplane for heterogeneous multicomputers , 1989, ASPLOS III.

[9]  Wesley W. Chu,et al.  Task Allocation and Precedence Relations for Distributed Real-Time Systems , 1987, IEEE Transactions on Computers.

[10]  Nicholas Carriero,et al.  Linda and Friends , 1986, Computer.

[11]  Ashok K. Agrawala,et al.  An optimal algorithm for mutual exclusion in computer networks , 1981, CACM.

[12]  Viktor K. Prasanna,et al.  Efficient VLSI Implementation of Iterative Solutions to Sparse Linear Systems , 1993, Parallel Comput..

[13]  Pen-Chung Yew,et al.  An effective synchronization network for large multiprocessor systems , 1991, [1991] Proceedings. The Fifth International Parallel Processing Symposium.

[14]  Shahid H. Bokhari.  Partitioning Problems in Parallel, Pipelined, and Distributed Computing , 1988, IEEE Trans. Computers.

[15]  S. G. Skinner,et al.  Development of a mixed MIMD-SIMD architecture using the AMT DAP , 1991, [1991] Proceedings. The Fifth International Parallel Processing Symposium.

[16]  David P. Reed,et al.  Synchronization with eventcounts and sequencers , 1979, CACM.

[17]  H. T. Kung,et al.  Warp as a machine for low-level vision , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[18]  Kemal Efe,et al.  Heuristic Models of Task Assignment Scheduling in Distributed Systems , 1982, Computer.

[19]  M. M. Eshaghian,et al.  Optimal geometric algorithms for digitized pictures on an optical mesh , 1988, Proceedings., 2nd Symposium on the Frontiers of Massively Parallel Computation.

[20]  Richard F. Freund.  Superconcurrent Processing - A Dynamic Approach to Heterogeneous Parallelism , 1990 .

[21]  Constantine D. Polychronopoulos,et al.  Parallel programming and compilers , 1988 .

[22]  A. Gottleib,et al.  The nyu ultracomputer- designing a mimd shared memory parallel computer , 1983 .

[23]  Allen D. Malony,et al.  Faust: an integrated environment for parallel programming , 1989, IEEE Software.

[24]  Gul A. Agha,et al.  Supporting Multiparadigm Programming on Actor Architectures , 1989, PARLE.

[25]  Virgílio A. F. Almeida,et al.  Cost-performance analysis of heterogeneity in supercomputer architectures , 1990, Proceedings SUPERCOMPUTING '90.

[26]  Kai Hwang.  Advanced parallel processing with supercomputer architectures , 1987, Proceedings of the IEEE.

[27]  Butler W. Lampson,et al.  Distributed Systems — Architecture and Implementation , 1982, Lecture Notes in Computer Science.

[28]  Gregory F. Pfister,et al.  A Methodology for Predicting Multiprocessor Performance , 1985, International Conference on Parallel Processing.

[29]  S. Tanimoto,et al.  Structured computer vision: Machine perception through hierarchical computation structures , 1980 .

[30]  David Notkin,et al.  Heterogeneous computing environments: report on the ACM SIGOPS workshop on accommodating heterogeneity , 1987, CACM.

[31]  Viktor K. Prasanna,et al.  Optimal VLSI Sorting with Reduced Number of Processors , 1991, IEEE Trans. Computers.

[32]  David Fernández-Baca,et al.  Allocating Modules to Processors in a Distributed System , 1989, IEEE Trans. Software Eng..

[33]  Viktor Prasanna,et al.  Proceedings of the 6th International Parallel Processing Symposium , 1992 .

[34]  Andrzej Duda,et al.  Primitives for Distributed Computing in a Heterogeneous Local Area Network Environment , 1989, IEEE Trans. Software Eng..

[35]  Alan Weiss,et al.  Allocating Independent Subtasks on Parallel Processors , 1985, IEEE Transactions on Software Engineering.

[36]  Thomas L. Casavant,et al.  Experimental Analysis of a Mixed-Mode Parallel Architecture Using Bitonic Sequence Sorting , 1991, J. Parallel Distributed Comput..

[37]  Janak H. Patel,et al.  A Parallel Processing Architecture for an Integrated Vision System , 1988, ICPP.

[38]  Gérard Giraudon,et al.  Image processing on a SIMD/SPMD architecture: OPSILA , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[39]  Ralph Grishman,et al.  The NYU Ultracomputer—Designing an MIMD Shared Memory Parallel Computer , 1983, IEEE Transactions on Computers.

[40]  Dionysios I. Reisis,et al.  Image Computations on Meshes with Multiple Broadcast , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Ashfaq A. Khokhar,et al.  Stereo and image matching on fixed size linear arrays , 1993, [1993] Proceedings Seventh International Parallel Processing Symposium.

[42]  Harold S. Stone,et al.  Multiprocessor Scheduling with the Aid of Network Flow Algorithms , 1977, IEEE Transactions on Software Engineering.

[43]  Virginia Mary Lo,et al.  Heuristic Algorithms for Task Assignment in Distributed Systems , 1988, IEEE Trans. Computers.

[44]  Azriel Rosenfeld,et al.  Parallel Image Processing Using Cellular Arrays , 1983, Computer.

[45]  Adi Shamir,et al.  Shear Sort: A True Two-Dimensional Sorting Techniques for VLSI Networks , 1986, ICPP.

[46]  R. Cooper.  Pilgrim: A Debugger for Distributed Systems , 1987, ICDCS.

[47]  Wesley W. Chu,et al.  Task Allocation in Distributed Data Processing , 1980, Computer.

[48]  Anthony P. Reeves,et al.  Parallel computer architectures for image processing , 1984, Comput. Vis. Graph. Image Process..

[49]  Ronald H. Perrott,et al.  A supercomputer program development system , 1987, Softw. Pract. Exp..

[50]  Brian N. Bershad,et al.  A remote computation facility for a heterogeneous environment , 1988, Computer.

[51]  Jih-Kwon Peir,et al.  A Performance Evaluation Methodology for Coupled Multiple Supercomputers , 1990, International Conference on Parallel Processing.

[52]  W. Daniel Hillis,et al.  The connection machine , 1985 .

[53]  Milos D. Ercegovac,et al.  Heterogeneity in supercomputer architectures , 1988, Parallel Comput..

[54]  Mario J. Gonzalez.  Deterministic Processor Scheduling , 1977, CSUR.

[55]  Andrew P. Black,et al.  Interconnecting heterogeneous computer systems , 1988, CACM.

[56]  Jack J. Dongarra,et al.  Programming methodology and performance issues for advanced computer architectures , 1988, Parallel Comput..

[57]  Ken Kennedy,et al.  Performance of parallel processors , 1989, Parallel Comput..

[58]  David J. DeWitt,et al.  GAMMA - A High Performance Dataflow Database Machine , 1986, VLDB.

[59]  John R. Kender,et al.  Low-Level Image Analysis Tasks on Fine-Grained Tree-Structured SIMD Machines , 1987, J. Parallel Distributed Comput..

[60]  C. E. Houstics.  Module allocation of real-time applications to distributed systems , 1990 .

[61]  Michael P. Mariani,et al.  Distributed data processing system design - A look at the partitioning problem , 1978, COMPSAC.

[62]  Michael J. Flynn,et al.  Very high-speed computing systems , 1966 .

[63]  Carl A. Sunshine,et al.  A Platform for Heterogeneous Interconnection Network Management , 1990, IEEE J. Sel. Areas Commun..

[64]  Charles C. Weems,et al.  A multiple-level heterogeneous architecture for image understanding , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[65]  M. M. Eshaghian.  Cluster-M parallel programming model , 1992 .

[66]  JOHN B. ANDREWS,et al.  An Analytical Approach to Performance/Cost Modeling of Parallel Computers , 1991, J. Parallel Distributed Comput..

[67]  Jake K. Aggarwal,et al.  A Mapping Strategy for Parallel Processing , 1987, IEEE Transactions on Computers.

[68]  Viktor K. Prasanna,et al.  Parallel Geometric Algorithms for Digitized Pictures on Mesh of Trees , 1986, ICPP.