Outline of Material for Test #1

Architectural Models of Parallel Computers

• Flynn's Taxonomy (SISD,SIMD,MIMD)
• Understand discussion on pages 25-27 in text.
• Also understand SPMD (Single Program Multiple Data)
• Memory Access Architectures
Understand:
• Shared Memory Multiprocessor System [Figure 1.3]
• single address space
• uniform memory access (UMA)/ nonuniform memory access (NUMA)
• distributed shared memory system
• Message Passing Multicomputer
• How is this different from a shared memory system?
• What are the advantages/disadvantages of message passing v.s. shared memory
• Network Models
• Be able to compute Diameter, Bisection Width, and Cost for Linear, Ring, Mesh, Torus, Tree, and Hypercube networks.
• Understand how to construct a d-dimensional hypercube recursively.
• Be able to define and compute Diameter, Connectivity, Bisection Width, Bisection Bandwidth, and Cost for any arbitrary network.
• Communication Cost Models for Static Interconnection Networks
Know and Understand:
• communication latency ts+m tw
• startup time, ts
• Per-word transfer time tw
• How does bisection width relate to tw in the above formula?

Performance Models of Parallel Computers

• PRAM Models
• Be able to discuss the PRAM machine architecture and assumptions (See Handout from Algorithms text)
• Understand Types of PRAM models:
Combining CRCW, Priority CRCW, Arbitrary CRCW, Common CRCW, CREW, and CREW.
• Understand and be able to apply Brent's theorem.
• Review Homework problem!
• LogP Model
• Be able to define the Model Parameters, L, o, g, and P.
• Understand the discussion of estimating running time of the broadcast/summing under the LogP model as discussed in class and the LogP paper.
• Be able to use the LogP model to compute running times of programs.
• Review Homework problem!
• BSP Model
• Be able to discuss the BSP execution model. (What is the BSP execution model and what does it consist of?)
• Be able to use the cost model for BSP [w + hg + l] to estimate running times of BSP algorithms.
• What is a h-relation? A 1-relation? A p-relation?
• What is a super-step in the BSP model?
• What is parallel slackness?
• What role does the barrier operation play in the BSP model?
• Review Homework problem!

MPI and Message Passing

• Be able to discuss how deadlock is possible when using MPI_Send() and MPI_Recv() as discussed in class.
• Be able to describe the minimum set of calls required of any MPI program, e.g. MPI_Init, MPI_Finalize, etc.
• Be able to discuss the role of buffering in message passing computations.
• Review Homework problems!

Programming Patterns

• Background Information
• What is a task-dependency graph?
• What is a decomposition of a task-dependency graph?
• What is fine-grained v.s coarse grained?
• What is degree of concurrency?
• What is the critical path of a task graph?
• What is a task-interaction-graph and how does it relate to a task dependency graph?
• What is a process and processor? How are they different?
• Decomposition Techniques
• Be able to define and give examples of:
• recursive decomposition
• data decomposition
• exploratory decomposition
• speculative decomposition
• In data-decomposition, how does data partitioning relate to task partitioning? What is the meaning of partitioning input data or output data? What is the owner computes rule?
• What problems are suitable to exploratory decomposition?
• What problems are suitable for speculative decomposition?
• Static Mapping Techniques
• What are static mapping techniques?
• Be able to define and describe:
• block distributions
• cyclic distributions
• block-cyclic
• random
• How do we partition irregular data structures?
• Dynamic Mapping Techniques
• What is the differences between centralized and distributed schemes? What are the advantages/disadvantages of each?
• What is self scheduling and chunk scheduling?
• Why not use dynamic mapping for all problems?
• Methods for containing overheads
• Be able to discuss, compare, and contrast the following:
• Maximizing Data Locality
• Minimizing Volume of Data-Exchange
• Minimize Frequency of Interactions
• Minimizing Contention and Hot Spots
• Overlapping Computations and communications
• Replicating Data or Computations
• Using optimized collective interaction operations
• Overlapping Interactions with other interactions
• Parallel Programming Patterns
• Be able to discuss, compare, and contrast the following:
• The Data Parallel Model
• The Task Graph Model
• The Work Pool Model
• The Master-Slave Model
• The Pipeline or Producer-Consumer Model
• Hybrid models (be able to give an example of a hybrid approach)