Outline of Material for Test #2

Performance and Scalability of Parallel Systems

• General Principles
• Speedup
• Parallel Speedup, S = t1/tp
• Ideal Speedup, S_ideal = p
• Actual Speedup, S_actual = t_best/tp = S*E_a = S_ideal*E_p*E_a
• Efficiency
• Algorithmic Efficiency, E_a = T_best/t1
• Parallel Efficiency, E_p = S/S_ideal = S/p
• System Efficiency, E = E_a*E_p
• Amdahl's Law
If a parallel algorithm consists of a fully parallel part and a fully serial part such that the fraction of instructions that are fully serial is given by f , then the speedup for this parallel algorithm can be given by:

tp = ((1-f)/p + f)*t1

In other words speedup is given by

S = 1/((1-f)/p + f).

Taking the limit as p increases to infinity gives an upper bound on speedup such that

S <= 1/f

• Be able to define Speedup, Efficiency, Cost, Cost Optimal, Problem Size, Overhead Function, and Isoefficiency Function.
• Be able to determine if an algorithm is Cost Optimal.
• Be able to determine if an algorithm has an Isoefficiency function
• Be able to derive the overhead function and use this function to determine the existence of an Isoefficiency function. Be able to show that the overhead function is related to the isoefficiency function when one exists.
• Understand why there is a lower bound on the isoefficiency function as described in section 5.4.4 from text.
• What is the degree of concurrency of a parallel algorithm?
• What is the relationship between degree of concurrency and the isoefficiency function? See section 5.4.5 from text.
• What are the sources of parallel overhead? ANS: Communication time, Load Imbalance, and additional computations performed by the parallel algorithm.
• What is scaled speedup? How do we measure it?
• What is memory and time-constrained scaled speedup? How do we measure these?
• Study the Homework Assignments!

Parallel Sorting Algorithms

• Fundamentals
• Be able to define internal and external sorting algorithms.
• What is the compare-exchange operation? How is it implemented in parallel
• What is the compare-split operation? How much time does it require?
• What is a comparator? How do you draw one? What is a increasing or decreasing comparator and how do you represent it (draw it).
• Sorting Networks
• Be able to define a bitonic sequence, bitonic split and bitonic merge.
• What is a bitonic merging network? What does it do? What is its size? What is its depth? What does it look like?
• What is a bitonic sorting network? What does it do? What is its size? What is its depth? What does it look like?
• How do we map a bitonic sorting network onto a hypercube? What does the communication patterns look like?
• Be able to derive execution times for the bitonic sorting algorithm on a hypercube architecture. Be able to discuss both mapping one element per processor and many elements per processor.
• Be able to perform a scalability analysis of the bitonic sorting algorithm on the hypercube architecture.
• Quick-sort
• Why is it difficult to develop an efficient parallel quick-sort algorithm by simply running independent subproblems in parallel?
• For the shared memory quicksort algorithm, how is the array partitioned?
• For the shared memory quicksort algorithm, how is load balanced?
• What is the prefix sum operation used for in the shared memory quicksort algorithm?
• Be able to derive the running time of the quicksort algorithm.
• Be able to derive the iso-efficiency function of the quicksort algorithm.
• How do we modify the shared memory quicksort algorithm to generate a message-passing formulation?
• How does pivot selection affect the algorithm performance? Is pivot selection more or less important for parallel quicksort? Why?
• Sample Sort
• What are splitters?
• How large a sample is selected to find the splitters?
• What communication operation is used after the splitters have been found?
• What are the steps in the sample sort algorithm?
• What is the serial and parallel run time of this algorithm
• As described in the text
• Using a bitonic sort to sort the splitters
• What is the isoefficiency function of this sorting algorithm?

General Algorithm Analysis Knowledge

• Be able to compute timings for algorithms using the network model.
• Be prepared to use standard communication algorithms (broadcast, reduction, etc) when describing and analyzing parallel algorithms
• Be prepared to perform isoefficiency and cost optimality analysis on any of the algorithms covered in the sorting chapter.
• In short, you are responsible for the basic analysis knowledge that we have covered in Test#1