CSE 8163

PARALLEL AND DISTRIBUTED SCIENTIFIC COMPUTING

  Prof. Ioana Banicescu.

 HOURS:

CLASS: Tue/Thu 12:30-1:45, Butler 102.

OFFICE: 320 Butler Hall, Tue/Thu 2:00-3:00. Other times by appointment only. Email:ioana@cse.msstate.edu.

 325-2756.

RECOMMENDED TEXT BOOK AND READINGS FROM:

Uwe Naumann and Olaf Schenk, Combinatorial Scientific Computing, CRC Press, Taylor and Francis Group, Chapman and Hall, 2012

Ananth Grama, George Karypis, Vipin Kumar, and Anshul Gupta Introduction to Parallel Computing (2nd Edition), Addison-Wesley, 2003

Rob Bisseling, Parallel Scientific Computations - A structured approach using BSP and MPI, Oxford University Press, 2004

Michael T. Heath, Scientific Computing - An Introductory Survey(Second Edition) , McGraw Hill, 2002

Oliver Sinnen, Task Scheduling for Parallel Systems, John Wiley & Sons, 2007

Additional readings from research papers will be indicated in class.

PREREQUISITES:

CS4163/6163 Designing Parallel Algorithms course in high performance computing is a must. Two courses in high performance computing, and/or additional courses in algorithms, compilers, operating systems, computer architecture, and experience with message passing interface and programming with threads, are preferred.

OBJECTIVES:

This course evolves from a series of other high performance computing courses and related topics taught at the Mississippi State University. It is targeted towards graduate students in sciences and engineering who are interested in solving computationally intensive problems on parallel and distributed machines. The class will survey recent advances in parallel algorithms for scientific computing and will aim to discover their impact on improving performance of scientific applications. A special emphasis will be placed on adaptive algorithms for unstructured and irregular problems, algorithms used in combinatorial scientific computing, performance optimization of problems in computational science via load balancing, performance modeling, and present heterogeneous computing technologies (such as, multi-core, many-core, clusters, grids), as well as emerging technologies (such as autonomic computing). Since students come from different departments and backgrounds, the lectures will attempt to emphasize an even spread of topics in algorithm design, numerical methods, computer architecture, software and runtime environments. Students will spend time on particular parallel algorithms for scientific problems and their implementations on parallel and distributed machines. There might be a few invited talks given by recognized researchers (i.e., affiliated with the MSU research centers such as the ERC, or other institutions). The invited talks will be related to recent research advances in preferred topics related to numerical analysis, scientific applications, runtime environments, performance analysis, and others.

COURSE REQUIREMENTS:

The class will study a variety of algorithms, their applications, and tradeoffs between different solutions. Issues are performance analysis, evaluation and prediction will be emphasized. Recent advances in task scheduling and load balancing algorithms will also be introduced and discussed in detail. There will also be discussions on issues such as parallel computer architectures (memory hierarchy, interconnection networks, latency and bandwidth, parallel I/O), and software systems, with the aim of understanding their capabilities, costs and limitations. Students are expected to make use of recent technology through a number of software packages and programming environments appropriate to the topics addressed. These will be used in comparing the competitiveness and evaluating the performance of different implementations through a variety of criteria. Students will draw conclusions regarding preferred algorithms, methods, programming paradigms, and programming environments and tools for parallel and distributed computing. Among sample problems used in class, the following case studies could be enumerated: linear algebra and large systems of differential equations arising in global climate modeling, finite element, finite volume methods, and hierarchical methods for particle simulations in computational field simulation (i.e., computational and molecular modeling, computational fluid dynamics, plasma physics). You may find the course outline at my homepage at:  CSE8163.

GRADING: Assignments 20%, Presentations 10%, Midterm/Papers/Projects 30%, Final 40%.

Attendance: more than three absences will influence grades at border line.

 ANNOUNCEMENTS:

Students are responsible for checking the announcements regularly. For important announcements, click here.

COURSE OUTLINE:

The course will cover material from the following topics:

Fundamentals: Introduction to Scientific Computing.

scientific problems and their characteristics

the impact of parallel programming on performance and numerical properties

Fundamentals of parallel and distributed computing: theory and practice

models of parallel computation (i.e., PRAM, BSP, LOGP, etc)

parallel programming paradigms (data parallel, message passing and shared memory with multithreading)

mapping algorithms onto architectures (partitioning, allocation, scheduling)

performance analysis, evaluation and prediction (load balancing, communication, overhead)

A closer look at numerical properties, parallel algorithms and architectures: methods and tradeoffs

The impact of numerical properties on the performance of scientific applications cases studied (i.e., discrete Poisson equation using Jacobi or Multigrid methods, SOR, Conjugate Gradient and the FFT; hierarchical methods for the N-Body problem, etc)

Algorithms of common use (i.e., parallel sorting, parallel matrix multiplication, graph partitioning, fast algorithms using trees)

Recent technological advances in parallel architectures and their impact in the programming design methodology (i.e., cluster computing, parallelizing compilers, etc).

Scheduling and load balancing algorithms: theory and practice

Advanced scheduling methods, dynamic task scheduling, and loop scheduling

Load balancing methods (deterministic and stochastic algorithms, optimality analysis, methods using combinatorial optimization)

Performance modeling and analysis using process algebra

Review of the process algebra methods (CCS and CSP)

Introduction to quantitative modeling (PEPA)

Modeling task scheduling with PEPA

Important issues visited in case studies

Recent progress in high accuracy and high performance of linear algebra algorithms

Sources of parallelism and locality in simulation

The impact of memory hierarchy in the algorithm design

Scalability analysis of parallel applications

Combinatorial aspects in scientific computing

The impact of new technology in the design of parallel scientific applications

ACADEMIC HONESTY:

Students are expected to follow MSU as well as the CS department's policies on academic misconduct and on academic honesty with regard to homeworks, projects, exams, papers. These policies can be found on the MSU ethics in research and other scholarly activities and Computer Science and Engineering Departmental policy regarding professional conduct pages, respectively. Also, as of fall semester 2007, Mississippi State University has an approved Honor Code that applies to all students, and reads as follows: As a Mississippi State University student I will conduct myself with honor and integrity at all times. "I will not lie, cheat, or steal, nor will I accept the actions of those who do". Upon accepting admission to Mississippi State University, a student immediately assumes a commitment to uphold the Honor Code, and accepts responsibility for learning and following the philosophy and rules of the Honor Code. For additional information, please see the following links:

www.cse.msstate.edu/academics/gradstud/honesty.php

students.msstate.edu/honorcode/

ATTENDANCE AND ADD/DROP POLICIES:

Attendance is required. Roll will be taken, and absences will be listed on midterm and final grade reports. Attendance may influence grades at border line. Makeup exams are allowed only in special circumstances where absences are certified by the Dean of Students office. At the beginning of the semester, students are encouraged to also review the Add/Drop policy.

Students who are auditing this course must attend at least 75% of the class lectures to avoid a grade of F.