CP7006 PARALLEL PROGRAMMING PARADIGMS

CP7006     PARALLEL PROGRAMMING PARADIGMS

UNIT I   FOUNDATIONS OF PARALLEL PROGRAMMING

Motivation for parallel programming - Concurrency in computing – basics of processes, multiprocessing, and threads – cache – cache mappings – caches and programs – virtual memory – instruction level parallelism – hardware multi-threading – SIMD – MIMD – interconnection networks – cache coherence – shared-memory model – issues in shared-memory model – distributed-memory model – issues in distributed-memory model – hybrid model – I/O – performance of parallel programs – parallel program design 

UNIT II         MESSAGE PASSING PARADIGM

Basic MPI programming – MPI_Init and MPI_Finalize – MPI communicators – SPMD programs – message passing – MPI_Send and MPI_Recv – message matching – MPI I/O – parallel I/O – collective communication – MPI_Reduce – MPI_Allreduce – broadcast – scatter – gather – allgather – derived types – remote memory access – dynamic process management – MPI for grids – performance evaluation of MPI programs  

UNIT III       SHARED MEMORY PARADIGM: PTHREADS

Basics of Pthreads – thread synchronization – critical sections – busy-waiting – mutexes – semaphores – barriers and condition variables – read-write locks – Caches, cache coherence and false sharing – thread safety – Pthreads case study   

UNIT IV       SHARED MEMORY PARADIGM: OPENMP

Basic OpenMP constructs – scope of variabls – reduction clause – parallel for directive – loops in OpenMP – scheduling loops – synchronization in OpenMP – Case Study:  Producer-Consumer problem – cache issues – threads safety in OpenMP – OpenMP best practices  

 34

UNIT V      GRAPHICAL PROCESSING PARADIGMS: OPENCL AND CUDA                9 Introduction to CUDA – CUDA programming examples – CUDA execution model – CUDA memory hierarchy – CUDA case study - introduction to OpenCL – OpenCL programming examples – Programs and Kernels – Buffers and Images – Event model – OpenCL case study  

REFERENCES: 

1. Peter S. Pacheco, “An introduction to parallel programming”, Morgan Kaufmann, 2011. 

2. M. J. Quinn, “Parallel programming in C with MPI and OpenMP”, Tata McGraw Hill, 2003. 

3. W. Gropp, E. Lusk, and R. Thakur, “Using MPI-2: Advanced features of the message passing interface”, MIT Press, 1999. 

4. W. Gropp, E. Lusk, and A. Skjellum, “Using MPI:  Portable parallel programming with the message passing interface”, Second Edition, MIT Press, 1999. 

5. B. Chapman, G. Jost, and Ruud van der Pas, “Using OpenMP”, MIT Press, 2008. 

6. D. R. Butenhof, “Programming with POSIX Threads”, Addison Wesley, 1997. 

7. B. Lewis and D. J. Berg, “Multithreaded programming with Pthreads”, Sun Microsystems Press, 1998. 

8. A. Munshi, B. Gaster, T. G. Mattson, J. Fung, and D. Ginsburg, “OpenCL programming guide”, Addison Wesley, 2011. 

9. Rob Farber, “CUDA application design and development”, Morgan Haufmann, 2011.