This Spring, I completed the Intro to High Performance Computing course (CSE 6220) as part of OMSCS. It is one of the hardest (4.5/5) and also the highest rated (4.8/5) course in the program as per OMS Central. Based on my experience, I concur with both ratings.
At a high level, the course covers the algorithmic aspects of maximizing the performance of your code. This includes things like parallelizing your code across all processors or across multiple machines, exploiting the memory hierarchy to your advantage etc. The other ‘high performance’ course in the program – High Performance Computer Architectures (CS 6290), in contrast, discusses maximizing performance more at a processor architecture level.
Prof. Vuduc requires special mention. He has put a lot of effort in making the course videos easy-to-understand and interesting. His hilarious antics make you laugh even while discussing the most complex topics. He is also very active in Piazza and participates in the office hours regularly.
There were 5 hands-on projects in total, all in C/C++, with one due every two weeks. These were really the time-sinks. Interestingly, these were also the most fun part of the course in my experience. These involved implementing the algorithms taught in the lectures, making everything you learn more ‘real’.
At a broad level, these were the key concepts I learned from the course:
- Shared memory model (aka dynamic multithreading model)
- Distributed memory models
- Parallel computing across network using message passing.
- The Alpha-Beta model (aka latency & inverse-bandwidth model) for analyzing distributed parallel programs.
- Introduction to OpenMPI library.
- Two level memory model
- I/O aware algorithms that can exploit the cache and main memory structures.
- Cache oblivious algorithms that still achieve optimal performance without being aware of the cache/memory structures.
Overall, I really enjoyed the course and am happy that I decided to take it though it was not directly related to my specialization (Interactive Intelligence).