Benchmarks

From HPC Wiki
Jump to navigation Jump to search

General

Benchmarks are applications designed to characterize different performance aspects of a system. The results can then be used to compare and rank different systems (e.g. TOP500 list). Furthermore, they can be used to identify problems and monitor the progress of fixing them. They are also used by vendors to check if the performance requested by the customer is actually achieved. An important aspect for benchmarks is portability, as the main reason for not using "real" appliactions is usually that it takes too long and too many changes to get them to run on a different system. System-related adjustments should be avoided or at least kept to a minimum.

Benchmark types

Individual benchmarks can usually be placed in one of the following categories:

Category Example Description
Raw machine performance Tflops machine specifications, e.g. FP peak performance
Microbenchmarks Stream "small" basic components, e.g. latency, bandwidth, communication overhead...
Algorithmic benchmarks Linpack simple algorithms with increasing problem sizes, close to the "kernels"
Compact Apps/Kernels MINI-Apps/NPB more realistic applications that consider more aspects
Application Suites SPEC set of real key applications with different limiting factors
User-specific applications Custom benchmarks run your own application and use hardware counters to manually compute metrics of your choice

Resources

SPEC (Standard Performance Evaluation Cooperation)

STREAM (Sustainable Memory Bandwidth in High Performance Computers)