Micro benchmarking

Microbenchmarking is about measuring the time or performance of small to very small building blocks of real programs. This can be a common data access pattern, a sequence of operations or even a single instruction.

Introduction

Microbenchmarking is an indispensable tool in performance engineering, which fulfills many purposes. Among other things it :

• provides upper performance limits for sustained performance
• creates knowledge about performance behavior
• helps finding performance bugs in architectures
• provides undocumented processor performance properties
• quantifies the cost of programming model constructs or runtime environments
• provides input for performance models
• helps to learn how software interacts with the hardware

One important feature of microbenchmarking is that it is not a black box but a tool to create knowledge and deeper understanding.

Recommended Tools

The difficulty in microbenchmarking is to really measure what you are interested in. Because the things you want to measure are usually very small correct timing is a problem. Also separation of influences may be difficult to guarantee. If, e.g., implementing a microbenchmark in a programming language one must assure that the language does not add overhead that influences the results. Therefore it is usually recommended to use available benchmarks or tools which make it easier to produce meaningful results.

STREAM benchmark

The STREAM benchmark is the industry standard for measuring node-level sustained memory bandwidth. It is a very simple single file implementation of simple streaming loop kernels and should reach peak memory bandwidth on any architecture. Threading is implemented using OpenMP. For meaningful results one has to employ thread affinity control. Measuring main memory bandwidth is the sole purpose of this benchmark.