Difference between revisions of "Application benchmarking"
Jump to navigation
Jump to search
(Created page with "== Overview == Application benchmarking is an elementary skill for any performance engineering effort. Because it is the base for any other acitivity it is crucial to measure...") |
|||
| Line 5: | Line 5: | ||
* '''Timing''': How to accuratly measure time in software. | * '''Timing''': How to accuratly measure time in software. | ||
* '''Documentation''': Because there are many influences it is essential to document all possible performance relvant influences. | * '''Documentation''': Because there are many influences it is essential to document all possible performance relvant influences. | ||
| − | * '''System | + | * '''System configuration''': Modern systems allow to adjust many performance relevant settings as clock speed, memory settings, cache organisation as well as OS settings. |
| − | * ''' | + | * '''Resource allocation and affinity control''': What resources are used and how is worked mapped on resources. |
| + | |||
| + | Because so many things can go wrong while benchmarking it is imporatant to have a sceptical attitude against good results. Especially for very good results one has to check if the result is reasonable. Further results must be deterministic and reproducable, if required statistic distribution over multiple runs has to be documented. | ||
| + | |||
| + | In the following all examples use the [https://github.com/RRZE-HPC/likwid Likwid Performance Tools] for tool support. | ||
| + | |||
| + | == Timing == | ||
| + | |||
| + | |||
| + | == Documentation == | ||
| + | |||
| + | |||
| + | == System configuration == | ||
| + | |||
| + | |||
| + | == Affinity control == | ||
Revision as of 14:55, 16 January 2019
Overview
Application benchmarking is an elementary skill for any performance engineering effort. Because it is the base for any other acitivity it is crucial to measure result in an accurate, deterministic and reproducible way. The following components are required for meaningful application benchmarking:
- Timing: How to accuratly measure time in software.
- Documentation: Because there are many influences it is essential to document all possible performance relvant influences.
- System configuration: Modern systems allow to adjust many performance relevant settings as clock speed, memory settings, cache organisation as well as OS settings.
- Resource allocation and affinity control: What resources are used and how is worked mapped on resources.
Because so many things can go wrong while benchmarking it is imporatant to have a sceptical attitude against good results. Especially for very good results one has to check if the result is reasonable. Further results must be deterministic and reproducable, if required statistic distribution over multiple runs has to be documented.
In the following all examples use the Likwid Performance Tools for tool support.