SynchronizationOverhead

From HPC Wiki
Jump to: navigation, search

Description

The pattern "Synchonization overhead" describes the performance limitation caused by frequent synchronization calls in parallel environments. Each synchronization causes threads that finished earlier with their workload have to wait for slower threads.


Symptoms

  • Speedup going down as more cores are added
  • No speedup with small problem sizes
  • Cores are busy but low FP performance


Detection

  • Large non-FP instruction count (growing with used number of cores)
  • Low (good) CPI although misleading

The CPI is measured by LIKWID in all groups. For the instruction/FP ratio, use the FLOPS_DP/FLOPS_SP groups.

Possible optimizations and/or fixes

  • Reduce number of synchronization points


Applicable applications or algorithms or kernels