Binding/Pinning - Revision history

Dirk-ribbrock-3133@tu-dortmund.de: fix logical error - OMP_PROC_BIND is not already set at the start of the OMP_PROC_BIND section

2025-05-30T21:07:31Z

fix logical error - OMP_PROC_BIND is not already set at the start of the OMP_PROC_BIND section

Christian-wassermann-3f22@rwth-aachen.de: Fix OMP_PLACES examples

2022-03-23T13:35:42Z

Fix OMP_PLACES examples

Daniel-schurhoff-de23@rwth-aachen.de at 13:40, 3 September 2019

2019-09-03T13:40:58Z

Stefan-erlbeck-05df@rwth-aachen.de at 10:06, 20 November 2018

2018-11-20T10:06:16Z

Nina-loseke-fd7a@rwth-aachen.de at 13:35, 20 April 2018

2018-04-20T13:35:37Z

Nina-loseke-fd7a@rwth-aachen.de at 13:35, 20 April 2018

2018-04-20T13:35:29Z

Nina-loseke-fd7a@rwth-aachen.de: /* Options for Binding in Intel MPI */

2018-04-20T06:59:34Z

Options for Binding in Intel MPI

Nina-loseke-fd7a@rwth-aachen.de: /* Options for Binding in Intel MPI */

2018-04-20T06:59:22Z

Options for Binding in Intel MPI

Nina-loseke-fd7a@rwth-aachen.de: /* Options for Binding in Intel MPI */

2018-04-20T06:54:30Z

Options for Binding in Intel MPI

Nina-loseke-fd7a@rwth-aachen.de: /* Options for Binding in Intel MPI */

2018-04-20T06:52:46Z

Options for Binding in Intel MPI

@@ Line 54: / Line 54: @@
 === <code>OMP_PROC_BIND</code> ===
-Now that you have set <code>OMP_PROC_BIND</code>, you can now define the order in which the places should be assigned. This is especially useful for NUMA systems (see [[#References|references]] below) because some threads may have to access remote memory, which will slow your application down significantly. If <code>OMP_PROC_BIND</code> is not set, your system will distribute the threads across the nodes and cores randomly.
+Now that you have set <code>OMP_PROC_PLACES</code>, you can now define the order in which the places should be assigned. This is especially useful for NUMA systems (see [[#References|references]] below) because some threads may have to access remote memory, which will slow your application down significantly. If <code>OMP_PROC_BIND</code> is not set, your system will distribute the threads across the nodes and cores randomly.
 {| class="wikitable" style="width:60%;"

@@ Line 13: / Line 13: @@
 == <span id="Pin_OMP"></span> How to Pin Threads in OpenMP ==
-[[File:Omp_places.png|thumb|350px|Schematic of how <code>OMP_PLACES={0}:8:2</code> would be interpreted]]
+[[File:Omp_places.png|thumb|350px|Schematic of how <code>OMP_PLACES={0}:4:2</code> would be interpreted]]
 [[File:Proc_bind_close.PNG|thumb|350px|Schematic of how <code>OMP_PROC_BIND=close</code> would be interpreted on a system comprising 2 nodes with 4 hardware threads each]]
@@ Line 43: / Line 43: @@
 {| class="wikitable" style="width:60%;"
-| Example hardware || <code>OMP_PLACES</code> || Places
+| Example hardware || Desired binding || <code>OMP_PLACES</code> || Places
 |-
-| 24 cores with one hardware thread each, starting at core 0 and using every 2nd core || <code>{0}:24:2</code> or <code>{0:1}:24:2</code> || <code>{0}, {2}, {4}, {6}, {8}, {10}, {12}, {14}, {16}, {18}, {20}, {22}</code>
+| 24 cores with one hardware thread each || starting at core 0 use every 2nd core || <code>{0}:12:2</code> or <code>{0:1}:12:2</code> || <code>{0}, {2}, {4}, {6}, {8}, {10}, {12}, {14}, {16}, {18}, {20}, {22}</code>
 |-
-| 12 cores with two hardware threads each, starting at the first two hardware threads on the first core ({0,1}) and using every 4th core || <code>{0,1}:12:4</code> or <code>{0:2}:12:4</code> || <code>{0,1}, {4,5}, {8,9}, {12,13}, {16,17}, {20,21}</code>
+| 12 cores with two hardware threads each || starting at the first two hardware threads on the first core ({0,1}) use every 4th core || <code>{0,1}:6:4</code> or <code>{0:2}:6:4</code> || <code>{0,1}, {4,5}, {8,9}, {12,13}, {16,17}, {20,21}</code>
 |}

← Older revision		Revision as of 13:40, 3 September 2019
Line 1:		Line 1:
		+	[[Category:HPC-User]]
	== Basics ==		== Basics ==

@@ Line 10: / Line 10: @@
-== How to Pin Threads in OpenMP ==
+== <span id="Pin_OMP"></span> How to Pin Threads in OpenMP ==
 [[File:Omp_places.png|thumb|350px|Schematic of how <code>OMP_PLACES={0}:8:2</code> would be interpreted]]
@@ Line 69: / Line 69: @@
 |}
-== Options for Binding in Open MPI ==
+== <span id="Pin_MPI"></span> Options for Binding in Open MPI ==
 Binding processes to certain processors can be done by specifying the options below when executing a program. This is a more advanced way of running an application and also requires knowledge about your system's architecture, e. g. how many cores there are (for an overview of your hardware topology, use <code>$ lstopo</code>). If none of these options are given, default values are set.

@@ Line 5: / Line 5: @@
 The terms "thread pinning" and "thread affinity" as well as "process binding" and "process affinity" are used interchangeably.
 You can bind processes by specifying additional options when [[How_to_Use_MPI#How_to_Run_an_MPI_Executable|executing]] your [[MPI]] application.
 __TOC__
 == How to Pin Threads in OpenMP ==

@@ Line 104: / Line 104: @@
 |-
 | <code>I_MPI_PIN_ORDER=<order></code> || order in which the MPI processes will be associated with the domains; <code>order</code> can be one of: range, scatter, compact, spread, bunch; <code>compact</code>, for example, will put those domains next to each other that share the same resources
 | <code>I_MPI_PIN_PROCESSOR_LIST=<list></code> || <code>list</code> can be set to a list of CPUs to start processes on, e. g. <code>2,4-6</code> which will select CPUs 2, 4, 5, 6. It can also be set to a procset, which can be set to all, allcores, allsocks, and the processes can be mapped in more detail by passing more parameters (see [[#References|references]] below)
 |}

@@ Line 103: / Line 103: @@
 | <code>I_MPI_PIN_DOMAIN=<arg></code> || defines a separate set of CPUs (domain) on a single node; one process will be started per domain; targeted mainly at hybrid programs; arg can be one of: core, socket, node, cache1/2/3, cache; to define a custom pattern, put: <code><size>[:<layout>]</code> with size as the number of CPUs per domain, and layout as the distance inbetween domain members (e. g. scatter, compact)
 |-
 | <code>I_MPI_PIN_PROCESSOR_LIST=<list></code> || <code>list</code> can be set to a list of CPUs to start processes on, e. g. <code>2,4-6</code> which will select CPUs 2, 4, 5, 6. It can also be set to a procset, which can be set to all, allcores, allsocks, and the processes can be mapped in more detail by passing more parameters (see [[#References|references]] below)
 |}

@@ Line 101: / Line 101: @@
 | <code>I_MPI_PIN_CELL=<cell></code> || defines granularity of pinning resolution, where "unit" specifies a logical CPU and "core" a physical core; if <code>I_MPI_PIN_DOMAIN</code> is set, the cell will be "unit"; if you are using <code>I_MPI_PIN_PROCESSOR_LIST</code>, cell can be either, depending on the number of processes and cores available on your machine
 |-
-| <code>I_MPI_PIN_DOMAIN=<arg></code> || defines a separate set of CPUs(domain) on a single node; one process will be started per domain; targeted mainly at hybrid programs; arg can be one of: core, socket, node, cache1/2/3, cache; to define a custom pattern, put: <code><size>[:<layout>]</code> with size as the number of CPUs per domain, and layout as the distance inbetween domain members (e. g. scatter, compact)
+| <code>I_MPI_PIN_DOMAIN=<arg></code> || defines a separate set of CPUs (domain) on a single node; one process will be started per domain; targeted mainly at hybrid programs; arg can be one of: core, socket, node, cache1/2/3, cache; to define a custom pattern, put: <code><size>[:<layout>]</code> with size as the number of CPUs per domain, and layout as the distance inbetween domain members (e. g. scatter, compact)
 |-
-| <code>I_MPI_PIN_PROCESSOR_LIST=<list></code> || <code>list</code> can be set to a list, e. g. <code>2,4-6</code> which will select CPUs 2, 4, 5, 6. It can also be set to a procset, which can be set to all, allcores, allsocks, and can be further mapped by passing more parameters (see [[#References|references]] below)
+| <code>I_MPI_PIN_PROCESSOR_LIST=<list></code> || <code>list</code> can be set to a list of CPUs to start processes on, e. g. <code>2,4-6</code> which will select CPUs 2, 4, 5, 6. It can also be set to a procset, which can be set to all, allcores, allsocks, and the processes can be mapped in more detail by passing more parameters (see [[#References|references]] below)
 |}

@@ Line 97: / Line 97: @@
 | <code>I_MPI_PIN=<arg></code> || these values for arg enable binding: enable, yes, on, 1, and these disable binding: disable, no, off, 0
 |-
-| <code>I_MPI_PIN_MODE=<pinmode></code> || if set to mpd or pm, the available process manager will take care of the pinning, if pinmode=lib, the Intel MPI library pins the processes(per default, pinmode is set to "pm")
+| <code>I_MPI_PIN_MODE=<pinmode></code> || if set to mpd or pm, the available process manager will take care of the pinning, if pinmode=lib, the Intel MPI library pins the processes (per default, pinmode is set to "pm")
 |-
 | <code>I_MPI_PIN_CELL=<cell></code> || defines granularity of pinning resolution, where "unit" specifies a logical CPU and "core" a physical core; if <code>I_MPI_PIN_DOMAIN</code> is set, the cell will be "unit"; if you are using <code>I_MPI_PIN_PROCESSOR_LIST</code>, cell can be either, depending on the number of processes and cores available on your machine