Benchmarking & Scaling Tutorial/Automated Benchmarking - Revision history

Marc-andre-hermanns-bc32@rwth-aachen.de: Setting 'next' link to JUBE page

2022-11-17T14:39:06Z

Setting 'next' link to JUBE page

Sebastian-potthoff-3c73@uni-muenster.de at 13:11, 23 September 2021

2021-09-23T13:11:42Z

Sebastian-potthoff-3c73@uni-muenster.de at 12:14, 23 September 2021

2021-09-23T12:14:08Z

Sebastian-potthoff-3c73@uni-muenster.de at 12:10, 23 September 2021

2021-09-23T12:10:35Z

Sebastian-potthoff-3c73@uni-muenster.de at 12:07, 23 September 2021

2021-09-23T12:07:25Z

Sebastian-potthoff-3c73@uni-muenster.de: Created page with "{{DISPLAYTITLE:Automated Benchmarking using a Job Script}} {{Syllabus Benchmarking & Scaling}} TOC == Automate test runs == While still being logged..."

2021-09-23T11:35:32Z

Created page with "{{DISPLAYTITLE:Automated Benchmarking using a Job Script}}<nowiki /> {{Syllabus Benchmarking & Scaling}}<nowiki /> __TOC__ == Automate test runs == While still being logged..."

New page

{{DISPLAYTITLE:Automated Benchmarking using a Job Script}}<nowiki />
{{Syllabus Benchmarking & Scaling}}<nowiki />
__TOC__

== Automate test runs ==

While still being logged in at our interactive session on the compute node, we can start writing a Bash script to automate running simulations with different amounts of cores. After all we want to know how our simulation scales and determine the speedup.

We will start by loading all needed modules and defining a variable GMX to hold our complete GROMACS command

<syntaxhighlight lang="bash">
#!/bin/bash

# Load all needed modules (adjust to your specific site!)
module load GROMACS

GMX="gmx_mpi -quiet mdrun -deffnm MD_5NM_WATER -nsteps 10000 -ntomp 1 -pin on "

</syntaxhighlight>

Next we will create a loop over different amounts of cores our simulation should run with. It is advisable to start with the highest core count first, as these take the least amount of time. Which core counts you want to test depends on the system you are running on. Make sure to include the serial case (1 core) and the full node. In the first example we will use all ''logical'' cores (i.e. use hyperthreading). It is also a good idea to test a full ''socket'', in this case 36 cores.

<syntaxhighlight lang="bash">
for P in 72 66 60 54 48 42 36 30 24 18 12 8 4 2 1; do
srun --cpu-bind=threads -n $P $GMX
done
</syntaxhighlight>

----
'''Next''': [[Benchmarking_%26_Scaling_Tutorial/Results | Plotting and Interpreting Results ]]

'''Previous''': [[Benchmarking_%26_Scaling_Tutorial/Manual_Benchmarking | Interactive Manual Benchmarking ]]

← Older revision		Revision as of 13:11, 23 September 2021
Line 79:		Line 79:


		+	We can save this script in a file called e.g. <code>job.sh</code> and run it via the bash command. To see the output in the terminal and save it to file at the same time we pipe it to the <code>tee</code> command
		+
		+	bash job.sh \| tee timings.dat
		+
		+	The output will look similar to this
		+
		+	[[File:Benchmark_tutorial_gromacs_5.png\|600px]]
		+
		+
		+	== Transform to a job script ==
		+
		+	We can easily transform this into a SLURM job script by adding the necessary <code>SBATCH</code> pragmas. Here we also show you the job script to only use ''physical'' cores instead of all threads per core by using <code>--cpu-bind=cores</code> and reducing the total amount of cores to 36. Note that we still request ''all logical cores'' to receive a full node. Adjust the <code>SBATCH</code> pragmas for your site!
		+
		+	<syntaxhighlight lang="bash">
		+	#!/bin/bash
		+
		+	#SBATCH --nodes=1
		+	#SBATCH --ntasks-per-node=72
		+	#SBATCH --partition=express
		+	#SBATCH --time=01:00:00
		+	#SBATCH --job-name=benchmark
		+	#SBATCH --output=benchmark_results_noHT.out
		+
		+	# Load all needed modules (adjust to your specific site!)
		+	module load GROMACS
		+
		+	GMX="gmx_mpi -quiet mdrun -deffnm MD_5NM_WATER -nsteps 10000 -ntomp 1 -pin on "
		+
		+	echo "# Cores Time/s"
		+	for P in 36 30 24 18 12 8 4 2 1; do
		+	for i in $(seq 5); do
		+	START=$(date +%s.%N)
		+	srun --cpu-bind=core -n $P $GMX > gromacs.log 2>&1
		+	END=$(date +%s.%N)
		+	RUNTIME=$(echo "$END - $START" \| bc -l)
		+	printf "%3d %5.2f\n" $P $RUNTIME
		+	done
		+	done
		+
		+	</syntaxhighlight>

	----		----

@@ Line 67: / Line 67: @@
 echo "# Cores Time/s"
 for P in 72 66 60 54 48 42 36 30 24 18 12 8 4 2 1; do
-     for i in $(seq 5); doe
+     for i in $(seq 5); do
          START=$(date +%s.%N)
          srun --cpu-bind=threads -n $P $GMX > gromacs.log 2>&1

@@ Line 3: / Line 3: @@
 __TOC__
-== Automate test runs ==
+== Writing a Bash script ==
 While still being logged in at our interactive session on the compute node, we can start writing a Bash script to automate running simulations with different amounts of cores. After all we want to know how our simulation scales and determine the speedup.
@@ Line 41: / Line 41: @@
-We also want to write out the used number of cores and the measured time for each run to the console (stdout) using a <code>printf</code> statement. In addition we add a little header to know want the numbers actually mean afterwards
+We also want to write out the used number of cores and the measured time for each run to the console (stdout) using a <code>printf</code> statement. In addition we add a little header before the loop to know want the numbers actually mean
 <syntaxhighlight lang="bash">

@@ Line 121: / Line 121: @@
 ----
-'''Next''': [[Benchmarking_%26_Scaling_Tutorial/Results | Plotting and Interpreting Results ]]
+'''Next''': [[Benchmarking_%26_Scaling_Tutorial/Automated_Benchmarking_JUBE | Automated Benchmarking using JUBE ]]
 '''Previous''': [[Benchmarking_%26_Scaling_Tutorial/Manual_Benchmarking | Interactive Manual Benchmarking ]]

@@ Line 20: / Line 20: @@
-Next we will create a loop over different amounts of cores our simulation should run with. It is advisable to start with the highest core count first, as these take the least amount of time. Which core counts you want to test depends on the system you are running on. Make sure to include the serial case (1 core) and the full node. In the first example we will use all ''logical'' cores (i.e. use hyperthreading). It is also a good idea to test a full ''socket'', in this case 36 cores.
+Next we will create a loop over different amounts of cores our simulation should run with. It is advisable to start with the highest core count first, as these take the least amount of time. Which core counts you want to test depends on the system you are running on. Make sure to include the serial case (1 core) and the full node. In the first example we will use all ''logical'' cores (i.e. use hyperthreading). It is also a good idea to test a full ''socket'', in this case 36 cores. We will also tell ''srun'' to bind processes to threads to make sure we are actually using all threads of a physical core even with lower core counts.
 <syntaxhighlight lang="bash">
@@ Line 27: / Line 27: @@
 done
 </syntaxhighlight>
 ----