SLURM - Revision history

Matthias-hubl-45fe@rwth-aachen.de: fixed a typo

2025-06-24T08:37:34Z

fixed a typo

Roderick-dedekind-84ab@uni-koeln.de: updated dead link

2024-09-05T08:16:09Z

updated dead link

Marcus-wagner-bd2e@rwth-aachen.de at 15:33, 28 February 2023

2023-02-28T15:33:00Z

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:29:37Z

Array and Chain Jobs

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:28:47Z

Array and Chain Jobs

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:27:41Z

Array and Chain Jobs

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:21:20Z

Array and Chain Jobs

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:19:07Z

Array and Chain Jobs

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:14:41Z

Array and Chain Jobs

Christian-griebel-aab2@tu-darmstadt.de: /* Array and Chain Jobs */

2021-11-04T01:14:04Z

Array and Chain Jobs

@@ Line 320: / Line 320: @@
 * OMP_NUM_THREADS envvar '''must not''' be set/overwritten on RWTH's clusters in OpenMP and Hybrid jobs; this envvar is set by the system automatically.
 * access to hardware performance counters in order to use [https://doc.itc.rwth-aachen.de/display/CC/likwid <code>likwid-perfctr</code>] or [https://doc.itc.rwth-aachen.de/display/CC/intelvtune Intel VTune] is available using the <code>--hwctr=likwid</code> or  <code>--hwctr=vtune</code>  parameter, respectively.
-* in order to start MPI or Hybrid application please use '''$MPIEXEC $FLAGS_MPI_BATCH ./a.out''' instead of ''srun'' sommand; these envvars are set accorfingly to used MPI vendor by the module system.
+* in order to start MPI or Hybrid application please use '''$MPIEXEC $FLAGS_MPI_BATCH ./a.out''' instead of ''srun'' sommand; these envvars are set accordingly to used MPI vendor by the module system.
 * the shebang of batch script '''must be''' <code>#!/usr/bin/zsh</code>  (otherwise the modules are not accesssible)

@@ Line 327: / Line 327: @@
 [https://doku.lrz.de/display/PUBLIC/Example+parallel+job+scripts+on+the+Linux-Cluster/ Advanced SLURM jobscript examples]
-[http://www.nersc.gov/users/computational-systems/cori/running-jobs/example-batch-scripts/ Detailled guide to more advanced scripts]
+[https://docs.nersc.gov/jobs/examples/ Detailled guide to more advanced scripts]
 [https://slurm.schedmd.com/sbatch.html SBATCH documentation]

@@ Line 321: / Line 321: @@
 * access to hardware performance counters in order to use [https://doc.itc.rwth-aachen.de/display/CC/likwid <code>likwid-perfctr</code>] or [https://doc.itc.rwth-aachen.de/display/CC/intelvtune Intel VTune] is available using the <code>--hwctr=likwid</code> or  <code>--hwctr=vtune</code>  parameter, respectively.
 * in order to start MPI or Hybrid application please use '''$MPIEXEC $FLAGS_MPI_BATCH ./a.out''' instead of ''srun'' sommand; these envvars are set accorfingly to used MPI vendor by the module system.
-* the shebang of batch script '''must be''' <code>#!/usr/local_rwth/bin/zsh</code>  (otherwise the modules are not accesssible)
+* the shebang of batch script '''must be''' <code>#!/usr/bin/zsh</code>  (otherwise the modules are not accesssible)
 == References ==

@@ Line 145: / Line 145: @@
 ; Time series
 Job arrays can also be used to create series of jobs, starting one after another.
 Imagine a cluster with very few nodes allowing jobs up to 7 days runtime, and a lot of nodes dedicated to jobs <24h. If you submit a job with <tt>--time=7-00:00:00</tt>, this job would have to wait in "pending" for a long time.

@@ Line 133: / Line 133: @@
 Slurm will create '''1''' job with '''1000''' elements (subjobs = array tasks), each of these being
 * independent of each other
-* scheduled in/on any free time slot on any free node
+* scheduled in any free time slot on any free compute node
 * run as many in parallel as are nodes & time slots free
 * with less than a tenth of the Slurm-internal efforts as was necessary for distinct single jobs (those created by the above shell loop)

@@ Line 145: / Line 145: @@
 ; Time series
 Job arrays can also be used to create series of jobs, starting one after another.
-Imagine a cluster with very few nodes allowing jobs up to 7 days runtime, and a lot of nodes dedicated to jobs <24h. If ypou submit a job with <tt>--time=7-00:00:00</tt>, this job would have to wait in "pending" for a long time.
+Imagine a cluster with very few nodes allowing jobs up to 7 days runtime, and a lot of nodes dedicated to jobs <24h. If you submit a job with <tt>--time=7-00:00:00</tt>, this job would have to wait in "pending" for a long time.
-If you have a huge simulation to run, using a program capable of [[Snapshotting Jobs|CPR]], you can create a time series of 24h jobs:
+If you have to run a very huge and long simulation, using a program capable of [[Snapshotting Jobs|CPR]], you can create a time series of 24h jobs:
 <syntaxhighlight lang="bash">
   #SBATCH --array=1-19%1
@@ Line 154: / Line 154: @@
   mySim ... -statefile=Simulation19d.state
 </syntaxhighlight>
 Each array task will
 * run for one day, continuously saving the state of the simulation in "Simulation19d.state"
 * be killed by Slurm after 24:00:01 runtime
-* just to be followed by the next array task, which picks up where its predecessor left (by reading in "Simulation19d.state")
+* just to be followed by the next array task, which picks up right at where its predecessor left (by reading in "Simulation19d.state")
-That way, you run your 19 days of simulation in '''19''' single-day chunks, using the many more compute nodes available in the 24h queue!
+That way, you run your 19 day simulation in '''19''' single-day chunks, using the many more compute nodes available in the 24h queue!

@@ Line 114: / Line 114: @@
 == Array and Chain Jobs ==
-Arrays are the best way to submit ''many similar'' jobs. In short: whenever you are tempted to write a shell loop like
+Arrays are the best way to submit ''many similar'' jobs. In short: whenever you are tempted to write a shell loop around <tt>sbatch</tt> like
 <syntaxhighlight lang="bash">
   for i in {1..1000} ; do

@@ Line 119: / Line 119: @@
       sbatch myJobScript ${i}.jpg
   done
 do ''not'' do it - instead, use Slurm's '''job array''' feature.
 ; High-Throughput computing

@@ Line 114: / Line 114: @@
 == Array and Chain Jobs ==
 <syntaxhighlight lang="zsh">
@@ Line 121: / Line 169: @@
 </syntaxhighlight>
-This creates an array job with *2* subjobs (numbered 1..4 with step of 2) where only *one* may be executed at a time in a random order. An explicit order can be forced by either submitting each subjob at the end of the one before (which may prolong pending) or using the dependencies feature, which results in a chain job.
+This creates an array job with *2* subjobs (numbered 1..4 with a stepping of 2) where only *one* may be executed at a time, in a random order.
 <syntaxhighlight lang="zsh">