Difference between revisions of "Batch-Scheduler"

From HPC Wiki
Jump to: navigation, search
(Created page with "This page shall provide a general overview of the problems with a batch scheduler: Different types of jobs: * seriell * parallel (shared memory) * parallel (mehrere nodes, mp...")
 
Line 1: Line 1:
This page shall provide a general overview of the problems with a batch scheduler:
+
This page gives an overview of what a Batch-Scheduler can do and what pitfalls may exist. A more general description of why Batch-Schedulers are needed can be found [[Scheduling_Basics|here]]. There are different Schedulers around, e.g. [[SLURM]], [[LSF]] and [[Torque]]. Click [[Schedulers|here]] to figure out which one you need.
  
Different types of jobs:
+
__TOC__
* seriell
 
* parallel (shared memory)
 
* parallel (mehrere nodes, mpi)
 
* parallel hybrid
 
  
Hier evtl. am [https://doc.itc.rwth-aachen.de/pages/viewpage.action?pageId=3473705 Primer] orientieren
+
== Usage ==
 +
If you want to execute a program in the batch system, you need to submit the [[Jobscript]] you have written for the Scheduler used on the batch system. In this Jobscript, the Scheduler needs to learn:
 +
* how many resources your program needs (e.g. time and memory)
 +
* how you want to parallelize your program
 +
There are in general four types of parallelization:
 +
* serial (no parallelization)
 +
* shared memory only (e.g. OpenMP)
 +
* distributed memory only (e.g. MPI)
 +
* hybrid parallelisation
 +
 
 +
== Serial Jobs ==
 +
== Shared Memory Parallelization ==
 +
== Distributed Memory Parallelization ==
 +
== Hybrid Parallelization ==
 +
== Advanced Usage ==
 +
Here is stuff about
 +
*brief mentioning of non-mpi-multi-noding
 +
*that you should split long-runners (aka Chain Jobs) and why
 +
*a brief mentioning of array jobs

Revision as of 14:46, 14 November 2018

This page gives an overview of what a Batch-Scheduler can do and what pitfalls may exist. A more general description of why Batch-Schedulers are needed can be found here. There are different Schedulers around, e.g. SLURM, LSF and Torque. Click here to figure out which one you need.

Usage

If you want to execute a program in the batch system, you need to submit the Jobscript you have written for the Scheduler used on the batch system. In this Jobscript, the Scheduler needs to learn:

  • how many resources your program needs (e.g. time and memory)
  • how you want to parallelize your program

There are in general four types of parallelization:

  • serial (no parallelization)
  • shared memory only (e.g. OpenMP)
  • distributed memory only (e.g. MPI)
  • hybrid parallelisation

Serial Jobs

Shared Memory Parallelization

Distributed Memory Parallelization

Hybrid Parallelization

Advanced Usage

Here is stuff about

  • brief mentioning of non-mpi-multi-noding
  • that you should split long-runners (aka Chain Jobs) and why
  • a brief mentioning of array jobs