Scheduling Basics - Revision history

Karsten-siegmund-92ed@uni-ulm.de: /* Scheduling Algorithms */

2024-09-11T12:06:27Z

Scheduling Algorithms

Christian-griebel-aab2@tu-darmstadt.de: /* Illustration */

2019-11-03T13:15:48Z

Illustration

Daniel-schurhoff-de23@rwth-aachen.de at 13:07, 3 September 2019

2019-09-03T13:07:24Z

Rene-sitt-3d1d@uni-marburg.de: Fixed spelling error

2019-08-02T14:28:23Z

Fixed spelling error

Christian-griebel-aab2@tu-darmstadt.de: /* General */

2019-06-03T14:44:31Z

General

Christian-griebel-aab2@tu-darmstadt.de: /* General */

2019-06-03T14:43:05Z

General

Christian-griebel-aab2@tu-darmstadt.de: /* General */

2019-06-03T14:42:34Z

General

Stefan-erlbeck-05df@rwth-aachen.de at 11:52, 14 November 2018

2018-11-14T11:52:19Z

Stefan-erlbeck-05df@rwth-aachen.de at 11:42, 14 November 2018

2018-11-14T11:42:01Z

Stefan-erlbeck-05df@rwth-aachen.de at 11:19, 14 November 2018

2018-11-14T11:19:49Z

@@ Line 50: / Line 50: @@
 '''First Come, First Serve'''
-Jobs are run in the exact same order in which they first enter the queue. The advantage is that every job will definitely be run, however, very tiny jobs might wait for an inadequately long time compared to their actual execution time.
+Jobs are run in the exact same order in which they first enter the queue. The advantage is that every job will definitely be run, however, very tiny jobs might wait for an inadequately long time compared to their actual execution time or all jobs would need to wait at some point if one user submits several thousand jobs at the same time.
 '''Shortest Job First'''

@@ Line 40: / Line 40: @@
 Therefore, the jobs may not be distributed among the nodes in the same order in which they first entered the queue.
 The space that a job takes up is determined by the time and number of nodes required for executing it.
 == Scheduling Algorithms ==

← Older revision		Revision as of 13:07, 3 September 2019
Line 1:		Line 1:
		+	[[Category:Basics]]
	This is an overview over the basic concepts/goals of a scheduler. An overview which HPC centers uses which scheduler, can be found [[Schedulers\|here]]. Further Information about [[Batch-Scheduler\|Batch-Schedulers]] are available, as well as specific information about the schedulers [[SLURM]], [[LSF]] and [[Torque]].		This is an overview over the basic concepts/goals of a scheduler. An overview which HPC centers uses which scheduler, can be found [[Schedulers\|here]]. Further Information about [[Batch-Scheduler\|Batch-Schedulers]] are available, as well as specific information about the schedulers [[SLURM]], [[LSF]] and [[Torque]].

@@ Line 36: / Line 36: @@
 Assuming that the batch system you are using consists of 6 nodes, this is how the scheduler could place the nine jobs in the queue onto the available nodes.
-The goal is to eleminiate wasted resources, which can be identified on the right by looking at the free areas depicting nodes without any job execution on them.
+The goal is to eliminate wasted resources, which can be identified on the right by looking at the free areas depicting nodes without any job execution on them.
 Therefore, the jobs may not be distributed among the nodes in the same order in which they first entered the queue.
 The space that a job takes up is determined by the time and number of nodes required for executing it.

@@ Line 12: / Line 12: @@
 In contrast to the login nodes (for compiling and testing user software) and their interactive usage, these compute nodes are usually not directly accessible (via <code>ssh</code>).
-The batch system is thus the interface for the users on the [[Nodes#Login|login nodes]] to send work to compute nodes.
+The scheduler is thus the interface for the users on the [[Nodes#Login|login nodes]] to send work to the compute nodes.
 This requires the user to ask the scheduler for time and memory resources and to specify the application inside a [[jobscript]].

@@ Line 10: / Line 10: @@
 The scheduler stores the batch jobs, evaluate their resource requirements and priorities, and distributes the jobs to suitable ''compute nodes''. These work horses make up the majority of HPC clusters (about 98%), being their most powerful, but also the most power consuming parts.
-In contrast to the login nodes (for compiling and testing user software) and their interactive usage, these compute nodes are usually not directly accessible (via <code>ssh>/code>).
+In contrast to the login nodes (for compiling and testing user software) and their interactive usage, these compute nodes are usually not directly accessible (via <code>ssh</code>).
 The batch system is thus the interface for the users on the [[Nodes#Login|login nodes]] to send work to compute nodes.

@@ Line 7: / Line 7: @@
 [[File:Batch_System.PNG|thumb|500px|Schematic of how users can access the batch system]]
-A scheduler is software that controls a batch system. Such a batch system makes up the majority of a cluster (about 98%). It is the most powerful, but also most power consuming part of the whole cluster, and hence intended for executing memory demanding and long running applications.
+A scheduler is software that implements a ''batch system'' on a HPC (cluster). Users do not run their calculations directly and interactively (as they do on their personal workstations or laptops), instead they submit non-interactive ''batch jobs'' to the scheduler.<br />
-Such a batch system can be viewed as the counterpart of [[Nodes#Login|login nodes]]. A user can simply log in to so-called "front-end" nodes and type in commands that are immediately executed on the same machine. A batch system, on the other hand, consists of "back-end" nodes, which cannot be accessed by the user directly. In order to run an application on there, the user has to ask for time and memory resources and specify the application inside a [[jobscript]].
+In contrast to the login nodes (for compiling and testing user software) and their interactive usage, these compute nodes are usually not directly accessible (via <code>ssh>/code>).
-This jobscript can be submitted to the batch system via the scheduler, which will first add the job to a job queue. Based on the resources the job needs, the scheduler will decide when the job will leave the queue, and on which part of the back-end nodes it will run.
+The batch system is thus the interface for the users on the [[Nodes#Login|login nodes]] to send work to compute nodes.
 Be careful about the resources you request and know your system's limits. For example, if you
-* demand less time than your job actually needs to finish, the scheduler will simply kill the job once the given time is up.
+* demand less time than your job actually needs to finish, the scheduler will simply kill the job once the time allocated is up.
-* specify more memory than there is available on the system, your job might be stuck in queue forever.
+* specify more memory than there is available on the system, your job might be stuck in the queue forever.
 == Purpose ==

← Older revision		Revision as of 11:52, 14 November 2018
Line 1:		Line 1:
−	This is an overview over the basic concepts/goals of a scheduler. An overview which HPC centers uses which scheduler, can be found [[Schedulers\|here]]. ~~Futher~~ Information about [[Batch-Scheduler\|Batch-Schedulers]] are available, as well as specific information about the schedulers [[SLURM]], [[LSF]] and [[Torque]].	+	This is an overview over the basic concepts/goals of a scheduler. An overview which HPC centers uses which scheduler, can be found [[Schedulers\|here]]. Further Information about [[Batch-Scheduler\|Batch-Schedulers]] are available, as well as specific information about the schedulers [[SLURM]], [[LSF]] and [[Torque]].

	__TOC__		__TOC__

@@ Line 38: / Line 38: @@
 == Scheduling Algorithms ==
-There two very basic strategies that schedulers can use to determine which job to run next. Note that modern schedulers do not stick strictly to just one of these algorithms, but rather employ a combination of the two.
+There are two very basic strategies that schedulers can use to determine which job to run next. Note that modern schedulers do not stick strictly to just one of these algorithms, but rather employ a combination of the two.
 Besides, there are many more aspects a scheduler has to take into consideration, e. g. the current system load.

← Older revision		Revision as of 11:19, 14 November 2018
Line 1:		Line 1:
−	This is an overview over the basic concepts/goals of a ~~sheduler~~. An overview which HPC centers uses which ~~sheduler~~, can be found [[Schedulers\|here]]. Futher Information about [[Batch-Scheduler\|Batch-Schedulers]] are available, as well as specific information about the ~~shedulers~~ [[SLURM]], [[LSF]] and [[Torque]].	+	This is an overview over the basic concepts/goals of a scheduler. An overview which HPC centers uses which scheduler, can be found [[Schedulers\|here]]. Futher Information about [[Batch-Scheduler\|Batch-Schedulers]] are available, as well as specific information about the schedulers [[SLURM]], [[LSF]] and [[Torque]].

	__TOC__		__TOC__