Ds019135:

== [[Access|Access]] or "How-to-be-allowed-onto-the-supercomputer" ==
Depending on the specific supercomputer, one has to either register to get a user account or write a project proposal and apply for computing resources that way. The respective pages are linked in [[Access|this overview]].

After this is done and login credentials are supplied, one can proceed to [[ Getting_Started#Login_or_.22How-to-now-actually-connect-to-the-supercomputer.22 | login ]].

== [[Nodes#Login|Login]] or "How-to-now-actually-connect-to-the-supercomputer" ==
Most HPC Systems are unix-based environments with [[shell]] (commandline) access.

To log in, one usually uses [[ssh]] to reach the respective [[Nodes#Login|Login Nodes]] (computers reserved for people just like you that want to connect to the supercomputer). Sometimes this access is restricted, so you can only connect, when you are within the university/facility and its network. To still access the Login Nodes externally, one can 'pretend to be inside the network' by using a [[VPN|Virtual Private Network (VPN)]].

Once there, the user can interact with the system and run (small) programs to generally test the system/software.

== [[File_Transfer|File Transfer]] or "How-to-get-your-data-onto-or-off-the-supercomputer" ==
To get your data (files) onto the supercomputer or back to your local machine, there are usually different ways. Sometimes there are computers specifically reserved for this purpose called [[Nodes#Copy|copy nodes]].

If available to you, it is recommened to use these copy nodes to move data to or from the supercomputer, since this will result in a better connection and disturb other users less. Additionally the tools mentioned below might only work on these nodes. If there are no dedicated copy nodes, you can usually use the [[Nodes#Login|Login Nodes]] for this purpose.

Commonly used and widely supported copying tools are [[rsync]] which mirrors directories (folders) between the supercomputer and your local machine. [[scp]] which is useful for a few single files or specified file-lists, and lastly the commonly used [[ftp]] or the encrypted version sftp (or ftps).
A little bit more information can be found in the [[File_Transfer|File Transfer]] article.

== [[Scheduler|Schedulers]] or "How-To-Run-Applications-on-a-supercomputer" ==
To run any significant program or workload on a supercomputer, generally [[Scheduler|schedulers]] are employed. Alongside the above-mentioned Login Nodes there are usually far more Backend Nodes in the system (computers exclusively reserved for computing, to which you cannot connect directly, also referred to as "batch system"). A program called scheduler decides who gets how many of those compute resources for which amount of time. Please use the Backend Nodes for everything which is not a simple small test and only runs for a few minutes., otherwise you will block the Login Nodes for everybody when you run your calculations there. These Backend Nodes make up more than 98% of a supercomputer and can only be accessed via the scheduler.

When you log into a supercomputer, you can run commands on the Login Nodes interactively. You type, you hit return, the command gets executed. Schedulers work differently. You submit a series of commands (in form of a file) and tell it, how much resources it will approximately need in terms of:

* time: If the specified time runs out, before your application finishes and exits, it will be terminated by the scheduler.
* compute resources: how many cpus ('calculation thingies'), sockets ('cpu-houses') and nodes ('computers')
* memory resources: how much RAM ('very fast memory, similar to the few books you have at home')

This combination of specified commands and required resources is commonly referred to as a "(batch) job".

If later compute resources become free, which match the requirements of your application, the scheduler will run your specified commands on the requested hardware. This is usually delayed (sometimes you have to wait a day or two) and not instant, because other users are currently using the compute resources and you have to wait until their program runs finish. Furthermore you cannot change the series of commands after submitting, but just terminate the job and submit a new one in case of an error.

The file specifying this series of commands and the required resources is called a [[jobscript]]. Its format and syntax depends on the installed scheduler. When you have this jobscript ready with the help of [[jobscript-examples]], colleagues or your local [[support]], you can submit it to the respective [[Schedulers|scheduler of your facility]]. The scheduler then waits until a set of nodes (computers) are free and later allocates those to execute your job as soon as possible. Sometimes there is (an optional) email notification, which is send when your job starts execution/finished running.

Be aware that your specified requirements have to fit within the boundaries of the system of your facility. If you ask for more than there is, chances are, the scheduler will accept your job and wait until missing hardware is bought and installed - although this will not happen in 99.9% of cases. Information over the available hardware can be found in the [https://gauss-allianz.de/de/hpc-ecosystem overview of the Gauss Allianz] or the [[Site-specific_documentation|documentation of the different sites]]. You can find more information about [[Getting_Started#Parallel_Programming_or_.22How-To-Use-More-Than-One-Core.22|parallelizing programs here]]. Also there is an [[Schedulers|overview of the schedulers used at the different sites]].

== [[Modules|Modules]] or "How-To-Use-Software-Without-installing-everything-yourself" ==
Since a lot of applications rely on 3rd party software, there is a program on most supercomputers, called the [[Modules|Module system]]. With this system, other software, like compilers or special math libraries, are easily loadable and usable. Depending on the institution, different modules might be available, but there are usually common ones like the [[Compiler#Intel_Compiler|Intel]] or [[Compiler#Gnu_Compiler_Collection|GCC]] [[Compiler|Compilers]].

A few common commands, to enter into the supercomputer commandline and talk to the module system, are
{| class="wikitable" style="width: 40%;"
| module list || lists loaded modules
|-
| module avail || lists available (loadable) modules
|-
| module load/unload x || loads/unloads module x
|-
| module switch x y || switches out module x for module y
|}

If you recurrently need lots of modules, this loading can be automated with an [[sh-file]], so that you just have to execute the file once and it loads all modules, you need.

== [[Parallel_Programming|Parallel Programming]] or "How-To-Use-More-Than-One-Core" ==
Currently development of computers is at a point, where you cannot just make a processor run faster (e.g. by increasing its clock frequency), because limits of physics have been reached in semiconductor development. Therefore the current approach is to split the work into multiple, ideally independent parts, which are then executed in parallel. Similar to cleaning your house, where everybody takes care of a few rooms, on a supercomputer this is usually done with parallel programming paradigms like [[OpenMP|Open Multi-Processing (OpenMP)]] or [[MPI|Message Passing Interface (MPI)]]. However like the fact that you only have one vacuum cleaner in the whole house which not everybody can use at the same time, there are limits on how fast you can get, even with a big number of processing units/cpus/cores (analogous to people in the metaphor) working on your problem (cleaning the house) in parallel.

Getting Started

2018-07-31T15:22:24Z

Ds019135:

asdf
== [[Access|Access]] or "How-to-be-allowed-onto-the-supercomputer" ==
Depending on the specific supercomputer, one has to either register to get a user account or write a project proposal and apply for computing resources that way. The respective pages are linked in [[Access|this overview]].

After this is done and login credentials are supplied, one can proceed to [[ Getting_Started#Login_or_.22How-to-now-actually-connect-to-the-supercomputer.22 | login ]].

== [[Nodes#Login|Login]] or "How-to-now-actually-connect-to-the-supercomputer" ==
Most HPC Systems are unix-based environments with [[shell]] (commandline) access.

To log in, one usually uses [[ssh]] to reach the respective [[Nodes#Login|Login Nodes]] (computers reserved for people just like you that want to connect to the supercomputer). Sometimes this access is restricted, so you can only connect, when you are within the university/facility and its network. To still access the Login Nodes externally, one can 'pretend to be inside the network' by using a [[VPN|Virtual Private Network (VPN)]].

Once there, the user can interact with the system and run (small) programs to generally test the system/software.

== [[File_Transfer|File Transfer]] or "How-to-get-your-data-onto-or-off-the-supercomputer" ==
To get your data (files) onto the supercomputer or back to your local machine, there are usually different ways. Sometimes there are computers specifically reserved for this purpose called [[Nodes#Copy|copy nodes]].

If available to you, it is recommened to use these copy nodes to move data to or from the supercomputer, since this will result in a better connection and disturb other users less. Additionally the tools mentioned below might only work on these nodes. If there are no dedicated copy nodes, you can usually use the [[Nodes#Login|Login Nodes]] for this purpose.

Commonly used and widely supported copying tools are [[rsync]] which mirrors directories (folders) between the supercomputer and your local machine. [[scp]] which is useful for a few single files or specified file-lists, and lastly the commonly used [[ftp]] or the encrypted version sftp (or ftps).
A little bit more information can be found in the [[File_Transfer|File Transfer]] article.

== [[Scheduler|Schedulers]] or "How-To-Run-Applications-on-a-supercomputer" ==
To run any significant program or workload on a supercomputer, generally [[Scheduler|schedulers]] are employed. Alongside the above-mentioned Login Nodes there are usually far more Backend Nodes in the system (computers exclusively reserved for computing, to which you cannot connect directly, also referred to as "batch system"). A program called scheduler decides who gets how many of those compute resources for which amount of time. Please use the Backend Nodes for everything which is not a simple small test and only runs for a few minutes., otherwise you will block the Login Nodes for everybody when you run your calculations there. These Backend Nodes make up more than 98% of a supercomputer and can only be accessed via the scheduler.

When you log into a supercomputer, you can run commands on the Login Nodes interactively. You type, you hit return, the command gets executed. Schedulers work differently. You submit a series of commands (in form of a file) and tell it, how much resources it will approximately need in terms of:

* time: If the specified time runs out, before your application finishes and exits, it will be terminated by the scheduler.
* compute resources: how many cpus ('calculation thingies'), sockets ('cpu-houses') and nodes ('computers')
* memory resources: how much RAM ('very fast memory, similar to the few books you have at home')

This combination of specified commands and required resources is commonly referred to as a "(batch) job".

If later compute resources become free, which match the requirements of your application, the scheduler will run your specified commands on the requested hardware. This is usually delayed (sometimes you have to wait a day or two) and not instant, because other users are currently using the compute resources and you have to wait until their program runs finish. Furthermore you cannot change the series of commands after submitting, but just terminate the job and submit a new one in case of an error.

The file specifying this series of commands and the required resources is called a [[jobscript]]. Its format and syntax depends on the installed scheduler. When you have this jobscript ready with the help of [[jobscript-examples]], colleagues or your local [[support]], you can submit it to the respective [[Schedulers|scheduler of your facility]]. The scheduler then waits until a set of nodes (computers) are free and later allocates those to execute your job as soon as possible. Sometimes there is (an optional) email notification, which is send when your job starts execution/finished running.

Be aware that your specified requirements have to fit within the boundaries of the system of your facility. If you ask for more than there is, chances are, the scheduler will accept your job and wait until missing hardware is bought and installed - although this will not happen in 99.9% of cases. Information over the available hardware can be found in the [https://gauss-allianz.de/de/hpc-ecosystem overview of the Gauss Allianz] or the [[Site-specific_documentation|documentation of the different sites]]. You can find more information about [[Getting_Started#Parallel_Programming_or_.22How-To-Use-More-Than-One-Core.22|parallelizing programs here]]. Also there is an [[Schedulers|overview of the schedulers used at the different sites]].

== [[Modules|Modules]] or "How-To-Use-Software-Without-installing-everything-yourself" ==
Since a lot of applications rely on 3rd party software, there is a program on most supercomputers, called the [[Modules|Module system]]. With this system, other software, like compilers or special math libraries, are easily loadable and usable. Depending on the institution, different modules might be available, but there are usually common ones like the [[Compiler#Intel_Compiler|Intel]] or [[Compiler#Gnu_Compiler_Collection|GCC]] [[Compiler|Compilers]].

A few common commands, to enter into the supercomputer commandline and talk to the module system, are
{| class="wikitable" style="width: 40%;"
| module list || lists loaded modules
|-
| module avail || lists available (loadable) modules
|-
| module load/unload x || loads/unloads module x
|-
| module switch x y || switches out module x for module y
|}

If you recurrently need lots of modules, this loading can be automated with an [[sh-file]], so that you just have to execute the file once and it loads all modules, you need.

== [[Parallel_Programming|Parallel Programming]] or "How-To-Use-More-Than-One-Core" ==
Currently development of computers is at a point, where you cannot just make a processor run faster (e.g. by increasing its clock frequency), because limits of physics have been reached in semiconductor development. Therefore the current approach is to split the work into multiple, ideally independent parts, which are then executed in parallel. Similar to cleaning your house, where everybody takes care of a few rooms, on a supercomputer this is usually done with parallel programming paradigms like [[OpenMP|Open Multi-Processing (OpenMP)]] or [[MPI|Message Passing Interface (MPI)]]. However like the fact that you only have one vacuum cleaner in the whole house which not everybody can use at the same time, there are limits on how fast you can get, even with a big number of processing units/cpus/cores (analogous to people in the metaphor) working on your problem (cleaning the house) in parallel.

Getting Started

2018-07-31T15:20:18Z

Ds019135:

Getting Started

2018-07-31T15:20:10Z

Ds019135:

test
== [[Access|Access]] or "How-to-be-allowed-onto-the-supercomputer" ==
Depending on the specific supercomputer, one has to either register to get a user account or write a project proposal and apply for computing resources that way. The respective pages are linked in [[Access|this overview]].

After this is done and login credentials are supplied, one can proceed to [[ Getting_Started#Login_or_.22How-to-now-actually-connect-to-the-supercomputer.22 | login ]].

== [[Nodes#Login|Login]] or "How-to-now-actually-connect-to-the-supercomputer" ==
Most HPC Systems are unix-based environments with [[shell]] (commandline) access.

To log in, one usually uses [[ssh]] to reach the respective [[Nodes#Login|Login Nodes]] (computers reserved for people just like you that want to connect to the supercomputer). Sometimes this access is restricted, so you can only connect, when you are within the university/facility and its network. To still access the Login Nodes externally, one can 'pretend to be inside the network' by using a [[VPN|Virtual Private Network (VPN)]].

Once there, the user can interact with the system and run (small) programs to generally test the system/software.

== [[File_Transfer|File Transfer]] or "How-to-get-your-data-onto-or-off-the-supercomputer" ==
To get your data (files) onto the supercomputer or back to your local machine, there are usually different ways. Sometimes there are computers specifically reserved for this purpose called [[Nodes#Copy|copy nodes]].

If available to you, it is recommened to use these copy nodes to move data to or from the supercomputer, since this will result in a better connection and disturb other users less. Additionally the tools mentioned below might only work on these nodes. If there are no dedicated copy nodes, you can usually use the [[Nodes#Login|Login Nodes]] for this purpose.

Commonly used and widely supported copying tools are [[rsync]] which mirrors directories (folders) between the supercomputer and your local machine. [[scp]] which is useful for a few single files or specified file-lists, and lastly the commonly used [[ftp]] or the encrypted version sftp (or ftps).
A little bit more information can be found in the [[File_Transfer|File Transfer]] article.

== [[Scheduler|Schedulers]] or "How-To-Run-Applications-on-a-supercomputer" ==
To run any significant program or workload on a supercomputer, generally [[Scheduler|schedulers]] are employed. Alongside the above-mentioned Login Nodes there are usually far more Backend Nodes in the system (computers exclusively reserved for computing, to which you cannot connect directly, also referred to as "batch system"). A program called scheduler decides who gets how many of those compute resources for which amount of time. Please use the Backend Nodes for everything which is not a simple small test and only runs for a few minutes., otherwise you will block the Login Nodes for everybody when you run your calculations there. These Backend Nodes make up more than 98% of a supercomputer and can only be accessed via the scheduler.

When you log into a supercomputer, you can run commands on the Login Nodes interactively. You type, you hit return, the command gets executed. Schedulers work differently. You submit a series of commands (in form of a file) and tell it, how much resources it will approximately need in terms of:

* time: If the specified time runs out, before your application finishes and exits, it will be terminated by the scheduler.
* compute resources: how many cpus ('calculation thingies'), sockets ('cpu-houses') and nodes ('computers')
* memory resources: how much RAM ('very fast memory, similar to the few books you have at home')

This combination of specified commands and required resources is commonly referred to as a "(batch) job".

If later compute resources become free, which match the requirements of your application, the scheduler will run your specified commands on the requested hardware. This is usually delayed (sometimes you have to wait a day or two) and not instant, because other users are currently using the compute resources and you have to wait until their program runs finish. Furthermore you cannot change the series of commands after submitting, but just terminate the job and submit a new one in case of an error.

The file specifying this series of commands and the required resources is called a [[jobscript]]. Its format and syntax depends on the installed scheduler. When you have this jobscript ready with the help of [[jobscript-examples]], colleagues or your local [[support]], you can submit it to the respective [[Schedulers|scheduler of your facility]]. The scheduler then waits until a set of nodes (computers) are free and later allocates those to execute your job as soon as possible. Sometimes there is (an optional) email notification, which is send when your job starts execution/finished running.

Be aware that your specified requirements have to fit within the boundaries of the system of your facility. If you ask for more than there is, chances are, the scheduler will accept your job and wait until missing hardware is bought and installed - although this will not happen in 99.9% of cases. Information over the available hardware can be found in the [https://gauss-allianz.de/de/hpc-ecosystem overview of the Gauss Allianz] or the [[Site-specific_documentation|documentation of the different sites]]. You can find more information about [[Getting_Started#Parallel_Programming_or_.22How-To-Use-More-Than-One-Core.22|parallelizing programs here]]. Also there is an [[Schedulers|overview of the schedulers used at the different sites]].

== [[Modules|Modules]] or "How-To-Use-Software-Without-installing-everything-yourself" ==
Since a lot of applications rely on 3rd party software, there is a program on most supercomputers, called the [[Modules|Module system]]. With this system, other software, like compilers or special math libraries, are easily loadable and usable. Depending on the institution, different modules might be available, but there are usually common ones like the [[Compiler#Intel_Compiler|Intel]] or [[Compiler#Gnu_Compiler_Collection|GCC]] [[Compiler|Compilers]].

A few common commands, to enter into the supercomputer commandline and talk to the module system, are
{| class="wikitable" style="width: 40%;"
| module list || lists loaded modules
|-
| module avail || lists available (loadable) modules
|-
| module load/unload x || loads/unloads module x
|-
| module switch x y || switches out module x for module y
|}

If you recurrently need lots of modules, this loading can be automated with an [[sh-file]], so that you just have to execute the file once and it loads all modules, you need.

== [[Parallel_Programming|Parallel Programming]] or "How-To-Use-More-Than-One-Core" ==
Currently development of computers is at a point, where you cannot just make a processor run faster (e.g. by increasing its clock frequency), because limits of physics have been reached in semiconductor development. Therefore the current approach is to split the work into multiple, ideally independent parts, which are then executed in parallel. Similar to cleaning your house, where everybody takes care of a few rooms, on a supercomputer this is usually done with parallel programming paradigms like [[OpenMP|Open Multi-Processing (OpenMP)]] or [[MPI|Message Passing Interface (MPI)]]. However like the fact that you only have one vacuum cleaner in the whole house which not everybody can use at the same time, there are limits on how fast you can get, even with a big number of processing units/cpus/cores (analogous to people in the metaphor) working on your problem (cleaning the house) in parallel.

MediaWiki:Sidebar

2018-07-11T13:17:44Z

Ds019135:

2018-04-23T15:14:12Z

Ds019135:

SLURM

2018-04-23T12:15:23Z

Ds019135: Reverted edits by 134.130.1.109 (talk) to last revision by Ds019135

== General ==

SLURM is a workload manager / job [[scheduler]]. To get an overview of the functionality of a scheduler, go [[Scheduler#General|here]].

__TOC__

== #SBATCH Usage ==

If you are writing a [[jobscript]] for a SLURM batch system, the magic cookie is "#SBATCH". To use it, start a new line in your script with "#SBATCH". Following that, you can put one of the parameters shown below, where the word written in <...> should be replaced with a value.

Basic settings:
{| class="wikitable" style="width: 40%;"
| Parameter || Function
|-
| --job-name=<name> || job name
|-
| --output=<path> || path to the file where the job (error) output is written to
|}

Requesting resources:
{| class="wikitable" style="width: 60%;"
| Parameter || Function
|-
| --time=<runlimit> || runtime limit in the format hours:min:sec; once the time specified is up, the job will be killed by the [[scheduler]]
|-
| --mem=<memlimit> || job memory request per node, usually an integer followed by a prefix for the unit (e. g. --mem=1G for 1 GB)
|}

Parallel programming (read more [[Parallel_Programming|here]]):

Settings for OpenMP:
{| class="wikitable" style="width: 60%;"
| Parameter || Function
|-
| --nodes=1 || start a parallel job for a shared-memory system on only one node
|-
| --cpus-per-task=<num_threads> || number of threads to execute OpenMP application with
|-
| --ntasks-per-core=<num_hyperthreads> || number of hyperthreads per core; i. e. any value greater than 1 will turn on hyperthreading (the possible maximum depends on your CPU)
|-
| --ntasks-per-node=1 || for OpenMP, use one task per node only
|}

Settings for MPI:
{| class="wikitable" style="width: 60%;"
| Parameter || Function
|-
| --nodes=<num_nodes> || start a parallel job for a distributed-memory system on several nodes
|-
| --cpus-per-task=1 || for MPI, use one task per CPU
|-
| --ntasks-per-core=1 || disable hyperthreading
|-
| --ntasks-per-node=<num_procs> || number of processes per node (the possible maximum depends on your nodes)
|}

Email notifications:
{| class="wikitable" style="width: 60%;"
| Parameter || Function
|-
| --mail-type=<type> || type can be one of BEGIN, END, FAIL, REQUEUE or ALL (where a mail will be sent each time the status of your process changes)
|-
| --mail-user=<email_address> || email address to send notifications to
|}

== Job Submission ==

This command submits the job you defined in your [[Jobscript|jobscript]] to the batch system:

$ sbatch jobscript.sh

Just like any other incoming job, your job will first be queued. Then, the scheduler decides when your job will be run. The more resources your job requires, the longer it may be waiting to execute.

You can check the current status of your submitted jobs and their job ids with the following shell command. A job can either be pending <code>PD</code> (waiting for free nodes to run on) or running <code>R</code> (the jobscript is currently being executed). This command will also print the time (hours:min:sec) that your job has been running for.

$ squeue -u <user_id>

In case you submitted a job on accident or realised that your job might not be running correctly, you can always remove it from the queue or terminate it when running by typing:

$ scancel <job_id>

== Jobscript Examples ==

This serial job will run a given executable, in this case "myapp.exe".
<syntaxhighlight lang="bash">
#!/bin/bash

### Job name
#SBATCH --job-name=MYJOB

### File for the output
#SBATCH --output=MYJOB_OUTPUT

### Time your job needs to execute, e. g. 15 min 30 sec
#SBATCH --time=00:15:30

### Memory your job needs per node, e. g. 1 GB
#SBATCH --mem=1G

### The last part consists of regular shell commands:
### Change to working directory
cd /home/usr/workingdirectory

### Execute your application
myapp.exe
</syntaxhighlight>

If you'd like to run a parallel job on a cluster that is managed by SLURM, you have to clarify that. Therefore, use the command "srun <my_executable>" in your jobscript.

This OpenMP job will start the [[Parallel_Programming|parallel program]] "myapp.exe" with 24 threads.
<syntaxhighlight lang="bash">
#!/bin/bash

### Job name
#SBATCH --job-name=OMPJOB

### File for the output
#SBATCH --output=OMPJOB_OUTPUT

### Time your job needs to execute, e. g. 30 min
#SBATCH --time=00:30:00

### Memory your job needs per node, e. g. 500 MB
#SBATCH --mem=500M

### Use one node for parallel jobs on shared-memory systems
#SBATCH --nodes=1

### Number of threads to use, e. g. 24
#SBATCH --cpus-per-task=24

### Number of hyperthreads per core
#SBATCH --ntasks-per-core=1

### Tasks per node (for shared-memory parallelisation, use 1)
#SBATCH --ntasks-per-node=1

### The last part consists of regular shell commands:
### Set the number of threads in your cluster environment to the value specified above
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

### Change to working directory
cd /home/usr/workingdirectory

### Run your parallel application
srun myapp.exe
</syntaxhighlight>

This MPI job will start the [[Parallel_Programming|parallel program]] "myapp.exe" with 12 processes.
<syntaxhighlight lang="bash">
#!/bin/bash

### Job name
#SBATCH --job-name=MPIJOB

### File for the output
#SBATCH --output=MPIJOB_OUTPUT

### Time your job needs to execute, e. g. 50 min
#SBATCH --time=00:50:00

### Memory your job needs per node, e. g. 250 MB
#SBATCH --mem=250M

### Use more than one node for parallel jobs on distributed-memory systems, e. g. 2
#SBATCH --nodes=2

### Number of CPUS per task (for distributed-memory parallelisation, use 1)
#SBATCH --cpus-per-task=1

### Disable hyperthreading by setting the tasks per core to 1
#SBATCH --ntasks-per-core=1

### Number of processes per node, e. g. 6 (6 processes on 2 nodes = 12 processes in total)
#SBATCH --ntasks-per-node=6

### The last part consists of regular shell commands:
### Set the number of threads in your cluster environment to 1, as specified above
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

### Change to working directory
cd /home/usr/workingdirectory

### Run your parallel application
srun myapp.exe
</syntaxhighlight>

== Site specific notes ==

=== RRZE ===

* <code>--output=</code> ''should not'' be used on RRZE's clusters; the submit filter already sets suitable defaults automatically
* <code>--mem=<memlimit></code> '''must not''' be used on RRZE's clusters
* the first line of the job script ''should be'' <code>#/bin/bash -l</code> otherwise <code>module</code> commands won't work in te job script
* to have a clean environment in job scripts, it is recommended to add <code>#SBATCH --export=NONE</code> '''and''' <code>unset SLURM_EXPORT_ENV</code> to the job script. Otherwise, the job will inherit some settings from the submitting shell.
* access to the parallel file system has to be specified by <code>#SBATCH ---constraint=parfs</code> or the command line shortcut <code>-C parfs</code>
* access to hardware performance counters (e.g. to be able to use <code>likwid-perfctr</code>) has to be requested by <code>#SBATCH ---constraint=hwperf</code> or the command line shortcut <code>-C hwperf</code>. Only request that feature if you really want to access the ardware performance counters as the feature interferes with the automatic system monitoring.
* multiple features have to be requested in a single <code>--constraint=</code> statement, listing all required features separated by ampersand, e.g. <code>hwperf&parfs</code>
* for Intel MPI, RRZE recommends the usage of <code>mpirun</code> instead of <code>srun</code>; if <code>srun</code> shall be used, the additional command line argument <code>--mpi=pmi2</code> is required. The command line option <code>-ppn</code> of <code>mpirun</code> only works if you <code>export I_MPI_JOB_RESPECT_PROCESS_PLACEMENT=off</code> before.
* for <code>squeue</code> the option <code>-u user</code> does not have any effect as you always only see your own jobs

== References ==

[https://www.lrz.de/services/compute/linux-cluster/batch_parallel/example_jobs/ Advanced SLURM jobscript examples]

[http://www.nersc.gov/users/computational-systems/cori/running-jobs/example-batch-scripts/ Detailled guide to more advanced scripts]

[https://slurm.schedmd.com/sbatch.html SBATCH documentation]

[https://user.cscs.ch/getting_started/running_jobs/jobscript_generator/#slurm-jobscript-generator SLURM jobscript generator]

2018-04-17T11:54:16Z

Ds019135:

Vim

2018-04-16T14:47:53Z

Ds019135:

== General ==

Vi or Vim (VI IMproved) is a visual text editor that can be run inside a [[shell]]. This has the advantage, that you can quickly write or edit text files in the [[shell]], even over an [[ssh]] connection without needing a graphical user interface. Alternatively there is the option to edit files locally (on your desktop computer) in your favourite editor or IDE (e.g. Notepad++, VSCode, ...) and then transfer the files to the remote (super)computer. Visit the [[File_Transfer|file transfer]] page for more information on that topic. As these tansfers can be a lot of effort, just to fix a missing <code>;</code> or typo, Vi/Vim can come in handy, as they are available on practically every Unix system.

While it's usage is far from intuitive, a few basics are generally helpful. For a more complete coverage of this editor, why people use it and why you should probably do that too, please take a look at the [[#Absolute_Basics|<code>vimtutor</code>]], the [[#References|References]] or the internet in general. There are lots of great resources, presenting reasons for using Vi/Vim and giving detailed instructions for its effective usage.

== Modes ==

In Vi/Vim there are six modes from which the four most important ones are listed below.

{| class="wikitable"
! Mode Name !! Description !! How to Reach the Mode from Normal
|-
|normal || For navigation and manipulation of text. This is the mode that vim will usually start in, which you can usually get back to with pressing <code>ESC</code>. || you are already there or <code>ESC</code>
|-
|insert || For inserting new text. Insert mode works more or less like you would expect a normal text editor to work with arrow keys for navigation, backspace for deletion and characters for text. || <code>a</code>
|-
|visual || For navigation and manipulation of text selections, this mode allows you to perform most normal commands, and a few extra commands, on selected text. || <code>v</code>
|-
|command-line || For entering editor commands. || <code>:</code>
|}

More information about different modes and their uses can be found in the [[#References|References]] or the internet.

== Absolute Basics ==
To open yourfile.txt in Vi/Vim simply enter

<syntaxhighlight lang="bash">
$ vim yourfile.txt
</syntaxhighlight>

From normal mode (press <code>Esc</code> to get there) you can type

{| class="wikitable" style="width: 40%;"
| <code>:q</code> || to quit Vi/Vim
|-
| <code>:w</code> || to save the current file
|-
| <code>:q!</code> || to quit Vi/Vim without saving (discarding changes)
|-
| <code>:x</code> || to quit Vi/Vim with saving changes
|}

For a tutorial you can call the

<syntaxhighlight lang="bash">
$ vimtutor
</syntaxhighlight>

and follow the instructions on screen.

== References ==
[http://www.openvim.com/ Interactive Vim Tutorial]

[https://wiki.gentoo.org/wiki/Vim/Guide Vim/Guide in the Gentoo wiki]

[http://www.viemu.com/a-why-vi-vim.html Why should you use vi (or vim)]

[https://vim-adventures.com/ Vim Game]

2018-04-06T08:20:00Z

Ds019135:

== Basics ==

This will give you a general overview of how to compile and execute a program that has been [[Parallel_Programming|parallelized]] with [[MPI]]. Many of the options listed below are the same for both Open MPI and Intel MPI, however, be care if they do differentiate.

== How to Compile MPI Code ==

Before continuing, please make sure that the openmpi or intelmpi module is loaded (go [[Modules|here]] to see how to load/switch modules).

There are several so called MPI "compiler wrappers", e.g. "mpicc". These take care of including the correct MPI libraries for each programming language etc. for you. But they share most command line options. Depending on whether your code is written in C, C++ or Fortran, follow the instructions in one of the tables below. Make sure to replace the arguments inside <…> with specific values.

=== Open MPI ===

Use the following command to specify the program you would like to compile (replace <src_file> with a path to your code, e. g. ./myprog.c).
{| class="wikitable" style="width: 40%;"
| Language || Command
|-
| C || <code>$ mpicc <src_file> -o <name_of_executable></code>
|-
| C++ || <code>$ mpicxx <src_file> -o <name_of_executable</code>
|-
| Fortran || <code>$ mpifort <src_file> -o <name_of_executable></code>
|}

You can also type the command <code>$ mpicc [options]</code>, <code>$ mpicxx [options]</code> or <code>$ mpifort [options]</code>. There are a few options that come with Open MPI, however, options are more important for running your program. The compiler options might be useful to fetch more information about the Open MPI module you're using.
{| class="wikitable" style="width: 40%;"
|Options || Function
|-
| -showme:help || print a short help message about the usage and lists all compiler options
|-
| -showme:version || show Open MPI version
|}

For RWTH cluster users:
Instead of typing the compiler wrapper <code>mpicc</code> etc., you can simply put one of the environment variables <code>$MPICC</code>, <code>$MPICXX</code> or <code>$MPIFC</code> for Fortran codes. They are already set by the module system so that you do not have to worry about which compiler module to use.

=== Intel MPI ===

Use the following command to specify the program you would like to compile (replace <src_file> with a path to your code, e. g. ./myprog.c).
{| class="wikitable" style="width: 40%;"
| Compiler Driver || C || C++ || Fortran
|-
| GCC || <code>$ mpicc <src_file> -o <name></code> || <code>$ mpicpc <src_file> -o <name></code> || <code>$ mpifort <src_file> -o <name></code>
|-
| Intel || <code>$ mpiicc <src_file> -o <name></code> || <code>$ mpiicpc <src_file> -o <name></code> || <code>$ mpiifort <src_file> -o <name></code>
|}

You can also type the command <code>$ mpicc [options] <src_file> -o <name></code> etc., where [options] can be replaced with one or more of the ones listed below. Intel MPI comes with rather advanced compiler options, that are mainly aimed at optimization and analyzing your code with the help of Intel tools.
{| class="wikitable" style="width: 40%;"
|Options || Function
|-
| -g || enable debug mode
|-
| -O || enable compiler optimization
|-
| -v || print compiler version
|}

For RWTH cluster users:
Instead of typing the compiler wrapper <code>mpicc</code> etc., you can simply put one of the environment variables <code>$MPICC</code>, <code>$MPICXX</code> or <code>$MPIFC</code> for Fortran codes. They are already set by the module system so that you do not have to worry about which compiler module to use.

== How to Run an MPI Executable ==

Ensure that the correct MPI module is loaded (go [[Modules|here]] to see how to load/switch modules). Once again, the command line options slightly differ between Intel MPI and Open MPI.
In order to start any MPI program, type the following command where <executable> specifies the path to your application:
$ mpirun -n <num_procs> [options] <executable>
Note that <code>mpiexec</code> and <code>mpirun</code> are synonymous in Open MPI, in Intel MPI it's <code>mpiexec.hydra</code> and <code>mpirun</code>.

Don’t forget to put the “-np” or “-n” option as explained below. All the other options listed below are not mandatory.

=== Open MPI ===

{| class="wikitable" style="width: 60%;"
| Option || Function
|-
| -np <num_procs> or -n <num_procs> || number of processes to run
|-
| -npersocket <num_procs> || number of processes per socket
|-
| -npernode <num_procs> || number of processes per node
|-
| -wdir <directory> || change to directory specified before executing the program
|-
| -nw || complete command when all MPI processes have been launched successfully
|-
| -path <path> || look for executables in the directory specified
|-
| -q or -quiet || suppress helpful messages
|-
| -output-filename <name> || redirect output into the file <name>.<rank>
|-
| -x <env_variable> || export the specified environment variable to the remote nodes where the program will be executed
|-
| --help || list all options available with an explanation
|}

=== Intel MPI ===

{| class="wikitable" style="width: 60%;"
| Option || Function
|-
| -n <num_procs> || number of processes to run
|-
| -ppn <num_procs> || number of processes per node
|-
| -wdir <directory> || change to directory specified before executing the program
|-
| -path <path> || look for executables in the directory specified
|-
| -outfile-pattern <name> || redirect stdout to file
|-
| --help || list all options available with an explanation
|}

=== Process Binding in Open MPI ===

Binding processes means telling your system how to place the processes onto the architecture. This can be done by adding command-line options when calling "mpiexec" and may enhance the performance of your application. In order to learn more about that, go [[Binding/Pinning#Options_for_Binding_in_Open_MPI|here]].

== References ==

[https://software.intel.com/en-us/mpi-developer-reference-linux-compiler-command-options Intel MPI compiler options]

[https://www.open-mpi.org/doc/v2.0/man1/mpiexec.1.php Manual page for Open MPI's mpiexec]

MediaWiki:Sidebar

2018-04-06T08:19:09Z

Ds019135:

Binding/Pinning

2018-04-06T08:18:20Z

Ds019135: Ds019135 moved page Thread Pinning to Binding/Pinning

== Basics ==

Pinning threads for shared-memory [[Parallel_Programming|parallelism]] or binding processes for distributed-memory parallelism is an advanced way to control how your system distributes the threads or processes across the available cores. It is important for improving the performance of your application by avoiding costly remote memory accesses and keeping the threads or processes close to each other. Threads are "pinned" by setting certain [[OpenMP]]-related environment variables, which you can do with this command:
$ export <env_variable_name> = <value>
The terms "thread pinning" and "thread affinity" as well as "process binding" and "process affinity" are used interchangeably.
You can bind processes by specifying additional options when [[How_to_Use_MPI#How_to_Run_an_MPI_Executable|executing]] your Open [[MPI]] application.

== How to Pin Threads in OpenMP ==

[[File:Omp_places.png|thumb|350px|Schematic of how <code>OMP_PLACES={0}:8:2</code> would be interpreted]]

[[File:Proc_bind_close.PNG|thumb|350px|Schematic of how <code>OMP_PROC_BIND=close</code> would be interpreted on a system comprising 2 nodes with 4 hardware threads each]]

[[File:Proc_bind_spread.PNG|thumb|350px|Schematic of <code>OMP_PROC_BIND=spread</code> and an remote memory access, if thread 0 and 1 work on the same data]]

OMP_PLACES is employed to specify places on the machine where the threads are put. However, this variable on its own does not determine thread pinning completely, because your system still won't know in what pattern to assign the threads to the given places. Therefore, you also need to set OMP_PROC_BIND.

OMP_PROC_BIND specifies a binding policy which basically sets criteria by which the threads are distributed.

If you want to get a schematic overview of your cluster's hardware, e. g. to figure out how many hardware threads there are, type: <code>$ lstopo</code>.

=== OMP_PLACES ===

This variable can hold two kinds of values: a name specifying (hardware) places, or a list that marks places.

{| class="wikitable" style="width:50%;"
| Abstract name || Meaning
|-
| <code>threads</code> || a place is a single hardware thread, i. e. the hyperthreading will be ignored
|-
| <code>cores</code> || a place is a single core with its corresponding amount of hardware threads
|-
| <code>sockets</code> || a place is a single socket
|}

In order to define specific places by an interval, OMP_PLACES can be set to <code><lowerbound>:<length>:<stride></code>.
All of these three values are non-negative integers and must not exceed your system's bounds. The value of <code><lowerbound></code> can be defined as a list of hardware threads. As an interval, <code><lowerbound></code> has this format: <code>{<starting_point>:<length>}</code> that can be a single place, or a place that holds several hardware threads, which is indicated by <code><length></code>.

{| class="wikitable" style="width:60%;"
| Example hardware || OMP_PLACES || Places
|-
| 24 cores with one hardware thread each, starting at core 0 and using every 2nd core || <code>{0}:24:2</code> or <code>{0:1}:24:2</code> || <code>{0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22}</code>
|-
| 12 cores with two hardware threads each, starting at the first two hardware threads on the first core ({0,1}) and using every 4th core || <code>{0,1}:12:4</code> or <code>{0:2}:12:4</code> || <code>{{0,1}, {4,5}, {8,9}, {12,13}, {16,17}, {20,21}}</code>
|}

You can also determine these places with a comma-separated list. Say there are 8 cores available with one hardware thread each, and you would like to execute your application on the first four cores, you could define this: <code>$ export OMP_PLACES={0, 1, 2, 3}</code>

=== OMP_PROC_BIND ===

Now that you have set OMP_PROC_BIND, you can now define the order in which the places should be assigned. This is especially useful for NUMA systems (see [[#References|references]] below) because some threads may have to access remote memory, which will slow your application down significantly. If OMP_PROC_BIND is not set, your system will distribute the threads across the nodes and cores randomly.

{| class="wikitable" style="width:60%;"
| Value || Function
|-
| <code>true</code> || the threads should not be moved
|-
| <code>false</code> || the threads can be moved
|-
| <code>master</code> || worker threads are in the same partition as the master
|-
| <code>close</code> || worker threads are close to the master in contiguous partitions, e. g. if the master is occupying hardware thread 0, worker 1 will be placed on hw thread 1, worker 2 on hw thread 2 and so on
|-
| <code>spread</code> || workers are spread across the available places to maximize the space inbetween two neighbouring threads
|}

== Options for Binding in Open MPI ==

Binding processes to certain processors can be done by specifying the options below when executing a program. This is a more advanced way of running an application and also requires knowledge about your system's architecture, e. g. how many cores there are (for an overview of your hardware topology, use <code>$ lstopo</code>). If none of these options are given, default values are set.
By overriding default values with the ones specified, you may be able to improve the performance of your application, if your system distributes them in a suboptimal way per default.

{| class="wikitable" style="width: 80%;"
| Option || Function || Explanation
|-
| --bind-to <arg> || bind to the processors associated with hardware component; arg can be one of: none, hwthread, core, l1cache, l2cache, l3cache, socket, numa, board; default values: "core" for np <= 2, "socket" otherwise || e. g.: in case of "l3cache" the processes will be bound to those processors that share the same L3 cache
|-
| --bind-to-core || bind processes to cores || bind each process to a core
|-
| --bind-to-socket || bind processes to sockets || put each process onto a processor socket
|-
| --bind-to-none || bind no processes || do not bind any processes, but distribute them freely
|-
| --cpus-per-proc <num_cpus> || bind each process to the given number of CPUs || if set to 3, each process will take up 3 CPUs
|-
| --report-bindings || print any bindings for launched processes to the console
|}

== References ==

[http://pages.tacc.utexas.edu/~eijkhout/pcse/html/omp-affinity.html Thread affinity in OpenMP]

[https://docs.oracle.com/cd/E60778_01/html/E60751/goztg.html More information on OMP_PLACES and OMP_PROC_BIND]

[https://doc.itc.rwth-aachen.de/download/attachments/35947076/03_OpenMPNumaSimd.pdf Introduction to OpenMP from PPCES (@RWTH Aachen) Part 3: NUMA & SIMD]

[https://www.open-mpi.org/faq/?category=tuning#using-paffinity-v1.4 FAQ about process affinity in Open MPI]

Ssh keys

2018-04-05T14:56:21Z

Ds019135:

An ssh key is a way of identifying (authenticating) yourself when connecting to a server per [[ssh]]. A different popular authentication method is via a password.

== Why should I use it?==
When you connect to a server, authenticating via a password there are two main problems:
* Someone could bruteforce or guess the password, since many passwords are commonly weak, hard to remember or used for multiple applications and then cracked/leaked.
* Someone could intercept/crack your password, since it has to be send to the server at some point in some form.

== How-to-use-it ==
=== Generate a key ===
You should start by generating a key pair:
$ ssh-keygen -b 4096
where you can specify the max length of the key up to 16384 bits.

You can then optionally protect your key with a passphrase. (Your key is basically just a file sitting on your computer and a passphrase protects your key, if someone happens to steal/copy that file).

If you did not specify a different file, the key normaly gets generated into the folder
~/.ssh
with the files '''id_rsa''' being your private and '''id_rsa.pub''' being your public key.

=== Copy the public key to the server ===
This public key now has to be copied to the server to the
~/.ssh/authorized_keys
file. This can be done, by opening an [[ssh]] connection via password and then using an editor (e.g. [[vim]]) to paste the key into the file (creating the '''.ssh''' directory if it does not exist):
$ mkdir ~/.ssh
$ vim ~/.ssh/authorized_keys

The next time you [[ssh]] to the server, it should use the key and instead of prompting the pass'''word''' for the server, prompt for the pass'''phrase''' of the key, if you chose to employ a passphrase.

=== Troubleshooting ===
If it still asks for your password, something went wrong. In that case you should check, whether the '''authorized_keys''' file really contains the key by executing:
$ cat ~/.ssh/authorized_keys
on the '''server''' and
$ cat ~/.ssh/id_rsa.pub
on '''your local machine'''. If the key of your local machine is not contained in the '''authorized_keys''' on the server, repeat the steps of copying the key to the server.

== How-it-works ==
The basic principle is that of public and private keys. A public key is like an indestructible piggy bank: Everybody can put something (data) into it and nobody can get it back out again. A private key is the key for this. In this way you can distribute all the piggy banks you like and if someone put something in there and sends it back, only you can open it with your private key.

Now since you gave the server the public key (=piggy bank), it can encrypt something (say a random number), send it back and only you can decrypt this number, since only you have the private key.

For more detailed information on how this works, head over to the References.

== References ==
[https://wiki.archlinux.de/title/SSH-Authentifizierung_mit_Schl%C3%BCsselpaaren SSH keys on the archlinux wiki]

[https://medium.com/@vrypan/explaining-public-key-cryptography-to-non-geeks-f0994b3c2d5 Public and private keys easily explained]

[https://www.digitalocean.com/community/tutorials/understanding-the-ssh-encryption-and-connection-process More detailed explanation of the connection and encryption process of ssh]

2018-04-05T14:13:52Z

Ds019135:

Ssh keys

2018-04-05T14:02:28Z

Ds019135: Created page with "An ssh key is a way of identifying (authenticating) yourself when connecting to a server per ssh. A different popular authentication method is via a password. == Why shou..."

SSH

2018-04-05T13:37:21Z

Ds019135:

== General ==
Secure Shell (ssh) is a [[shell|commandline]]-tool for logging into a different computer over some network (e.g. the internet) and for executing commands on that machine, as if one would be sitting there instead of the own computer. So you use ssh to build a connection to the other computer and can then interact with it, using it's [[shell]]. It is commonly used to [[Getting_Started#Log-in_or_.22How-to-now-actually-connect-to-the-supercomputer.22|log-in]] to the [[Nodes#Log-In|frontent Nodes]] of a supercomputer.

== Access ==
OpenSSH is the standard ssh client on Linux and MAC and it is freely available for everyone. On Windows you can use Putty or the Gitbash (coming with ssh) which is also free.

Sometimes this access is restricted, so you can only connect, when you are within the university/facility and it's Network. To still access the Log-in Nodes externally, one can 'pretend to be inside the network' by using a [[VPN|Virtual Private Network (VPN)]].

== Usage ==
Logging in with OpenSSH on a Linux Cluster is done with:

$ ssh -l <login> <cluster>

Here login is your user name and [[Nodes#Log-In|frontent Nodes]] is one of the frontend nodes of the system.

If you need to start graphical applications you need to enable X11 forwarding/X11 tunneling by your ssh client.
For OpenSSH this is done by giving it the "-X" command line option:

$ ssh -X -l <login> <cluster>

which might or might not work depending on the your Operating System.

For more Security and easy of use you should consider setting up authentication via [[ssh_keys|ssh keys]].

== FAQs ==
[https://wickie.hlrs.de/platforms/index.php/Secure_Shell_ssh slightly longer description]

[https://www.freebsd.org/cgi/man.cgi?query=ssh&sektion=1 ssh manpage]

2018-04-05T09:08:26Z

Ds019135:

Getting Started

2018-04-05T09:06:57Z

Ds019135:

== [[Access|Access]] or "How-to-be-allowed-onto-the-supercomputer" ==
Depending on the specific supercomputer, one has either to register to get a user account or write a project proposal and apply for computing resources that way. The respective pages are linked in [[Access|this overview]].

After this is done and login credentials are supplied, one can proceed to

== [[Nodes#Log-In|Log-in]] or "How-to-now-actually-connect-to-the-supercomputer" ==
Most HPC Systems are unix-based environments with [[shell]] (commandline) access.

To log in, one usually uses [[ssh]] to reach the respective [[Nodes#Log-In|Log-in Nodes]] (Computers reserved for people just like you that want to connect to the supercomputer). Sometimes this access is restricted, so you can only connect, when you are within the university/facility and it's Network. To still access the Log-in Nodes externally, one can 'pretend to be inside the network' by using a [[VPN|Virtual Private Network (VPN)]].

Once there, the user can interact with the system and run (small) programs to generally test the system/software.

== [[File_Transfer|File Transfer]] or How-to-get-your-data-onto-or-off-the-supercomputer ==
To get your data (files) onto the supercomputer or back to your local machine, there are usually different ways. Sometimes there are computers specifically reserved for this purpose called [[Nodes#Copy_Nodes|copy Nodes]]

Please use these nodes to copy your data to or off the supercomputer, if available, since you will get a better/faster connection and disturb other users less. Also the tools mentioned below might only work on those nodes. If there are no dedicated copy nodes, you can usually use the [[Nodes#Log-In|Log-in Nodes]] for this purpose

Commonly used and widely supported copying tools are [[rsync]] which mirrors directories (folders) between the supercomputer and your local machine. [[scp]] which is useful for a few single files or specified file-lists, and lastly the commonly used [[ftp]] or the encrypted version sftp (or ftps).
A little bit more information can be found in the [[File_Transfer|File Transfer]] article.

== [[Schedulers|Schedulers]] or "How-To-Run-Applications-on-a-supercomputer" ==
To run any significant program or workload on a supercomputer, generally schedulers are employed. Except from the above-mentioned Login Nodes there are usually far more Backend Nodes (Computers exclusively reserved for computing, where you can not directly connect to). A programm called scheduler decides who gets how many of those for what time. Please use the scheduler for everything which is not a simple small test, which only runs for a few minutes. More than 98% of the power of a supercomputer can only be accessed via the scheduler and you will block the Login-Nodes for everybody when you run your calculations there :/

When you log in, you can run commands on the Log-in nodes interactive. You type, you hit return, the command gets executed. Schedulers works different: You submit a series of commands (in form of a file) and tell it, how long you think these commands will take (also in the file) and how much resources it will approximately need in terms of:

* time: If the specified time runs out, before your application finishes and exits, it will be terminated by the scheduler.
* compute resources: how many cpus ('calculation thingies'), sockets ('cpu-houses') and nodes ('computers')
* memory resources: how much RAM ('very fast memory, similar to the few books you have at home')
* how to actually execute your application

and the scheduler then runs this series of commands when there is a part of the supercomputer free, which fits your requirements. This is usually delayed (sometimes you have to wait a day or two) and not instant. Furthermore you can not change the series of commands, but just terminate 'job' and submit a new one in case of an error.

The file specifying this series of commands and the requirements is called a [[Jobscript]] and specific/different depending on the scheduler. When you have this jobscript ready with the help of [[jobscript-examples]], colleagues or your local [[Support]], you can submit it to the respective [[Scheduler]] of your facility. The Scheduler then waits until a set of nodes (computers) are free and then allocated those to compute your 'job'. Sometimes there is (an optional) Email notification on when your job starts/finishes computation.

The requirements you specified obviously have to fit within the boundaries of the system of your facility. If you ask for more than there is, chances are, the scheduler will take your job and wait until you buy and install the missing hardware in your facility -> probably forever. Information over the available hardware can be found in the [https://gauss-allianz.de/de/hpc-ecosystem Overview of the Gauss Allianz] or the [[Site-specific_documentation|documentation of the different sites]]. You can find more information about [[Getting_Started#Parallelizing_or_.22How-To-Use-More-Than-One-Core.22|parallelizing programs here]]

== [[Modules|Modules]] or "How-To-Use-Software-Without-installing-everything-yourself" ==
Since a lot of applications rely on 3rd party software, there is a program on most supercomputers, called the [[Modules|Module system]]. With this system, other software, like compilers or special math libraries, are easily loadable and usable. Depending on the institution, different modules might be available, but there are usually common ones like the [[Compiler#Intel_Compiler|Intel]] or [[Compiler#Gnu_Compiler_Collection|GCC]] [[Compiler|Compilers]].

A few common commands, to enter into the supercomputer commandline and talk to the module system, are
{| class="wikitable" style="width: 40%;"
| module list || lists loaded modules
|-
| module avail || lists available (loadable) modules
|-
| module load/unload x || loads/unloads modul x
|-
| module switch x y || switches out module x for module y
|}

If you recurrently need lots of modules, this loading can be automated with an [[sh-file]], so that you just have to execute the file once and it loads all modules, you need.

== [[Parallel_Programming|Parallel Programming]] or "How-To-Use-More-Than-One-Core" ==
Currently development of computers is at a point, where you can not just make a processor run faster, because the semiconductor physics simply dont work that way. Therefore the current solution is to split the work into multiple, ideally independent parts, which are then executed in parallel. Similar to cleaning your house, where everybody takes care of a few rooms, on a supercomputer this is usually done with parallel programming paradigms like [[OpenMP|Open Multi-Processing (OpenMP)]] or [[MPI|Message Passing Interface (MPI)]]. However like the fact that you only have one vacuum cleaner in the whole house which not everybody can use at the same time, there are limits on how fast you can get, even with a big number of processing units/cpus/cores (analogous to people in the metaphor) working on your problem (cleaning the house) in parallel.