Difference between revisions of "Modules"

From HPC Wiki
Jump to: navigation, search
m
 
(8 intermediate revisions by 3 users not shown)
Line 1: Line 1:
The module system is a program, running on most supercomputers, that simplifies using different software in a precise and controlled manner. Oftentimes a supercomputer has more software installed than the average user will ever use. Frequently there are also multiple version of the same program available at the same time, to enable backward compatibility. A prominent example of this are [[Compiler|Compilers]], which sometimes have the tendency to break older programs with new versions.
+
[[Category:Basics]]
 +
The ''module'' system is a concept available on most supercomputers, simplifying the use of different software (versions) in a precise and controlled manner.<br />
 +
 
 +
In most cases, a supercomputer has far more software installed than the average user will ever use. Each of these software packages need different settings in terms of <code>$PATH</code>, <code>$LD_LIBRARY_PATH</code> and other [[HPC-Dictionary#Environment_Variables|environment variables]], which can adversely affect each other or even be mutually exclusive. Secondly, different user (groups) need different versions of the same software, which in general cannot be installed nor used in parallel on the same system.
 +
 
 +
Therefore, the settings for all these software packages and their supported versions are encapsulated in “environment modules” maintained by the module system. ''These modules are in no way related to 'perl' or 'python' modules and should not be confused with that''.
 +
 
 +
By means of the module system, all software currently available on your cluster can be listed, loaded, and unloaded, by using the command <code>module</code>.  
 +
 
  
 
== Usage ==
 
== Usage ==
To get a list of all currently loaded modules, one can employ
+
To get a list of all currently loaded modules:
  $ module list
+
  $ '''module list'''
and might receive something like:
 
 
  Currently Loaded Modulefiles:
 
  Currently Loaded Modulefiles:
 
  1) DEVELOP                3) intelmpi/2017.4
 
  1) DEVELOP                3) intelmpi/2017.4
 
  2) intel/16.0            4) likwid/system-default
 
  2) intel/16.0            4) likwid/system-default
detailing that from the DEVELOP group the Intel [[Compiler]] ICC in version 16, the Intel implementation of [[How_to_Use_MPI|MPI]] in Version 2017.4 and the system default version of [[Likwid]] are currently loaded and usable.
+
detailing that from the category DEVELOP, the Intel [[Compiler]] ICC in version 16, the Intel implementation of [[How_to_Use_MPI|MPI]] in Version 2017.4 and the system default version of [[Likwid]] are currently loaded and usable.
  
 
Calling
 
Calling
  $ module avail
+
  $ '''module avail'''
lists all available (loadable) modules and module groups. With the information of these two commands, one can select module to load/unload/switch with
+
lists all available (loadable) modules and module groups. With the information of these two commands, one can:
  $ module load/unload x
+
* load a specific module
to load or unload module x or
+
  $ '''module load x'''
  $ module switch x y
+
 
to switch out module x for module y. Switching is especially useful, to switch to a different version of the same program.
+
* unload a specific module
 +
  $ '''module unload x'''
 +
 
 +
* swap a specific module for another one (especially useful to switch between different versions of the same program)
 +
$ '''module switch x y'''
 +
Using these commands, the module system takes care of setting all the system and environment variables, so as to allow the desired applications to find themselves and all their prerequisites.
 +
 
 +
When there are multiple versions of the same program available, you can append <code>/X.Y.Z</code> to your "module load ..." command, exactly specifying what version number you want. In most cases, this is to enable backward compatibility (a prominent example being [[Compiler|Compilers]], where newer versions tend to break (or fail to compile) older programs).
  
If you recurrently need lots of modules, this loading can be automated with an [[sh-file]], so that you just have to execute the file once and it loads all modules, you need.
+
If you need lots of modules time and again, this loading can be automated with a [[sh-file]], so that you just have to execute the file once and it will load all modules you need.
  
Using these commands, the module system takes care to set all the system variables, so the desired applications are referenced and called correctly.
+
== References ==
 +
[http://modules.sourceforge.net/ Modules Homepage]

Latest revision as of 14:07, 3 September 2019

The module system is a concept available on most supercomputers, simplifying the use of different software (versions) in a precise and controlled manner.

In most cases, a supercomputer has far more software installed than the average user will ever use. Each of these software packages need different settings in terms of $PATH, $LD_LIBRARY_PATH and other environment variables, which can adversely affect each other or even be mutually exclusive. Secondly, different user (groups) need different versions of the same software, which in general cannot be installed nor used in parallel on the same system.

Therefore, the settings for all these software packages and their supported versions are encapsulated in “environment modules” maintained by the module system. These modules are in no way related to 'perl' or 'python' modules and should not be confused with that.

By means of the module system, all software currently available on your cluster can be listed, loaded, and unloaded, by using the command module.


Usage

To get a list of all currently loaded modules:

$ module list
Currently Loaded Modulefiles:
1) DEVELOP                3) intelmpi/2017.4
2) intel/16.0             4) likwid/system-default

detailing that from the category DEVELOP, the Intel Compiler ICC in version 16, the Intel implementation of MPI in Version 2017.4 and the system default version of Likwid are currently loaded and usable.

Calling

$ module avail

lists all available (loadable) modules and module groups. With the information of these two commands, one can:

  • load a specific module
$ module load x
  • unload a specific module
$ module unload x
  • swap a specific module for another one (especially useful to switch between different versions of the same program)
$ module switch x y

Using these commands, the module system takes care of setting all the system and environment variables, so as to allow the desired applications to find themselves and all their prerequisites.

When there are multiple versions of the same program available, you can append /X.Y.Z to your "module load ..." command, exactly specifying what version number you want. In most cases, this is to enable backward compatibility (a prominent example being Compilers, where newer versions tend to break (or fail to compile) older programs).

If you need lots of modules time and again, this loading can be automated with a sh-file, so that you just have to execute the file once and it will load all modules you need.

References

Modules Homepage