Streaming scientific software with EESSI - Revision history

Martin-errenst-7986@uni-wuppertal.de at 06:09, 23 September 2025

2025-09-23T06:09:02Z

2025-09-23T06:06:01Z

2024-04-12T13:19:46Z

Proof of Concept at the University of Wuppertal

2024-04-12T13:19:35Z

2024-04-12T13:18:34Z

2024-04-12T13:17:26Z

2024-04-12T13:12:51Z

2024-04-12T13:12:07Z

2024-04-12T13:11:34Z

Add performance considerations and Wuppertal PoC

2024-03-28T12:34:47Z

Fix typo

@@ Line 87: / Line 87: @@
 This is likely due to worse performance of the local parallel filesystem (BeeGFS), which is not optimized for software distribution and can be affected by other running jobs in the HPC system.
-So using software distributed by EESSI can lead equal or better performance than using a local software stack.
+So using software distributed by EESSI can to lead equal or better performance than using a local software stack.
 Changes to the job scripts are minimal. Users just have to make sure to source the EESSI init script and use the correct module naming scheme.

@@ Line 48: / Line 48: @@
 == Adding Missing Software ==
 EESSI is a community driven project built on top of EasyBuild to manage reproducible build recipes.
-If an important software module is missing or if you would like to distribute scientific software under your are developing, you can [https://www.eessi.io/docs/adding_software/overview/ contribute by opening a pull request].
+If an important software module is missing or if you would like to distribute scientific software that you are developing, you can [https://www.eessi.io/docs/adding_software/overview/ contribute by opening a pull request].
 After the pull request is reviewed and approved, an automatic process is building and ingesting the software into the EESSI stack.

← Older revision		Revision as of 13:19, 12 April 2024
Line 90:		Line 90:

	Changes to the job scripts are minimal. Users just have to make sure to source the EESSI init script and use the correct module naming scheme.		Changes to the job scripts are minimal. Users just have to make sure to source the EESSI init script and use the correct module naming scheme.
−

	== Additional Resources ==		== Additional Resources ==

@@ Line 87: / Line 87: @@
 This is likely due to worse performance of the local parallel filesystem (BeeGFS), which is not optimized for software distribution and can be affected by other running jobs in the HPC system.
-So using Software distributed by EESSI can lead equal or better performance than using a local software stack.
+So using software distributed by EESSI can lead equal or better performance than using a local software stack.
 Changes to the job scripts are minimal. Users just have to make sure to source the EESSI init script and use the correct module naming scheme.

@@ Line 67: / Line 67: @@
 EESSI is distributed with the default configurations of CVMFS clients, so any site already offering CVMFS on worker nodes can evaluate software streaming through EESSI without any changes.
-In a simple proof of concept, we've compared the execution of a trivial computational fluid dynamics simulation with the same version of OpenFOAM 11.
+In a simple proof of concept, we've compared the execution of a trivial computational fluid dynamics simulation in OpenFOAM 11.
 The test is running 4 MPI processes on two separate nodes, to also test the OpenMPI version shipped with EESSI in combination with local InfiniBand hardware.

@@ Line 61: / Line 61: @@
 EESSI provides optimised builds for many architectures and the correct version is automatically loaded.
-This can result in even performance when the locally provided alternatives have not been built with architecture optimisations in mind.
+This can result in even better performance when the locally provided alternatives have not been built with architecture optimisations in mind.
 With community contributions from multiple people, you can benefit from synergy effects in the maintenance of a common software stack.

@@ Line 70: / Line 70: @@
 The test is running 4 MPI processes on two separate nodes, to also test the OpenMPI version shipped with EESSI in combination with local InfiniBand hardware.
-In the following table, you can observe the results of three measurements:
+In the following table, you can observe the results of three measurements each:
-<pre># EESSI Module:
+<pre># EESSI modules via CVMFS:
       JobID    State       Elapsed  CPUEff   MemEff
    26269014  COMPLETED    00:04:25   44.2%     0.2%

@@ Line 71: / Line 71: @@
 In the following table, you can observe the results of three measurements:
-<blockquote># EESSI Module:
+<pre># EESSI Module:
       JobID    State       Elapsed  CPUEff   MemEff
    26269014  COMPLETED    00:04:25   44.2%     0.2%
@@ Line 82: / Line 82: @@
    26269018  COMPLETED    00:04:25   41.8%     0.2%
    26269019  COMPLETED    00:04:31   41.7%     0.2%
-</blockquote>
+</pre>
 Both approaches result in comparable elapsed job times, whereas the EESSI version is slightly faster.

@@ Line 16: / Line 16: @@
 Another reason is the heterogeneous hardware environment which may require special software installations and configurations, e.g. optimized builds for x86 CPUs, ARM CPUs, FPGAs or GPUs, and libraries for fast interconnect technologies.
 == Loading Software Modules ==
@@ Line 49: / Line 51: @@
 After the pull request is reviewed and approved, an automatic process is building and ingesting the software into the EESSI stack.
 == Additional Resources ==

@@ Line 42: / Line 42: @@
 </syntaxhighlight>
-On a laptop, you would have to [https://www.eessi.io/docs/getting_access/native_installation/ install CVMFS] or use a [https://www.eessi.io/docs/getting_access/eessi_container/ EESSI container].
+On a laptop, you would have to [https://www.eessi.io/docs/getting_access/native_installation/ install CVMFS] or use an [https://www.eessi.io/docs/getting_access/eessi_container/ EESSI container].
 == Adding Missing Software ==