Admin Guide File System Separation

From HPC Wiki
Admin Guide File System Separation /
Revision as of 18:31, 30 October 2020 by Robert-schade-e757@uni-paderborn.de (talk | contribs) (Created page with "Category:HPC-Admin Category:HPC.NRW-Best-Practices At most HPC sites/clusters there are two separate filesystems available on both gateway servers and all compute nod...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search


At most HPC sites/clusters there are two separate filesystems available on both gateway servers and all compute nodes:

  • /home
  • /work

In addition, every compute node provides a separate /scratch filesystem for local, temporary data storage.

On both, /home and /work, disk quotas per user are enabled. Available disk space quota and current quota usage are automatically shown when logging in. At this point, no file quota beyond filesystem limitations exist.

/home

/home could have, for example, a quota of 32 GiB for user data, but usually its content is backed up on tape such that in case of a filesystem problem the /home filesystem and its data can be restored. The tape backup is usually done on a daily basis, deleted files are kept for a maximum of 60 days or similar durations. /home is usually provided by a redundant infrastructure, e.g., two redundant NFS servers and is hence a network filesystem, but not a parallel filesystem. To avoid excessive load on the /home filesystem, it can be mounted write-protected on all compute nodes.

/work

/work has different characteristics: it usually has a much larger quota for user data, but the files are not saved externally. It is provided by several redundant file servers and uses some kind of parallel filesystem. /work can be read from and written to on both gateway servers and all compute nodes.

Extras

To mimic the filesystem layout of some other faculties a link /home/$USER/nobackup -> /work/$USER is added in each home directory.