Difference between revisions of "Introduction to Linux in HPC/Various tips"

From HPC Wiki
Introduction to Linux in HPC/Various tips
Jump to navigation Jump to search
(Created page with "__TOC__ === Video === <!--T:5--> <youtube width="600" height="400" right>IfD9IPixgpo</youtube> [https://git-ce.rwth-aachen.de/hpc.nrw/ap2/tutorials/linux/-/blob/master/Slid...")
 
m (Tweak page sorting)
 
(28 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
[[Category:Tutorials|Various Tips for Working with Linux]]<nowiki />
 +
{{DISPLAYTITLE:Various Tips for Working with Linux}}<nowiki />
 +
{{Syllabus Introduction to Linux}}<nowiki />
 
__TOC__
 
__TOC__
 +
 +
This video covers a range of miscellaneous topics that are useful, but were not addressed before. For example, there are tools to print the space left on a disk or the recent command history. Other tools regularly execute commands for you or manipulate text.
 +
Finally, an overview of common package managers is given.
  
 
=== Video === <!--T:5-->
 
=== Video === <!--T:5-->
  
<youtube width="600" height="400" right>IfD9IPixgpo</youtube>
+
<youtube width="600" height="340" right>8w16X2d2T54</youtube>
 +
 
 +
([[Media:HPC.NRW_Introduction_to_Linux_in_HPC_14_Various_Tips.pdf | Slides as pdf]])
 +
 
 +
=== Quiz === <!--T:5--> 
 +
 
 +
{{hidden begin
 +
|title = Does <code>df</code> show the size of a given file?
 +
}}
 +
<quiz display=simple>
 +
{
 +
|type="()"}
 +
+  No
 +
|| Explanation: <code>df</code> (disk free) shows how much space is free on a disk. <code>du</code> (disk used) shows how much space is taken up by a given file or directory.
 +
-  Yes
 +
|| Explanation: <code>df</code> (disk free) shows how much space is free on a disk. <code>du</code> (disk used) shows how much space is taken up by a given file or directory.
 +
</quiz>
 +
{{hidden end}}
 +
 
 +
 
 +
{{hidden begin
 +
|title = What does <code>ln -s /tmp .</code> do?
 +
}}
 +
<quiz display=simple>
 +
{
 +
|type="()"}
 +
- Create a symbolic link of the current directory in <code>/tmp</code>
 +
||
 +
+ Create a symbolic link of <code>/tmp</code> in the current directory named <code>./tmp</code>
 +
||
 +
- Print a list of files located in <code>/tmp</code> and <code>.</code>
 +
symbolic links
 +
||
 +
</quiz>
 +
{{hidden end}}
 +
 
 +
 
 +
{{hidden begin
 +
|title = What is the <code>watch</code> command used for?
 +
}}
 +
<quiz display=simple>
 +
{
 +
|type="()"}
 +
- To open a file of unknown format
 +
||
 +
+ To regularly execute a command and see its output
 +
||
 +
- To watch video files
 +
||
 +
</quiz>
 +
{{hidden end}}
 +
 
 +
 
 +
{{hidden begin
 +
|title = What are <code>sed</code> or <code>awk</code> used for?
 +
}}
 +
<quiz display=simple>
 +
{
 +
|type="()"}
 +
+ Manipulate and process text
 +
||
 +
- Set system configuration settings
 +
||
 +
</quiz>
 +
{{hidden end}}
 +
 
 +
 
 +
{{hidden begin
 +
|title = How will you find a specific command in history ? HINT: use grep with history, how to use it?
 +
}}
 +
<quiz display=simple>
 +
{
 +
|type="()"}
 +
+  Click and submit to see the answer
 +
|| <code>history | grep cd</code>
 +
</quiz>
 +
{{hidden end}}
  
[https://git-ce.rwth-aachen.de/hpc.nrw/ap2/tutorials/linux/-/blob/master/Slides/Linux_Intro/Linux_Intro.pdf Linux Introduction]  Slides 130 - 136 (7 pages)
 
  
 +
{{hidden begin
 +
|title = How can you compare two files and display a list of the difference? HINT: <code>man diff</code>
 +
}}
 +
<quiz display=simple>
 +
{
 +
|type="()"}
 +
+  Click and submit to see the answer
 +
|| <code>diff file1 file2</code>
 +
</quiz>
 +
{{hidden end}}
  
=== Slide Layout === <!--T:5-->
 
  
 +
{{Warning|mode=warn|text= '''Please make backups regularly! Hard drives can fail on any computer'''}}
  
 +
=== Exercises in Terminal === <!--T:5--> 
  
 +
1. Find out the number of CPUs and amount of memory you have.
 +
  {| role="presentation" class="wikitable mw-collapsible mw-collapsed"
 +
    | <strong>Answer:</strong>
 +
    |-
 +
    |
 +
      There are multiple ways to find out the number of CPUs and amount of memory on Linux computer.
 +
      the number of CPUs.
 +
      1. <code>lscpu</code>: the following is an example output of the  <code>lscpu</code> command:
 +
      <syntaxhighlight lang="bash">
 +
        CPU(s):                40
 +
        On-line CPU(s) list:  0-39
 +
      </syntaxhighlight>
 +
      Brief description:
 +
            <code>CPU(s):                40</code>: this computer contains 40 CPU cores.
 +
            <code>On-line CPU(s) list:  0-39</code>: all 40 CPU cores are active.
 +
      2. <code>grep -c processor /proc/cpuinfo</code>: This command uses  <code>grep</code> to count the number of lines that match  <code>processor</code> in  <code>/proc/cpuinfo</code>. The following is an example output:
 +
      <syntaxhighlight lang="bash">
 +
        $ grep -c processor /proc/cpuinfo
 +
        40
 +
      </syntaxhighlight>
 +
      As can be seen, there are 40 CPU cores in this computer.
 +
      the amount of memory
 +
      1. <code>vmstat -s</code>: the following is an example output of the <code>vmstat</code> command:
 +
        <syntaxhighlight lang="bash">
 +
          196698736 K total memory
 +
          4982596 K used memory
 +
        </syntaxhighlight>
 +
        <code>vmstat</code> reports the virtual memory statistics.
 +
        The <code>-s</code> option displays a table for memory statistics.
 +
        In this example there are 196 GB memory in total in the computer and 5 GB memory is used.
 +
      2. <code>grep Mem[TF] /proc/meminfo</code>: This command uses <code>grep</code> to search for either <code>MemT</code> or <code>MemF</code> in <code>/proc/meminfo</code> and the following is an example output:
 +
        <syntaxhighlight lang="bash">
 +
          $ grep Mem[TF] /proc/meminfo
 +
          MemTotal:      196698740 kB
 +
          MemFree:        113479460 kB
 +
        </syntaxhighlight>
 +
          As can be seen, MemT and MemF are matched by MemTotal and MemFree, respectively. The total amount of memory in the computer is 196 GB and there are 113 GB are freely available.
 +
    |
 +
    |}
  
 +
2. Permanently change your vim color scheme.
 +
  {| role="presentation" class="wikitable mw-collapsible mw-collapsed"
 +
    | <strong>Answer:</strong>
 +
    |-
 +
    | 
 +
      Your personal vim configuration file is ~/.vimrc. In this file you can change the vim color scheme permanently. For example, the following setting uses the desert color scheme:
 +
      <code>colorscheme desert</code>
 +
      But beware of this permanent change of vim color scheme, because different color schemes may be preferred by different people. The vim color scheme recommended by others may not be the best choice for you.
 +
    |}
  
    page 1:
+
{{Tutorial Navigation
        Please make backups regularly!
+
| previous = [[Introduction_to_Linux_in_HPC/SSH_Graphics_File_Transfer | SSH: Graphics and File Transfer ]]
    page 2:
+
| main = [[Introduction_to_Linux_in_HPC | Overview ]]
        du command
+
}}
        df command
 
    page 3:
 
        history command
 
        the ~/.bash_history file
 
    page 4:  
 
        ln command and syntax
 
    page 5:
 
        watch command and the usage
 
    page 6:
 
        $(( )) as integer calculator
 
    page 7: 60 sec
 
        very brief into to sed
 
        just mention awk
 

Latest revision as of 17:27, 4 December 2020

Tutorial
Title: Introduction to Linux in HPC
Provider: HPC.NRW

Contact: tutorials@hpc.nrw
Type: Multi-part video
Topic Area: HPC Platforms
License: CC-BY-SA
Syllabus

1. Background and History
2. The Command Line
3. Linux Directory Structure
4. Files
5. Text display and search
6. Users and permissions
7. Processes
8. The vim text editor
9. Shell scripting
10. Environment variables
11. System configuration
12. SSH Connections
13. SSH: Graphics and File Transfer
14. Various tips

This video covers a range of miscellaneous topics that are useful, but were not addressed before. For example, there are tools to print the space left on a disk or the recent command history. Other tools regularly execute commands for you or manipulate text. Finally, an overview of common package managers is given.

Video

( Slides as pdf)

Quiz

Does df show the size of a given file?

No
Yes


What does ln -s /tmp . do?

Create a symbolic link of the current directory in /tmp
Create a symbolic link of /tmp in the current directory named ./tmp
Print a list of files located in /tmp and .


What is the watch command used for?

To open a file of unknown format
To regularly execute a command and see its output
To watch video files


What are sed or awk used for?

Manipulate and process text
Set system configuration settings


How will you find a specific command in history ? HINT: use grep with history, how to use it?

Click and submit to see the answer


How can you compare two files and display a list of the difference? HINT: man diff

Click and submit to see the answer


Warning:  Please make backups regularly! Hard drives can fail on any computer

Exercises in Terminal

1. Find out the number of CPUs and amount of memory you have.
2. Permanently change your vim color scheme. 


<< SSH: Graphics and File Transfer

Overview