Difference between revisions of "User:Mukund-pondkule-6a11@uni-paderborn.de"

Revision as of 14:36, 15 September 2020

An Introduction to the Linux Operating System and Command Line. The course provides students with a hands-on approach for learning Linux through assignments and projects.

Test

Historical Background

Video: Slide 17 - 23 (7 pages), 5 min 45 sec (Text) + 15 sec (Quiz)

Linux Introduction

Text

Quiz:

1. Are all Top 500 supercomputers using Linux in HPC?
  A: Yes
  B: No

Answer:

A

Explanation: see https://top500.org/statistics/list (access on 27.08.2020) List in June 2020 shows all Top 500 supercomputers are using Linux.

2. What does GNU stands for?
  A. GNU's Not Unix
  B. Geek Needed Unix
  C. General Unix
  D. General Unix
  E. None of the Above

Answer:

A

Tips: no tips in this section

Warning: no warning in this section

Exercises in Terminal: no exercises in this section

The Command Line

Video: Slide 17 - 23 (7 pages), 5 min 45 sec (Text) + 15 sec (Quiz)

Linux Introduction

Text

Quiz:

1. Which keys can be used for command history?
  A: up- and down-arrow keys
  B: Page-up and Page-down keys

Answer:

A

Explanation: up- and down-arrow keys (↑ and ↓) can be used for command history. The behavior of Page-up and Page-down keys depends on the setting of a terminal. But normally the command history is not available by using the Page-up and Page-down keys.

Tips: Working directory in console reminds user, where they are. (page 3)

Warning: 1) In command line user may forget where they are (page 2). 2) Child processes may stop, if parent shell exits (page 2).

Exercises in Terminal:

1. What's your username on a Linux computer?
  There are two ways to find your username on a Linux computer

Answer:

     1. In the command line prompt, e.g. [username@hostname ~]$ , you can find your username as username directly.
     2. The Linux command whoami can also show your username.

2. What's the hostname of a Linux computer?
  There are two ways to find the hostname on a Linux computer.

Answer:

     1. In the command line prompt, e.g. [username@hostname ~]$ , you can find the hostname as hostname directly.
     2. The Linux command hostname can also give you the hostname.

3. What's your current working directory on a Linux computer?
   There are two ways to find your current working directory on a Linux computer.

Answer:

      1. In the command line prompt, e.g. [username@hostname ~]$ , you can find your current working directory is ~, which means your $HOME directory.
      2. The Linux command pwd can also show your current working directory.

4.  Use up- and down-arrow keys to see the command history.

Explanation:

      The up- and down- arrow keys can be used to navigate command history.

5.  Run sleep 1h command and wait, then use Ctrl-C to kill it.

Explanation:

      sleep 1h puts the terminal into idle, e.g. terminal is doing nothing, for 1 hour.
      Ctrl-C kills the current command. In this exercise it's sleep 1h.

Directory structure

Video: Slide 33 - 48 (16 pages), 5 min 45 sec (Text + Demo + Quiz) + 15 sec (Tips and Warning)

Linux Introduction

Text

Quiz:

1. Which one is the top directory in Linux?
  A: /
  B: /home
  C: C:\

Answer:

A

Explanation: The tree structure for directory is used in Linux system. Therefore the top directory in Linux is /. The /home directory may be an upper level directory for all non-root users. The last option C:\ represents the C drive on Windows.

2. The command 'cd ' without arguments : if you start in var/log/ and run cd with no arguments, what do you expect will happen?
  A: Nothing it stays in /var/log/.
  B: It goes to your home directory.
  C: It goes to the filesystem root
  D: The shell prompt turns into a shark and eats you.
  E: It’s an error.
  F: The shell stops having a working directory.

Answer:

B

It goes to your home directory. cd without arguments is a shortcut to take you home. As long as your home directory exists, you can always go home.

Tips: no tips in this section

Warning: no undo and make sure what you want to do (page 16)

Exercises in Terminal (slide 49):

1. Go to a specific subfolder of a folder (example: cd Documents/courses/ ) and get back to the home directory using cd command. List 3 different ways to do it using one command. check after every action your path with pwd command.

Answer:

       cd
       cd ~
       cd $HOME
       the last option $HOME is an enviornment variable. You will learn about enviornment variables later.

2. Go to the directory /tmp and jump between /tmp and your home directory back and forth. check after every action your path with pwd command.

Answer:

       cd /tmp
       cd -
       cd - with cd - you change back to the previous working directory, pass the dash (-) character as an argument to the cd command.

Files

Video: Slide 51 - 56 (6 pages), 5 min 30 sec (Text + Demo + Quiz) + 15 sec (Tips and Warning)

Linux Introduction

Text

Quiz:

1. Can you copy directory with cp <old_dir> <new_dir> ? 
  A: Yes
  B: No, -r option is needed for copying directory

Answer:

B

Explanation: Without any option the cp command can only be used to copy files (text files as well as binary files). To copy a directory, the -r option must be used and it means to copy a given directory in a recursive manner e.g. one file after another in a directory.

Tips: no tips in this section

Warning: 1. rm -f: no confirmation! Use with care (page 3). 2. Wildcards expand before given to program (page 6).

Exercises in Terminal:

1. Use the find command to search for files, whose name ends with dat, in your home directory.

Answer:

     The complete command is find $HOME -type f -name "*dat" 
         find can be used to search files and directories.
         $HOME is your home directory.
         -type f means only file (not directory) will be shown.
         -name "*dat" specifies the required filename and "*dat" restricts the filename ends with dat, while it can begin with any characters (the * wild cards).

2. Use the find command to search for directories, whose name begins with test and ends with _dir, in your home directory.

Answer:

     The complete command is find $HOME -type d -name "test*_dir"
         find can be used to search files and directories.
         $HOME is your home directory.
         -type d means only directory (not file) will be shown.
         -name "test*_dir" specifies the required directory name, which begins with test and ends with _dir. The wild card * matches any characters in between.

Text display, search

Video: Slide 58 - 64 (7 pages), 5 min 40 sec (Text + Demo) + 15 sec (Quiz)

Linux Introduction

Text

Quiz:

1. Which command can be used to append text to filename?
  A: command > filename
  B: command >> filename

Answer:

B

Explanation: The > operator redirects the output of command into filename. If filename exists already, its contents will be overwritten. The >> operator can redirect the output of command into filename as well, except that if filename exists, the new data are appended instead of overwritten.

2. How do I find all files containing specific text on Linux?
   HINT: use grep command

Answer:

      grep -rnw '/path/to/somewhere/' -e 'pattern'
           -r or -R is recursive.
           -n is line number.
           -w stands for match the whole word.
           -l (lower-case L) can be added to just give the file name of matching files.

Tips: no tips in this section

Warning: no warning in this section

Exercises in Terminal (slide 65):

1. Create a file using cat (some filename) and write some text in it as below. than use grep to find and display the line number and the line with the word Alan in the file.
        the best way to predict
        the future is to invent it
        –  Alan Kay

Answer:

      $ cat > quote.txt <<"EOF" \ > the best way to predict \ > the future is to invent it
      >  - Alan Kay
      > EOF
      EOF mean end of file. Press enter to get a new line while writing text. With cat > filename you write the file content in the file quote.txt.

      Further use grep as below
      $ grep -in alan quote.txt
      3: - Alan Kay 
      grep command option -
           - i to ignore case distinctions, so that characters that differ only in case match each other.
           - n for line number

Processes and permissions

Video: Slide 67 - 88 (22 pages), 7 min 40 sec (Text + Demo)

Linux Introduction

Text

Quiz:

1. Which command can prevent other users not in your group from writing to input.dat? 
  A: chmod u-w input.dat
  B: chmod o-w input.dat
  C: chmod g-w input.dat

Answer:

B

Explanation: The chmod command changes file permission.

           u represents the user who owns the file. 
           g means other users in your group.
           o stands for other users not in your group. 
           -w means to remove the write permission. 
     Therefore B is the correct answer.

Tips: no tips in this section

Warning: no warning in this section

Exercises in Terminal (slide 89):

1. Create an empty file, make it read-only, make it executable.

Answer:

     Create an empty file: touch file.txt.
     make it read-only: chmod a-w file.txt; chmod a-x file.txt; chmod a+r file.txt.
     make it executable: chmod a+x file.txt.

     Explanation: The touch command can be used to create an empty file. In this example it is file.txt.
          chmod a-w file.txt; chmod a-x file.txt; chmod a+r file.txt means to execute the three commands subsequently. 
          chmod a-w file.txt removes the write permission for all users. 
          chmod a-x file.txt removes the executable permission for all users.
          chmod a+r file.txt adds the read permission for all users.
          chmod a+x file.txt makes file.txt executable for all users.

2. Start a process (e.g. sleep 10m)

Answer:

     Use a second console to look at it in top.
     Kill it.
     Hint:
        To kill the started process (e.g. sleep 10m): 
        1. Go to the terminal that started the process.
        2. Press Ctrl-C to kill it.

3. Start a process, bring it into background/foreground.

Hint:

     To bring a process to background:
        Press Ctrl-Z (pause the process).
        Type bg (bring it to background).
     To bring a process to foreground:
        Type fg (bring it to foreground).

The vim text editor

Video: Slide 91 - 99 (9 pages), 5 min 0 sec (Text + Demo) + 15 sec (Quiz)

Linux Introduction

Text

Quiz:

1. How to enter the insert mode of vim ? 
  A: press enter key
  B: press i key
  C: press Esc key

Answer:

B Explanation: If you are not in the insert mode, the i key means insert and by pressing it you can enter the insert mode. On the other hand, the Esc key brings you out of the insert mode. The behavior of the enter key depends on in which mode of vim you are. If you are in the normal mode, you navigate to the next line. If you are in the insert mode already, you start a new line.

2. How would you open a file in read-only mode using the vim editor ? 
  Hint: In terminal man vim

Answer:

$ vim -R filename

Tips: If you forget which mode you are in while using vim, just keep pressing Esc.

Warning: no warning in this section

Exercises in Terminal (slide 100):

1.  Create a vim file and write some text in it with insert (pressing i) and than undo and redo the changes.

Answer:

     $ vim test.txt 
     In vim editor press i to insert text and write few lines.
     Press Esc than press u (for undo) after that press ctrl + r on keyboard for redo.
        - u: undo last change (can be repeated to undo preceding commands).
        - Ctrl-r: Redo changes which were undone (undo the undos).

Shell scripting, environment variables

Video: Slide 102 - 114 (13 pages), 7 min 0 sec (Text + Demo)

Linux Introduction

Text

Quiz:

1. Which bash command below assigns "value" to variable var? 
  A: var="value"
  B: var = "value"
  C: var=="value"

Answer:

A Explanation: The space character in bash is important. To assign value to a variable, there must not be any space characters in the assignment. Therefore, A is correct. B results in a bash syntax error. C assigns the value =value to variable var. The first = sign means assignment. The second = sign is part of the value. The two double-quotes around value state that value is a character string.

Tips: no tips in this section

Warning: no warning in this section

Exercises in Terminal (slides 115 - 116) and solution (slide 117):

1. a. Write a script that
   b. Prints an environment variable
   c. Saves the output of the date command to a variable
   d. Sleeps briefly
   e. Prints the new and old date and time

Answer:

     One possible way to write the script is

      #!/bin/bash

      echo "I am $USER, my home directory is $HOME"

      olddate=$(date)
      sleep 10s
      echo "old date: ${olddate}"
      echo "new date: $(date)"

     Explanation:
        #!/bin/bash: the shebang line for bash script
        echo "I am $USER, my home directory is $HOME": the variables $USER and $HOME are your username and home directory, respectively. This line prints a sentence embedded with your username and home directory.
        olddate=$(date) assigns the output of the date command to the variable olddate as a string.
        sleep 10s puts the terminal into idle for 10 seconds.
        echo "old date: ${olddate}" prints the previously saved date in the olddate variable. The form of ${olddate} is for using (or referencing) this variable.
        echo "new date: $(date)" prints the current date, in which the output of the command date will take the place of $(date).

2. Find a way to execute a script without setting execute permissions.

Answer:

bash script.sh Explanation: bash will be used as an interpreter to execute script.sh and the commands contained in script.sh will be executed sequentially.

3. Find out how to do other programming things in bash (e.g. functions, classes). How convenient do they look?

Answer:

    To define a function hello in bash, which prints a message "hello, world" on the terminal. bash script.sh

       function hello {
           echo "hello, world"
       }

    To use this function in bash (after defining it):

       hello

4. Look at different ways you can define if conditions.

Answer:

     Both single square brackets [] and double square brackets [[]] can be used to define if conditions. But the effects are different. For example, we define a variable str by using a string with spaces:

        str="a  b"

     The if condition with double square brackets works fine.

       if [[ $str = "a  b" ]]; then
          echo "it works"
       fi

     However, the if condition with single square brackets will emit a syntax error.

       if [ $str = "a  b" ]; then
          echo "it works"
       fi

     To have the correct syntax for if condition with single square brackets, the variable str must be double quoted inside the if condition.

       if [ "$str" = "a  b" ]; then
          echo "it works"
       fi

5. Find out what different types of quotes (single ' vs. double ") do?

Answer:

     single quote ' gives literal string, the variable will not be interpreted, e.g.

       var=abc
       echo '$var'

     This script prints the literal string $var (instead of its value abc) to terminal.
     double quote " allows the variable to be interpreted, e.g.

       var=abc
       echo "$var"

     This script prints the value of var, which is abc (instead of the literal string $var) to terminal.

6. create an shell variable MYIDENTITY and export it as below: 
   $ export MYIDENTITY=whoami
   How will you list the shell variable MYIDENTITY?
   Execute the shell variable MYIDENTITY, what is the output?

Answer:

     you can list the variable by using the echo command as follows

        $ echo $MYIDENTITY
        whoami

     The arguments passed to echo are printed to the standard output.

        $ $MYIDENTITY     
        username

     The shell variable upon execution runs the command whoami, which is assigned to it. whoami command prints the user name of the effective user ID

System configuration files

Video: Slide 119 - 128 (10 pages), 6 min 50 sec (Text + Demo)

Linux Introduction

Text

Quiz:

1. In which system file could you find the CPU information of a Linux computer?  
  A: /dev/cpuinfo
  B: /sys/cpuinfo
  C: /proc/cpuinfo

Answer:

C Explanation: The /dev directory contains special files that represent devices, e.g. hard drive. None of them, however, gives the CPU information. The /sys directory is a virtual file system that stores the device information on a Linux system. However, the /sys/cpuinfo does not exist at all. The /proc directory is also a virtual file system that contains information about running process on a Linux system. From the file /proc/cpuinfo you can obtain the CPU information.

Tips: no tips in this section

Warning: Be very careful, when editing configuration files. A faulty configuration file may prevent a user from login to computer.

Exercises in Terminal: no exercises in this section

Various tips

Video: Slide 130 - 136 (7 pages), 5 min 10 sec (Text + Demo)

Linux Introduction

Text

Quiz:

1. How will you find a specific command in history ? HINT: use grep with history, how to use it?

Answer:

     history | grep cd

2. How can you compare two files and display a list of the difference? HINT: man diff

Answer:

     diff file1 file2

Tips: Please make backups regularly!

Warning: Please make backups regularly!

Exercises in Terminal (slides 137):

1. Find out the number of CPUs and amount of memory you have.

Answer:

     There are multiple ways to find out the number of CPUs and amount of memory on Linux computer. 
     the number of CPUs.
      1. lscpu: the following is an example output of the  lscpu command:

        CPU(s):                40
        On-line CPU(s) list:   0-39
      </syntaxhighlight lang="bash">
      Brief description:
            <code>CPU(s):                40</code>: this computer contains 40 CPU cores.
            <code>On-line CPU(s) list:   0-39</code>: all 40 CPU cores are active.
       2. <code>grep -c processor /proc/cpuinfo</code>: This command uses  <code>grep</code> to count the number of lines that match  <code>processor</code> in  <code>/proc/cpuinfo</code>. The following is an example output:
      <syntaxhighlight lang="bash">
        $ grep -c processor /proc/cpuinfo
        40

     As can be seen, there are 40 CPU cores in this computer.
     the amount of memory
      1. vmstat -s: the following is an example output of the vmstat command:

           196698736 K total memory
           4982596 K used memory

        vmstat reports the virtual memory statistics.
        The -s option displays a table for memory statistics.
        In this example there are 196 GB memory in total in the computer and 5 GB memory is used.
      2. grep Mem[TF] /proc/meminfo: This command uses grep to search for either MemT or MemF in /proc/meminfo and the following is an example output:

           $ grep Mem[TF] /proc/meminfo
           MemTotal:       196698740 kB
           MemFree:        113479460 kB

         As can be seen, MemT and MemF are matched by MemTotal and MemFree, respectively. The total amount of memory in the computer is 196 GB and there are 113 GB are freely available.

2. Permanently change your vim color scheme.

Answer:

     Your personal vim configuration file is ~/.vimrc. In this file you can change the vim color scheme permanently. For example, the following setting uses the desert color scheme:
      colorscheme desert
     But beware of this permanent change of vim color scheme, because different color schemes may be preferred by different people. The vim color scheme recommended by others may not the best choice for you.

Beyond the cluster

Video: Slide 139 - 141 (3 pages), 2 min 30 sec (Text)

Linux Introduction

Text

Tips: no tips in this section

Warning: no warning in this section

Exercises in Terminal: no exercises in this section

Linux in HPC

Video: Slide 3 - 40 (38 pages), 14 min 35 sec (Text)

Linux in HPC

Text

Quiz:

1. Which command you can use to do a secure copy from the Cluster to you local Linux machine?
   Hint:man scp

Answer:

scp

2. Label the interface elements in the terminal:
   1. shell command
   2. current prompt
   3. previous prompt
   4. cursor
   5. login message
   6. command output

Tip: Integrated in slides.

Warning: Integrated in slides.

Exercises for Linux in HPC: GO CP2K GO!

CP2K is a quantum chemistry and solid state physics software package that can perform atomistic simulations of solid state, liquid, molecular, periodic, material, crystal, and biological systems. In this exercise we are going to
   1. create CP2K input files with different cutoff values from a template input for simulation of 32 water molecules in a box using density functional theory (DFT) calculation.
   2 .analyse the simulation output files and summarize some important results.

Create CP2K input files
The CP2K template input file for simulation of 32 water molecules in a box using DFT calculation can be found in the Ex_LinuxHPC/01_CreateInput directory and the file name is template.inp.
A placeholder __CUTOFF__ is set on line 7 of this file.

     CUTOFF __CUTOFF__

With smaller cutoff value the DFT calculation runs faster, but the results may be less accurate. With larger cutoff value, on the other hand, the results become more accurate, but the DFT calculation can be slower.
In this exercise we create the CP2K input files based on the template (template.inp) for a range of different cutoff values, e.g. from 250 to 350 with a step size of 10. Please write a bash script that:
   1. creates individual subdirectories for the simulations with different cutoff values
   2. in each subdirectory creates the CP2K input file from the template file with the placeholder __CUTOFF__ being replaced by an appropriate cutoff value. The cutoff values are from 250 to 350 and the increment 
      is 10. e.g. the cutoff in the first input file may have

         CUTOFF 250

   and the cutoff in the second input file may look like

         CUTOFF 260

   and so on until CUTOFF 350 in the last CP2K input file.

Note: Due to the time limitation we cannot perform all these CP2K simulations during this exercise. However, the example CP2K input (run.inp) and output (run.out) files with different cutoff values can be 
found in Ex_LinuxHPC/02_AnalyseOutput.

Analyse CP2K output files

In the CP2K output file, e.g.run.out, the most important information is printed after every simulation step. For example:

 *******************************************************************************
 ENSEMBLE TYPE                =                                              NVE
 STEP NUMBER                  =                                                1
 TIME [fs]                    =                                         0.500000
 CONSERVED QUANTITY [hartree] =                              -0.545718508103E+03

                                              INSTANTANEOUS             AVERAGES
 CPU TIME [s]                 =                        8.34                 8.34
 ENERGY DRIFT PER ATOM [K]    =         -0.172713513639E+02   0.000000000000E+00
 POTENTIAL ENERGY[hartree]    =         -0.545966997800E+03  -0.545966997800E+03
 KINETIC ENERGY [hartree]     =          0.248489696633E+00   0.248489696633E+00
 TEMPERATURE [K]              =                     550.644              550.644
 *******************************************************************************

for step number 1 (see the line of STEP NUMBER above). Among these data the most useful results are:
   1. the time step for the simulation on the line beginning with TIME [fs].
   2. the potential energy for the step on the line beginning with POTENTIAL ENERGY[hartree]. Please note the energy value given in the column of INSTANTANEOUS is relevant.

In this exercise please:
   1. write a script that summarizes and prints the time step and potential energy for the CP2K output file.
   2. create a bash script to loop through all CP2K output files and print the time step and potential energy for each one automatically.

@@ Line 32: / Line 32: @@
 { Aristotle|Plato|plato|aristotle }
 </quiz>
+{{#widget:YouTube|id=DWef69ItVrU|height=360|width=480}}
 === Historical Background === <!--T:8-->

Difference between revisions of "User:Mukund-pondkule-6a11@uni-paderborn.de"

Revision as of 14:36, 15 September 2020

Contents

Test

Historical Background

The Command Line

Directory structure

Files

Text display, search

Processes and permissions

The vim text editor

Shell scripting, environment variables

System configuration files

Various tips

Beyond the cluster

Linux in HPC

Navigation menu

Search