View source for Git Tutorials/Creating and Changing Repositories

[[Category:Tutorials|Git Tutorials: Creating and Changing Repositories]]<nowiki />
{{DISPLAYTITLE:Git Tutorial: Creating and Changing Repositories}}<nowiki />
{{Syllabus Git Tutorials}}

Most of the users are only interested in having some kind of revision control for their own code. Therefore, we will start with basics, which will allow you to put your code under revision control in a few simple steps, and you will also learn how to easily add changes to the code (this will be done locally on your hard drive; utilizing GitHub follows later). At first, you will need to create a repository for your code. To fully grasp what a repository is, you can read the corresponding Wikipedia page, but for now it is sufficient to think of it as a box into which you will put everything that you want to have under revision control. Additionally, the box contains a list, which tracks everything put into the box, everything taken out of it and any changes made to the objects inside. 

This tutorial will consist of two blocks. The first one focuses on your local repository and the following contents (we will stick with the box analogy for now):

# <syntaxhighlight inline>init</syntaxhighlight>''ializing'' your repository: You '''create''' the box
# <syntaxhighlight inline>add</syntaxhighlight>''ing'' files to the repository: You '''note''' on the list what goes into the box and what changes to apply
# <syntaxhighlight inline>commit</syntaxhighlight>''ting'' your changes to the repository: You '''put''' the objects and '''apply the changes''' to the box as has been '''noted''' on the list
# Excluding files from your repository: Deciding what should never be '''noted''' for the box

The second block will teach you how to be ready for collaboration by using a remote repository - a box used by everyone. Concepts here are similar to those for the local repository, and the topics covered are:

# <syntaxhighlight inline>push</syntaxhighlight>''ing'' your latest <syntaxhighlight inline>commit</syntaxhighlight> to the remote repository: '''Applied changes''' to your local box should be applied to the remote box, too
# <syntaxhighlight inline>pull</syntaxhighlight>''ing'' the latest version from the remote repository: '''Apply changes''' from the remote box to my box

Keep in mind that <syntaxhighlight inline>push</syntaxhighlight> and <syntaxhighlight inline>pull</syntaxhighlight> usually require additional <syntaxhighlight inline>merge</syntaxhighlight> work, which will be covered in a later tutorial. However, if you stick with this tutorial, you should be fine.


Many of git's command line interactions can be easily reproduced through GitHub's GUI. We strongly recommend that, additionally to working through this tutorial, you take a peek at the suggested interactive GitHub tutorials in the [[Git_Tutorials/Creating_and_Changing_Repositories#Useful_Links|Useful Links section]]. They were chosen to closely match this Wiki tutorial.


== Creating a new Repository ==

Before we start, please make sure that you fulfill the following conditions:
* You have some kind of Linux distribution installed
* You have installed [https://git-scm.com/downloads Git]
* You have a folder (we will call it ''MyFolder''), that you want to put under revision control
* You have set your identity and email through the two commands
<syntaxhighlight lang="console">user@HPC.NRW:~$ git config --global user.name "Your Name Comes Here"
user@HPC.NRW:~$ git config --global user.email you@yourdomain.example.com</syntaxhighlight>

Once those conditions are met, go into ''MyFolder'' and initialize the repository by entering

<syntaxhighlight lang="console" line>
user@HPC.NRW:~$ cd MyFolder
user@HPC.NRW:~MyFolder/$
user@HPC.NRW:~MyFolder/$ git init
Initialized empty Git repository in /home/MyFolder/.git/
</syntaxhighlight>
Your repository has been initialized (line 4). Your box has been created, but currently it is still empty. You need to '''note''' with which content you want to fill the box/repository by using the <syntaxhighlight inline>git add</syntaxhighlight> command. You can simply '''note''' to add the whole directory with

<syntaxhighlight lang="console">
user@HPC.NRW:~MyFolder/$ git add .
</syntaxhighlight>

If you want to '''note''' the addition of one or more particular files you can do this through

<syntaxhighlight lang="console">
user@HPC.NRW:~MyFolder/$ git add someFiles* andMore
</syntaxhighlight>
which in this case would '''note''' to <syntaxhighlight inline>add</syntaxhighlight> the file ''andMore'' and any files beginning with ''someFiles'' into the repository (e.g., ''someFiles2'' or ''someFilesWithMoreTextAndNumb3rsAtTheEnd'').

Once you are sure what goes in, it is time to '''put''' it in the box and '''apply the changes'''. This is done through the <syntaxhighlight inline>git commit</syntaxhighlight> command. It has to be accompanied by a message, which describes the changes you have made to the repository. Using the imperative in <syntaxhighlight inline>commit</syntaxhighlight> messages is a widely suggested convention. The message can be directly written with the <syntaxhighlight inline>commit</syntaxhighlight> through the option <syntaxhighlight inline>-m</syntaxhighlight>:

<syntaxhighlight lang="console">
user@HPC.NRW:~MyFolder/$ git commit -m "Create repository/box and fill it with some initial files"
</syntaxhighlight>
This command will be followed by information regarding your <syntaxhighlight inline>commit</syntaxhighlight> like the amount of files and changes it contained. At this point, it will also inform you that you have committed to your master branch. Branches will be covered later, so you do not need to worry about them.

== Modifying your Repository ==

You actually already know everything you need to apply further changes to your repository/box. Let's assume you want to <syntaxhighlight inline>add</syntaxhighlight> a new file ''newFile'' to the repository and that you have also made some adjustments to the file ''andMore'', which is already inside the box. As the box is already '''created''', you only need to '''note''' what you want to add and change:

<syntaxhighlight lang="console">
user@HPC.NRW:~MyFolder/$ git add andMore newFile
</syntaxhighlight>

Of course, you can also simply use <syntaxhighlight inline>git add .</syntaxhighlight> to '''note''' all the changes that have been made so far. Afterwards, '''put''' everything in the box and '''apply the changes''' through {{Avoid wrap|<syntaxhighlight inline>git commit -m "My commit message"</syntaxhighlight>}}. If you have not added any new files, but only made changes to existing ones, then you can type

<syntaxhighlight lang="console">
user@HPC.NRW:~MyFolder/$ git commit -a -m "My commit message with no new files added"
</syntaxhighlight>
This will automatically perform the <syntaxhighlight inline>git add</syntaxhighlight> command on files which have changed since your last <syntaxhighlight inline>commit</syntaxhighlight> (again: No new files will be <syntaxhighlight inline>add</syntaxhighlight>''ed'' this way!).

== Excluding files from the Repository ==

Usually you want to keep your repository tidy without unnecessary file bloat. For example, this can happen when you compile your repository's code and keep the binaries inside. Entering <syntaxhighlight inline>git add .</syntaxhighlight> would result in binaries being <syntaxhighlight inline>add</syntaxhighlight>''ed'' to the repository, which is usually not desirable as they lead to larger repository size. 
The easiest way to control what should not go in your repository is the <syntaxhighlight inline>.gitignore</syntaxhighlight> file. Simply create the file in your repository and note inside, what you want to exclude from revision control. If you do not want to track any <syntaxhighlight inline>.txt</syntaxhighlight> files and the <syntaxhighlight inline>logo.jpg</syntaxhighlight> file in particular, you could do the following:

<syntaxhighlight lang="console" line>
user@HPC.NRW:~MyFolder/$ touch .gitignore
user@HPC.NRW:~MyFolder/$ echo -e "*.txt\nlogo.jpg" > .gitignore
</syntaxhighlight>

Naturally, you can exclude anything, but below you find a <syntaxhighlight inline>.gitignore</syntaxhighlight> example which contains a comprehensive list of file types that can usually be ignored.

<div style="border:2px solid #bcc0c4; background-color:#bcc0c4;" class="mw-collapsible mw-collapsed" data-expandtext="&#9660;" data-collapsetext="&#x25B2;">
<span class="mw-customtoggle-toggle" style="float:right;"></span>
<span style="text-align:center; padding:2px 5px; display:block;">Example of a comprehensive <syntaxhighlight inline>.gitignore</syntaxhighlight> file</span>
<div id="mw-customtoggle-toggle" class="mw-collapsible-content" style="background-color:#f2f2f2; padding:2px 5px; ">
<syntaxhighlight lines>
# SLURM output #
################
*.out
*.err

# Compiled source #
###################
*.com
*.class
*.dll
*.exe
*.o
*.so

# Packages #
############
*.7z
*.dmg
*.gz
*.iso
*.jar
*.rar
*.tar
*.zip

# Logs and databases #
######################
*.log
*.sql
*.sqlite

# OS generated files #
######################
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
</syntaxhighlight>
</div>
</div>

== Going back to a previous Version ==

One advantage of revision control is the possibility of easily going back to an older version of your repository. As with every <syntaxhighlight inline>git add</syntaxhighlight> you have '''noted''' the changes made to the repository's contents, git can use these notes to switch back to a previous state/version. In order to see all existing versions, use the <syntaxhighlight inline>git log</syntaxhighlight> command:

<syntaxhighlight lang ="console" line>
user@HPC.NRW:~MyFolder/$ git log
commit aab2d012c5a5965d14c440a6727191c19625e6e3 (HEAD -> master)
Author: user <tutorials@hpc.nrw>
Date:   Thu Jul 5 14:15:09 2000 +0200

    add .gitignore file

commit 30763897a2fc05b13f3ecb3197a9243b4a7941d8
Author: user <tutorials@hpc.nrw>
Date:   Wed Jul 1 03:43:57 2000 +0200

    Create repository/box and fill it with some initial files
</syntaxhighlight>

This yields information about the authors who made changes to the repository, dates, related <syntaxhighlight inline>commit</syntaxhighlight> messages (lines 6 and 12) and <syntaxhighlight inline>commit</syntaxhighlight> names/references (lines 2 and 8, the string after <syntaxhighlight inline>commit </syntaxhighlight>). These automatically generated references are the SHA-values of each <syntaxhighlight inline>commit</syntaxhighlight>. You can use {{Avoid wrap|<syntaxhighlight inline>git show <commit reference></syntaxhighlight>}} for detailed information on a particular <syntaxhighlight inline>commit</syntaxhighlight>. The long reference is actually not required, and the shortened version you get with <syntaxhighlight inline>git log --oneline</syntaxhighlight> can be used instead. Remember to inspect the different options you can pass to <syntaxhighlight inline>git log</syntaxhighlight> for extended or filtered output.

Once you know which version you want to go back to, use the {{Avoid wrap| <syntaxhighlight inline>git checkout <commit reference> .</syntaxhighlight>}} command (with <syntaxhighlight inline>.</syntaxhighlight> at the end):
<syntaxhighlight lang="console" line>
user@HPC.NRW:~MyFolder/$ git checkout 3076389 .
Updated 1 path from 83ca202
</syntaxhighlight>

Please keep in mind that you should attach an appropriate message to your next <syntaxhighlight inline>commit</syntaxhighlight> to keep everybody aware of version revert due to a <syntaxhighlight inline>checkout</syntaxhighlight>. Generally, such reverts should not be done without caution, even more so when cooperating on a project. It is preferable to use branches in such cases - a [[Git_Tutorials/Branching|topic touched later upon]].

{| role="presentation" class="wikitable mw-collapsible mw-collapsed" style="background-color:white; width:54em" data-expandtext="&#9660;" data-collapsetext="&#9650;"
|+ style="background-color:#2e3871; color:white;text-align: center; width:10em"| <strong>Further remarks</strong>
|-
| <syntaxhighlight inline>git reset --hard <name of commit></syntaxhighlight> can also be used to go back to a previous version and discard all changes that have been made since then.
|-
| Revert changes of a specific version, but keep other changes (that have been done afterwards) by using {{Avoid wrap|<syntaxhighlight inline>git revert <name of commit></syntaxhighlight>}}. Internally, this does not revert files, but commits changes through which the files become identical to their reverted state.  
|}

== Using a remote Repository ==

Although Git is useful for keeping track of your own projects, it becomes most valuable when collaborating with others on the same code. In such cases, it is useful to set up a remote repository to which everyone has access and can contribute their changes. This tutorial will focus on how to set up your remote repository with the help of GitHub. At the end, we will also show you how to <syntaxhighlight inline>clone</syntaxhighlight> a remote repository from GitHub and start contributing to an existing project. Please note that you can set the Git remote repository to be anywhere, e.g., a location in your facilities own network, but for simplicity's sake we stick with GitHub only.
Before we start, please make sure that you have

* registered a GitHub account
* an [[Git_Tutorials/Creating_and_Changing_Repositories#Creating_a_new_Repository|existing Git repository]] to upload to GitHub (you can use your local repository from the previous section)

Now, log into your account and do the following preparations:

# Start creating a new GitHub repository
# Give the repository an appropriate name, we choose "Wikipages"
# Set yourself as the owner
# Add some short description
# Set the repository to private
# Do not add any <syntaxhighlight inline>.ignore</syntaxhighlight> or <syntaxhighlight inline>README</syntaxhighlight> file, as you are going to use your existing local repository, which should at least have an <syntaxhighlight inline>.ignore</syntaxhighlight> file
# Finish creating the repository

Afterwards, you associate your local repository with your remote one and <syntaxhighlight inline>push</syntaxhighlight> your latest <syntaxhighlight inline>commit</syntaxhighlight> to GitHub's repository:

<syntaxhighlight lang="console" line>
user@HPC.NRW:~MyFolder/$ git remote add origin https://github.com/HPC.NRW-User/Wikipages.git
user@HPC.NRW:~MyFolder/$ git branch -M main
user@HPC.NRW:~MyFolder/$ git push -u origin main
Username for 'https://github.com': HPC.NRW-User
Password for 'https://HPC.NRW-User@github.com': ********
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 12 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 341 bytes | 341.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/HPC.NRW-User/Wikipages.git
   75895d2..b4e0116  main -> main
</syntaxhighlight>

The <syntaxhighlight inline>origin</syntaxhighlight> in line 1 is an arbitrary name (though by convention) chosen for your remote repository. In theory, you could have several remote repositories and always chose which one to use. For now, it should be sufficient to stick with conventions and use only one remote repository named ''origin''. Line 2 generates a branch called ''main'' to which you will be automatically set. Branches will be covered in [[Git_Tutorials/Branching|the next tutorial]], but it is beneficial to start using them early on, in particular when collaborating on a project. Line 3 is the actual <syntaxhighlight inline>push</syntaxhighlight>, meaning that afterwards the same files and changes as in your local repository will be '''put''' into the remote one. Do not forget to <syntaxhighlight inline>commit</syntaxhighlight> your changes before <syntaxhighlight inline>push</syntaxhighlight>''ing'' them to make sure that everything is up-to-date. However, if you were following this tutorial step by step, then no <syntaxhighlight inline>commit</syntaxhighlight> was necessary here, because no changes have been performed before the <syntaxhighlight inline>push</syntaxhighlight>.
GitHub provides an interactive tutorial for moving your local repository to GitHub. You can check it out in the [[Git_Tutorials/Creating_and_Changing_Repositories#Useful_Links|Useful Links section]].


The next step is to learn how you can join an existing remote repository. Usually, you have to make sure that others (or you yourself) are allowed to contribute to a remote repository and to download from it. In our case, you are the only user and therefore no further settings are required. For practicing purposes, we will continue using the current remote repository.
The first thing you often want to do is to <syntaxhighlight inline>clone</syntaxhighlight> an existing repository, which basically means, you are going to '''create''' an empty box and '''put''' into it exactly what is in the remote repository box:


[[File:Git_tutorial_-_clone_commit_push.png|thumb|500px|Figure 1: Visualization of the worklflow with different <syntaxhighlight inline>git</syntaxhighlight> operations acting on a remote and a local repository. Different colors represent different/modified content. As can be seen, it is suggested that a <syntaxhighlight inline>push</syntaxhighlight> is only performed after sufficient changes have been implemented in the local repository. However, for the local repository it is better to <syntaxhighlight inline>commit</syntaxhighlight> often.]]

<syntaxhighlight lang="console" line>
user@HPC.NRW:~MyFolder/$ mkdir ~/myClonedFolder
user@HPC.NRW:~MyFolder/$ cd ~/myClonedFolder
user@HPC.NRW:~MyFolder/$ git init
user@HPC.NRW:~MyFolder/$ git clone https://github.com/HPC.NRW-User/Wikipages.git
Cloning into 'Wikipages'...
Username for 'https://github.com': HPC.NRW-User
Password for 'https://HPC.NRW-User@github.com': ******** 
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 44 (delta 23), reused 42 (delta 21), pack-reused 0
Unpacking objects: 100% (44/44), done.
</syntaxhighlight>

That’s it. You have successfully <syntaxhighlight inline>clone</syntaxhighlight>''d'' a remote repository and can start working on it. Figure 1 visualizes how <syntaxhighlight inline>clone</syntaxhighlight>, <syntaxhighlight inline>commit</syntaxhighlight> and <syntaxhighlight inline>push</syntaxhighlight> operations affect local and remote repositories.


The last thing you will learn in this tutorial is how to get <syntaxhighlight inline>push</syntaxhighlight>''ed'' '''changes''' from a remote repository. The prerequisite here is, that you already have a local version of a remote repository (through either <syntaxhighlight inline>git clone</syntaxhighlight> or <syntaxhighlight inline>git remote add</syntaxhighlight>). If you have been following the tutorial up to this point, this should be a given.
What we will do now is:

# Go to the <syntaxhighlight inline>~/myClonedFolder</syntaxhighlight> directory
# Change a file in your <syntaxhighlight inline>clone</syntaxhighlight>''d'' repository in <syntaxhighlight inline>~/myClonedFolder</syntaxhighlight> and add a new file ''clonedRepoFile'' to it
# <syntaxhighlight inline>add</syntaxhighlight> (or '''note''') the changes you want to do
# <syntaxhighlight inline>commit</syntaxhighlight> the changes
# <syntaxhighlight inline>push</syntaxhighlight> the last <syntaxhighlight inline>commit</syntaxhighlight> to the remote repository
# Go back to the initial local repository in <syntaxhighlight inline>~/MyFolder</syntaxhighlight>
# Get the recently <syntaxhighlight inline>push</syntaxhighlight>''ed'' changes from the remote repository into the local repository in <syntaxhighlight inline>~/MyFolder</syntaxhighlight>

The above points correspond to the following lines:

<syntaxhighlight lang="console" line>
user@HPC.NRW:~MyFolder/$ cd ~/myClonedFolder
user@HPC.NRW:~MyFolder/$ touch clonedRepoFile
user@HPC.NRW:~MyFolder/$ git add clonedRepoFile someChangedFile
user@HPC.NRW:~MyFolder/$ git commit -m “add file clonedRepoFile and make some small changes”
user@HPC.NRW:~MyFolder/$ git push origin main
user@HPC.NRW:~MyFolder/$ cd ~/myFolder
user@HPC.NRW:~MyFolder/$ git pull origin main
</syntaxhighlight> 

Afterwards, the contents of your folders ''myFolder'' and ''myClonedFolder'' should be identical. <syntaxhighlight inline>pull</syntaxhighlight> is most commonly used to be up-to-date with changes other contributors have <syntaxhighlight inline>push</syntaxhighlight>''ed'' and not for going back to an older version. The latter will more often than not lead to conflicts with other versions which need to be painstakingly resolved and that will propagate to future versions for your colleagues, once you have <syntaxhighlight inline>commit</syntaxhighlight>''ted'' and <syntaxhighlight inline>push</syntaxhighlight>''ed'' your changes after <syntaxhighlight inline>pull</syntaxhighlight>''ing'' an older version.

== Best Practices ==

* This cannot be understated: [[Git_Tutorials/Branching|'''WORK THROUGH THE NEXT TUTORIAL ON BRANCHES''']]
* Do not <syntaxhighlight inline>push</syntaxhighlight> every <syntaxhighlight inline>commit</syntaxhighlight> to the remote repository. Do <syntaxhighlight inline>commit</syntaxhighlight>''s'' regularly to your local repository, but only <syntaxhighlight inline>push</syntaxhighlight> them, once a significant amount of changes has been made.
* <syntaxhighlight inline>checkout</syntaxhighlight> and <syntaxhighlight inline>pull</syntaxhighlight> are not meant for jumping back and forth between different versions of code. <syntaxhighlight inline>checkout</syntaxhighlight> should be used sparsely, while <syntaxhighlight inline>pull</syntaxhighlight> is intended to get your local repository up-to-date with other's contributions to the remote repository.
* <syntaxhighlight inline>commit</syntaxhighlight> messages should be used in imperative form ("add" instead of "added" or "Write new method for..." instead of "Wrote new method for..."). However, this is just a convention.
* <syntaxhighlight inline>commit</syntaxhighlight> messages should give an idea of what the functional idea of the * <syntaxhighlight inline>commit</syntaxhighlight> is. Therefore, a detailed description of new methods should not be part of the message, because that is what the code itself is for. In general the messages should be short (less than 50 characters).


== Glossary ==

* <syntaxhighlight inline>add</syntaxhighlight>: Keeps track of what should be adjusted in the local repository with your next <syntaxhighlight inline>commit</syntaxhighlight>.
* <syntaxhighlight inline>commit</syntaxhighlight>: Files marked for tracking (aka staging) with <syntaxhighlight inline>add</syntaxhighlight> will be adjusted in the current branch of your repository (other branches can be specified).
* <syntaxhighlight inline>push</syntaxhighlight>: Take your most recent <syntaxhighlight inline>commit</syntaxhighlight> and ask for your changes to be integrated into the most recent remote repository version of the desired branch. Usually, when collaborating on a project, this does '''not''' mean that the remote version will be identical to yours afterwards, because others have been pushing their changes, too.
* <syntaxhighlight inline>checkout</syntaxhighlight>: Adjust files in your current directory to a different version (or a different branch).
* <syntaxhighlight inline>pull</syntaxhighlight>: Similar to <syntaxhighlight inline>checkout</syntaxhighlight>, but you access a version from a remote repository. Usually requires some <syntaxhighlight inline>merge</syntaxhighlight> adjustments (which have not been covered, yet).



== Useful Links ==

* [https://lab.github.com/githubtraining/uploading-your-project-to-github Interactive GitHub course]: uploading your project to GitHub
* [https://lab.github.com/githubtraining/introduction-to-github Interactive GitHub course]: first steps