Difference between revisions of "Python/pip"

From HPC Wiki
Jump to navigation Jump to search
m
Line 5: Line 5:
  
 
== Usage ==
 
== Usage ==
In Linux, there is a version of Python boxed in the distribution, delivered as RPM. This Python and some Py-modules (NumPy, SciPy, Matplotlib, ...) are available by default. The RPMs installed in exactly that versions which are available by OS, also typically older versions. The RPMs are updated accordingly to the OS distribution update rules. We do not offer any other versions/updates/fixes besides the distribution RPMs. Users cannot modify this installation. However, you are welcome to add needed Py-modules in your $HOME directory, e.g. by 'pip' or 'easy_install'.  
+
Python and its most commonly used libraries (NumPy, SciPy, Matplotlib) are usually available as a module. Please notice that an older version may be set as default. If the version you need is not available, you can install it in your directory by using 'pip' or 'easy_install'. You might also want to consider using a GCC compiler as Python software with it. This should be done before adding any Py-Modules.
Python software often rely on GCC compilers and do not work with Intel compilers; thus type 'module switch intel gcc' before adding any Py-Modules.
 
  
Examples:
+
 
 +
Examples (RWTH CLAIX):
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
$ module switch intel gcc
 
$ module switch intel gcc
Line 25: Line 25:
 
Note: the installation is in $HOME/.local/[bin|lib] directories. Beware of clash if using multiple Py-modules!
 
Note: the installation is in $HOME/.local/[bin|lib] directories. Beware of clash if using multiple Py-modules!
  
== Other Python versions ==
+
== Pip ==
If you need yet another version of Python as the one available by default, take a look at the output of 'module avail python' command. It lists a number of Python installations available in the HPC Cluster. E.g. type 'module load python/3.6.0'  for loading of the the appropriate version of Python3.
+
Pip is a package management system for Python and is automatically included since Python 2.7.9/3.4. It enables managing lists of packages and their versions
 +
 
 +
Getting the list of the installed packages:
 +
<syntaxhighlight lang="bash">
 +
$ pip list
 +
</syntaxhighlight>
  
Note: the name of the Python binary version 3 is 'python2' and not just 'python'!
+
Pip can also list the packages with outdated versions or available prereleases:
 +
<syntaxhighlight lang="bash">
 +
$ pip list --outdated
 +
$ pip list --pre
 +
</syntaxhighlight>
  
You can add Py-Modules in your $HOME in same ways as described above. In addition,
+
Packages can easily be installed through the pip install command:
some Py-Modules (NumPy, SciPy, ...) are bundled in these installations, and made available by PYTHONPATH environment variable (set by loading the python environment module). Whenever modifying this environment variable, you will likely wish to enhance the value instead to overwrite it, e.g. 'export PYTHONPATH=$HOME/my/newpatho/added:$PYTHONPATH'
 
By unsetting the PYTHONPATH envvar you have the ultimate power to install any wished versions of these bundled Py-modules in your $HOME.
 
  
Intel also offers Python releases, available via the environment modules. List available versions with  'module avail pythoni' and load one by e.g. 'module load pythoni/3.5' for Python3 version.
+
<syntaxhighlight lang="bash">
All notes above about installing software in $HOME and PYTHONPATH apply. Note that Intel also call Python-3 binary 'python' in addition to of 'python3', which differs the Intel release of python from other ones.
+
$ pip install my-package
 +
</syntaxhighlight>
  
You're also free to install your own version of Python in your $HOME. All notes above about installing software in $HOME and PYTHONPATH apply.
+
The same goes for uninstalling packages:
 +
 
 +
<syntaxhighlight lang="bash">
 +
$ pip uninstall my-package
 +
</syntaxhighlight>
 +
 
 +
The present packages can be stored as a 'requirements' file. If properly formatted, this file can then be used to recreate the given environment on another system with exactly the same packages and versions:
 +
 
 +
<syntaxhighlight lang="bash">
 +
$ pip install -r requirements.txt
 +
</syntaxhighlight>
  
 
== Tutorials ==
 
== Tutorials ==
Line 50: Line 68:
 
== References ==
 
== References ==
 
https://www.python.org/about/
 
https://www.python.org/about/
 +
https://pypi.org/project/pip/

Revision as of 11:29, 10 December 2018

General

Python is an Open Source, interpreted high-level programming language which is popular for prototyping high end simulations and data science applications. In addition to a large standard library there are many thirdparty libraries available (e.g. Numpy, Scipy). However, it is a single threaded environment and therefore may not be most suitable for very large simulations.

Usage

Python and its most commonly used libraries (NumPy, SciPy, Matplotlib) are usually available as a module. Please notice that an older version may be set as default. If the version you need is not available, you can install it in your directory by using 'pip' or 'easy_install'. You might also want to consider using a GCC compiler as Python software with it. This should be done before adding any Py-Modules.


Examples (RWTH CLAIX):

$ module switch intel gcc
$ export MYPY=$HOME/SomeDirForPythonInstall
$ export PYTHONPATH=$PYTHONPATH:$MYPY/lib/python2.7/site-packages
$ mkdir -p $MYPY/lib/python2.7/site-packages
$ easy_install --prefix $MYPY  theano

Note: you need to set PYTHONPATH anytime you wish to use 'teano' Py-module!

$ module switch intel gcc
$ pip install --user theano

Note: the installation is in $HOME/.local/[bin|lib] directories. Beware of clash if using multiple Py-modules!

Pip

Pip is a package management system for Python and is automatically included since Python 2.7.9/3.4. It enables managing lists of packages and their versions

Getting the list of the installed packages:

$ pip list

Pip can also list the packages with outdated versions or available prereleases:

$ pip list --outdated
$ pip list --pre

Packages can easily be installed through the pip install command:

$ pip install my-package

The same goes for uninstalling packages:

$ pip uninstall my-package

The present packages can be stored as a 'requirements' file. If properly formatted, this file can then be used to recreate the given environment on another system with exactly the same packages and versions:

$ pip install -r requirements.txt

Tutorials

As there are plenty of python tutorials available online we decided not to create our very own but instead simply list a few we consider to be most helpful.

https://docs.python.org/3/tutorial/

https://www.w3schools.com/python/

https://www.tutorialspoint.com/python/

References

https://www.python.org/about/ https://pypi.org/project/pip/