Mukund-pondkule-6a11@uni-paderborn.de at 17:52, 9 December 2020

2020-12-09T17:52:31Z

Robert-schade-e757@uni-paderborn.de at 15:38, 2 November 2020

2020-11-02T15:38:12Z

Robert-schade-e757@uni-paderborn.de at 14:14, 2 November 2020

2020-11-02T14:14:56Z

Robert-schade-e757@uni-paderborn.de at 17:22, 30 October 2020

2020-10-30T17:22:50Z

Robert-schade-e757@uni-paderborn.de: Created page with "Category:HPC-Admin Category:HPC.NRW-Best-Practices"

2020-10-30T16:42:17Z

Created page with "Category:HPC-Admin Category:HPC.NRW-Best-Practices"

New page

[[Category:HPC-Admin]]
[[Category:HPC.NRW-Best-Practices]]

@@ Line 1: / Line 1: @@
-[[Category:HPC-Admin|Customizeable Remote Desktop with NoVNC]]
+[[Category:HPC-Admin|Customizeable Remote Desktop with NoVNC]]<nowiki />
-[[Category:HPC.NRW-Best-Practices|Customizeable Remote Desktop with NoVNC]]
+[[Category:HPC.NRW-Best-Practices|Customizeable Remote Desktop with NoVNC]]<nowiki />
-[[Category:Remote-Desktop]]
+[[Category:Remote-Desktop]]<nowiki />
-[[Category:NoVNC]]
+[[Category:NoVNC]]<nowiki />
 == Purpose ==

@@ Line 1: / Line 1: @@
 [[Category:HPC-Admin|Customizeable Remote Desktop with NoVNC]]
 [[Category:HPC.NRW-Best-Practices|Customizeable Remote Desktop with NoVNC]]
 == Purpose ==

@@ Line 1: / Line 1: @@
-[[Category:HPC-Admin]]
+[[Category:HPC-Admin|Customizeable Remote Desktop with NoVNC]]
-[[Category:HPC.NRW-Best-Practices]]
+[[Category:HPC.NRW-Best-Practices|Customizeable Remote Desktop with NoVNC]]
 == Purpose ==

← Older revision		Revision as of 17:22, 30 October 2020
Line 1:		Line 1:
	[[Category:HPC-Admin]]		[[Category:HPC-Admin]]
	[[Category:HPC.NRW-Best-Practices]]		[[Category:HPC.NRW-Best-Practices]]
		+
		+	== Purpose ==
		+
		+	In the situation of workshops or schools with many participants (tested under real-world-conditions with 60 participants for one week) that are supposed to do something on the cluster systems easily happens. Especially in the case of groups with heterogeneous HPC knowledge or if graphical applications need to be used, an operating system-independent solution is preferred.
		+
		+	A good example was the “International Winter School on Electronic Structure Calculations” which has been organized in February 2020 by PC² at Paderborn University. The expected participants (chemists, physicists, material scientists) from experience have a rather heterogeneous composition of operations systems. However, it was expected that most of them have at least some experience with the shell and how to submit jobs on a large cluster system.
		+
		+	Similar solutions can be built with [https://xpra.org/ XPRA] or [https://wiki.x2go.org/doku.php x2go].
		+
		+	Additional challenges that might arise:
		+
		+	* Not sufficiently large computer rooms or only computer rooms with an unsuitable layout for tutorials. Thus, the use of personal laptops of the participants is a good idea.
		+	* Heterogeneity of the operating system or installed versions of operating systems of the participants.
		+	* Use of graphical applications for visualization
		+	* Effort for initial configuration of ssh clients, scp-clients, x-forwarding and others
		+	* Due to the rather high traffic on the wireless access points in the room, special care has to be taken for graphical applications. For example, in this case, the latency-sensitivity of X-forwarding will make work cumbersome.
		+	* Copying of data back and forth between laptops and clusters in other scenarios.
		+
		+	== How the final solution will look to the participants ==
		+
		+	* The participants will get the link to a landing page plus some account information (or a link to an electronic form where they can generate accounts)
		+	* They select the link to their account an are greeted by noVNC (can also be themed):
		+	** [[File:novnc_landing.png]]
		+
		+	* They enter their password:
		+	** [[File:novnc_novnc-pw.png\|500px\|NoVNC Login prompt]]
		+
		+	* And have a desktop environment in their web browser:
		+	** [[File:novnc_novnc.png\|500px\|NoVNC Desktop in a webbrowser]]
		+
		+	* They can submit jobs from the terminal, use the module environment from the cluster, use graphical programs,….
		+
		+	== Solution: ==
		+
		+	In order to solve the above problems, the following setup has proven to be stable, efficient, and easy to use:
		+	* The overall setup looks like:
		+	* [[File:novnc_novnc-setup.png\|500px\|Schematic NoVNC Setup in an HPC-system]]
		+	* There are five important components:
		+	1. The laptops of the participants that have to have some network connection
		+	2. The landing, login page, and apache
		+	3. The websocket to vnc proxies (websockify)
		+	4. Some vnc servers
		+	5. The cluster
		+
		+	=== laptops of the participants ===
		+
		+	* required properties:
		+	** wireless access (eduroam or some event-specific wlan)
		+	** webbrowser with Java Script
		+
		+	=== The landing page and login page ===
		+
		+	<ul>
		+	<li>A very rudimentary landing page could look like:</li>
		+	<li>[[File:novnc_landing.png\|500px\|Very rudimentary NoVNC Landing page]]<br />
		+	</li>
		+	<li>At the beginning of the hands-on tutorials in the workshop or school, the participants are directed to an electronic form where they input their information and agree to the terms-of-use and data privacy terms as well as to the license conditions of licensed software. As a return, they get an account number as well as access information. Because they are getting generic cluster accounts pre-generated this way, there is no latency or waiting time that might otherwise exist for guest accounts.</li>
		+	<li>The links on the landing page are simply links to static html from noVNC (https://github.com/novnc/noVNC) served by the webserver.
		+	<ul>
		+	<li>https://gateway/noVNC/vnc.html?host=gateway&port=443&path=websocketpath01</li>
		+	<li>websocketpath01 is individual for every account and has to have a corresponding ProxyPass directive in the webserver, i.e.,</li></ul>
		+
		+	<pre> ProxyPass /websocketpath01 ws://127.0.0.1:6000/websockify retry=3
		+	ProxyPassReverse /websocketpath01 ws://127.0.0.1:6000/websockify retry=3
		+	ProxyPass /websocketpath02 ws://127.0.0.1:6001/websockify retry=3
		+	ProxyPassReverse /websocketpath02 ws://127.0.0.1:6001/websockify retry=3
		+	...</pre>
		+	<ul>
		+	<li>Due to the proxy functionality the individual ports websocket ports (6000,…) don’t have to be reachable from the client and the webserver takes care of ssl-encryption.</li>
		+	<li>Upon visiting the link to the account on the landing page the user is greeted by the novnc-login screen which can also be themed</li>
		+	<li>[[File:novnc_novnclogin.png\|500px\|NoVNC Login prompt]]<br />
		+	</li></ul>
		+	</li>
		+	<li>The landing page can also be used so that the participants can download their files as packaged archives after the workshop or school.</li></ul>
		+
		+	=== The websocket to vnc proxies (websockify) ===
		+
		+	The websocket to vnc proxies (websockify) can either run on the same machine as the webserver or on different machines. In our experience even on an old dual-socket Westmere-Xeon system with 12 CPU-cores can easily be used for at least 60 participants when the webserver and also the websockify proxies run on it.
		+
		+	==== Websockify proxies ====
		+
		+	* Even though there is a C-based implementation of the websockify-proxy (https://github.com/novnc/websockify-other) which needs less performance than the python version, we recommend to use the python version because of security considerations (https://github.com/novnc/websockify-other/issues/4).
		+	* Even though there is the possibility to use only one websockify proxy for all participants, web recommend to use individual ones instead for better stability and performance.
		+	* After cloning noVNC from github (https://github.com/novnc/noVNC) run
		+
		+	<pre>bash utils/launch.sh</pre>
		+	once to pull the websockify proxy and run it for testing purposes. * At this point we recommend to create a token file that contains information on the locations of the vnc servers, websocketpaths, vnc ports and vnc passwords. This servers as a data basis for the automated setup of everything. An example is the format:
		+
		+	<pre>[accountid=accountname/number] [vnchost=hostname of vnc server] [websocketpath] [websocketport] [vnc server port] [vnc password]</pre>
		+	* The basic commands for execution of the websocket proxies is now:
		+
		+	<pre>accountid=$1
		+	websocketport=$2
		+	vnchost=$3
		+	vncport=$4
		+	while true;
		+	do
		+	./utils/websockify/run 127.0.0.1:$websocketport $vnchost:$vncport 2>&1 > logs/$accountid.log
		+	sleep 1
		+	done</pre>
		+	* There has to run exactly one websocket proxy for every account. We recommend to set up a systemd-service that reads the token file and sets up the websocket proxies accordingly.
		+	*
		+
		+	=== Some vnc servers ===
		+
		+	At the vnchosts some vncserver has to run. Many different vncserver can be used. We have good experience with the TigerVNC server. Although we haven’t tested it yet, also solutions for hardware-accelerated graphics like VirtualGL should work for example with Tigervnc. The authentication used when logging in is actually the vnc authentication. Hence, a vnc password has to be set automatically from the tokenlist above. For simplicity, a few nodes of the cluster can be reserved for the duration of the workshop or school for the vncservers. For the following we assume that this is the case. Because a complete desktop environment, for example, MATE desktop, has to be installed for the vnc server, we recommend to create a Singularity container, that contains the desktop environment, graphical programs and so on. This container can be prepared by the organizer of the workshop and then copied to the cluster. The vnc server is then started from within the Singularity container. All required paths to make job submission, the module environment of the cluster available and the parallel file system available in the container have to be set. In case different file systems for home directories and data (parallel file system) are used on the cluster, we recommend to set the home directory to somewhere on the parallel file system to avoid the need for additional explanations and confusion among participants. A nice additional feature is that the nodes with the vnc servers can, if necessary, be further restricted in terms of allowed network connections.
		+
		+	=== The cluster ===
		+
		+	This setup assumes that the compute jobs of the participants are executed by the normal cluster nodes. Other setups, e.g., with a small virtualized cluster and a separate workload manager are possible if stronger isolation of the participants is required or desired.
		+
		+	== Advantages of this setup: ==
		+
		+	* Very flexible and customizable to many situations.
		+	* Only an up-to-date web browser with JavaScript is needed.
		+	* Firewall-settings and other possible problems on laptops don’t matter.
		+	* No software installation needed on laptops of participants.
		+	* Performance class of laptops is almost irrelevant. (We have successfully tried it on old Intel atom-based laptops.)
		+	* The setup is very flexible. Different levels of isolation of the participants can be realized.
		+	* Hardware-accelerated rendering with solutions like VirtualGL can be integrated easily.
		+	* Can also be used in computer rooms with existing computers instead of laptops of participants.

Admin Guide Customizeable Remote Desktop with NoVNC - Revision history

Mukund-pondkule-6a11@uni-paderborn.de at 17:52, 9 December 2020

Robert-schade-e757@uni-paderborn.de at 15:38, 2 November 2020

Robert-schade-e757@uni-paderborn.de at 14:14, 2 November 2020

Robert-schade-e757@uni-paderborn.de at 17:22, 30 October 2020

Robert-schade-e757@uni-paderborn.de: Created page with "Category:HPC-Admin Category:HPC.NRW-Best-Practices"