<
Advanced Research Computing


Accessing Condor and HPC systems


Contents


Introduction

In the interests of security, access to the Condor server and HPC clusters such as chadwick is restricted to registered users and the way in which these services are accessed will depend on whether users are connected to the campus network directly or working remotely off-campus. Users have a choice of client applications which can be used to login to the services and to transfer files to/from them. The choice of which to use is really down to personal taste and experience.

MobaXterm is a very versatile graphical application which can be used to login to remote services and to upload/download files. One its main advantages is that it can act as an X Windows server allowing the users to run graphical Windows-type interfaces to applications. The multitude of features provided by MobaXterm may be off putting to beginners and it can be replaced by a pair of simpler applications - namely PuTTY for loging in to services and CoreFTPlite for transferring files. Another alternative for users who already work in a command line UNIX environment is the built-in ssh and scp commands for loging in and file transfer respectively.

All of the above clients can be used either on or off-campus however for off campus access you will need to employ an additional tool to allow you to "get through" the campus firewall - this is because we do not allow direct access from off site to Condor or chadwick for security reasons. This is provided by the CSD Virtual Private Network (VPN). An alternative is to use AppsAnywhere however this will not allow you transfer files - only to login.


Local access via the Managed Windows Service (MWS)

PuTTY

The easiest way to login to Condor or other servers when using the MWS is via the PuTTY terminal client. If this is not already present on the desktop, it can be installed from the Install University Applications shortcut. Select Internet from the list of Categories and PuTTY 0.60 from the list of Applications.

Once PuTTY is installed, a shortcut to it should appear on the desktop. To access a service, first double click on the PuTTY shortcut. The following window should appear:



In the Hostname box enter the server hostname (e.g. condor.liv.ac.uk or chadwick.liv.ac.uk), ensure that the Connection type is set to SSH and then click on the Open button as shown below:







A window similar to that shown below should appear (if a window prompting you to accept the SSH session key appears, click YES to confirm). To login, enter your normal username and password as prompted:



When you are finished with your login session type exit at the command line prompt to logout.


CoreFTPLite

The simplest way to transfer files to and from a server is to use the CoreFTPLite client. Unfortunately this seems to have disappeared from Install University Apps, however it can be downloaded and installed from http://www.coreftp.com/download.html.

Once Core FTP Lite is installed, a shortcut to it should appear on the desktop. To access Condor or other servers, first double click on the Core FTP Lite shortcut. The following window should appear:



Set Host to the server name (e.g. condor.liv.ac.uk or chadwick.liv.ac.uk), fill in the Username and Password boxes and ensure that the Connection Type is set to SSH/SFTP. Finally click the Connect button as shown below below:









If a window prompting you to accept the SSH session key appears, click YES to confirm. You should then see a window similar to this:



Files on the desktop PC will appear in the left hand pane and those on the server on the right. To transfer files, navigate to the directory/folder containing the files you want to move. In the example shown below files are uploaded from c:\tmp on the PC to /condor_data/smithic/matlab_test on the Condor server.



Select the files to be uploaded in the left hand pane:



and click the right-pointing arrow to start the upload:



Instead of using the arrow icons, files can be transferred by dragging from one pane to the other. Files can be downloaded by a similar method to uploading.


MobaXterm

The MobaXterm software supports all of the features of PuTTY and CoreFTPLite in a single application and also supports graphical user interface applications based on the UNIX X Windows system. If you don't already have MobaXterm installed, it can be installed on the MWS using:

Start -> Install Uni Apps -> Utilities -> MobaXterm 10.2

Start MobaXterm by double-clicking on the desktop icon and then click on the Session icon in the top left hand corner and then the SSH icon in the pop-up window as shown below:

Next enter the name of the remote host you want to connect to e.g. condor.liv.ac.uk or chadwick.liv.ac.uk (in this case it is bioinf1.liv.ac.uk) and enter your username:

You will then be prompted for your password and, once this has been entered correctly, you should see another Xterm window similar to the one below:

You can use the Xterm to run commands as you would do with any other terminal emulator however, in addition, you can now open multiple Xterms on your desktop as well as run programs that have graphical interfaces. To run another Xterm, enter this command

$ xterm &

You should now see another Xterm open on your desktop which you can use at the same time as the original one. You can also create additional Xterms by clicking Session and SSH again. Select Terminal | Detach from the menu to separate an Xterm from the original MobaXterm window thus allowing you to move it around your desktop as shown below:

Local access using a Mac or Linux-based machine

If you want to access ARC services from an Apple Mac or Linux-based machine which is connected to the campus network, simply open a terminal window and issue a command such as :

                    $ ssh -l <username> chadwick.liv.ac.uk
                
where <username> is your username on the chadwick server and the option is "minus el". Then enter your usual password. To transfer files to and from chadwick you can use the sftp (secure file transfer protocol) client in a similar fashion. Open a terminal window and enter the command:
                    $ sftp chadwick.liv.ac.uk
                
then enter your chadwick username and password. Once connected, you can transfer files using normal sftp commands (see standard Linux documentation for details). The same commands can be used for Condor with condor.liv.ac.uk as the host name.


Local access using a non-MWS laptop

If you are accessing the campus network using a Windows-based laptop (e.g. via EduRoam) and this does not run the MWS, the easiest way to access Condor will be still be to use PuTTY and Core FTP Lite. These can be downloaded and installed via the web from the following sites:

http://www.putty.org/

http://www.coreftp.com/

or you can use MobaXterm instead by downloading it from: http://mobaxterm.mobatek.net

Users of Apple and Linux-based laptops should follow the instructions given above.


Accessing ARC services from off-campus

It is possible to access ARC services from outside the University network however unsecured connections made directly to the machines such as Condor or chadwick are prohibited for security reasons. This makes off-campus access trickier than local access. In practise, a connection first needs to be made to another server on the campus network or, alternatively, a secure (so-called "tunnelled") connection needs to be made to the required service (e.g. Condor or chadwick) via the University virtual private network.

If you have a Windows-based machine then the easiest way to login to a server is via Apps Anywhere. Start Internet Explorer and then go to:

https://citrix.liv.ac.uk

Then start PuTTY from the Utilities group and login as you would using the MWS (see above).

Transfer of files to and from the a server will require use of the VPN and a secure FTP client such as Core FTP Lite. The latter can be downloaded directly via the web from:

http://www.coreftp.com/

You can replace PuTTY and CoreFTPLite by downloading and installing MobaXterm from:

http://mobaxterm.mobatek.net

To install the VPN client, follow the instructions on this page:

http://www.liv.ac.uk/csd/vpn/

Before connecting using Core FTP Lite or MobaXterm, you will first need to start the VPN client. Once this is working you can connect to the Condor or chadwick etc with Core FTP Lite as you would when logged in locally via the MWS (see above).

If you are using a Linux or Mac-based machine from off-campus, then you can still access ARC servers by first installing the VPN client and then using ssh or sftp from a terminal window as described above.

Any problems with VPN access will need to be referred to the CSD helpdesk (servicedesk@liverpool.ac.uk) as this is outside the control of the ARC team (please mark the request as a VPN problem not an ARC one.)


Off-campus access via the Sun UNIX Service

If you encounter any difficulties with the VPN or with Apps Anywhere, a workaround is available making use of the Sun UNIX service as a "stepping stone". First make sure you have a ssh client such as PuTTY installed. Then login to the Sun UNIX service at this address:

unix.liv.ac.uk

You should then be able to login to the Condor service with the command:

                    $ ssh condor
                
or chadwick with:
                    $ ssh chadwick
                
File transfer is more difficult and you will first need to copy any files to/from the Sun UNIX service and then transfer them to/from the server. To upload files, for example, first connect to unix.liv.ac.uk and upload the files to /tmp, then use a command similar to this to copy the files to the server (in this case Condor):
                    $ scp /tmp/uploaded_files condor:/condor_data/username
                
To download files use the same process but in reverse. The standard UNIX documentation gives details on how to use scp. Please delete any files from /tmp after use.


Moving multiple files about more easily

If you have a large amount of input and/or output files to transfer between your PC and a server, then selecting each one with Core FTP Lite can be cumbersome and prone to error. Here it is much easier to bundle the files together into a ZIP format file which can be uploaded or downloaded in one action.

For example, say you have a number of MAT format output files from a Condor MATLAB simulation called output0.mat, output1.mat, output2.mat ... which are to be downloaded. These could first be bundled into a single ZIP file (here called simulation_results.zip) using a command similar to this:

                    $ zip simulation_results.zip output*.mat
                
The ZIP format file can then be downloaded easily and the individual files extracted. To extract the files in Windows, select the ZIP file in Windows Explorer then right-click and select Extract All....

A similar method can be used to upload the input files. First copy the input files to an empty folder on the PC. Then right-click on the folder name and select Send To | Compressed (zipped) folder and enter the ZIP file name. After uploading the ZIP format file the individual files can be extracted with the unzip command e.g.

                $ unzip input_files.zip
                
Mac and Linux users should find that the zip and unzip programs are available from the command line and there may be other graphical tools to do the job as well.