This guide is intended for creating Singularity containers on a computer where you have root (administrative) privileges and then using the container on the HPC resources.
Installation of Singularity and its dependencies
Singularity cannot be directly installed on windows. The user needs any VM software to run a linux system under their windows env. A thorough guide for installation via apt, yum, rpmor from source can be found at https://docs.sylabs.io/guides/3.0/user-guide/installation.html
Creating a container
In order to run singularity containers on the HPC resources we will need to create .sif image files which are compressed and immutable which ensures reproducibility of results. Since the process of creating sif files requires root privileges it needs to be done on a machine where the creator has root access(meaning that it cannot be done on HPC resources) and then transferred to your personal HPC directory.
Images can either be created using Definition files(.def) or a writable environment can be created and modified to specifications. Both the .def file and the writable environment can be converted to sif files.
Using existing container
Singularity allows the use of existing containers from the Container library, Docker hub or locally stored containers, these premade containers can come with preinstalled software depending on the specifications needed.
Premade containers can be either used as is or can be a base to modify and/or add any other required software. We recommend searching for containers that most closely match the requirements of the project.
Commands for building and running
Building a sif image file from a definition file:
sudo singularity build IMAGENAME.sif DEFINITIONFILE.def
Building a sif image file from a premade container with either:
sudo singularity build IMAGENAME.sif docker://godlovedc/lolcow
sudo singularity build IMAGENAME.sif library://sylabs-jms/testing/lolcow
Building a writable container:
sudo singularity build --sandbox CONTAINERNAME/ docker://godlovedc/lolcow
Running a writable container after creating it:
sudo singularity shell -writable CONTAINERNAME/
Converting a writable container to sif image:
sudo singularity build IMAGENAME.sif CONTAINERNAME/
The singularity documentation contains a thorough guide on creating definition files here.
Running the container
Containers can either be called to run scripts/commands directly using exec:
singularity exec IMAGENAME.sif COMMAND
OR they can be used as an environment to work with using shell:
singularity shell IMAGENAME.sif
also they can be used to run predefined scripts with run:
singularity run IMAGENAME.sif
Sending the container to your directory
This section assumes that an HPCF account exists and the ssh keys are setup properly.
scp IMAGENAME.sif email@example.com:~/IMAGENAME.sif
The path after ":" can be changed to any folder in your Cyclone home directory.