UGE HPC Configuration
nf-core pipelines have been successfully configured for use on the UGE HPC cluster.
To use the UGE profile, run the pipeline with -profile uge. This will download and apply ‘uge.config’ which has been configured for the UGE HPC cluster. This profile will allow all Nextflow processes to run within singularity containers, which will be downloaded and converted from docker containers, if needed.
Note: This profile does not configure resources but does partition jobs based on runtime to make use of
short.qandall.qnodes. If you require resources outside of the limits specified in the pipeline (ex. more memory, CPUS, or walltime), you will need to provide a custom config specifying needed resources.
The UGE HPC has Nextflow pre-installed on the HPC cluster, and can be accessed by running module load nextflow or module load nextflow/<version> prior to running your pipeline. Additional singularity variables may need to be configured, such as a scratch directory, temp directory, cache directory, etc. If assistance is needed, please contact the Scientific Computing and Bioinformatics department (SciComp).
Config file
/*
* ----------------------------------------------
* nf-core/configs: Univa Grid Engine
* ----------------------------------------------
*/
// Basic params of config file
params {
config_profile_description = 'Univa Grid Engine config file'
config_profile_contact = 'Gregory Sprenger (@gregorysprenger)'
}
// Use Singularity
singularity {
enabled = true
autoMounts = true
}
// Submit up to 100 jobs at once with 15 secs between to avoid spamming the cluster
executor {
queueSize = 100
pollInterval = '15 sec'
}
// Process specific resource requirements
process {
// Error and retry handling
errorStrategy = { task.exitStatus in [143, 137, 104, 134, 139, 71, 255] ? 'retry' : 'finish' }
maxRetries = 3
// Executor and queue information
executor = 'sge'
penv = 'smp'
queue = { task.time <= 4.h ? 'short.q' : task.time > 5.h ? 'all.q' : 'short.q' }
}
executor
contact
