Janelia Research Campus Configuration
To use, run the pipeline with -profile janelia. This will download and launch the janelia.config which has been pre-configured with a setup suitable for the Janelia Research Campus LSF cluster.
Running workflows on the Janelia Research Campus cluster
The latest version of Nextflow is not installed by default on the cluster. You will need to install it into your cluster account by following the installation instructions:
Remember to move the nextflow executable to ~/bin or otherwise ensure that it’s in your PATH.
Nextflow manages each process as a separate job that is submitted to the cluster by using the bsub command. Janelia’s cluster has Singularity installed on all login and compute nodes, and the profile specifies that Singularity will be used to execute processes.
Nextflow should never be run directly on a login node. You can run it via an interactive cluster session (bsub -Is /bin/bash), or make a wrapper shell script like the example below, and submit with bsub < $PWD/my_script.sh
#!/bin/bash
#BSUB -o /path/to/a/log/dir/%J.o
#BSUB -e /path/to/a/log/dir/%J.e
#BSUB -n 2
export NXF_ANSI_LOG=false
export NXF_VER=22.10.7-5853
export NXF_JAVA_HOME=/misc/sc/jdks/zulu17
export NXF_SINGULARITY_CACHEDIR=$HOME/.singularity_cache
nextflow run \
/path/to/nf-core/pipeline/main.nf \
-w /path/to/some/dir/work \
-profile janelia \
-params-file my_params.jsonUsing Nextflow Tower (web UI) to launch and monitor workflows
You can use Janelia’s internal Nextflow Tower instance to run workflows on the cluster. Additional instructions for using this resource can be found on the Janelia Wiki.
Config file
params {
config_profile_contact = 'Konrad Rokicki (rokickik@janelia.hhmi.org)'
config_profile_name = 'Janelia Compute Cluster'
config_profile_description = 'IBM Platform LSF Cluster at Janelia Research Campus'
config_profile_url = 'https://www.janelia.org'
max_cpus = 48
max_memory = '768.GB'
max_time = '48.h'
lsf_opts = ''
lsf_queue_size = 500
schema_ignore_params = 'genomes,lsf_opts,lsf_queue_size'
validationSchemaIgnoreParams = "genomes,lsf_opts,lsf_queue_size,schema_ignore_params"
}
singularity {
enabled = true
autoMounts = true
}
process {
resourceLimits = [
memory: 768.GB,
cpus: 48,
time: 48.h
]
executor = 'lsf'
scratch = '/scratch/$USER'
clusterOptions = params.lsf_opts
}
executor {
perTaskReserve = false
perJobMemLimit = true
queueSize = params.lsf_queue_size
}
executor
contact
