Set Up Your Workspace

1 What you need to understand first

Before running the pipeline, it helps to know what the tools are so the commands make more sense.

1.1 What is Nextflow?

Nextflow is a workflow manager. Instead of running each analysis step yourself one by one, Nextflow runs all the steps automatically in the right order, handles errors, and lets you pick up where you left off if something fails.

nf-core/rnaseq is a Nextflow pipeline that was built by the bioinformatics community to process RNA-seq data in a standardized and reproducible way.

1.2 What is Apptainer?

Every step of the pipeline runs inside a container. A container is a self-contained package that includes the exact version of each software tool the pipeline needs. This means the pipeline runs the same way on any computer or cluster.

Apptainer (also called Singularity) is the container system used on Unity. When you run nf-core/rnaseq, Apptainer automatically downloads and uses the correct containers for each step. You do not need to install any bioinformatics tools manually.

1.3 What is SLURM?

SLURM is Unity’s job scheduler. On a shared cluster, many people are running jobs at the same time. SLURM manages the queue, it decides which job runs on which compute node, and when.

You interact with SLURM by writing a job script (a shell script with #SBATCH settings at the top) and submitting it with the sbatch command. SLURM then runs your script on a compute node when resources are available.

Note

You do not need to deeply understand any of these tools to follow this tutorial. Just know that Nextflow manages the analysis steps, Apptainer provides the software, and SLURM runs everything on the cluster.

2 Step 1: Create the log directory

Before anything else, create a folder to hold your SLURM job logs. These log files are where you will look when something goes wrong.

mkdir -p ~/rnaseq_nf_core/job-logs

This creates the folder rnaseq_nf_core/job-logs inside your home directory. The -p flag creates the full path even if rnaseq_nf_core does not exist yet.

3 Step 2: Check that Nextflow is available

On Unity, software is loaded with the module system. Check that the Nextflow module is available:

module avail nextflow

Load it:

module load nextflow/26.04.1

Check the version:

nextflow -version

You should see something like:

      N E X T F L O W
      version 26.04.1 build ...

If the module is not found, check the module explorer for the correct module name.

4 Step 3: Create a scratch workspace

The pipeline creates a lot of large temporary files. Your home directory does not have enough space for this. On Unity, you use a scratch workspace, a temporary directory with much more space.

Warning

Scratch workspaces expire after 30 days. They are meant for active work, not long-term storage. Always copy final results to /work/pi_lmangiamele_smith_edu/ before the workspace expires.

4.1 Check your existing workspaces

ws_list

Example output:

(base) kfloer_smith_edu@login1:~$ ws_list
id: simple
     workspace directory  : /scratch3/workspace/kfloer_smith_edu-simple
     remaining time       : 11 days 7 hours
     creation time        : Wed May 20 18:42:17 2026
     expiration date      : Fri Jun 19 18:42:17 2026
     filesystem name      : workspace
     available extensions : 9998

In this example, I already have one workspace from a different project called simple.

4.2 Create a new workspace

Create a workspace called nf-core-demo that lasts 30 days:

ws_allocate nf-core-demo 30

Example output:

(base) kfloer_smith_edu@login1:~$ ws_allocate nf-core-demo 30
Info: creating workspace.
/scratch4/workspace/kfloer_smith_edu-nf-core-demo
remaining extensions  : 10000
remaining time in days: 30

The important line is the workspace path, copy it, you will need it:

/scratch4/workspace/kfloer_smith_edu-nf-core-demo

4.3 Move into the workspace

cd /scratch4/workspace/kfloer_smith_edu-nf-core-demo

Use the path shown in your own ws_list output, your path may differ slightly.

4.4 Extend a workspace before it expires

If your workspace is about to expire and you still need it:

ws_extend nf-core-demo 30

Then confirm:

ws_list

4.5 Delete a workspace when you are done

Only do this when you have saved everything you need.

ws_release nf-core-demo

5 Step 4: Pull the nf-core/rnaseq pipeline

Downloading the pipeline definition once means Nextflow does not need to fetch it again every time you run.

module purge
module load nextflow/26.04.1

nextflow pull nf-core/rnaseq -r 3.26.0

This downloads the pipeline code (not the containers, those are downloaded later, when the pipeline actually runs).

6 You are ready

Once your workspace is created and the pipeline is pulled, you are ready to run your first job. Continue to the Run the Pipeline page.