Using tmpfs in Nextflow process (null value for variable with path)

Question

I need some help with a situation. I want to try the tmpfs in order to speed up my process because the actual one is taking so long time (probably due to IO in python script).

So I defined the containerOptions

process my_app {

    tag "$_id"
    publishDir "${outdir}/my_app", mode: params.publish_dir_mode

    containerOptions "--tmpfs ${BLASTDB}:rw"

    input:
        val param1
        path file1
        path blast_db
        env BLASTDB
        ...

the value of blast_db is a S3 URI defined in nextflow.config blast_db = "s3://${params.root_asset_bucket}/my_app/genome/BLASTDB_GRCH38"

Then it is retrieved in the workflow as a file (to stage the path and the resources) and also as a string for the env variable BLASTDB (that will be read later on by the python script):

workflow my_app_wf {
    take:
        param1
        file1

    main:
        my_app (
            param1=param1,
            file1="${file1}",
            blast_db=file("${params.quest_haystack_pincushion_app.blast_db}"),
            BLASTDB=blast_db.getFileName().toString(),
            ...
        )

This is not working and the process fails due to:

Error executing process > 'my_app_wf:my_app (null)'
Caused by:
  Found a malformed value 'null:rw' for --tmpfs option

I tried multiple options like:

containerOptions "--tmpfs ${BLASTDB}"
containerOptions "--tmpfs ${BLASTDB}:rw"
containerOptions "--tmpfs ${blast_db}"
containerOptions "--tmpfs ${blast_db}:rw"

but always get the same error. What I am missing?

Thank you very much in advance!

score 0 · Answer 1 · answered Aug 23 '23 at 12:43

containerOptions is a process directive, and process directives do not have access to channel values. If you make BLASTDB a global variable (maybe a param.BLASTDB in your config?) like you did with outdir both your script block and your process directives will be able to access it.

Using tmpfs in Nextflow process (null value for variable with path)

1 Answers1