Stack Overflow
Stack Overflow

Questions tagged [nextflow]

a framework for developing and managing scalable scientific data processing pipelines targeted for cloud or cluster computing

Nextflow is a framework for developing and managing scalable scientific data processing pipelines targeted for cloud or cluster computing. The framework provides a DSL that aims to extend the Unix pipe model with emphasis on functional composition, implementing a reactive dataflow programming model. To promote reproducibility, the framework supports Docker and Singularity containers.

Nextflow is developed by the Comparative Bioinformatics group at the Barcelona Center for Genomic Regulation (CRG).

It is open source software available on GitHub and distributed under the GPL3 license.

348 questions
3
votes
1 answer

Nextflow: publishDir, output channels, and output subdirectories

I've been trying to learn how to use Nextflow and come across an issue with adding output to a channel as I need the processes to run in an order. I want to pass output files from one of the output subdirectories created by the tool (ONT-Guppy) into…
dthorbur
  • 155
  • 1
  • 11
3
votes
1 answer

The most NextFlow-like (DSL2) way to incorporate a former bash scheduler submission script to a NextFlow workflow

New to NextFlow, here, and struggling with some basic concepts. I'm in the process of converting a set of bash scripts from a previous publication into a NextFlow workflow. I'm converting a simple bash script (included below for convenience) that…
Mark Ebbert
  • 441
  • 3
  • 13
3
votes
1 answer

How to call a forward the value of a variable created in the script in Nextflow to a value output channel?

i have process that generates a value. I want to forward this value into an value output channel. but i can not seem to get it working in one "go" - i'll always have to generate a file to the output and then define a new channel from the…
tristan
  • 105
  • 1
  • 2
  • 12
3
votes
1 answer

change the location of the .nextflow folder?

when I run nextflow, I get a .nextflow folder, but I can't find a way to change its location (i.e. it is't -work-dir). How can I change the location of the .nextflow folder? I have looked at launchDir but it seems that is a read-only implicit…
Pablo
  • 983
  • 10
  • 24
3
votes
1 answer

Snakemake explicit handling for Out Of Memory (OOM) failures

A Snakemake workflow can re-attempt for each restart after any type of failure, including if the error is of an Out Of Memory (OOM) doing e.g. def get_mem_mb(wildcards, attempt): return attempt * 100 rule: input: ... output: ... …
pmb59
  • 31
  • 3
3
votes
1 answer

`errorStrategy` setting to stop current process but continue pipeline

I have a lot of samples that go through a process which sometimes fail (deterministically). In such a case, I would want the failing process to stop, but all other samples to still get submitted and processed independently. If I understand…
Alexlok
  • 2,999
  • 15
  • 20
3
votes
1 answer

How to specify a scratch output directory in Nextflow?

I have started to read Nexflow's documentation and found that one can specify a scratch directory for the execution. Once the task is complete, one can use the stageOutMode directive to copy the output files from scratch to storeDir. The output…
Botond
  • 2,640
  • 6
  • 28
  • 44
3
votes
1 answer

nextflow does not find all my python modules

I am trying to make a Nextflow script that utilizes a python script. My python script imports a number of modules but within Nextflow python3 does not find two (cv2 and matplotlib) of 7 modules and crashes. If I call the script directly from bash it…
TheCodeNovice
  • 750
  • 14
  • 35
3
votes
1 answer

Docker CannotCreateContainerError: Thin Pool has 0 free data blocks

I am trying to run a Nextflow pipeline using AWS (an EC2 instance) which requires using docker, but the following error appears: CannotCreateContainerError: Error response from daemon: devmapper: Thin Pool has 0 free data blocks which is less than…
Ruben S.
  • 33
  • 4
3
votes
1 answer

Extract sample ids from nextflow fromPath()

I am new to nextflow and here is a practice that I wanted to test for a real job. #!/usr/bin/env nextflow params.cns = '/data1/deliver/phase2/CNVkit/*.cns' cns_ch = Channel.fromPath(params.cns) cns_ch.view() The output of this script is: N E X T F…
David Z
  • 6,641
  • 11
  • 50
  • 101
3
votes
1 answer

How to collect channels hierarchically?

How can I run a process with one instance for each pair of values in two lists, then collect the output of those instances along only one of the lists at a time? For example, if you run this Nextflow script: numbers = Channel .from(1..2) …
Danica Scott
  • 185
  • 2
  • 6
3
votes
1 answer

Running Rscript in Nextflow pipeline

I'm trying to run a Rscript in Nextflow pipeline. This Rscript has a package "ggolot2". I have this error: Error in library(ggplot2): there is no package called "ggplot2", Execution halted. Why do I get that error? Here's the code: process…
Shikan
  • 93
  • 6
3
votes
2 answers

How to run the python script within a docker container through nextflow seemlessly without any path or env related issues?

I am trying to run a python script using nextflow and docker. I am using a dockerfile (as shown below) to create a docker image. Nextflow script has a simple launch of a python script. The issue is when I run the same python command from within the…
deepesh
  • 53
  • 1
  • 4
3
votes
1 answer

Overriding Nextflow Parameters with Commandline Arguments

Given the following nextflow.config: google { project = "cool-project" region = "europe-west4" lifeSciences { bootDiskSize = "200 GB" debug = true preemptible = true } } Is it possible to override one or more of…
r0f1
  • 2,717
  • 3
  • 26
  • 39
3
votes
1 answer

Nextflow manipulate variables between processes

I am redesigning a workflow, basically it starts off with a process which spawns multiple other processes. Initially I had the variables before starting my workflow and so I made a tuple of these variables and then passed it as input to a process.…
DUDANF
  • 2,618
  • 1
  • 12
  • 42
1
2
3
…
23 24