Questions tagged [pipeline]

A pipeline is a sequence of functions (or the equivalent thereof), composed so that the output of one is input for the next, in order to create a compound transformation. Famously, a shell pipeline looks like "command | command2 | command3" (but use the tag "pipe" for this). It's also used in computer architecture to define a sequence of serial stages that execute in parallel over elements being fed into a pipe, in order to increase the overall throughput.

In a command line interface or shell, a pipeline uses the pipe operator ("|") to take output from one function or command and input it to another. This is done in a series like "command1 | function1 | command2". For questions related to the pipe operator use the tag.

In computer architecture, a pipeline is a process consisting of a sequence of stages that must be performed in serial order over each element passing the pipe, but may execute in parallel over the elements inside, such that the overall throughput does not depend on the length of the pipe. This is utilized by most CPUs' hardware to process instructions.

A similar technique is also done in software (software-pipelining) in order to optimize the parallelism of a given loop by reordering it to arrange data dependencies in a pipelined manner.

More broadly, "pipeline" is synonymous with "workflow."

See also:

5444 questions
1
vote
1 answer

How to add jenkins build status as a comments in jira issues using pipeline script

I'm shifting from maven jobs to pipeline jobs. in maven jobs we have used Jira update relevant issues plugin in post build actions step which will update the jenkins current build status to all the jira issues mentioned in the commits messages. I'm…
1
vote
0 answers

GStreamer image to rtsp using rtspclientsink

New to GStreamer, I want to stream a simple JPEG to rtspclientsink gst-launch-1.0 -v multifilesrc location="image.jpg" caps="image/jpeg,framerate=1/1" ! jpegdec ! videoconvert ! videoscale ! video/x-raw,width=1280,height=1024,framerate=1/1 ! x264enc…
AmirTallap
  • 11
  • 3
1
vote
1 answer

Setting the nextflow saveAs directive to save files in different directories

I am working on a nextflow pipeline where a program used in a process generates all the results inside a subfolder. I save the output files I need within an output tuple for later processes, and I am attempting at saving the output files in…
schmat_90
  • 572
  • 3
  • 22
1
vote
1 answer

'Vect' not defined sklearn logistic regression error message

So I have this pipeline i used for a text classifier that works fine. from sklearn.feature_extraction.text import TfidfTransformer from sklearn.feature_extraction.text import CountVectorizer from sklearn.metrics import accuracy_score from…
1
vote
2 answers

Kedro Pipeline contains no nodes after applying all provided filters

ValueError: Pipeline contains no nodes after applying all provided filters src/pipelines/data_processing
src/pipelines/data_science
src/pipelines/data_project
src/pipelines/pipeline2
Kedro build-docs-> dont'work pip install…
1
vote
1 answer

Get-ChildItem with Logging and Copy-Item in one command

Basically my goal is copy content of a folder to another folder with exclusion one name and also log everything what has been copied. I'm stock on logging Get-ChildItem command combined with Pipeline --> Copy-Item Bellow command will put to log file…
MikeZetPL
  • 97
  • 5
1
vote
0 answers

Got the "Unable to load vocabulary from file." while using pipelines

I have been trying to use the "csebuetnlp/mT5_multilingual_XLSum" model for summarization purposes. The code I tried is listed as below: !pip install transformers !pip install sentencepiece import transformers text_example = """ En düşük emekli…
1
vote
2 answers

Where does github actions stores the log files?

On my github action, it logged the following: The full lint text report is located at: /home/runner/work/honk/honk/app/build/intermediates/lint_intermediate_text_report/debug/lint-results-debug.txt honk is the name of my repo, but where do I find…
Leonardo Sibela
  • 1,613
  • 1
  • 18
  • 39
1
vote
2 answers

How to measure software security?

I want to start measure the software security, meaning that, I want to understand if my application is secure or not and improve from month to month. It would be also really useful to have some suggestion of the tools. I use sonar for detecting the…
1
vote
1 answer

About merging/grouping documents with composite key in Mongodb 4.4

i am new to mongodb, and this issue make me feel confuse and i have no idea how to do so. Heres 3 documents example in mongodb collection. I tried multiple way but still not success, can any one help with following sample? Does it really possible to…
Edmund
  • 35
  • 1
  • 7
1
vote
1 answer

ColumnTransformer' object has no attribute '_name_to_fitted_passthrough

enter image description here although I used a virtual environment. I got the following error. GitHub repo: I have latest version of sklearn @dataclass class DataTransformationConfig: …
1
vote
0 answers

Compare the last overall coverage to the current overall coverage precentage in GitLab

I am looking for a way to use the overall coverage percentage used and shown in the history of coverage in GitLab and compare in a way so the pipeline will fail if the difference between the current overall coverage and the last one is more than X,…
1
vote
1 answer

How to call bash script in nextflow and manage multiple inputs

I am trying to integrate a bash script into my nextflow pipeline. The bash script reads bed files from two directories and performs intersect on them. My directory structure before running the pipeline is as follows: results ├── extract_gimme │ …
1
vote
0 answers

Raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")

I am trying to make a text summarization tool using facebook/bart-large-cnn model. I tried the same code in windows and it worked perfectly, now, I am trying the same code in MAC and I am getting below error. Traceback (most recent call last): …
1
vote
1 answer

In a Powershell advanced function, in what block should I open an external file?

I am writing an advanced function called Get-Store, where it will be used within other scripts (on the pipeline) or by users at the prompt (regular old parameters). The user will be entering a 4-character alphanumeric code representing one or more…
Todd A
  • 11
  • 3
1 2 3
99
100