Stack Overflow
Stack Overflow

Questions tagged [lsf]

LSF, aka Load Sharing Facility is software that executes batch jobs on networked Unix and Windows systems on many different architectures. It is commonly used in HPC Clusters in many universities and research centers around the world.

The Load Sharing Facility or LSF is a set of tools for distributing jobs across a set of networked systems. If was developed initially by the Platform Computing company, which was acquired by IBM in 2012. It is now called IBM Spectrum LSF

Resources:

229 questions
3
votes
0 answers

How to limit number of threads for python interpreter?

I want to know whether it is possible to limit the number of threads python interpreter can spawn when running a script. I am not using threading myself, but my script uses modules like pandas, numpy, and others which I assume would be using…
Manish Goel
  • 843
  • 1
  • 8
  • 21
3
votes
1 answer

script to log the completion of an LSF (bsub) job

I have a script called by cron to run an LSF job. I would like to be told when that job is submitted and when it completes. The-Powers-That-Be have decided to disable email notifications. So I am writing this script to output the relevant…
trubliphone
  • 4,132
  • 3
  • 42
  • 66
3
votes
2 answers

LSF job submission - stdout & stderr redirection

I've submitted my job by the following command: bsub -e error.log -o output.log ./myScript.sh I have a question: why are the output and errors logs available only once the job ended? Thanks
Fab
  • 1,145
  • 7
  • 20
  • 40
3
votes
1 answer

Apply in a single bsub command the Bourne shell background ampersand

I would like to employ the Bourne shell background ampersand as part of ONE bsub execution command. Note that I don't want to use a .bash file but rather one bsub command line for executing the entire procedure. Note that I also don't want to…
Roy
  • 723
  • 2
  • 8
  • 21
3
votes
1 answer

Pass SIGTSTP signal to all processes in a job in LSF

The problem statement in short: Is there a way in LSF to pass a signal SIGCONT/SIGTSTP to all processes running within a job? I have a Perl wrapper script that runs on LSF (Version 9.1.2) and starts a tool (Source not available) on the same LSF…
Abhishek
  • 77
  • 6
3
votes
3 answers

specifying job arrays in LSF

My objective is to repeatedly run an R script, each time with a different set of parameters. To do so, I have been using a bash script to pass the command-line parameters to the R script by looping through an input file, in which each line contains…
Tom
  • 151
  • 9
3
votes
1 answer

LSF: Submit one Python script that uses multiprocessor module *or* submit several scripts at once that are "pre-split"?

I have a single task to complete X number of times in Python and I will be using LSF to speed that up. Is it better to submit a job containing several Python scripts which can be run separately in parallel or one Python script that utilizes the…
Ryan Schuster
  • 494
  • 4
  • 15
3
votes
1 answer

Recommendation on how to write a good python wrapper LSF

I am creating a python wrapper script and was wondering what'd be a good way to create it. I want to run code serially. For example: Step 1. Run same program (in parallel - the parallelization is easy because I work with an LSF system so I just…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
3
votes
1 answer

Run matlab command line and automate running unix server bjobs

I'm running a chunk of matlab code called fit.m this way, clear all; load('/pathtomatfile/alldata.mat') count = rxw1; count = double(count); x_cov = ltgbd; alldata.mat has several data of interest, i.e., rxw1 rbg2 rfd1 wop3, ltgbd, etc. I can…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
2
votes
1 answer

LSF queueing output in crontab problem

I have a bash script that monitors the jobs in a cluster from the output of the LSF queueing system command "bqueues". The script runs fine. Now I wanted to create a crontab entry that calls this script. The problems is that everythinh related with…
Open the way
  • 26,225
  • 51
  • 142
  • 196
2
votes
1 answer

How to Run Docker Container Using IBM LSF

I am trying to trigger Docker Container using IBM LSF Spectrum bsub commands. OS I am using is RHEL 8.1 I have started IBM LSF , checked it is running using lsid command which shows "cluster name & master name". under configdir/lsb.params I have …
Dheeraj
  • 59
  • 1
  • 10
2
votes
1 answer

How to automate the LSF waiting based on job name in perl

I have a perl code where I am submitting few jobs at once in parallel via LSF bsub command and once all these jobs finish want to submit a final job. For example I have these three bsub commands where first two bsub commands submits jobs t1 and t2…
Heyya
  • 57
  • 6
2
votes
1 answer

submit job arrays in LSF

I have 12 folders that all contains one R file with the same name (e.g., file.R). The name of the folders are m1, m2, ..., m12. To run each file, I run bsub -n 2 -q long -W 12:00 -R "rusage[mem=25000]" -M 25000 -hl R CMD BATCH file.R in each folder.…
Andrew
  • 678
  • 2
  • 9
  • 19
2
votes
1 answer

numpy import SystemError and segfault only through LSF (OpenBLAS blas_thread_init: pthread_create: Resource temporarily unavailable)

When I import numpy with blas 1.1 in a script running through LSF, it gives a SystemError and segmentation fault after repeated instances of OpenBLAS blas_thread_init: pthread_create: Resource temporarily unavailable and OpenBLAS blas_thread_init:…
gerrit
  • 24,025
  • 17
  • 97
  • 170
2
votes
1 answer

Can I use regular expression for job name dependency in IBM LSF?

I have a job which is supposed to be dependent on DONE state of multiple LSF Jobs having name prefix similar, let's say, job 'A' depends on jobs with names 'Job-1', 'Job-2', 'Job-3',....,'Job-100' Can I use following regex in defining job…
Vishal
  • 666
  • 1
  • 8
  • 30
1
2
3
…
15 16