Stack Overflow
Stack Overflow

Questions tagged [lsf]

LSF, aka Load Sharing Facility is software that executes batch jobs on networked Unix and Windows systems on many different architectures. It is commonly used in HPC Clusters in many universities and research centers around the world.

The Load Sharing Facility or LSF is a set of tools for distributing jobs across a set of networked systems. If was developed initially by the Platform Computing company, which was acquired by IBM in 2012. It is now called IBM Spectrum LSF

Resources:

229 questions
2
votes
2 answers

How to parse bhist log

I am using IBM LSF and trying to get usage statistics during a certain period. I found that bhist does the job, but the short form bhist output does not show all of the fields I need. What I want to know is: Is bhist's output field customizable?…
Seong
  • 556
  • 4
  • 18
2
votes
1 answer

How to optimize multithreaded program for use in LSF?

I am working on a multithreaded number crunching app, let's call it myprogram. I plan to run myprogram on IBM's LSF grid. LSF allows a job to scheduled on CPUs from different machines. For example, bsub -n 3 ... myprogram ... can allocate two CPUs…
Gowtham
  • 1,465
  • 1
  • 15
  • 26
2
votes
2 answers

Force lshosts command to return megabytes for "maxmem" and "maxswp" parameters

When I type "lshosts" I am given: HOST_NAME type model cpuf ncpus maxmem maxswp server RESOURCES server1 X86_64 Intel_EM 60.0 12 191.9G 159.7G Yes () server2 X86_64 Intel_EM 60.0 12 191.9G 191.2G Yes…
Christopher Sousa
  • 51
  • 8
2
votes
1 answer

host availability for a user with limits consideration

Say I have a host with 10 slotsn I have applied a user limit to that host as following: SLOT = 5 JOB = 3 Is there a command to find whether this host can run more jobs from that user? OR, Is there a command to find whether the said host is closed…
user5005786
  • 65
  • 2
2
votes
1 answer

Python redirecting output from an LSF command

I am trying to run an LSF command , 'bjobs' inside a python code using subprocess and I am unable to get the output into a variable Ways I have already tried and failed are as follows: proc = subprocess.Popen(['bjobs'],stdout=subprocess.PIPE) print…
Krishna
  • 131
  • 2
  • 11
2
votes
1 answer

Redirect stderr through grep -v in LSF batch job

I'm using a library that generates a whole ton of output to stderr (and there is really no way to suppress the output directly in the code; it is ROOT's Minuit2 minimizer which is known for not having a way to suppress the output). I'm running batch…
DenverCoder9
  • 473
  • 3
  • 9
2
votes
1 answer

LSF issue with large scale runs

On our cluster, when I submit jobs requesting more than (including) 40 nodes or 640 cores, the $LSB_HOSTS gets empty and so the job stops. I use this variable to generate a nodelist file which I use with the mpirun command line as the…
Éric
  • 419
  • 5
  • 17
2
votes
1 answer

running R in emacs on LSF

I would like to run R in emacs using ESS. My R session runs interactively but is submitted to a different host using LSF. Is there a way to make this work? That is, R is not being run locally. However, ESS seems to run a local copy which doesn't…
Alex
  • 19,533
  • 37
  • 126
  • 195
2
votes
3 answers

os.system for submitting command

I am using os.system to submit a command to the system. I.e., import os os.system(my_cmd) But I was wondering how could I obtain the output, i.e., let us say i am in the bash and I type in my cmd, I'd get an output of this form: Job <57960787> is…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
2
votes
2 answers

"qsub -now" equivalent using bsub

In SGE , we have qsub -now yes/no By "-now yes" the job is scheduled immediately(if possible) or not at all . We are not put in pending queue . By "-now no " the job is put in pending queue if it cannot be executed immediately . But in…
baky
  • 631
  • 1
  • 8
  • 17
2
votes
1 answer

LSF parent job waiting for child

I am using LSF bsub command to submit jobs in Unix environment. However the LSF job is waiting for child jobs to finish. Here is an example (details about sample scripts below): Without LSF: If I submit parent.ksh in Unix without using LSF, i.e in…
Rakesh Sharma
  • 21
  • 4
2
votes
1 answer

lsf awk command quotes

When I run this command in the bash terminal it works: awk '{ sum += $1 } END { print sum }' /user/dnaiel/test.txt > /user/dnaiel/sum.txt But when I run this: bsub -q priority -R'rusage[mem=2000]' -oo bin${count}.out -eo bin${count}.err \ "awk '{…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
2
votes
2 answers

LSF wget not working with bsub

I am trying to run a script on a server through their lsf system for submitting jobs. When I run wget without using bsub, wget successfully downloads the file I'm looking to get. However, when I run the identical command inside of bsub, the job…
pzratnog
  • 43
  • 3
2
votes
1 answer

Referencing job index in LSF job array

I'm trying to pass the index of a job in a job array as a parameter to another bash script. numSims=3 numTreatments=6 # uses numTreatments top rows of parameters.csv maxFail=10 j=1 while [ $j -le $numSims ]; do bsub -q someQueue -J…
Sarah
  • 1,614
  • 1
  • 23
  • 37
2
votes
2 answers

Can i decide how much memory to allocate in LSF queue

Is there any option to decide how much memory I can allocate in LSF? I tried bsub -R "rusage[mem=10000]" sleep 1000s But when i checked resource using "bjobs -l " I get this: Job <203180>, User , Project , Status , Queue…
Ani
  • 918
  • 1
  • 10
  • 25
1 2 3
…
15 16