Questions tagged [htcondor]
9 questions
1
vote
1 answer
submit to htcondor from outside the cluster (without condor_submit)
Usually to submit to condor I have to ssh to a machine which is inside the htcondor cluster and then run condor_submit. Now I would like to do the same, but from a machine outside the cluster (e.g. my laptop), where I don't have condor (and…
Ruggero Turra
- 16,929
- 16
- 85
- 141
1
vote
0 answers
How to break a nested Python for-loop into many individual jobs and then run them on HTCondor?
I have a nested for loop with this kind of logic:
As, Bs, Cs = [...], [...], [...]
for a in As:
for b in Bs:
for c in Cs:
result = function(a, b, c)
I want to break this into many HTCondor jobs, where each one takes an a, b,…
Liam
- 11
- 1
0
votes
0 answers
HTCondor: How to suspend long-running jobs if new ones come in?
I've been trying to get this running for the past couple days but I can't get it to work.
I have an HTCondor cluster with 5 nodes that are often filled by my users with jobs that sometimes run a very long time (i.e. days). When new jobs are…
rokyo401
- 1
0
votes
0 answers
htcondor: can a python executable started via condor_submit access all the values of its condor descriptors?
Here a trivial submission file:
executable = /path/to/myexecutable
error = test.err
output = test.out
log = test.log
request_memory = 1024
request_cpus = 1
queue
myexecutable is a python executable
#!/usr/bin/env python …
Vince
- 3,979
- 10
- 41
- 69
0
votes
1 answer
How to use all machine's CPUs for HTCondor jobs (parallel run)?
I have a submit file my_file.sub for HTCondor with multiple (~100) jobs to be done. File looks pretty simple (I know it can be simplified with $(Process) but it is generated automatically and cannot be changed):
executable = my_script.sh
arguments =…
kerzhy
- 9
- 3
0
votes
1 answer
HT_condor removing/putting on hold only jobs with a certain batch_name
Suppose I am running 200 jobs on HT_Condor with BATCH_NAME = "Ranking_noVV", and each job has a different JOB_ID, starting, for instance, at 4333042 and ending at 4333242. I would like to remove or put on hold all the jobs with BATCH_NAME ==…
Pleba
- 3
- 3
0
votes
1 answer
HTCondor: DAGMan input file priority vs job submit description file priority
In HTCondor, what priority has greater precedence (if both are provided): the one specified for a job in the DAGMan input file or the one provided in the job submit description file?
For example, if one has in the DAGMan input file:
JOB MyJob…
S.V
- 2,149
- 2
- 18
- 41
0
votes
1 answer
Resubmit failed condor jobs
When submitting condor jobs, typically a few or more jobs can fail for unknown reasons, and these jobs have to be resubmitted. so I was wondering: What's the most efficient way of resubmitting failed condor jobs? i.e. with having to fish one by one…
StackExchanger
- 101
0
votes
1 answer
How to log the errors both stdout and sterr files in the HTCondor HPC manager?
I want to log the errors to both stderr and stdout, how does one do this?
This is my current script:
####################
#
# Experiments script
# Simple HTCondor submit description file
#
# reference:…
Charlie Parker
- 5,884
- 57
- 198
- 323