0

I'm programming script for the operations on 2 clusters.

My aim is to focus only one cluster, it means every script source code is only on one of the cluster.

For example in Cluster A I have all scripts installed, now I want to ssh to Cluster B to run these scripts without having to install them again on Cluster B.

I knew with bash script like ssh root@ClusterB 'bash -s' < local_ClusterA_script.sh then Cluster B run local_ClusterA_script.sh from ClusterA.

Now there is a further challenge that I have to use ssh to submit a batch job on Cluster B from Cluster A, this batch job needs a script from Cluster A.

# run.sh :

#!/bin/bash

INPUT_ARGS="$@"
qsub -v argv="$INPUT_ARGS" -l arch=x86_64 -l walltime=10:00:00 -l vmem=8GB -l nodes=1:ppn=6 $CLUSTERA_BIN/run.script

# run.script
runprogram()
{
  #Input
  INPUT_ARGS=`echo $argv`
  $CLUSTERA_BIN/runprogram.py "$INPUT_ARGS" ;
}

So my problem is where does Cluster B find $CLUSTERA_BIN/runprogram.py ? How to make the code in run.script run successfully on Cluster B ?

One command I thought it would be ssh root@ClusterB 'bash -s' < run.sh $MYARGS , but obviously it won't work like this.

tomriddle_1234
  • 3,145
  • 6
  • 41
  • 71

1 Answers1

2

Cluster B has to access those files it needs. So you have some options:

  1. Change your mind and copy (means scp, rsync, ...) files to Cluster B. You may also consider using a version control system (like svn, git, ...) to sync files between multiple hosts.
  2. When Cluster B needs a file, get it from Cluster A on the fly (for example with password-less scp).
  3. Put your files under one dir of Cluster A and share (means NFS, SMB, ...) the dir to Cluster B.
  4. ... ...
pynexj
  • 19,215
  • 5
  • 38
  • 56
  • My experience is that placing scripts on a filesystem that can be reached through the network is how people commonly do this. – dbeer Mar 28 '13 at 16:23