0

I am working on translating some code I have written into a parallel process to be distributed on a computer cluster at my home university. In order to prepare to write my script for the cluster, I began by reading one of the sample snippets of Python code provided by the cluster:

#! /usr/bin/python

# This script replicates a given test with varied parameters
# creating unique submit scripts and executing the submission to the CRC SGE queue

# Imports
import os
import shutil
import string

basedir=os.getcwd()
origTestFol='wwd'
templFol='template'
origTestDir= basedir + '/' + origTestFol
templFolDir= basedir + '/' + templFol

steps=[0,1,2,3,4,5,6,7,8,9]
primes=[2,3,5,7,11,13,17,19,23,29,31]
trials=[6,7,8,9,10]

for step in steps:
    newTestDir= origTestDir + '_nm_' + str(step)
    if not os.path.exists(newTestDir):
        os.mkdir(newTestDir)
    os.chdir(newTestDir)
    for trial in trials:
       newTestSubDir= newTestDir + '/' + str(trial)
       if not os.path.exists(newTestSubDir):
            shutil.copytree(templFolDir,newTestSubDir)
            os.chdir(newTestSubDir)   
            os.system('sed -i \'s/seedvalue/' + str(primes[trial]) + '/g\' wwd.nm.conf')
            os.system('sed -i \'s/stepval/' + str(step) + '/g\' qsubScript.sh')
            os.system('qsub qsubScript.sh')
            os.chdir(basedir)

I can follow the code up to the last four lines [e.g. up to "os.system('sed -i ...)"] but then have a hard time following the code. Is there any chance others could help me understand what these last four lines are doing. Is there a way to describe the lies in pseudocode? So far as I can tell, the first sed line attempts to replace "seedvalue" with the value of primes[trial], but I'm not sure what the seedvalue is. I'm also not sure how to understand the "stepval" in the subsequent line. Any light others can shed on these questions will be most appreciated.

duhaime
  • 25,611
  • 17
  • 169
  • 224

2 Answers2

1

You could do a little search on sed: http://en.wikipedia.org/wiki/Sed#In-place_editing

In short: sed -i 's/old/new/g' file replaces all the occurrence of old with new in file. The -i flag tells it do it in-line, modifying the file itself.

In your code, seedvalue and stepval are nothing but two words in the text files wwd.nm.conf and qsubScript.sh. Those commands are replacing those words just as you'd do in your text editor or word processor.

emnoor
  • 2,528
  • 2
  • 18
  • 15
  • Thank you for the concise definition of the s/ command and the -i flag, @emnoor. This simple explanation helped me to grasp the function of the last four lines of code. – duhaime Sep 01 '13 at 11:20
1

seedvalue is a string, so its value is itself, same with stepval.

The first sed line will replace all the seedvalue strings in wwd.nm.conf (on all lines, that's what /g does, it stands for "global") with the value str(primes[trial]). Think about it like this:

Wherever the text stepvalue is found in the file wwd.nm.conf, put the value str(primes[trial]) (whatever that evaluates to in Python).

The second sed call will do something similar but with stepval and the result of step and it will replace text in the file qsubScript.sh.

Phillip Cloud
  • 24,919
  • 11
  • 68
  • 88
  • Many thanks, @Phillip! I was sure that seedvalue and stepval were strings, but was confused because the code doesn't define those strings. I'm just now grasping that the code leaves those as undefined variables such that users can replace "seedvalue" with some line of code they wish to change in their "wwd.nm.conf" script. The point of the script above, if I understand it correctly, is to slightly modify a single script and its associated .job file and then submit the new job file (with qsub) to run the modified script. Thank you for helping me to grasp this. – duhaime Sep 01 '13 at 11:18