humpy.genfromtxt input fname argument from list

Question

I have a number of text files as output from a calculation from which I wish to extract data:

(Note: Because some of the files are rather mangled, I have placed copies in my Dropbox. The URL is https://www.dropbox.com/sh/h774f8jzjb5l0wx/AAAqhvHsmPAhK_svdQG2Ou9Ha?dl=0)

=======================================================================
PSOVina version 2.0                                                    
Giotto H. K. Tai & Shirley W. I. Siu                                   

Computational Biology and Bioinformatics Lab                           
University of Macau                                                    

Visit http://cbbio.cis.umac.mo for more information.                   

PSOVina was developed based on the framework of AutoDock Vina.         

For more information about Vina, please visit http://vina.scripps.edu. 

=======================================================================

Output will be 14-7_out.pdbqt
Reading input ... done.
Setting up the scoring function ... done.
Analyzing the binding site ... done.
Using random seed: 768314908
Performing search ... done.

Refining results ... done.

mode |   affinity | dist from best mode
     | (kcal/mol) | rmsd l.b.| rmsd u.b.
-----+------------+----------+----------
   1    -9.960902669      0.000      0.000
   2    -8.979504781      1.651      2.137
   3    -8.942611364      3.051      6.898
   4    -8.915523010      2.146      2.875
   5    -8.736508831      2.908      7.449
   6    -8.663387139      2.188      2.863
   7    -8.410739711      5.118      7.281
   8    -8.389146347      2.728      3.873
   9    -8.296798909      2.416      3.846
  10    -8.168454106      3.809      8.143
  11    -8.127990818      3.712      8.109
  12    -8.127103774      3.084      4.097
  13    -7.979090739      3.798      4.959
  14    -7.941872682      4.590      8.294
  15    -7.900766215      3.300      8.204
  16    -7.881485228      2.953      4.224
  17    -7.837826485      3.005      4.125
  18    -7.815909505      4.390      8.782
  19    -7.722540286      5.695      9.851
  20    -7.720346742      3.362      4.593
Writing output ... done.

This works:

 import numpy as np

    print('${d}')

    data = np.genfromtxt("14-7.log", usecols=(1), skip_header=27, 
    skip_footer=1, encoding=None)

    print(data)

    np.savetxt('14-7.dG', data, fmt='%12.9f', header='14-7')
    print(data)

which produces:

runfile('/home/comp/Apps/Python/PsoVina/DeltaGTable_V_s.py', 
 wdir='/home/comp/Apps/Python/PsoVina', current_namespace=True)
 ${d}
 [-9.96090267 -8.97950478 -8.94261136 -8.91552301 -8.73650883 -8.66338714
  -8.41073971 -8.38914635 -8.29679891 -8.16845411 -8.12799082 -8.12710377
  -7.97909074 -7.94187268 -7.90076621 -7.88148523 -7.83782648 -7.8159095
  -7.72254029 -7.72034674]
 [-9.96090267 -8.97950478 -8.94261136 -8.91552301 -8.73650883 -8.66338714
  -8.41073971 -8.38914635 -8.29679891 -8.16845411 -8.12799082 -8.12710377
  -7.97909074 -7.94187268 -7.90076621 -7.88148523 -7.83782648 -7.8159095
  -7.72254029 -7.72034674]

Note; the print statements are for a quick check o the output, which is:

# 14-7
 -9.960902669
 -8.979504781
 -8.942611364
 -8.915523010
 -8.736508831
 -8.663387139
 -8.410739711
 -8.389146347
 -8.296798909
 -8.168454106
 -8.127990818
 -8.127103774
 -7.979090739
 -7.941872682
 -7.900766215
 -7.881485228
 -7.837826485
 -7.815909505
 -7.722540286
 -7.720346742

Also, this bash script works:

    #!/bin/bash

    # Run.dG.list_1

    while IFS= read -r d
    do
        echo "${d}.log"

   done <ligand.list

which returns the three log file names:

 14-7.log
 15-7.log
 18-7.log

But, if I run this bash script:

#!/bin/bash

 # Run.dG.list_1

 while IFS= read -r d
 do
     echo "${d}.log"
     python3 DeltaGTable_V_sl.py
 done <ligand.list

where DeltaGTable_V_sl.py is:

 import numpy as np

    print('${d}')

    data = np.genfromtxt('${d}.log', usecols=(1), skip_header=27, 
    skip_footer=1, encoding=None)
    print(data)

    np.savetxt('${d}.dG', data, fmt='%12.9f', header='${d}')
    print(data.dG)

I get:

 (base) comp@AbNormal:~/Apps/Python/PsoVina$ sh ./Run.dG.list_1.sh
     14-7.log
     python3: can't open file 'DeltaGTable_V_sl.py': [Errno 2] No such file 
     or directory
     15-7.log
     python3: can't open file 'DeltaGTable_V_sl.py': [Errno 2] No such file 
     or directory
     18-7.log
     python3: can't open file 'DeltaGTable_V_sl.py': [Errno 2] No such file 
     or directory
     C-VX3.log
     python3: can't open file 'DeltaGTable_V_sl.py': [Errno 2] No such file 
     or directory

So, it would appear that the log file labels are in the workspace, but '${d}.log' is not being recognized as fname by genfromtxt. Although I have googled every combination of terms I can think of I am obviously missing something.

As I have potentially hundreds of files to process, I would appreciate pointers towards a solution to the problem.

Thanks in advance.

Answer 1

Python does no now know ${d} as used in the shell script.

If you want do use a command-line argument passed to your Python script you can use argparse or the sys module. argparse is a more mighty, so you could first try sys:

sys.argv[0] # name of the Python script.
sys.argv[1] # command line arguments 1
sys.argv[n] # command line arguments n

See here.

Answer 2

I can create your error message with:

0029:~/mypy$ python3 foobar
python3: can't open file 'foobar': [Errno 2] No such file or directory

foobar is a random name, and clearly not present in the Python path.

So you haven't even started DeltaGTable_V_sl.py, much less run into problems with genfromtxt. So most of your question isn't relevant.