-1

I have a python script which I call through the .bat (batch) file. For the Python script to work I pass hardcoded arguments in the .bat file. But I need to pass the arguments dynamically as each time argument's value (name) would be different.

The python script's logic is not subject to my question, as it works properly. The question here is to how dynamically pass such arguments in the .bat file so that the specified in the same .bat file python script would except such dynamically passed arguments.

sys.argv[1] and sys.argv[1] are the arguments in the python file (see code below) which are hardcoded and passed in the .bat file. But I need to pass the arguments dynamically (as the the names will be different).

This is the .bat file that works properly as the arguments are hardcoded.

python "E:\Export\Test.py" "E:\Config\Test_Upload_10-15-2020.csv" "./Test_Upload_10-15-2020.csv" "E:\BulkInsert_DataAcquisition\SAM\Log"

The arguments are fully qualified file path and file name. "E:\Export\Test_Upload_10-15-2020.csv" is the sys.argv[1] "Test_Upload_10-15-2020.csv" is the sys.argv[2]

So I need to automate it to allow each csv file starting with the "Test_Upload_" to be processed.

I tried to replace the date stamp with the "" sign in the .bat (see code below) python "E:\Export\TEST.py" "E:\Config\Test_Upload_.csv" "./Test_Upload_*.csv" "E:\logFolderPath"

But I'm getting an error when execute this .bat file

Error occurred: [Error 123] The filename, directory name, or volume label syntax is incorrect: 'E:\Export\Test_Upload_*.csv'

It works though with the exact file name "Test_10-15-2020.csv"

The Python script below works fine as long as I pass exact CVC name in the argumetns

import pysftp
import sys
import datetime

if __name__ == "__main__":
    localFilePath = sys.argv[1] #"\\MyDBServer\E:\Config\Test_Upload_%.csv" % name
    remoteFilePath = sys.argv[2] #"./Test_Upload_%.csv" % name.csv" 
    logFolderPath = sys.argv[3] #"E:\logFolderPath"


    myHostname = "SFTP.MyWebSite.com"
    myUsername = "MyLogin"
    myPassword = "MyPassword"

    try:
        with pysftp.Connection(host=myHostname, username=myUsername, password=myPassword) as sftp:
            print ("Connection succesfully stablished ... ")
    
            sftp.put(localFilePath, remoteFilePath)
    
    except Exception as e:
        logfile = open(logFolderPath+"\Error_"+datetime.datetime.now().strftime("%Y%m%d_%H%M%S")+".log", "w")
        logfile.write("Error occured: "+ str(e))
        print(str(e))
        logfile.close
        sys.exit(1)
Data Engineer
  • 795
  • 16
  • 41
  • Thanks for updating your question. It seems to be a simple duplicate, though; if you want Python to expand the wildcard, you'll have to do that in your code, like in https://stackoverflow.com/questions/3348753/search-for-a-file-using-a-wildcard even if the wildcard string is populated from the command line rather than inside your script. – tripleee May 13 '21 at 11:07
  • In the duplicate question which I needed to create again since this one was closed I have already get the answer. Thanks for your participance. https://stackoverflow.com/questions/67513911/dynamically-pass-argument-file-name-in-bat-file-when-calling-python-script – Data Engineer May 13 '21 at 17:35

1 Answers1

0

You can search your "E:\Export\Test_*.csv" and get the newest or oldest file

paths = pathlib.Path('E:\Export).glob('Test_*.csv')
newest = max([path for path in paths], key=os.path.getctime)

or

oldest = min([path for path in paths], key=os.path.getctime)
Klim Bim
  • 484
  • 2
  • 7
  • There won't be any oldest or newest one in the directory. I simply need to process all that have 'Test_*.csv' in their name in the given directory @Klim Bim – Data Engineer May 11 '21 at 16:09
  • @enigma6205: Ok, `min` and `max` isn´t necessary. However, the `paths`-expression should help. – Klim Bim May 12 '21 at 07:07