How can i add to top of a txtfile in python?

Question

I have a sql query txt file i would like to append to (at the top).

The items i want to add are from this yml file:

data: 
    START_DT: '202001'
    END_DT: '202104'

i call this yml file in my python script using:

import yaml

with open("data.yml", "r") as ymlfile:
    cfg = yaml.load(ymlfile)

the top of my sql file is:

set START_DT='202001';
set END_DT='202104';

i want to change the top of the sql file such that whatever is in the config yaml file for strt date and end date will change the value of the start_dt, end_dt in the sql query. How can i replace the values after SET for each variable (str date, end date) whilst using only the values from the config?

If i cannot replace the values after START_DT then another option is to remove the SET statments and rewrite from scratch using config.yml variables but if i use this way how can i make the dates strings and how can i append this to top of the sql file or create a new tmp file?:

for k,v in cfg['data'].items():
    print("SET" ,str(k)+':'+ v)

prints: 
SET START_DT:201901
SET END_DT:202104

but the dates aren't in "'" i would prefer: SET START_DT:'201901'. Again i am not sure how i can append the above print statment to top if the txtfile/sql file.

Any suggestions?

Go with the rewrite from scratch approach. It's safe. You can't append at the top. That would be *prepend* and is not available for files. That's why it's called *append* to a file. — Thomas Weller, Jun 02 '21 at 10:57
i am saying with the second approach that i am unsure how to append to the top of the file with my print statments — Maths12, Jun 02 '21 at 11:02
for k,v in cfg['data].items(): with open ('data.sql', 'a') as f: f.write(("SET" ,k+':'+ str(v)) does not work — Maths12, Jun 02 '21 at 11:10

Thomas Weller · Answer 1 · 2021-06-02T17:03:13.517

It's really straight forward:

you read the YAML configuration
you write the first lines from YAML into a temp file
you read all lines from the SQL file
you write all lines into the temp file
you delete the old file
you move the temp file to replace the SQL file

In code that is

import os
import yaml

# read the YAML configuration
with open("data.yml", "r") as ymlfile:
    cfg = yaml.load(ymlfile)

# Create a temporary file
with open("data.tmp", "w") as tempfile:
    # Write the lines from YAML
    for k, v in cfg["data"].items():
        tempfile.write(f"SET {k}='{v}';\n")
    # Copy the contents of the old file ...
    with open("data.sql") as oldfile:
        for line in oldfile:
            # ... as long as they haven't been set by YAML
            set_by_yaml = False
            for k, v in cfg["data"].items():
                if line.startswith(f"SET {k}="):
                    set_by_yaml = True            
            if not set_by_yaml:
                tempfile.write(line)
# Replace old file by new file
os.remove("data.sql")
os.rename("data.tmp", "data.sql")

thank you for this but when i call cfg["data"] i was thinking of doing it more dynamically otherwise it defeats purpose of bothering to do this in first place. e.g. for k,v in cfg['data'].items(): print("SET" ,str(k)+':'+ v) would ideally give me the print statements required but it doesn't keep the quotation marks of the date intact and this is issue i am having — Maths12, Jun 02 '21 at 12:12
@Maths12: the quotation marks are not part of the value in YAML. So you need to add them manually, like I did. I changed the code to match your k,v needs — Thomas Weller, Jun 02 '21 at 17:04

score 0 · Answer 2 · answered Jun 02 '21 at 11:31

import yaml
file = open('write_to.txt', 'r+')
temp_data = file.readlines()

with open('read_from.yaml') as f:
    data = yaml.load(f, Loader=yaml.FullLoader)
    data = data.get('data')
    START_DT = str(data.get('START_DT'))
    END_DT = str(data.get('END_DT'))

lines_of_interest = temp_data[:2]
rest_of_the_file = temp_data[2:]

ST_DT = lines_of_interest[0]
ST_DT = ST_DT.split('=')
ST_DT[1] = "'"+START_DT+"';\n"
lines_of_interest[0] = ''.join('=').join(ST_DT)


EN_DT = lines_of_interest[1]
EN_DT = EN_DT.split('=')
EN_DT[1] = "'"+END_DT+"';\n"
lines_of_interest[1] = ''.join('=').join(EN_DT)

lines = lines_of_interest + rest_of_the_file

file = open('write_to.txt', 'w+')
file.writelines(lines)

Here, note that the file which we're writing to is 'write_to.txt' (from which we also read in the beginning). We get the data necessary from 'read_from.yaml'

Thank you!

How can i add to top of a txtfile in python?

2 Answers2