Checking for Duplicates twice over in a File - Python

Question

config.yml example,

DBtables:
  CurrentMinuteLoad:
    CSV_File: trend.csv
    Table_Name: currentminuteload

GUI image, GUI

This may not be the cleanest route to take.

I'm making a GUI that creates a config.yml file for another python script I'm working with.

Using pysimplegui, My button isn't functioning the way I'd expect it to. It currently and accurately checks for the Reference name (example here would be CurrentMinuteLoad) and will kick it back if it exists, but will skip the check for the table (so the ELIF statement gets skipped). Adding the table still works, I'm just not getting the double-check that I want. Also, I have to hit the Okay button twice in the GUI for it to work?? A weird quirk that doesn't quite make sense to me.

def add_table():
  window2.read()
  with open ("config.yml","r") as h:
    if values['new_ref']  in h.read():
      sg.popup('Reference name already exists')  
    elif values['new_db']  in h.read():
      sg.popup('Table name already exists')
    else:
      with open("config.yml", "a+") as f:
        f.write("\n  " + values['new_ref'] +":")
        f.write("\n    CSV_File:" + values['new_csv'])
        f.write("\n    Table_Name:" + values['new_db'])
        f.close()
        sg.popup('The reference "' + values['new_ref'] + '" has been included and will add the table "' + values['new_db'] + '" to PG Admin during the next scheduled upload')

The recommended extention for YAML files has been `.yaml` since 2006 ([yaml.org](http://yaml.org/faq.html)). And you should consider using a YAML library to load and dump your data, that way you can much better circumvent problems when `values['new_ref']` is a new value, but a substring of some old value already in your YAML — Anthon, Apr 21 '22 at 19:47
I'm not quite sure how to do that. I've followed the procedure laid out in this answer, https://stackoverflow.com/questions/70162828/delete-blocks-in-yaml-file-using-python , for deleting sections using pyyaml, but am not sure how to proceed to add (especially with values from 3 different input boxes. — BabaZuri, Apr 22 '22 at 18:03
I added working code as an example. Since I don't use `pysimplegui` I hardcoded the input and used sys.stdout for messages. That should be easy to adapt. — Anthon, Apr 22 '22 at 18:59

score 2 · Answer 1 · answered Apr 21 '22 at 13:17

2

When you use h.read(), you should save the value since it will read it like a stream, and subsequent calls for this method will result in an empty string.

Try editing the code like this:

 with open ("config.yml","r") as h:
    content = h.read()
    if values['new_ref']  in content:
      sg.popup('Reference name already exists')  
    elif values['new_db']  in content:
      sg.popup('Table name already exists')
    else:
        # ...

answered Apr 21 '22 at 13:17

Drago96

1,265
10
19

That worked perfectly. Much appreciated. – BabaZuri Apr 21 '22 at 13:37
Happy it worked :) If you found it helpful, set this as the answer (click on the checkmark on the left of the answer) – Drago96 Apr 21 '22 at 13:56

score 1 · Answer 2 · answered Apr 22 '22 at 18:57

You should update the YAML file using a real YAML parser, that will allow you to check on duplicate values, without using in, which will give you false positives when a new value is a substring of an existing value (or key).

In the following I add values twice, and show the resulting YAML. The first time around the check on new_ref and new_db does not find a match although it is a substring of existing values. The second time using the same values there is of course a match on the previously added values.

import sys
import ruamel.yaml
from pathlib import Path

def add_table(filename, values, verbose=False):
    error = False
    yaml = ruamel.yaml.YAML()
    data = yaml.load(filename)
    dbtables = data['DBtables']
    if values['new_ref'] in dbtables:
        print(f'Reference name "{values["new_ref"]}" already exists') # use sg.popup in your code
        error = True
    for k, v in dbtables.items():
        if values['new_db'] in v.values():
            print(f'Table name "{values["new_db"]}" already exists')
            error = True
    if error:
        return
    dbtables[values['new_ref']] = d = {}
    for x in ['new_cv', 'new_db']:
        d[x] = values[x]
    yaml.dump(data, filename)
    if verbose:
        sys.stdout.write(filename.read_text())
    

values = dict(new_ref='CurrentMinuteL', new_cv='trend_csv', new_db='currentminutel')
add_table(Path('config.yaml'), values, verbose=True)
print('========')
add_table(Path('config.yaml'), values, verbose=True)

which gives:

DBtables:
  CurrentMinuteLoad:
    CSV_File: trend.csv
    Table_Name: currentminuteload
  CurrentMinuteL:
    new_cv: trend_csv
    new_db: currentminutel
========
Reference name "CurrentMinuteL" already exists
Table name "currentminutel" already exists

Very nice. It worked like a charm. Much appreciated Anthon. – BabaZuri Apr 22 '22 at 19:24 — BabaZuri, Apr 22 '22 at 19:24

Checking for Duplicates twice over in a File - Python

2 Answers2