Convert a folder of pipe-delimited text files to CSV in Python

Question

I have a folder of .txt files which contain "|" instead of commas and I am trying to get it into CSV format. I found some code that will supposedly work, but I keep getting the error "iterator should return strings, not bytes (did you open the file in text mode?)". The code I found was not nested in a for loop, could that be the issue?

The code:

import csv
import os

folder_path= r'C:\Users\%user%\Documents\data\Dataset'
txt_files = os.listdir(folder_path)

to_csv = []

for file in range(0, len(txt_files)):
    path_name = os.path.abspath(os.path.join(folder_path, txt_files[file]))
    to_csv.append(path_name)

for file in to_csv:
    with open(file, "rb") as f:
        with_pipes = csv.reader(f, delimiter='|')
        wo_pipes = list(with_pipes)

Answer 1

Change the open statement to:

with open(file, "r", encoding="utf-8") as f:

This will open the file in text mode, as opposed to binary mode, and the encoding allows you to read non-ASCII content

Answer 2

with open(output_file_name, 'w') as f_out:
    for line in source_lines:
        # get the count of delimiters in a line
        pipe_cnt = line.count('|')
        # replacing the delimiters in the line bases on count from previous step
        line = line.replace('|', ',', pipe_cnt)
        f_out.write(line)