How to avoid double quotes at the end of lines with .to_csv()

Question

I have .txt file as next:

Red Blue Green, 1354,8676,38
------------------------------
Yellow Or,ange Black

TU    CD    CL
0.26  0.265 0.25
-------------------------------

I need to read it. For that purpose, I use pd.read_csv() and I avoid the last two lines with skipfooter = 2 argument. So, I have:

path_file1 = pathlib.Path.cwd() / ("folder1") / ("file1.txt")

df_folder1 = pd.read_csv(str(path_file1), engine = 'python', index_col = False, skipfooter = 2)

Then, I would like to paste all tha info in other file using .to_csv():

path_file2 = pathlib.Path.cwd() / ("folder1") / ("file2.txt")

df_folder2 = df_folder1.to_csv(str(path_file2), index = False, quoting=csv.QUOTE_NONE)

However, I end up with this result:

Red Blue Green, 1354,8676,38
------------------------------,,,
Yellow Or,ange Black,,
TU     CD   CL,,,

So I cannot get rid of the commas at the end of each line and the empty line is omitted. Is it possible to achieve this using to_csv()? Should I use csv.writer() as in this thread? write csv file with double quotes for particular column not working Is there any chance of specifying a lineterminator?

Answer 1

the problem you're facing is that csv is a representation of tabular data.

In computing, a comma-separated values file is a delimited text file that uses a comma to separate values. A CSV file stores tabular data in plain text. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format. https://en.wikipedia.org/wiki/Comma-separated_values

And the additional commas at the end of your lines represent empty fields in that column. Even though I'm not sure that what you ask is impossible I would argue that it would be out of spec.

Therefore, I suggest, if you what you really want is to remove the last to lines from that file, you read the file with

with open('file') as f:
    for i in range(len(f.readlines()) - 2):
        # do what ever you want

and write it back to file.