Read lines from multiple files python

Question

I have a list of files ['file_a.txt', 'file_b.txt', 'file_c.txt' ....]

I want to read one line from each file and generate a new line by adding them together.

The output should look like:

file_a_line_1 file_b_line_1 file_c_line_1.... 
file_a_line_2 file_b_line_2 file_c_line_2....

Can zip be used to do this?

Yes, zip is the way to go for this. What do you want to have done if the files aren't of the same length? — L3viathan, Jan 22 '16 at 00:33
I am not able to find examples of using zip with multiple files (more than 2), if they are not of same length I would like to keep them adding lines from the ones which are still there in the same order. — Rahul, Jan 22 '16 at 00:35
"I am not able to find examples of using zip with multiple files (more than 2)," - What makes you think it would be any different for multiple files? — TigerhawkT3, Jan 22 '16 at 00:44
Also see my recent answer [here](http://stackoverflow.com/questions/34756145/most-pythonic-way-to-interleave-two-strings/34756161#34756161). — TigerhawkT3, Jan 22 '16 at 00:59

L3viathan · Accepted Answer · 2016-01-22T00:42:11.397

3

from itertools import zip_longest

files = [open(filename) for filename in file_list]

for lines in zip_longest(*files, fillvalue=''):
    print(" ".join(lines))

This should also work when the files don't have the same length. I would use izip_longest, if you're on Python 2, instead.

This will leave several spaces between values if some files are exhausted, so you might want to do more complicated stuff than the join, but that's the easier part.

edited Jan 22 '16 at 00:42

answered Jan 22 '16 at 00:37

L3viathan

26,748
2
58
81

I didn't know about zip_longest. Awesome! – Rahul Jan 22 '16 at 00:41
Ah, thanks, corrected. I didn't know about zip_longest either before, but in general it's good to look in itertools for iteration questions and functools for functional programming questions. – L3viathan Jan 22 '16 at 00:43

score 1 · Answer 2 · answered Jan 22 '16 at 00:42

Something like this might work. You'd open all the files, then read one line at a time from each of them. It's unclear what you want to do when one file has no lines left (stop entirely), keep going until all lines have no lines left?

file_list = ['file1', 'file2', 'file3']

fps = []
for fn in file_list:
    fps.append(open(fn, 'w'))
curLine = 'start'

while curLine:
    curLine = ''
    for fp in fps:
        myLine = fp.readline()
        if myLine:
            curLine += myLine + ' '
        else:
            break #can do what you want when you run out

for fp in fps:
    fp.close()

Remember to close your file handlers.

Read lines from multiple files python

2 Answers2