Python sum certain values from multiple text files

Question

I have multiple text files that contain multiple lines of floats and each line has two floats separated by white space, like this: 1.123 456.789123. My task is to sum floats after white space from each text file. This has to be done for all lines. For example, if I have 3 text files:

1.213 1.1
23.33 1

0.123 2.2
23139 0

30.3123 3.3
44.4444 444

Now the sum of numbers on the first lines should be 1.1 + 2.2 + 3.3 = 6.6. And the sum of numbers on second lines should be 1 + 0 + 444 = 445. I tried something like this:

def foo(folder_path):
    contents = os.listdir(folder_path)
    for file in contents:
        path = os.path.join(folder_path, file)
        with open(path, "r") as data:
            rows = data.readlines()
            for row in rows:
                value = row.split()
                second_float = float(value[1])

    return sum(second_float)

When I run my code I get this error: TypeError: 'float' object is not iterable. I've been pulling my hair out with this, and don't know what to do can anyone help?

Error - Syntactical Remorse · Answer 1 · 2019-11-17T18:04:33.820

Here is how I would do it:

def open_file(file_name):
    with open(file_name) as f:
        for line in f:
            yield line.strip().split() # Remove the newlines and split on spaces

files = ('text1.txt', 'text2.txt', 'text3.txt')
result = list(zip(*(open_file(f) for f in files)))
print(*result, sep='\n')

# result is now equal to:
# [
#     (['1.213', '1.1'], ['0.123', '2.2'], ['30.3123', '3.3']),
#      (['23.33', '1'], ['23139', '0'], ['44.4444', '444'])
# ]

for lst in result:
    print(sum(float(x[1]) for x in lst)) # 6.6 and 445.0

It may be more logical to type cast the values to float inside open_file such as:

yield [float(x) for x in line.strip().split()]

but I that is up to you on how you want to change it.

See it in action.

-- Edit --

Note that the above solution loads all the files into memory before doing the math (I do this so I can print the result), but because of how the open_file generator works you don't need to do that, here is a more memory friendly version:

# More memory friendly solution:
# Note that the `result` iterator will be consumed by the `for` loop.
files = ('text1.txt', 'text2.txt', 'text3.txt')
result = zip(*(open_file(f) for f in files))
for lst in result:
    print(sum(float(x[1]) for x in lst))

Python sum certain values from multiple text files

1 Answers1