How to prevent hardcoding when reading from txt-file to 3 different lists - python

Question

I am creating a code in which it reads information from a txt-file and stores the information on three separate lists. The code I have created does this, but in a pretty hardcoded manner. My question is if there is any better way to read from the file by for example using a for-loop or anything since the code at this point is repeating itself.

The code I have created is:

def load():
with open("kurser.txt") as share_price_time:
    share_price_list_1 = []
    share_price_list_2 = []
    share_price_list_3 = []
    row_number = 0
    for row in share_price_time:
        if 36 < row_number < 67:
            info_1 = row.strip().split("\t")
            share_price_list_1.append(info_1[1])
        elif 104 < row_number < 135:
            info_2 = row.strip().split("\t")
            share_price_list_2.append(info_2[1])
        elif 172 < row_number < 203:
            info_3 = row.strip().split("\t")
            share_price_list_3.append(info_3[1])
        row_number = row_number + 1

Thank you in advance for your help!

Moses Koledoye · Accepted Answer · 2017-08-12T13:33:18.333

1

You can use itertools.islice to read out those contiguous portions and then process each line into a list which is appended to a parent list:

from itertools import islice

indices = [(37, 67), (105, 135), (173, 203)]

def load():
   with open("kurser.txt") as share_price_time:
      lst = []
      for st, sp in indices:
         lst.append([r.strip().split("\t")[1] 
                               for r in islice(share_price_time, st, sp)])
         share_price_time.seek(0)

The list of indices are easily amendable and can easily be passed as part of the function arguments.

edited Aug 12 '17 at 13:33

answered Aug 12 '17 at 13:25

Moses Koledoye

77,341
8
133
139

Hmm it seems to give me an invalid syntax over the row: for r in islice(share_price_time, st, sp))] – Jurkka Aug 12 '17 at 13:31
@Jurkka Ooops typo. Fixed. – Moses Koledoye Aug 12 '17 at 13:33
Nice, yes it seems to work now :) but sorry if its a stupid question but where is the information stored now? I tried to print the list "lst", but nothing prints! Maybe im doing it wrong? EDIT: it works now haha sorry – Jurkka Aug 12 '17 at 13:39
@Jurkka It should not be empty. Did you already read from the file previously? Try running this code separately. – Moses Koledoye Aug 12 '17 at 13:41
This is not very Pythonic (Simple is better than complex.) why use `itertools` and a list comprehension when all you need is slicing a list? – Dror Av. Aug 12 '17 at 13:52
@droravr You totally have that one *Zen* misconstrued, and I don't see why using `islice` to read a file and then building a list from that is unpythonic. You have to evaluate the `islice` object at some point. However, reading the entire file into memory when you only need parts of it like you've done is rather **unpythonic**. – Moses Koledoye Aug 12 '17 at 13:55

Dror Av. · Answer 2 · 2017-08-12T14:10:23.780

0

You can read entire filr first and store slices:

with open('kurser.txt') as f:
    x = f.readlines()

a, b, c = x[37:67], x[105:135], x[173:203]

A slice is configured [Start Position (included):End Position (excluded)].

See Docs Here

edited Aug 12 '17 at 14:10

answered Aug 12 '17 at 13:28

Dror Av.

1,184
5
14

Reading the entire file into memory is not a very good idea here since you don't how large the file is, and you only need a part of it. Besides, the start and stop indices for the slices are wrong. – Moses Koledoye Aug 12 '17 at 14:01
@MosesKoledoye You're right didn't see he used larger than and not equal and larger, edited. – Dror Av. Aug 12 '17 at 14:04

score 0 · Answer 3 · edited Aug 12 '17 at 14:58

If rowcount is the only way of splitting the file, maybe something like this:

share_price_spec = [(37,67), (105,135), (173,203)]
share_price_lists = []

with open("c:\Python\data.txt") as share_price_time:
    test = share_price_time.readlines()
    for price in share_price_spec:
        row_start,row_end = price[0], price[1]
        share_price_lists.append( test[row_start: row_end])

print(share_price_lists)

First create a file spec containing the line blocks you are interested in. Then read the entire file into memory (assuming the file is not too large) and use the file spec to slice the file into the separate sections. Should be easy to maintain because you only have to change the file spec to read more blocks. There's a few extra unnecessarily lines in my sample code, but its worth it for readability IMO.

If you prefer, you can change share_price_spec into a list of directories or a list of named slices for future readability.

How to prevent hardcoding when reading from txt-file to 3 different lists - python

3 Answers3