read every 25 line of a csv file and pass to list using python

Question

I want to read a file and convert every 25 lines of that file into a list, that is, it should have 4 lists with 25 items in each (for 100 lines of a file). I am not being able to get the code for this problem. The input file looks like this , in actual it has 100 lines:

{'PutRequest': {'Item': {'id': {'S': 'E1DBEAE3'}, 'value': {'M': {'result': {'N': u'0.0015'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '31C6C'}, 'value': {'M': {'result': {'N': u'0.1129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '59D40'}, 'value': {'M': {'result': {'N': u'0.00129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': 'A2A9'}, 'value': {'M': {'result': {'N': u'0.05129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}

Also, I want to prepend and append a string to the very first element and the last element of every list respectively, prepend string like :

'{"test":[' and append string like: ']}'

After prepend and append it should look like for a list size of 3 for example:

{"test":[{'PutRequest': {'Item': {'id': {'S': 'E1DBEAE3'}, 'value': {'M': {'result': {'N': u'0.0015'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '31C6C'}, 'value': {'M': {'result': {'N': u'0.1129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': '59D40'}, 'value': {'M': {'result': {'N': u'0.00129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}
{'PutRequest': {'Item': {'id': {'S': 'A2A9'}, 'value': {'M': {'result': {'N': u'0.05129'}, 'lastupdatedtime': {'S': '2019-06-20'}}}}}}]}

I have tried this code:

from itertools import islice
list =[]
with open('output_of_json.json', 'r') as infile:
    lines_gen = islice(infile, 25)
    for line in lines_gen:
        list.append(line)

Unable to go past the first 25 lines of the file

`from itertools import islice` `list =[]` `with open(filename', 'r') as infile:` `lines_gen = islice(infile, 25)` `for line in lines_gen:` `list.append(line)` Not able to go further the first 25 lines @G.Anderson @Mureinik — shalini, Jun 20 '19 at 16:44
Put your code in an edited question, not in comments, please. — TomServo, Jun 20 '19 at 16:47
@shalini, `islice(infile, 25)` will iterate over the _first_ 25 lines of the file. It _slices off_ these 25 lines and iterates over them, so there's no wonder you're unable to go past the first 25 lines. — ForceBru, Jun 20 '19 at 16:49
That's not a CSV file. It's also not JSON, since it contains single quotes around strings. — Barmar, Jun 20 '19 at 16:55
@shalini, you can literally count the lines and start appending to a new list once the count becomes a multiple of 25 — ForceBru, Jun 20 '19 at 17:00
It looks like a Python literal, so you should use `ast.literal_eval()` to parse it. — Barmar, Jun 20 '19 at 17:00

score 0 · Answer 1 · answered Jun 20 '19 at 16:56

This can be made into a function. It's logically what you want. You can use enumerate to clean it up.

with open(filename,'r') as f:

    counter = 25
    iteration = -1
    out_dict = {}            

    for i in f.readlines():

        if counter == 25:

            if out_dict[iteration]:

                # append your list
                out_dict[iteration].append('string here')

            counter = 0
            iteration += 1

            # create new instance and pre-pend
            out_dict[iteration] = ['string here']

        out_dict[iteration].append(i)

Rory Daulton · Answer 2 · 2019-06-20T17:05:52.507

0

Look at the grouper function in itertools recipes.

import itertools

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return itertools.zip_longest(*args, fillvalue=fillvalue)

from itertools import islice
list =[]
with open('output_of_json.json', 'r') as infile:
    lines_gen = grouper(infile, 25, fillvalue='')
    for line in lines_gen:
        # whatever you want to do

Note that if the final block of lines has fewer than 25 lines, that code will fill out the 25 with blank lines.

edited Jun 20 '19 at 17:05

answered Jun 20 '19 at 16:59

Rory Daulton

21,934
6
42
50

You need to call `ast.literal_eval()` to parse the line into a dictionary. – Barmar Jun 20 '19 at 17:00
@Barmar: I'm not sure just what the questioner is trying to accomplish. I just showed how to group the lines into chunks of 25 lines. But thanks for your comment--I changed my code slightly to make it more general. – Rory Daulton Jun 20 '19 at 17:02
What about this part of the question: **Also, I want to prepend and append a string to the very first element and the last element of every list respectively, prepend string like** – Barmar Jun 20 '19 at 17:03

read every 25 line of a csv file and pass to list using python

2 Answers2