How to context manage BytesIO for multiple zip files?

Question

I am trying to use a context manager for a BytesIO stream while creating multiple zip files. I can find no way to "reset" the BytesIO object after the first zip file is written, so I can use the same BytesIO object to create the next zip file. I always get a "Can not open file ... as archive" error when trying to open the second zip file after it is written to disk. First zip file opens just fine. I have searched and cannot find a solution. Changing modes from write to append didn't help either. I can, of course, reinitialize to a new BytesIO object, but that defeats the context manager. Below is the code I thought should work. I'm using Anaconda Python 3.6.6 on Windows 10.

import io
import os
import zipfile

with io.BytesIO() as bytes_io:
    with zipfile.ZipFile(bytes_io, mode='w') as zf:
        filecount = 0
        for item in os.scandir(r'C:\Users\stephen\Documents'):
            if not item.is_dir():
                zf.write(item.path, item.name)
                filecount += 1
                if filecount % 3 == 0:
                    with open(r'C:\Users\stephen\Documents\\' + str(filecount // 3) + '.zip', 'wb') as f:
                        f.write(bytes_io.getvalue())
                    bytes_io.seek(0)
                    bytes_io.truncate()

blhsing · Accepted Answer · 2018-10-05T07:17:18.577

5

You can reuse the same BytesIO object, but you should create a new ZipFile object for every zip file you want to create:

with io.BytesIO() as bytes_io:
    filecount = 0
    for item in os.scandir(r'C:\Users\stephen\Documents'):
        if not item.is_dir():
            with zipfile.ZipFile(bytes_io, mode='w') as zf:
                zf.write(item.path, item.name)
            filecount += 1
            if filecount % 3 == 0:
                with open(r'C:\Users\stephen\Documents\\' + str(filecount // 3) + '.zip', 'wb') as f:
                    f.write(bytes_io.getvalue())
                bytes_io.seek(0)
                bytes_io.truncate()

edited Oct 05 '18 at 07:17

answered Oct 05 '18 at 05:49

blhsing

91,368
6
71
106

Not too sure what you mean here. The `ZipFile` needs to stay open in order for the bytes to be written to it, and only in the same condition should the `BytesIO` object be reset, so I don't see why or how the `ZipFile` should be closed any earlier. – blhsing Oct 05 '18 at 05:55
That's what a context manager is for. It seems that you do not know that the context manager closes the file handle automatically when the context manager block ends. Please read: https://docs.python.org/3.7/tutorial/inputoutput.html#reading-and-writing-files – blhsing Oct 05 '18 at 07:12
1

blhsing: I used your code and got multiple zip files that I can open, but only one file within each zip file. If I change the mode of the `ZipFile` zf from 'w' to 'a', then I get zip files containing as many files as whatever file count increment I set. Thank you! – Stephen Frost Oct 05 '18 at 13:52
What I still don’t understand is **why** creating the ZipFile object outside the loop doesn’t work. Also, why this **does** work for the first zip file created and works for that file only? – Stephen Frost Oct 07 '18 at 07:15

How to context manage BytesIO for multiple zip files?

1 Answers1