Why are some files are not being added do my zipfile

Question

I have the following code:

with ZipFile('deploy.zip', 'w') as deploy:
    if os.path.isfile(artifact.source):
        deploy.write(artifact.source, artifact.target)
    else:
        for base, dirs, files in os.walk(artifact.source):
            for file_ in files:
                source = os.path.join(base, file_)
                target = os.path.join(base[base.index(artifact.target):], file_)
                deploy.write(source, target)

when this code finish, only the files that match when the artifact.source is a file are added to the deploy.zip. And in some cases artifact.source will be a directory (i have tested this case too) ad the for part will be executed.

The result of the following lines are valid and source exists for every iteration:

source = os.path.join(base, file_)
target = os.path.join(base[base.index(artifact.target):], file_)

Here the full code that i'm working: https://gist.github.com/khaoz/9b04d87b0900fba780f0 Set config.project_root to something like "c:\temp" and remove the import config line. OBS: I'm a Python newbie, so ignore some crap code that you will see :P

And here a example of my csv file: https://gist.github.com/khaoz/e9a59390f415f22d46db

What i'm doing wrong ?

so when you print source and target before every ``deploy.write`` they're as you expect ? — sotapme, Feb 07 '13 at 20:24
Short answer: The `for` loop in the `else` clause is never being executed. You have a single `if` statement executing once in the `ZipFile` context. — martineau, Feb 07 '13 at 20:50
In the code shown in your question, it's only executed when `os.path.isfile(artifact.source)` isn't True. — martineau, Feb 07 '13 at 23:52
@martineau and this will happens :). The source for the `artifac.source` is a csv with files and directories to be copied. — Giuliani Sanches, Feb 08 '13 at 00:51
If `artifac.source` is a csv file, you will have to open it and process its contents, line by line. This will not happen automatically in the `ZipFile.write()` method. — martineau, Feb 08 '13 at 01:57
@martineau yeap, i know and it's exactly what i'm doing. :) I will poste my code and a sample csv to clarify things. — Giuliani Sanches, Feb 08 '13 at 11:16
You really need to include enough source code in your question from the start so everyone doesn't waste time guessing about what's going on. — martineau, Feb 08 '13 at 15:43

score 1 · Answer 1 · answered Feb 07 '13 at 20:47

JUST FYI

My interpretation of what you did, this seems to work.

from zipfile import ZipFile
from collections import namedtuple
import os

Artifact =  namedtuple('Artifact', ['source', 'target'])
artifact =  Artifact(source="Mongodb", target="g")

with ZipFile('deploy.zip', 'w') as deploy:
    if os.path.isfile(artifact.source):
        print "F"
        print "\n", artifact.source
        print "\n", artifact.target
        deploy.write(artifact.source, artifact.target)
    else:
        for base, dirs, files in os.walk(artifact.source):
            for file_ in files:
                print "base", base, file_
                source = os.path.join(base, file_)
                target = os.path.join(base[base.index(artifact.target):], file_)
                print "f"
                print "\t", source
                print "\t", target
                deploy.write(source, target)

unzip -l deploy.zip | tail

     2591  01-09-13 21:26   godb/Sortif/scratch.py
     2010  01-15-13 20:20   godb/Sortif/sortif_model.py
     2495  01-15-13 20:22   godb/Sortif/sortif_model.pyc
      161  01-15-13 20:45   godb/Sortif/sortif_scratch.py
        0  01-08-13 12:05   godb/Sortif/sortif/__init__.py
        0  01-08-13 12:05   godb/Sortif/sortif/models/__init__.py
     1408  01-21-13 18:05   godb/ZeroMQ/client.py
     3044  01-21-13 17:51   godb/ZeroMQ/controller.py
 --------                   -------
 11137644                   967 files

I'm unsure what you're trying to achieve with base[base.index(artifact.target):] are you wanting to change the prefix ? Because running it on my Mongodb directory target had to appear in the directory of the file base.

I'm unsure how you expect to drive the code as it seems that artifact.source is a constant. So first time around it finds a file and it will never do the part where it's looking for directory.

Shouldn't it be

with ZipFile('deploy.zip', 'w') as deploy:
    for artifact in articats:
        if os.path.isfile(artifact.source):
            ...

You're right about change the prefix; remove it to be more exact. It's not needed in my zip. `artifact.source` is not constant. It's a path that is loaded from a csv file. I will commit my code in my github repository tomorrow and share here to give more context. — Giuliani Sanches, Feb 07 '13 at 23:42

score 0 · Accepted Answer · answered Feb 08 '13 at 11:51

I discovered the problem. Sometimes, sleep is the best solution form some problems

I was doing:

for artifact in artifacts:
    if not artifact.name in contents:
        contents.append(artifact.name)

    with ZipFile('deploy.zip', 'w') as deploy:
        if os.path.isfile(artifact.source):
            deploy.write(artifact.source, artifact.target)
        else:
            for base, dirs, files in os.walk(artifact.source):
                for file_ in files:
                    source = os.path.join(base, file_)
                    target = os.path.join(base[base.index(artifact.target):], file_)
                    deploy.write(source, target)

But for every iteration in the artifacts close and open a new deploy.zip file.

The right way to do this is:

with ZipFile('deploy.zip', 'w') as deploy:
    for artifact in artifacts:
        if not artifact.name in contents:
            contents.append(artifact.name)

        if os.path.isfile(artifact.source):
            deploy.write(artifact.source, artifact.target)
        else:
            for base, dirs, files in os.walk(artifact.source):
                for file_ in files:
                    source = os.path.join(base, file_)
                    target = os.path.join(base[base.index(artifact.target):], file_)
                    deploy.write(source, target)

And everything works as expected.

Thank you very much for everyone that tried to help. Next time i will post the full source code or at last, some lines more. :)

We live and learn - at least you didn't drop the bomb "It doesn't work" then sit back and wait - sleeping on it is always a good thing. — sotapme, Feb 08 '13 at 13:13
Agreed. But not because the `else:` was not being executed. Anyway I apologize for not posting the entire source code in the initial post. :) — Giuliani Sanches, Feb 08 '13 at 15:41

Why are some files are not being added do my zipfile

2 Answers2