python tarfile writes tar to pipe

Question

I want to create a tar file and pipe it to a http upload.

However, seems python tarfile module performs seek which make it impossible to pipe to the next process.

Here is the code

tar = tarfile.open('named_pipe', mode='w')
tar.add('file1')
p.close()

'named_pipe' is a named pipe file created by mkfifo command when I run it and cat the named_pipe in another terminal, I got the error

tar = tarfile.open('named_pipe', mode='w')
  File "/usr/lib/python2.7/tarfile.py", line 1695, in open
    return cls.taropen(name, mode, fileobj, **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1566, in __init__
    self.offset = self.fileobj.tell()
IOError: [Errno 29] Illegal seek

Any ideas?

score 4 · Answer 1 · answered Aug 15 '14 at 19:49

We do something similar, but from the other side. We have a web app that feeds a tarfile to the visitors browser, which means we stream it over the http session. The trick is to pass the opened and writeable file handle to the tarfile.open() call, along with the mode set to "w|". Something like this:

# assume stream is set to your pipe

with tarfile.open(name="upload.tar", 
    mode = "w|", 
    fileobj = stream, 
    encoding = 'utf-8') as out:

    out.add(file_to_add_to_tar)

python tarfile writes tar to pipe

1 Answers1