python : post data within stringIO through poster?

Question

params = {'file': open("test.txt", "rb"), 'name': 'upload test'}
datagen, headers = poster.encode.multipart_encode(params)
request = urllib2.Request(upload_url, datagen, headers)
result = urllib2.urlopen(request)

I use poster library to POST for HTTP. It works well. I'm satisfied with that.

But I want to try something. As you see above, to send file data, I have to OPEN a file. But is there any way not to make a real file to do that? We can use a STREAM, like StringIO, to deal with data like a file, right? But, I don't know about poster deeply. So, I want to know the method to use a STREAM with poster.

ADDED

Actually, I tried to POST image data. I wrote this below

from PyQt4 import QtCore, QtGui
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
import urllib2, os

register_openers()
app = QtGui.QApplication(sys.argv)
pixmap = QtGui.QPixmap("c:/test_img.png")
byte_array = QtCore.QByteArray()
buffer = QtCore.QBuffer(byte_array)
buffer.open(QtCore.QIODevice.WriteOnly)
pixmap.save(buffer, "PNG")
from cStringIO import StringIO
datagen, headers = multipart_encode({"image": StringIO(str(byte_array.toBase64()))})
request = urllib2.Request(upload_url, datagen, headers)
_rnt = urllib2.urlopen(request)

But, I receive this error:

Traceback (most recent call last):
  File "<pyshell#25>", line 1, in <module>
    _rnt = urllib2.urlopen(request)
  File "C:\Python26\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python26\lib\urllib2.py", line 397, in open
    response = meth(req, response)
  File "C:\Python26\lib\urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python26\lib\urllib2.py", line 435, in error
    return self._call_chain(*args)
  File "C:\Python26\lib\urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "C:\Python26\lib\urllib2.py", line 518, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 500: Internal Server Error

This isn't relevant to your problem, but you realize that your code never closes text.txt, right? In CPython, it'll _probably_ get closed automatically as soon as `params` and `datagen` both go away, but no guarantees; in other implementations, it's even worse. — abarnert, Nov 13 '12 at 02:07
Again, possibly not related to your new problem, but: I don't think you need to base64 the data, since they're already going to be properly encoded in the MIME body—you'll need to do an extra base64 decode on the other side. Also, why are you creating a pixmap from a PNG file and then saving it to a buffer as a PNG instead of just reading the PNG file (or just passing it to `poster`)? — abarnert, Nov 13 '12 at 19:38
The error you're getting is "HTTP Error 500: Internal Server Error". That means a bug in the web server or, more likely, the back-end script it runs for your URL. If that URL is normally reliable, it could mean you've send garbage so far outside the realms of what anyone expected that you found a bug nobody ever tested for. Try saving the buffer to make sure it really is a PNG, saving the output of `datagen` and `headers` to make sure they're reasonable, and doing a simpler test without all the `Qt` stuff where you just send a `StringIO` with the data you want to end up with. — abarnert, Nov 13 '12 at 19:41
Is the `upload_url` a publicly-accessible service? Or, if not, is it something you have the source to and can post somewhere (along with setup instructions)? That would let other people help you test what's going wrong. — abarnert, Nov 13 '12 at 19:44
@abarnert Surely, I can save to PNG file and send it to poster. But, I hope to reduce processing time, so tried like that. :) And, I will check `upload_url`, but it' not my duty so it will take somehow. — Hyun-geun Kim, Nov 14 '12 at 00:49
Well, if one of your coworkers is responsible for the web service you're trying to upload to, submit a bug against his work—it's always wrong for a service to return 500 just because it got bad input. As for "reduce processing time"—it obviously takes longer to read a PNG, convert it into a `QPixmap`, render that `QPixmap` to a buffer, then encode and send that buffer than it does to just encode and send the PNG file directly. — abarnert, Nov 14 '12 at 01:05

score 4 · Accepted Answer · edited Apr 07 '16 at 17:50

The file param is where you pass the file object. So, what happens if you pass a file-like object instead?

>>> params = {'file': cStringIO.StringIO('upload test data'), 'name': 'upload test'}
>>> datagen, headers = poster.encode.multipart_encode(params)
>>> headers
{'Content-Length': '317', 'Content-Type': 'multipart/form-data; boundary=0c56082b1e134424a918b2b083391467'}

Looks like it worked.

What does the documentation say?

Values are either strings parameter values, or file-like objects to use as the parameter value. The file-like objects must support .read() and either .fileno() or both .seek() and .tell().

So, you can use StringIO objects because they support seek() and tell().

But you don't have to. You should be able to just use the raw string. Let's try it and see:

>>> params = {'file': 'upload test data', 'name': 'upload test'}
>>> datagen, headers = poster.encode.multipart_encode(params)
>>> headers
{'Content-Length': '317', 'Content-Type': 'multipart/form-data; boundary=0c56082b1e134424a918b2b083391467'}

Look at that, the documentation is correct.

@pydsigner: Well, the library is described as incomplete and one version away from having a stable API, and the documentation comes with a warning saying it may not match the current version… — abarnert, Nov 13 '12 at 19:31

python : post data within stringIO through poster?

ADDED

1 Answers1

Linked