Problem with __VIEWSTATE, __EVENTVALIDATION, __EVENTTARGET and scrapy & splash

Question

How do i handle __VIEWSTATE, __EVENTVALIDATION, __EVENTTARGET with scrapy/splash?

I tried with

return FormRequest.from_response(response,
    [...]
    '__VIEWSTATE': response.css(
    'input#__VIEWSTATE::attr(value)').extract_first(),

But this does not work.

Answer 1

You'll need to use a dict as the formdata keyword arg.

(I'd also recommend extracting into variables first for readability)

def parse(self, response):
    vs = response.css('input#__VIEWSTATE::attr(value)').extract_first()
    ev = # another extraction
    et = # a third extraction
    return scrapy.FormRequest.from_response(
        response,
        formdata={'__VIEWSTATE': vs,
            '__EVENTVALIDATION': ev,
            '__EVENTTARGET': et },
        callback=self.your_callback
    )

See this doc for more information.