2

I'm using WWW::Mechanize to fetch a web page that includes a Google Maps widget that receives constant data from a single response of type text/event-stream.

That kind of response is like a never ending response from the server that constantly returns updated data for the widget to work.

I'm trying to find out how to read that exact response from Perl. Using something like:

my $mech = WWW::Mechanize->new;

# Do some normal GET and POST requests to authenticate and set cookies for the session

# Now try to get that text/event-stream response

$mech->get('https://some.domain.com/event_stream_page');

But that doesn't work because the response never ends.

How can I make that request and start reading the response and do something with that data every time the server updates the stream?

Francisco Zarabozo
  • 3,676
  • 2
  • 28
  • 54
  • Your target website seems to be private. Can you find a public website that you can use to reproduce the problem so we can try? I agree that you're on the right path with handler, but I cannot debug without a working endpoint. – simbabque Jun 11 '16 at 09:25
  • @simbabque, that is a completely fake domain. – oldtechaa Jun 12 '16 at 01:01
  • @oldtechaa domain.com is not fake, it does exist. But the OP used it (falsely) to mask their their real domain. People usually do that when their stuff is in a private network or they do not want to disclose it because it's $work. That's why I asked for an example with the same behavior that's accessible publicly. The correct way to give an example domain that's obviously false (not fake) is _http://example.org_. – simbabque Jun 12 '16 at 18:23

1 Answers1

4

Found a way to do this. Using a handler from LWP, from which WWW::Mechanize inherits:

$mech->add_handler (
    'response_data',
    sub {
        my ($response, $ua, $h, $data) = @_;
        # Your chunk of response is now in $data, do what you need
        # If you plan on reading an infinite stream, it's a good idea to clean the response so it doesn't grow infinitely too!
        $response->content(undef);
        # Important to return a true value if you want to keep reading the response!
        return 1;
    },
);
Francisco Zarabozo
  • 3,676
  • 2
  • 28
  • 54