6

I'm trying to download a file using HTTP::UserAgent, but all my attempts thus far have ended with the following error.

Malformed UTF-8

I've tried to use the getstore sub, exported in the :simple subset.

getstore($upstream ~ %module<link>, $dist.absolute);

And using the HTTP::UserAgent class directly.

my $ua = HTTP::UserAgent.new;

$dist.spurt: $ua.get($upstream ~ %module<link>).content;

While looking through the source code of the module, I found a :bin argument for .get, so naturally I've tried that as well.

$dist.spurt: $ua.get($upstream ~ %module<link>, :bin).content;

But even with the :bin argument I get the Malformed UTF-8 error.

How would I go about downloading a binary file using HTTP::UserAgent?

EDIT: To give some more context, the $dist is just another IO::File object, created with tempdir.IO.add("dist.tar.gz"). I've also tried to .open this file and .spurt to the IO::Handle instead of the IO::File, but this returned the same error. Adding :bin to the $dist.open call does also not seem to have an effect.

with ($dist.open(:w)) {
    LEAVE { .close }
    .spurt: $ua.get($upstream ~ %module<link>, :bin).content;
}

The URL I'm working with in this particular case is https://cpan.metacpan.org/authors/id/V/VR/VRURG/Perl6/OO-Plugin-v0.0.5.tar.gz, but it doesn't seem to be limited to this URL only.

Tyil
  • 1,797
  • 9
  • 14
  • 1
    Are you sure you are receiving what you expected (can't tell from example -- no url)? Is `$dist` a handle opened to an incorrect encoding for binary data (can't tell from example -- lacking code context)? https://github.com/sergot/http-useragent/blob/master/t/220-binary-content.t – ugexe Jan 17 '19 at 22:51
  • 4
    FWIW, `getstore('https://cpan.metacpan.org/authors/id/V/VR/VRURG/Perl6/OO-Plugin-v0.0.5.tar.gz', 'OO-Plugin-v0.0.5.tar.gz');` WJFFM. Maybe you should provide a runnable minimal test case so we can see where it goes wrong. And also check which version of `HTTP::UserAgent` you are using – I used 1.1.46. – mscha Jan 18 '19 at 14:12
  • Argh, after updating the module (`zef upgrade HTTP::UserAgent`) it does indeed work. I hadn't thought about trying that, thanks for the tip! Should I close or delete this question, as it seems it's not a relevant question to anyone who updates their packages from time to time? – Tyil Jan 20 '19 at 08:13

1 Answers1

2

try the :bin option for the spurt routine. This works for me (with http)

'/tmp/test.tar.gz'.IO.spurt( $ua.get('http://cpan.metacpan.org/authors/id/V/VR/VRURG/Perl6/OO-Plugin-v0.0.5.tar.gz').content, :bin);
LuVa
  • 2,288
  • 2
  • 10
  • 20