How to convert Http url (image URL) to byte buffer or in byte?

Question

I have tried some functions to convert url image to byte buffer but it doesn't work. It shows an IOException error

String imagePath = "https://www.clearias.com/up/UPSC-Civil-Services-Mains-Exam-2018-Timetable-V1.png";

Try 1:

URL u = new URL(imagePath);
int contentLength = u.openConnection().getContentLength();
InputStream openStream = u.openStream();
byte[] binaryData = new byte[contentLength];
openStream.read(binaryData);
ByteBuffer imageBytes = ByteBuffer.wrap(openStream);

This shows error when i wrap openStream to ByteBuffer

"The method wrap(byte[]) in the type ByteBuffer is not applicable for the arguments (InputStream)"

Try 2:

    URL url = new URL(imagePath);
    ByteArrayOutputStream output = new ByteArrayOutputStream();

    try (InputStream inputStream = url.openStream()) {
        int n = 0;    
        byte[] buffer = new byte[1024];    
        while (-1 != (n = inputStream.read(buffer))) {
            output.write(buffer, 0, n);
        }
    }
    byte[] img = output.toByteArray();    
    ByteBuffer imageBytes = ByteBuffer.wrap(img);

I have also tried this function but it shows this error:

java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.clearias.com/up/UPSC-Civil-Services-Mains-Exam-2018-Timetable-V1.png

Try 3: And another is

byte[] img = Base64.encodeBase64(IOUtils.toByteArray((new URL(imagePath)).openStream()), true);

This line also gives me error

Try this https://stackoverflow.com/questions/2295221/java-net-url-read-stream-to-byte — Saif Ahmad, Jun 25 '18 at 11:15
I have tried these all functions https://stackoverflow.com/questions/2295221/java-net-url-read-stream-to-byte But it shows error. I have mentioned all errors in my question. — Umang Soni, Jun 25 '18 at 11:29
It's very weird that you'd get an HTTP 403 error, when any of us can just go visit that URL in our browser and obtain the image without a problem. Maybe this website noticed you were trying to get the image with Java, and decided it doesn't like you? — kumesana, Jun 25 '18 at 12:20
Ah, no, I'm stupid. Everyone always get a 403 when trying to access the image with Java. It's just that *Try 1* doesn't even compile in the first place. — kumesana, Jun 25 '18 at 12:26
I have also tried another url not only this but it gives same error. Please try another image and check it. — Umang Soni, Jun 25 '18 at 12:32

score 3 · Accepted Answer · answered Jun 25 '18 at 12:38

The HTTP error 403 means that the server has fully understood your intention to request this image and download it. However, it estimates that you have no right to do so, not because of a programming error, but because you shouldn't be allowed to do that.

Normally the conclusion should be that it is not possible to obtain this image from just its URL, and maybe you need to provide some form of authentication alongside the request to prove to the server that you should be allowed to get the image.

However it is clear, by just copy/pasting the URL in a browser, that in "normal" conditions the server accepts to give out the image unconditionally. The server only rejects the request when it is made from a Java program (and maybe other technologies too, I did not test.) The question remaining is, how does the server tell that the request is made by a Java program? Or, in more general terms, how does the server decide whether or not you have a right to make this request?

In theory we cannot guess everyone's intentions, but it is common for HTTP servers that want to deny requests from specific technologies, to do that based on the User-Agent HTTP header of the request. So I decided to modify the User-Agent that Java sends by default, and lie and pretend the request is done by Firefox (in a very naive way)

Here's the code:

URL url = new URL(imagePath);
ByteArrayOutputStream output = new ByteArrayOutputStream();
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent", "Firefox");

try (InputStream inputStream = conn.getInputStream()) {
  int n = 0;
  byte[] buffer = new byte[1024];
  while (-1 != (n = inputStream.read(buffer))) {
    output.write(buffer, 0, n);
  }
}
byte[] img = output.toByteArray();
ByteBuffer imageBytes = ByteBuffer.wrap(img);

It works.

score 2 · Answer 2 · answered Jun 25 '18 at 11:48

I guess there is some problem with URL.

Use Apache commons-io Apache commons-io

Below is the sample Code:

import org.apache.commons.io.IOUtils;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.util.Arrays;

public class DownloadImage {
    public static void main(String[] args) {

        try {
            URL url = new URL("https://en.wikipedia.org/wiki/Car#/media/File:401_Gridlock.jpg");
            System.out.println(Arrays.toString(downloadFile(url)));
        } catch (MalformedURLException e) {
            e.printStackTrace();
        }
    }

    private static byte[] downloadFile(URL url) {
        try {
            URLConnection conn = url.openConnection();
            conn.setConnectTimeout(5000);
            conn.setReadTimeout(5000);
            conn.connect();

            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            IOUtils.copy(conn.getInputStream(), baos);

            return baos.toByteArray();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
}

OUTPUT: [..., 47, 98, 111, 100, 121, 62, 10, 60, 47, 104, 116, 109, 108, 62, 10]

How to convert Http url (image URL) to byte buffer or in byte?

2 Answers2