Google cloud vision not accepting base64 encoded images python

Question

I'm having a problem with base64 encoded images sent to Google Cloud Vision. Funny thing is that if I send the image via URI, it works fine, so I suspect there is something wrong the way I'm encoding.

Here's the deal:

from google.cloud import vision
import base64
client = vision.ImageAnnotatorClient()
image_path ='8720911950_91828a2aeb_b.jpg'
with open(image_path, 'rb') as image:
    image_content = image.read()
    content = base64.b64encode(image_content)   
    response = client.annotate_image({'image': {'content': content}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})
    print(response)

The response I get always is:

error {
  code: 3
  message: "Bad image data."
}

If I try using URI instead:

response = client.annotate_image({'image': {'source': {'image_uri': 'https://farm8.staticflickr.com/7408/8720911950_91828a2aeb_b.jpg'}}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})

Response is ok...

label_annotations {
  mid: "/m/0168g6"
  description: "factory"
  score: 0.7942917943000793
}
label_annotations {
  mid: "/m/03rnh"
  description: "industry"
  score: 0.7761002779006958
}

I've followed the recommended way to encode from Google

Any idea what is wrong here?

@Leon I get this "TypeError: '/9j/4AAQSkZJRgABAQEA8ADwAAD/4gJASUNDX1BST0ZJTEUAAQEAAAIwQURCRQIQAABtbnRyUkdCIFhZWiAHzwAGAAMAAAAAAAB has type str, but expected one of: bytes " — AlejandroVK, Aug 15 '17 at 15:29
Have you tried the other way of doing it as in [this example](https://cloud.google.com/vision/docs/reference/libraries#client-libraries-usage-python), where encoding is obviously done automatically inside the `types.Image()` constructor. — Leon, Aug 15 '17 at 15:38
@Leon I don't want to use what they call shortcuts, I want to use the annotator client so I select what type of analysis, and more important, send more than one image per request — AlejandroVK, Aug 15 '17 at 15:42
My guess is that base64-encoding is performed by `vision.ImageAnnotatorClient.annotate_image()` automatically. [Their documentation on base64 encoding](https://cloud.google.com/vision/docs/base64) is only for the case when you create and send the HTTP request on your own. So try eliminating base64 encoding: `content = image.read()` — Leon, Aug 15 '17 at 15:46
@Leon I used a very similar approach with Google Cloud Vision not v1 and it worked fine... — AlejandroVK, Aug 15 '17 at 15:50
@Leon That solved the issue! I understood this wrong from docs https://googlecloudplatform.github.io/google-cloud-python/stable/vision/gapic/types.html#google.cloud.vision_v1.types.Image.content...looks like if no JSON is going to be used then pure binary representation can be used — AlejandroVK, Aug 15 '17 at 15:54

score 10 · Accepted Answer · answered Aug 15 '17 at 15:54

10

I don't have any experience with Google Cloud Vision, however after looking at their documentation and examples, my feeling is that the linked documentation page about base64 encoding of image data is for the case when you create and send the HTTP requests on your own, without using vision.ImageAnnotatorClient. The latter seems to encode the image data automatically, hence in your example double encoding is applied. Therefore I believe that you should remove the encoding step from your code:

from google.cloud import vision
import base64
client = vision.ImageAnnotatorClient()
image_path ='8720911950_91828a2aeb_b.jpg'
with open(image_path, 'rb') as image:
    content = image.read()
    response = client.annotate_image({'image': {'content': content}, 'features': [{'type': vision.enums.Feature.Type.LABEL_DETECTION}],})
    print(response)

answered Aug 15 '17 at 15:54

Leon

31,443
4
72
97

correct! you got it right :) More info here, for the curious minds https://googlecloudplatform.github.io/google-cloud-python/stable/vision/gapic/types.html#google.cloud.vision_v1.types.Image.content – AlejandroVK Aug 15 '17 at 16:22
@AlejandroVK It's a bit late but I'm trying to do the same thing and I was looking how to do it. The link that you gave is broken. Can you refresh it or share how can I encode the image to be used in Vision API? – Aka Jan 24 '18 at 04:39
@Aka let me know if this helps https://gist.github.com/internetmosquito/b1c1f00ac99d9aa1f79cfdc21be36b5b – AlejandroVK Jan 24 '18 at 16:17
@AlejandroVK Thanks. – Aka Jan 26 '18 at 00:50
AttributeError: module 'google.cloud.vision' has no attribute 'enums' how can i get rid of this error? – KrisH Jodu Oct 21 '21 at 01:56

score 4 · Answer 2 · edited Jan 27 '21 at 12:00

Well, if you still want to use base64 encoded image data, you will have to convert it into byte array using module before sending request to annotate image. This base64 to bytearray should be used when creating API or when you are receiving input in the form of encoded data without actual path/url. Otherwise, use as it is by providing path or url as pointed out by @Leon.

import binascii
content = binascii.a2b_base64(base64_encoded_image_data)

pass this content as value for content argument in annotate_image method. Then, you will get a correct response.

Google cloud vision not accepting base64 encoded images python

2 Answers2

Linked

Related