3

I'm trying to collect similar images of an image using Bing Visual Search API. I have the following code snippet and I'm supplying an insightsToken of this image to the API endpoint.


import requests
import json

BASE_URI = "https://api.bing.microsoft.com/v7.0/images/visualsearch"

SUBSCRIPTION_KEY = ''

HEADERS = {'Ocp-Apim-Subscription-Key': SUBSCRIPTION_KEY}

insightsToken = 'ccid_twIfDfmx*cp_BF680FB5127A96880FCA7D76CC402B60*mid_D6663701CA72F63CF255CBF7EB4998ED50CAABC8*simid_608045920540183139*thid_OIP.twIfDfmxxN4umi!_-sacdNAHaFo'

formData = '{"imageInfo":{"imageInsightsToken":"' + insightsToken + '", }, "knowledgeRequest":{"invokedSkills": ["SimilarImages"]}}'

file = {'knowledgeRequest': (None, formData)}

def main():

    try:
        response = requests.post(BASE_URI, headers=HEADERS, files=file)
        response.raise_for_status()
        print_json(response.json())

    except Exception as ex:
        raise ex


def print_json(obj):
    """Print the object as json"""
    print(json.dumps(obj, sort_keys=True, indent=2, separators=(',', ': ')))

Take a look at the first few images that bing.com/visualsearch shows for that image. I tried both uploading an image and providing a link of an image. The results were the identical.

enter image description here

Now take a look at the first few images that were collected with the code snippet via API endpoint.

enter image description here

For some reason, bing.com/visualsearch results are much better and precise than Bing Visual Search API. I know that Bing Image Search API has the market parameter (based on IP, and your cookies and browsing history). However, even when I tried to change my IP and use incognito mode in a browser, I still got identical results when using bing.com/visualsearch. I also tried to provide params = (('mkt', 'en-us')) to the API endpoint, but the results were still almost identical. So I think this parameter doesn't affect the outcome to such a large extent. So I don't supply anything, as I think this makes the result close to the web search.

Still, why are the results so different? Is Bing using one API version for their web search and another version for their cloud services?

VLAZ
  • 26,331
  • 9
  • 49
  • 67
lowlypalace
  • 137
  • 2
  • 9

1 Answers1

2

Is Bing using one API version for their web search and another version for their cloud services?

I can't speak for the internal workings of microsoft, but it appears that the endpoint API for bing.com/visualsearch and your code are different.

To determine an endpoint called from the browser, open inspect element in firefox on the image search page after uploading your image, and reload. In the network tab, you will see lots of requests. Search for api and for me a pointer to an ad service comes up, along with the endpoint for the api service thats used to find similar images.

enter image description here

The url used by microsoft visual search is https://www.bing.com/images/api/custom/knowledge Instead of your https://api.bing.microsoft.com/v7.0/images/visualsearch

In that same inspect element window, you can navigate to the Request tab within the networking tab and view the post request. The form used by the browser, surprisingly, is perfectly compatible with your code. I copied your code onto my machine with my own azure key and replaced the formData variable with the browsers form. You should try it yourself and judge the results. For me however, the results still weren't quite the same, although they seemed objectively closer, with some images overlapping between the browser vs python api results. The form my browser sent was this:

{
   "imageInfo":{
      "imageInsightsToken":"bcid_Tq7w6Pw5It0DxQ*ccid_rvDo/Dki",
      "source":"Gallery"
   },
   "knowledgeRequest":{
      "invokedSkills":[
         "ImageById",
         "BestRepresentativeQuery",
         "Offline",
         "ObjectDetection",
         "OCR",
         "EntityLinkingFace",
         "EntityLinkingDog",
         "EntityLinkingAnimal",
         "EntityLinkingPlant",
         "EntityLinkingLandmark",
         "EntityLinkingFood",
         "EntityLinkingBook",
         "SimilarProducts",
         "SimilarImages",
         "RelatedSearches",
         "ShoppingSources",
         "PagesIncluding",
         "TextAds",
         "ProductAds",
         "SponsoredAds",
         "Annotation",
         "Recipes",
         "Travel"
      ],
      "invokedSkillsRequestData":{
         "adsRequest":{
            "textRequest":{
               "mainlineAdsMaxCount":2
            }
         }
      },
      "index":1
   }
}

So in addition to similar images, the webpage also invoked a number of other skills including Object Detection, and BestRepresentativeQuery.

In summary, it appears that the endpoints used are different, and only a microsoft employee could tell you if they point to the same internal api. However, refining the form submitted in your post request to match what the browser uses will give you similar results to the official image search webpage.

Andrew James
  • 372
  • 1
  • 8