How to add Primary key of Azure Cosmos DB as search parameter in Azure search Service

Question

I am using Azure search service to search for the documents in my Azure CosmosDB Account.

Using the portal, I have created an Azure search service and given my existing CosmosDB as data source.

Following is the sample document stored in CosmosDB

{
    "id": "Engine",     
    "Sub-Components Price": [
        //Price list
    ],
    "Sub-Components": [
        "List of sub components here"
    ],      
    "Brand": "Brand Name here",     
}

When the CosmosDB containing above document is given as data source to the Azure search, id field is internally converted to some string (Automatic Indexing may be).

I am able to set other fields like Sub-Components, Brand as search parameter (using C#) and search only those specific fields. I want to apply the same to id field also. But id field encrypted/encoded to some other string as follows:

{
    "id": "UkVRX1ZFSF9DVVNUX0",     
    "Sub-Components Price": [
        //Price list
    ],
    "Sub-Components": [
        "List of sub components here"
    ],      
    "Brand": "Brand Name here",     
}

How to retrieve my original id and set it as search parameter?

Thanks in advance !!

`UkVRX1ZFSF9DVVNUX0` when I base64 decode it, I get `REQ_VEH_CUST_`. Does this decoded value make sense in context of your application? — Gaurav Mantri, Feb 07 '19 at 08:42
@GauravMantri Thank u so much. Yes it makes sense. That is what i needed. Can you share the snippet you decoded? — code-geek, Feb 07 '19 at 08:50
I used https://www.base64decode.org to decode `UkVRX1ZFSF9DVVNUX0`. — Gaurav Mantri, Feb 07 '19 at 08:52
But I would like to add this as search parameter to azure search. Will this decoded string be taken as search paramtere and gives proper results? — code-geek, Feb 07 '19 at 08:59
Endcode value you have like `System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes("REQ_VEH_CUST_"))` & use as search parameter, it should work since base64 encode is stored. — Pranav Singh, Feb 07 '19 at 09:11
`But I would like to add this as search parameter to azure search. Will this decoded string be taken as search paramtere and gives proper results?` - No it won't. You will need to base64 encode your search string and then send it to Azure Search. — Gaurav Mantri, Feb 07 '19 at 09:15

Gaurav Mantri · Accepted Answer · 2019-02-07T10:59:26.387

UkVRX1ZFSF9DVVNUX0 is a base64 encoded string and when you decode it you get REQ_VEH_CUST_.

Why the values are converting to base64 encoded string?

Please check the indexer details. Since there're limitations on the value in the key field (https://learn.microsoft.com/en-us/rest/api/searchservice/naming-rules - See Document Key), there's probably a setting in indexer (look under field mapping section and then check if base64Encode mapping function is applied on the id field mapping) which is converting and storing the value as base64 encoded string.

If you're confident that the value of id in source (i.e. key field in index) will not violate the rule for key field value, you can remove this base64encode mapping function, save the indexer, reset the indexer and run it again. This time the data will be saved as it is in the source.

score 1 · Answer 2 · answered Feb 07 '19 at 09:27

Based on comment of @GauravMantri comment since you id is base 64 encoded before stored so either you can remove encoding while storing Id if that data is unique key in itself without encoding as suggested.

Alternatively you can endcode value you already have like System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes("IdBeforeEncodingAsString")) & use as search parameter, it should work since base64 encoded value of this string is stored as Id.

How to add Primary key of Azure Cosmos DB as search parameter in Azure search Service

2 Answers2