Azure Search using REST c#

Question

I am trying to run the following code:

public class Item
{
    [JsonProperty(PropertyName = "api-key")]
    public string apikey { get; set; }

}

[[some method]]{
            var url = "https://[search service name].search.windows.net/indexes/temp?api-version=2016-09-01"; 

            using (var httpClient = new HttpClient())
            {
                using (var request = new HttpRequestMessage(HttpMethod.Put,url))
                {
                    request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));                        
                    var sItem = new Item { apikey = [AzureSearchAdminKey] };
                    var tststring = JsonConvert.SerializeObject(sItem);
                    var body=new  StringContent(tststring, Encoding.UTF8,"application/json" );



                    request.Content =  body;

                    request.Method = HttpMethod.Put;

                    using (HttpResponseMessage response = httpClient.SendAsync(request).Result)
                    {                                                       
                        var stringr = response.Content.ReadAsStringAsync().Result;
                        Console.WriteLine(stringr);
                        Console.ReadLine();
                    }

                }  
            }

}

I get the following error: "Error reading JObject from JsonReader. Path '', line 0, position 0."

Could someone from search team tell me what I did wrong?

actually the error I have received was:{StatusCode: 403, ReasonPhrase: 'Forbidden', Version: 1.1, Content: System.Net.Http.StreamContent, Headers: { Pragma: no-cache request-id: ------------- elapsed-time: 49 Strict-Transport-Security: max-age=15724800; includeSubDomains Cache-Control: no-cache Date: ------ Content-Length: 0 Expires: -1 }} — Nok Dev, Nov 10 '17 at 01:44

score 0 · Answer 1 · answered Nov 10 '17 at 02:20

0

It looks like you're trying to modify your index definition, but the body of the request contains the api-key instead of the JSON for the index definition. The api-key needs to be in the request headers, not the body.

You might find it simpler to use the Azure Search .NET SDK instead of calling the REST API directly.

answered Nov 10 '17 at 02:20

Bruce Johnston

8,344
3
32
42

The reason for REST is because .net SDK does not cover everything. And I was told by a few of Microsoft engineers that REST will be updated first followed by .NET SDK. So I have added the header information and I still get the same error. I would appreciate if you could modify the code I have posted and show me what I am supposed to write. – Nok Dev Nov 11 '17 at 04:48
The .NET SDK is at feature parity with the REST API. Version 3.0.4 of the SDK matches version 2016-09-01 of the REST API, while version 4.0.1-preview of the SDK matches version 2016-09-01-Preview of the REST API. There should be no REST API feature that isn't supported by one of these two SDKs. We try very hard to update the SDK within a month of the REST API whenever possible. If you still need to use the REST API directly and you're still getting a 403 error, please double-check that you are using the right api-key. – Bruce Johnston Nov 11 '17 at 17:05
I can't find SDK doc about creating indexer, where it is easy to find under REST. Not just search but all other Azure services give REST precedence over .net SDK. They are not always up to par. I have checked multiple times to make sure I am providing the right admin key. All I am trying to do is to update index and then I am going to try one for creating an indexer. I would appreciate if you could look at the REST code or create one for yourself and see if the index properly updates. I would appreciate any working source code that connects to the search service successfully using REST in c#. – Nok Dev Nov 12 '17 at 18:44
Sorry you haven't found what you need regarding indexers in the documentation. The team is working on new content for indexers, so this should get better over time. A new C# sample using the .NET SDK was just published on GitHub. That should help you: https://github.com/Azure-Samples/search-dotnet-getting-started/tree/master/DotNetHowToIndexers If you need the reference for the .NET SDK, it is located here: https://learn.microsoft.com/dotnet/api/microsoft.azure.search?view=azure-dotnet – Bruce Johnston Nov 12 '17 at 22:18
Thank you. However as you can see there are many instances where Microsoft fails to update documents in time and lowly developers like me has to suffer or wait for a response from stackexchange or look for some blog posts. That's why I want to use REST. This problem is not only present with Search but pretty much with all Azure services. So Please help me with making successful REST query to updating indexes. It's been taking days to find out why the call keeps returning connection refused, whereas it works using SDK. Same admin key has been used. – Nok Dev Nov 14 '17 at 00:02

score 0 · Accepted Answer · edited Jul 17 '19 at 17:07

The api key should be in the HTTP header, and the index definition should be in the HTTP body.

Here's some sample code for creating a data source, index, and indexer, which reads from an Azure SQL DB and indexes its rows.

class Program
{
    static void Main(string[] args)
    {
        var searchServiceName = "[search service name]";
        var apiKey = "[api key]";

        var dataSourceName = "[data source name]";
        var indexName = "[index name]";
        var indexerName = "[indexer name]";

        var azureSqlConnectionString = "[Azure SQL connection string]";
        var azureSqlTableName = "[table name]";

        using (var httpClient = new HttpClient())
        {
            var dataSourceDefinition = AzureSqlDatasourceDefinition(azureSqlConnectionString, azureSqlTableName);
            var putDataSourceRequest = PutDataSourceRequest(searchServiceName, apiKey, dataSourceName, dataSourceDefinition);
            Console.WriteLine($"Put data source {putDataSourceRequest.RequestUri}");
            Console.WriteLine();
            var putDataSourceResponse = httpClient.SendAsync(putDataSourceRequest).Result;
            var putDataSourceResponseContent = putDataSourceResponse.Content.ReadAsStringAsync().Result;
            Console.WriteLine(putDataSourceResponseContent);
            Console.WriteLine();

            var indexDefinition = IndexDefinition();
            var putIndexRequest = PutIndexRequest(searchServiceName, apiKey, indexName, indexDefinition);
            Console.WriteLine($"Put index {putIndexRequest.RequestUri}");
            Console.WriteLine();
            var putIndexResponse = httpClient.SendAsync(putIndexRequest).Result;
            var putIndexResponseContent = putIndexResponse.Content.ReadAsStringAsync().Result;
            Console.WriteLine(putIndexResponseContent);
            Console.WriteLine();

            var indexerDefinition = IndexerDefinition(dataSourceName, indexName);
            var putIndexerRequest = PutIndexerRequest(searchServiceName, apiKey, indexerName, indexerDefinition);
            Console.WriteLine($"Put indexer {putIndexerRequest.RequestUri}");
            Console.WriteLine();
            var putIndexerResponse = httpClient.SendAsync(putIndexerRequest).Result;
            var putIndexerResponseContent = putIndexerResponse.Content.ReadAsStringAsync().Result;
            Console.WriteLine(putIndexerResponseContent);
            Console.WriteLine();

            var runIndexerRequest = RunIndexerRequest(searchServiceName, apiKey, indexerName);
            Console.WriteLine($"Run indexer {runIndexerRequest.RequestUri}");
            Console.WriteLine();
            var runIndexerResponse = httpClient.SendAsync(runIndexerRequest).Result;
            Console.WriteLine($"Success: {runIndexerResponse.IsSuccessStatusCode}");
            Console.ReadLine();
        }
    }

    static HttpRequestMessage PutDataSourceRequest(string searchServiceName, string apiKey, string dataSourceName,
        string datasourceDefinition)
    {
        var request = new HttpRequestMessage(HttpMethod.Put,
            $"https://{searchServiceName}.search.windows.net/datasources/{dataSourceName}?api-version=2016-09-01");
        request.Headers.Add("api-key", apiKey);
        var body = new StringContent(datasourceDefinition, Encoding.UTF8, "application/json");
        request.Content = body;
        return request;
    }

    static HttpRequestMessage PutIndexRequest(string searchServiceName, string apiKey, string indexName,
        string indexDefinition)
    {
        var request = new HttpRequestMessage(HttpMethod.Put,
            $"https://{searchServiceName}.search.windows.net/indexes/{indexName}?api-version=2016-09-01");
        request.Headers.Add("api-key", apiKey);
        var body = new StringContent(indexDefinition, Encoding.UTF8, "application/json");
        request.Content = body;
        return request;
    }

    static HttpRequestMessage PutIndexerRequest(string searchServiceName, string apiKey, string indexerName,
        string indexerDefinition)
    {
        var request = new HttpRequestMessage(HttpMethod.Put,
            $"https://{searchServiceName}.search.windows.net/indexers/{indexerName}?api-version=2016-09-01");
        request.Headers.Add("api-key", apiKey);
        var body = new StringContent(indexerDefinition, Encoding.UTF8, "application/json");
        request.Content = body;
        return request;
    }

    static HttpRequestMessage RunIndexerRequest(string searchServiceName, string apiKey, string indexerName)
    {
        var request = new HttpRequestMessage(HttpMethod.Post,
            $"https://{searchServiceName}.search.windows.net/indexers/{indexerName}/run?api-version=2016-09-01");
        request.Headers.Add("api-key", apiKey);
        return request;
    }

    static string AzureSqlDatasourceDefinition(string connectionString, string tableName)
    {
        return @"
{
  ""description"": ""azure sql datasource"",
  ""type"": ""azuresql"",
  ""credentials"": { ""connectionString"": """ + connectionString + @""" },
  ""container"": { ""name"": """ + tableName + @""" },
  ""dataChangeDetectionPolicy"": {
    ""@odata.type"": ""#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy"",
    ""highWaterMarkColumnName"": ""highwatermark""
  },
  ""dataDeletionDetectionPolicy"": {
    ""@odata.type"": ""#Microsoft.Azure.Search.SoftDeleteColumnDeletionDetectionPolicy"",
  ""softDeleteColumnName"": ""deleted"",
  ""softDeleteMarkerValue"": ""true""
  }
}
";
    }

    static string IndexDefinition()
    {
        return @"
{
  ""fields"": [
    {
      ""name"": ""id"",
      ""type"": ""Edm.String"",
      ""key"": true,
      ""searchable"": true,
      ""sortable"": true,
      ""retrievable"": true
    },
    {
      ""name"": ""field1"",
      ""type"": ""Edm.String"",
      ""searchable"": true,
      ""retrievable"": true
    },
    {
      ""name"": ""field3"",
      ""type"": ""Edm.Int32"",
      ""retrievable"": true
    }
  ]
}
";
    }

    static string IndexerDefinition(string dataSourceName, string indexName)
    {
        return @"
{
  ""description"": ""indexer for azure sql datasource"",
  ""dataSourceName"": """ + dataSourceName + @""",
  ""targetIndexName"": """ + indexName + @""",
  ""schedule"": { ""interval"": ""P1D"" }
}
";
    }
}

The indexer is scheduled to run once per day. You can set it to run more frequently if the data changes often, but it might affect your search throughput.

This is the table definition if you're interested

CREATE TABLE [dbo].[testtable](
    [id] [int] IDENTITY(1,1) NOT NULL,
    [field1] [nchar](10) NULL,
    [field2] [nchar](10) NULL,
    [field3] [int] NULL,
    [highwatermark] [timestamp] NOT NULL,
    [deleted] [bit] NOT NULL
) ON [PRIMARY]

INSERT INTO [dbo].[testtable] (field1, field2, field3, deleted) VALUES ('abc', 'def', 123, 0)

Thank you. That's what I was looking for. Microsoft Docs are sometimes cryptic. It shows as though "application/json" should go into the header but it really doesn't say anything other than the phrase being required. I think at least it should explicitly state which part goes to the header and which ones to the body. — Nok Dev, Nov 16 '17 at 05:26

Azure Search using REST c#

2 Answers2

Linked