1

I am trying to get data from Pendo through REST API using Azure Data Factory. The PYTHON code for this activity is 

import requests
url = "https://app.pendo.io/api/v1/aggregation"
data = "{\"response\":{\"mimeType\":\"application/json\"},\"request\":{\"pipeline\":[{\"source\":{\"guideEvents\":null,\"timeSeries\":{\"first\":\"1506977216000\",\"count\":-10,\"period\":\"dayRange\"}}}]}}"
headers = {
    'x-pendo-integration-key': "[add_your_pendo_int_key_here]",
    'content-type': "application/json"
}
response = requests.post(url, data = data, headers = headers)

How do I define the data part of the code through Azure Data Factory? I've put the content-type and x-pendo-integration-key in the additional headers.

Mohit Verma
  • 5,140
  • 2
  • 12
  • 27
Joe
  • 35
  • 2
  • 6

2 Answers2

1

Problem solved. The data part of the PYTHON code(with out the \) is put in the Request Body of Copy Activity in Azure Data Factory and the Request Method is POST.

Joe
  • 35
  • 2
  • 6
0

You can use HTTP Connector to retrieve data from HTTP endpoint. This HTTP connector is supported for the following activities:

  • Copy activity with supported source/sink matrix
  • Lookup activity You can copy data from an HTTP source to any supported sink data store. For a list of data stores that Copy Activity supports as sources and sinks, see Supported data stores and formats.

Note: To test an HTTP request for data retrieval before you configure the HTTP connector in Data Factory, learn about the API specification for header and body requirements. You can use tools like Postman or a web browser to validate.

https://learn.microsoft.com/en-us/azure/data-factory/connector-http

Here is a sample Custom activity for POST:

{
    "name": "HttpSourceDataInput",
    "properties": {
        "type": "HttpFile",
        "linkedServiceName": {
            "referenceName": "<HTTP linked service name>",
            "type": "LinkedServiceReference"
        },
        "typeProperties": {
            "relativeUrl": "<relative url>",
            "requestMethod": "Post",
            "requestBody": "<body for POST HTTP request>"
        }
    }
}

You can check ADF related samples here.Also find this for additional reference. Hope it helps.

Mohit Verma
  • 5,140
  • 2
  • 12
  • 27
  • Thanks for the response. ADF now have connector for REST API. You just need to put the headers in and it will connect to the end point. I was questioning how to deal with the aggregations specified in the data section of the Python code. – Joe Sep 23 '19 at 22:01