How can I create a python webhook sender app?

Question

This is a follow up question to this post.

I have a data warehouse table exposed via xxx.com\data API endpoint

I have been querying this table using the following code and parsing it into a dataframe as follows;

import requests
import json
import http.client
import pandas as pd

url = "xxx.com\data?q=Active%20%3D1%20and%20LATITUDE%20%3D%20%20%220.000000%22%20and%20LONGITUDE%20%3D%20%220.000000%22&pageSize =300"
payload = {}
headers = {'Authorization': access_token}
response = requests.request("GET", url, headers=headers, data = payload)
j=json.loads(response.text.encode('utf8'))
df = pd.json_normalize(j['DataSet'])

The warehouse table gets periodically updated and I am required to create a webhook to be listened to by the following Azure httptrigger;

import logging
import os
import json
import pandas as pd
import azure.functions as func

def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')
    d={
    'Date' :['2016-10-30','2016-10-30','2016-11-01','2016-10-30'],
    'Time':['09:58:11', '10:05:34', '10:07:57', '11:15:32'],
    'Transaction':[2,3,1,1]
    }
    df=pd.DataFrame(d, columns=['Date','Time','Transaction'])
    output = df.to_csv (index_label="idx", encoding = "utf-8")
 
return func.HttpResponse(output)

When run,the httptrigger successfully listens to the following webhooker sender which I have created and am running locally on my disk.

    import logging
    import os
    import json
    import pandas as pd

data={'Lat': '0.000000',
   'Long': '0.000000',
   'Status': '1', 'Channel URL':"xxx.com\data"}

webhook_url="http://localhost:7071/api/HttpTrigger1"

r=requests.post(webhook_url, headers={'Content-Type':'application/json'}, data =json.dumps(l))

My question is;

1. How can I deploy the webhook sender to the cloud as an app so that every time "xxx.com\data" is updated with Lat==0,Long=00 and Status=1, a message is send to my webhook listener?

The app can either be Azure/Flask/postman or any other python based webhook builder.

Answer 1

A simple approach can be to wrap your sender code into a Timer Trigger Function which would poll your xxx.com\data at every x seconds (or whatever frequency you decide) and call your webhook (another http triggered function) if there is any change.

{
    "name": "mytimer",
    "type": "timerTrigger",
    "direction": "in",
    "schedule": "0 */5 * * * *"
}

import datetime
import logging
import os
import json
import pandas as pd

import azure.functions as func


def main(mytimer: func.TimerRequest) -> None:
    utc_timestamp = datetime.datetime.utcnow().replace(
        tzinfo=datetime.timezone.utc).isoformat()

    if mytimer.past_due:
        logging.warn('The timer is past due!')
 
    # "xxx.com\data" polling in real scenario
    l={'Lat': '0.000000',
          'Long': '0.000000',
          'Status': '1', 
          'Channel URL':"xxx.com\data"}

    webhook_url="{function app base url}/api/HttpTrigger1"

    r=requests.post(webhook_url, headers={'Content-Type':'application/json'}, data =json.dumps(l))

    logging.info('Python timer trigger function ran at %s', utc_timestamp)

At the end of the day, you can deploy both your webhook function (http trigger) and the sender (timer trigger polling) into a Function app.

You can also think of getting rid of the webhook function altogether (to save one intermediate hop) and do your stuffs into the same timer triggered function.

Answer 2

1. You currently have some polling logic (under querying this table using the following code). If you want to move that to "Cloud" then create a TimerTrigger function, and put all your poller code in it.

2. If you want to leave that poller code untouched, but want to call some code in "cloud" whenever poller detects a change (updated with Lat==0,Long=00 and Status=1), then you can create an HTTPTrigger function and invoke it from poller whenever it detects the change.

---

Confusing part is this: How do you detect this change today? Where is the poller code hosted and how often is it executed?

If data in DB is changing then only ways you can execute "some code" whenever the data changes is:

1. poll the DB periodically, say every 1 minute and if tehre is a change execute "some code" OR
2. some feature of this DB allows you to configure a REST API (HTTP Webhook) that is called by the DB whenever there is a change. Implement a REST API (e.g. as an HttpTrigger function) and put that "some code" that you want executed inside it. Now whenever there is a change the DB calls your webhook/REST-API and "some code" is executed.

and the way to read it is to call a REST API (xxx.com/data?q=...) then the only ways you can detect