0

I am trying to filter the content from a URL Web API and I am using a GET method to obtain the complete data set, then I apply some filters to that response and I get my desired results, but the complete process of retrieval - filter - display results takes around 3-5 mins which is too much waiting time for users. I want to apply a POST method to filter directly from the URL request instead of retrieving the complete data set, in that way, I will get rid of my custom filters and greatly reduce the waiting time. How can I achieve this?

This is the current code I have:

from django.shortcuts import render
from django.http import JsonResponse
from rest_framework.views import APIView
from rest_framework.response import Response
from collections import Counter
from datetime import datetime, timedelta
import json, urllib.request, dateutil.parser, urllib.parse


class ChartData(APIView) 
    def get(self, request,format=None): 


# Request access to the PO database and parse the JSON object
with urllib.request.urlopen(
   "http://10.21.200.98:8081/T/ansdb/api/rows/PO/tickets?User_0001=Pat%20Trevor",
            timeout=15) as url:
        complete_data_user_0001 = json.loads(url.read().decode())

     # Custom filter to the JSON response
     # Count the number of times the user has created a PO between two given dates where the PO is not equal to N/A values
     Counter([k['user_id'] for k in complete_data_user_0001 if
                   start_date < dateutil.parser.parse(
                       k.get('DateTime')) < end_date and
                                  k['PO_value'] != 'N/A'])
    return Response(data)

The filters that I would like to apply with a POST method are:

{
  "filter": {
    "filters": [
      {
        "field": "CreatedOnDate",
        "operator": "gte",
        "value": "2017-05-31 00:00:00"
      },
      {
        "field": "CreatedOnDate",
        "operator": "lte",
        "value": "2017-06-04 00:00:00"
      },
      {
        "field": "Triage_Subcategory",
        "operator": "neq",
        "value": "N/A"
      }
    ],
    "logic": "and"
  }
}

The structure of the JSON object is:

[{
user_id : 0001
CreatedOn: "2017-02-16 15:54:48",
Problem: "AVAILABILILTY",
VIP: "YES",
PO_value: N/A 
},
{
user_id : 0001
CreatedOn: "2017-01-10 18:14:28",
Problem: "AVAILABILILTY",
VIP: "YES",
PO_value: 00098324 
},
...
}]

Any suggestion, approach or piece of code is appreciated.

abautista
  • 2,410
  • 5
  • 41
  • 72
  • Can you show your API view function. Tell how you use queryset to obtain the results ? – Raja Simon Jun 13 '17 at 06:19
  • Hello Raja, what do you mean by queryset? I obtain my results based on the web url api but I cannot filter it. I updated the code but I am not sure if that info gives you any further clue. Let me know if more details are needed. – abautista Jun 13 '17 at 06:26
  • I have mistaken wrongly. I thought you have created a web API by yourself. You have mention about `POST` method, Is your API provider support POST method call ? – Raja Simon Jun 13 '17 at 06:27
  • Yes, the API provider supports the POST method and I was able to confirm this by using the mentioned filter in PostMan. I was able to retrieve the filtered data with that tool. – abautista Jun 13 '17 at 06:29
  • Okay. That's great news. Now what's stopping you to get the filtered result ? – Raja Simon Jun 13 '17 at 06:29
  • The Python syntax is stopping me, I do not know how to implement the mentioned filter into the correct syntax, so I can retrieve the filtered data. – abautista Jun 13 '17 at 06:32
  • 1
    Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/146487/discussion-between-raja-simon-and-alejandro-ramos). – Raja Simon Jun 13 '17 at 06:33
  • Indeed, I already moved to the chat. – abautista Jun 13 '17 at 06:38

1 Answers1

0

You have to create POST method and use the urllib post call to your API.

class ChartData(APIView) 
    def post(self, request,format=None):
        ---
        # Your post call to API

Alternatively you can use requests library to make the POST call

Raja Simon
  • 10,126
  • 5
  • 43
  • 74
  • Hello Raja, at the end I was not able to complete the code that you provided in your last post. is there any chance you can help me on this? – abautista Jul 25 '17 at 06:13
  • Please come to [chat](https://chat.stackoverflow.com/rooms/146487/discussion-between-raja-simon-and-alejandro-ramos) we continue discuss – Raja Simon Jul 25 '17 at 06:19
  • i tried to implement the requests library to filter my post request but it just seems that the filter that I created is being ignored. This is my latest post https://stackoverflow.com/questions/45201628/how-to-make-a-post-request-with-the-python-requests-library/45291125#45291125 – abautista Jul 25 '17 at 06:19
  • any other room available? – abautista Jul 25 '17 at 06:21
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/150048/discussion-between-raja-simon-and-alejandro-ramos). – Raja Simon Jul 25 '17 at 06:21