Is there a way to write single prepare function for all the KPIs in Gramex

Question

I have a common function which needs to be evaluated before executing all the KPIs. So I wrote a prepare function but prepare function is calling for all the KPIs separately. Instead I want a prepare function to be executed once for the each endpoint. My endpoint is like this

project-test:
    pattern: /$YAMLURL/test
    handler: FormHandler
    kwargs:
      auth: &AUTH
        login_url: /$YAMLURL/login
      kpi1:
        prepare: validations.validate_request(args, handler)
        url: $CONNECTION_STRING
        queryfunction: queries.query1(args) 
      kpi2:
        prepare: validations.validate_request(args, handler)
        url: $CONNECTION_STRING
        queryfunction: queries.query2(args)
      modify: project.refactor(data, handler)

I tried to giving prepare function in the kwargs but getting

AttributeError: 'str' object has no attribute 'get'

score 0 · Accepted Answer · answered Jun 22 '21 at 08:17

The approach you shared is correct. The error may be in one of the custom functions, e.g. validations.validate_request, queries.query* or project.refactor

To test this, I created the following data1.csv

a,b
1,2
3,4

... and data2.csv

a,b
4,5
6,7

This is my gramex.yaml

url:
  project-test:
      pattern: /$YAMLURL/test
      handler: FormHandler
      kwargs:
        kpi1:
          prepare: validations.validate_request(args, handler)
          url: data1.csv
        kpi2:
          prepare: validations.validate_request(args, handler)
          url: data2.csv

... and this is my validations.py:

def validate_request(args, handler):
    print(args, handler)
    args['_c'] = ['a']    # Filter column 'a' alone

When I visit /test, the output only shows column 'a':

{"kpi1":[{"a":1},{"a":3}],"kpi2":[{"a":4},{"a":6}]}

... and the command prompt shows that the prepare function is called once for each dataset:

{'_limit': [10000]} <gramex.services.FormHandler object at 0x000001B1C14A43C8>
{'_limit': [10000]} <gramex.services.FormHandler object at 0x000001B1C14A43C8>

Perhaps you could share the full error message about where the AttributeError appeared? That might help.

The endpoint which I wrote is working fine @S Anand, but I want to modify it something like below `url: project-test: pattern: /$YAMLURL/test handler: FormHandler kwargs: prepare: validations.validate_request(args, handler) kpi1: url: data1.csv kpi2: url: data2.csv` one prepare function that should apply for all the KPI's inside the endpoint which is causing the above error — Manoj, Jun 24 '21 at 07:51
Ah, I understand. No, we can't do that. You can re-use the same function but have to specify it independently for each dataset. That's because when there are multiple datasets, Gramex doesn't know whether `prepare:` is the name of a dataset/KPI, or a function to call. — S Anand, Jun 25 '21 at 14:31

score 0 · Answer 2 · answered Jun 22 '21 at 13:15

0

I guess this is more of a prepare function issue. Could you please check if you are trying to access a string using its key. Just a quick hypothesis - check if any of your dict is in json/ string format.

answered Jun 22 '21 at 13:15

Vinay Ranjan

294
3
14

Is there a way to write single prepare function for all the KPIs in Gramex

2 Answers2