Capture flask request when entering and leaving the application

Question

I'm struggling a bit with writing decorators to capture and modify requests within my flask application once the requests enters the application context and leaves the application context.

My function in a flask application looks like below. I want the control of requests at two points to add custom headers and params:

1. before the request goes inside get_foo function and,
2. just before requests.get('http://third-party-service/v1/api') so I can attach any custom header or query param just before this request leaves the application.

@app.route('/my/v1/api')
def get_foo():
   ...
   ...
   r = requests.get('http://third-party-service/v1/api')
   ...
   ...
   return 'success!'

The former part I'm able to do using @app.before_request decorator where I get control of the request once it reaches the get_foo function.

@app.before_request
def before_request_callback():
    method = request.method
    path = request.path

    print(method, path)

The latter part I'm not able to do. If I use @app.after_request decorator, I get the control once the request has been process and I have got the response from http://third-party-service.

I debugged sentry sdk and they're able to take control of the requests once the request leaves the application. I tried out to follow how they have implemented it by following their piece of code from the github repo (https://github.com/getsentry/sentry-python) but wasn't able to do so hence posting the question over here.

Answer 1

You could define a decorator, e.g. my_decorator, that takes two arguments, query_param (a dictionary of parameters to be passed to the GET request, which defaults to {}) and headers (a dictionary containing the headers to be passed to the GET request, which defaults to {}). The decorator defines two function on the decorated function, get_query_param and get_headers, that can be called to retrieve these arguments`:

from functools import wraps

def my_decorator(query_param={}, headers={}):
    def decorate(func):
        wraps(func)
        def wrapper():
            # Add here any additional code that you might want to execute
            # prior to calling the wrapped function:
            ...
            return func()

        # Define two functions on the wrapped function:
        wrapper.get_query_param = lambda: query_param
        wrapper.get_headers = lambda: headers

        return wrapper
    return decorate


@my_decorator(query_param={'x': 1})
@app.route('/my/v1/api')
def get_foo():
   ...
   ...
   # Retrieve arguments to my_decorator:
   query_param = get_foo.get_query_param()
   headers = get_foo.get_headers()
   r = requests.get('http://third-party-service/v1/api', params=query_param, headers=headers)
   ...
   ...
   return 'success!'

Update

I had first tried to create a simpler decorator that would not require creating new functions. For example:

from flash import g
from functools import wraps

def my_decorator(query_param={}, headers={}):
    def decorate(func):
        wraps(func)
        def wrapper():
            # Assign to g:
            g.query_param = query_param
            g.headers = headers
            return func()
        return wrapper
    return decorate


@app.route('/my/v1/api')
@my_decorator(query_param={'x': 1})
def get_foo():
   ...
   ...
   r = requests.get('http://third-party-service/v1/api', params=g.query_param, headers=g.headers)
   ...
   ...
   return 'success!'

Note that the order of specifying the decorators has been changed.

Unfortunateley, this generated the following error message:

werkzeug.routing.exceptions.BuildError: Could not build url for endpoint 'get_foo'. Did you mean 'static' instead?

This problem can be resolved by moving the logic that is in get_foo to a newly decorated function, e.g. do_get_foo, and we no longer need to use flask.g:

from functools import wraps

def my_decorator(query_param={}, headers={}):
    def decorate(func):
        wraps(func)
        def wrapper():
            return func(query_param, headers)
        return wrapper
    return decorate


@my_decorator(query_param={'x': 1})
def do_get_foo(query_param, headers):
   ...
   ...
   r = requests.get('http://third-party-service/v1/api', params=query_param, headers=headers)
   ...
   ...
   return 'success!'

@app.route('/my/v1/api')
def get_foo():
    return do_get_foo() # delegate to this function

Note that do_get_foo is called from get_foo with no arguments but do_get_foo is defined to take two arguments.

Answer 2

I'm not sure that api decorator is useful for your purpose.

From my understanding you are doing an http server that act as a client to 3rd party services.

Probably you want to wrap request client to a 3rd party service and not the exposed apis.

You can do your own wrapper to requests object and can be very simple.

---

In case of a different scenario you can consider to create a middleware that is supported by flask for single route, route set and all routes.

Answer 3

1. intercept incoming requests: it appears you've already found an acceptable solution

• flask's before_request lifecycle hook is called before any of your route functions. it seems like a good solution for generic cross-cutting concerns such as e.g. logging details about incoming requests, handling authentication like validating username/password before handing requests to your service.
• a decorator as proposed by others would be my preferred solution if you need more flexibility, e.g. in the context of authentication a user might have to be a member of certain required LDAP groups that you would configure on each endpoint/route. it might also be useful to implement( and hide away) your generic exception handling or an adapter of sorts(we used flask for local development of aws lambda functions where a decorator converted the flask request into an aws json event, and the route then just passed that event on to the lambda handler)

2. intercept outgoing requests: it appears you have already asked that question here: Add decorator for python requests get function? you are trying to intercept a statement in your code...which is not possible afaik. you could try to monkeypatch the requests library. if, as suggested by your question, you only need to modify headers/parameters, then the simplest and most flexible solution is to just use the headers and params arguments of the requests.get() function as intended by the api. however, for cross-cutting concerns I'd use requests' session object and register adapters: suppose you'd be using different external APIs using different mechanisms of authentication then you could write and register different adapters that auto-magically add the api-key or oauth2 authenticate the user. it is also useful for logging outgoing requests, handling errors (e.g. implement some retry behavior) or to mock external services in unit-tests...and just all sorts of boilerplate code you want to hide away.

    import sys
    import requests
    import requests.adapters
    from functools import wraps
    from flask import Flask, request
    import logging
    
    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s.%(msecs)03d [%(levelname)8s] [%(name)s.%(funcName)s:%(lineno)d] %(message)s",
        handlers=[logging.StreamHandler(sys.stdout)]
    )
    
    logger = logging.getLogger(__name__)
    
    app = Flask(__name__)
    
    class http_session(requests.adapters.HTTPAdapter):
        def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None) -> requests.Response:
            # pre-process request...
            request.headers.update({'X-APIKEY': "KEY"})
            result =  super().send(request, stream, timeout, verify, cert, proxies)
            # post-process response...
            return result
    
        def __enter__(self):
            self._session = requests.Session()
            self._session.mount("http://httpbin.org", self)
            self._session.mount("https://httpbin.org", self)
            return self._session
    
        def __exit__(self, exc_type, exc_value, exc_tb):
            logger.info(exc_type, exc_value, exc_tb)
    
    
    @app.route('/v1/api/<value>')
    def get_foo(value):
        # http://127.0.0.1:5000/v1/api/abra
        with http_session() as session:
            r = session.get('https://httpbin.org/headers', headers={'abra': value})
            return r.json()

Answer 4

Python decorator is good to solve this, for example:

from functools import wraps

def attach_route(*options, **default_kwargs):
    """
    ## you can set any parameters.
    """

    def wrapper(view_func):
        @wraps(view_func)
        def view(*args, **kwargs):
            ### do before enter the flask.request                                   
            print('before: ', options, default_kwargs)
            method = request.method
            path = request.path                        
            
            result = view_func(*args, **kwargs)
            
            ### do after leave the flask.request
            print('after: ', result)
            
            return result 

        return view

    return wrapper


@app.route('/my/v1/api')
@attach_route()
def test_api():
    ### do whatever you want 
    print("test_api1")
    
    r = requests.get('http://third-party-service/v1/api')
    return 'success'
   

@app.route('/my/v2/api/<name>')
@attach_route(1,2,3,version=2,role='xxxx')
def test_api2(name):
    ### do whatever you want 
    print("test_api2:", name)
    return name

Answer 5

From reading your question, you might just want to write an abstraction to wrap the third-party service request; This might in fact prove to be simpler than adding more decorators.

From your two required points:

1. before the request goes inside get_foo function and,
2. just before requests.get('http://third-party-service/v1/api') so I can attach any custom header or query param just before this request leaves the application.

As you mentioned you can handle the first, I would suggest for the second you simply abstract the request on a function (or set of functions in a module) such as:

def third_party_get_request(query_param, headers):
    # you could add query and header modification here
    return requests.get('http://third-party-service/v1/api', params=query_param, headers=headers)

then you can use it in your route handler:

@app.route('/my/v1/api')
def get_foo():
    ...
    ...
    # you could add query and header modification here
    r = third_party_get_request(your_expanded_query, additional_headers)
    ...
    ...
    return 'success!'

Any additional thing you would add will have to be written in any case, both with decorators or this.. if you do not want to write the "expanding your query" part in the handler itself, then write it in the third_party_get_request function; A pro for this approach is that it makes the code easier to read rather than having 3 decorators on top of a route handler

Answer 6

you can write your own custom decorator and use it to decorate your function for example;

def headers(func):# just like in all decorators, the decorated functions are passed as argument.
   def wrapper(func):
     #modify the request headers here through the func argument
   return wrapper

you can use the above decorator in your code. it will take whatever your function is returning and modify it before it is finally returned.