If you have not read the previous post, please go through it because it is a prerequisite for this. In this post, we will go into some more depth and learn about decorators with arguments.
We learned from the previous blog post that when a function is decorated with a decorator, the representation of it will be changed to decorator.<locals>.wrapper at 0x109e233a0> which looks ugly. How can we keep the original string representation of the function? Here comes functools.wraps the decorator. It is a decorator — again, so many decorators huh? — which restores the original function name. We will use it like the one below.
import functools
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
pass
return wrapperIn a nutshell, what it does is simply sets __name__ , __doc__ and __module__ properties of the wrapper function to the original one.
wrapper.__name__ = func.__name__
wrapper.__doc__ = func.__doc__
wrapper.__module__ = func.__module__First, let’s write a retry decorator which makes an API call and retries on failure.
import requests
import time
import logging
logger = logging.getLogger(__name__)
def retry(make_http_call):
def wrapper(*args, **kwargs):
curr_retry_count = 0
max_retries = 5
retry_interval = 10
while init_retry_count <= max_retries:
try:
response = make_http_call(*args, **kwargs)
if response.status_code in [200, 201]:
return response
except requests.RequestException:
logger.error("error in making call")
time.sleep(retry_interval * (math.pow(2, curr_retry_count - 1))) # retry_interval * 2^(n-1) -> exponential backoff
curr_retry_count += 1
raise requests.RequestException("problem in making HTTP call please check logs")
return wrapperIn the above code snippet, retry is a function accepting make_http_call as a parameter. Assume that make_http_call triggers an HTTP request. The code is a no-brainer, it makes an HTTP request in a loop until the request is a success or the maximum retries are reached. It waits for some time until the next retry i.e. exponential backoff.
We want to make retry customizable by accepting max_retry_count and retry_interval because they may vary depending upon the usecase. How do we do that? simple, we will create another function which takes two parameters and simply wrap the above function inside it.
import requests
import time
import logging
logger = logging.getLogger(__name__)
def retry(max_retries, retry_interval):
def inner(make_http_call):
def wrapper(*args, **kwargs):
curr_retry_count = 0
while init_retry_count <= max_retries:
try:
response = make_http_call(*args, **kwargs)
if response.status_code in [200, 201]:
return response
except requests.RequestException:
logger.error("error in making call")
time.sleep(retry_interval * (math.pow(2, curr_retry_count - 1))) # retry_interval * 2^(n-1) -> exponential backoff
curr_retry_count += 1
raise requests.RequestException("problem in making HTTP call please check logs")
return wrapper
return inner
@retry(10, 60)
def make_http_request(url, params=None, data=None):
passMathematically, the above function make_http_request becomes make_http_request = retry(10, 60)(make_http_request) and calling that function would be like
make_http_request(
"https://api.twitter.com/v2/users",
params={"username": "lsanapalli"}
) = retry(10, 60)(make_http_request)(
"https://api.twitter.com/v2/users",
params={"username": "lsanapalli"}
)Can we decorate a function with multiple decorators? yes. What will be the order of execution? simple, let’s consider an example and stick to our basics.
@decorator3
@decorator2
@decorator1
def myfunc(a, b):
passConsider we have a function myfunc decorated with 3 decorators as above. Firstly, myfunc will be passed as an argument to decorator1 and the result of that will be passed to decorator2 and so on…
myfunc = decorator3(decorator2(decorator1(myfunc)))
If you prefer a video version, check out 👇