Python Pieces: Decorators

As some of you know – Im a big believer that we all learn differently. You may read something the first time and immediately grasp the topic whereas I may read it and miss the point entirely. For me, decorators have been one of those things that I felt like I was always close to understanding but still not quite getting it. Sure – some of the examples I read made sense but then I’d find another one that didn’t. In my quest to understand them, I spent a lot of time reviewing a lot of examples and asking a lot of very patient friends for help. At this point, I feel like I know enough to try and explain the topic in a manner that might hopefully help someone else who was having a hard time with the concept. With my learning philosophy out of the way, let’s jump right in….

I want to jump right into a real (albeit not super useful) example of decorators using the full decorator (or shorthand) syntax. Let’s start with this…

def a_decorator(a_function):
    print("You've been decorated!")
    return a_function

@a_decorator
def print_name_string(your_name):
    name_string = "Your name is: " + your_name
    return name_string

print(print_name_string("Jon"))

The code has two functions – a_decorator and print_name_string. The @a_decorator syntax directly above the print_name_string definition decorates that function with the function a_decorator (doing it this way is what I was referring to as the shorthand syntax). Lastly – we call our decorated function. Before we look at the output – let’s define what a decorator is. From all my reading the definition I like the most comes from this article where they define a decorator as…

A function that takes another function as an argument, generates a new function, augmenting the work of the original function, and returning the generated function so we can use it anywhere

With that said – and knowing that we are decorating the print_name_string function – any guesses as to what the output might be?

You've been decorated!
Your name is: Jon

That perhaps wasn’t hard to guess. If we apply our definition to this exact code example it might say…

a_decorator takes the print_name_string function as an argument, generates a new function, augmenting the work of print_name_string, and returns the generated function so we can use it anywhere

That said, you may look at the code above and think to yourself that when we call print_name_string Python sees the decorator and then runs the decorator function. The output we’re seeing certainly supports this. But now – let’s remove the print statement and try again…

def a_decorator(a_function):
    print("You've been decorated!")
    return a_function

@a_decorator
def print_name_string(your_name):
    name_string = "Your name is: " + your_name
    return name_string

#print(print_name_string("Jon"))

Our output is now….

You've been decorated!

Well – that’s discouraging. Why is the decorator function being called when we aren’t calling a function that’s been decorated with it? The answer is that decorators are rendered at runtime. The instant that the Python interpreter reads lines 5-8 above it runs them. So what does “run” mean in this context? I can boil it down to say that the new function based on the decorator is being created the instant the code is processed. This is not unlike any other function we create. If you’re in the Python CLI you have to create a function before you can call it. When the Python file is processed by the interpreter it’s done so top down. In our case, the interpreter sees the function is decorated and does the work to create a decorated function at that time so later in the code we can call it. So let’s look a little harder at what the decorator does…

def a_decorator(a_function):
    print("You've been decorated!")
    return a_function

It looks like it takes in a variable, does a print, and then returns the same variable. Based on the naming, we can discern that the variable being taken in, as well as returned, is a function. In this case a_function is the function that was being decorated (print_name_string). All that our decorator is doing is returning the same function. Seems like a waste returning the same function doesn’t it? That said – the decorator does need to return a function otherwise this will all fall apart. For instance…

def a_decorator(a_function):
    print("You've been decorated!")
    #return a_function

@a_decorator
def print_name_string(your_name):
    name_string = "Your name is: " + your_name
    return name_string

print(print_name_string("Jon"))

Commenting out the decorator function return gives us this…

You've been decorated!
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    print(print_name_string("Jon"))
TypeError: 'NoneType' object is not callable

The message is pretty clear – but since our decorator is returning nothing (None) we can’t actually call the function since None is not callable. Again – it’s important to think of the decorator as replacing or augmenting the decorated function. Having the decorator not return a usable function is sort of like taking your car for an oil change and finding out afterwards your car is gone (Im surprised that’s the best example I can come up with). We pass the decorator a function, allow it to do some work, but in the end we expect it to return us a usable function. So it seems we have a hard rule defined here – decorators must return a function. But what function can they return if it’s not the one it is inherently passed as part of the decoration? Well we can create sub functions of our decorator function. A simple example might look like this…

def a_decorator(a_function):
    def add_some_flair(a_name):
        return "******* " + a_name + " *******"
    return add_some_flair

@a_decorator
def print_name_string(your_name):
    name_string = "Your name is: " + your_name
    return name_string

print(print_name_string("Jon"))

Here we add a sub-function to a_decorator called add_some_flair. This function adds a series of asterisks around the variable it receives and returns that string. Any ideas what the output of this might look like?

******* Jon *******

This is a good example of a decorator replacing the functionality of an existing function. Notice that our output doesn’t list the “Your name is: Jon” line. In this case, we decorated the function print_name_string but the decorator itself decided to return a function that does something entirely different. Again – this seems like it has little use if we are just fully overriding the functions that we are decorating. But it does go to show you that the decorator has the ability to completely override the initial function call if it wants to. We can prove that we can still get to the decorated function just by calling it as part of the add_some_flair function…

def a_decorator(a_function):
    def add_some_flair(a_name):
        print(a_function(a_name))
        return "******* " + a_name + " *******"
    return add_some_flair

@a_decorator
def print_name_string(your_name):
    name_string = "Your name is: " + your_name
    return name_string

print(print_name_string("Jon"))

Now our output would look like…

Your name is: Jon
******* Jon *******

You may have noticed that our function add_some_flair is taking in a single variable called a_name. This is not optional. Since we are decorating a function that takes in a single variable (your_name) our decorator return function must also take in a single variable. For instance, let’s try adding another variable to our print_string_name function…

def a_decorator(a_function):
    def add_some_flair(a_name):
        print(a_function(a_name))
        return "******* " + a_name + " *******"
    return add_some_flair

@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name
    return name_string

print(print_name_string("Jon"))

Now when we run this we get…

Traceback (most recent call last):
  File "test.py", line 12, in <module>
    print(print_name_string("Jon"))
  File "test.py", line 3, in add_some_flair
    print(a_function(a_name))
TypeError: print_name_string() missing 1 required positional argument: 'your_age'

The traceback points out our problem. We are trying to call the function print_your_name but aren’t giving it the right amount of variables. This is another case that we have to step back and think about this for a moment. When we call print_name_string what are we actually doing? Since it’s decorated with a_decorator which returns add_some_flair we are really calling add_some_flair. In doing so, we are passing it a variable a_name. If we wish to call the original function print_string_name and wish for it to have access to both your_name and your_age then we need to pass those to print_name_string which is really calling add_some_flair. You still with me? It’s important to think of the decorator being the main function being called. If you wish to call print_name_string again from within the decorator, you need to have the variables available to do so which means passing them into the decorator. Now – since we aren’t currently doing that, we can certainly make up a variable to meet the requirement…

def a_decorator(a_function):
    def add_some_flair(a_name):
        print(a_function(a_name,"bah"))
        return "******* " + a_name + " *******"
    return add_some_flair

@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name + " Your age is: " + your_age
    return name_string

print(print_name_string("Jon"))

In this case we are still only passing a_name to the add_some_flair function and when we go to call print_name_string(a_function) we simply pass it “bah” as the your_age parameter…

Your name is: Jon Your age is: bah
******* Jon *******

The right thing to do here though is to pass the variable through. If we just start passing a second variable on line 12 let’s see what happens…

def a_decorator(a_function):
    def add_some_flair(a_name):
        print(a_function(a_name,"bah"))
        return "******* " + a_name + " *******"
    return add_some_flair

@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name + " Your age is: " + your_age
    return name_string

print(print_name_string("Jon","99"))

Gives us…

Traceback (most recent call last):
  File "test.py", line 12, in <module>
    print(print_name_string("Jon","99"))
TypeError: add_some_flair() takes 1 positional argument but 2 were given

This output should help solidify the point I was trying to make above. We are now passing two variables to print_name_string and if we look at that function we see that it certainly does take two variables on line 8. However, since our decorator is replacing print_name_string with add_some_flair which does not take two arguments, the interpreter doesn’t know what to do. To fix this, we need to accept both arguments in the decorator return function add_some_flair. Once we do that we can start consuming it…

def a_decorator(a_function):
    def add_some_flair(a_name, a_age):
        print(a_function(a_name, a_age))
        return "******* " + a_name + " *******"
    return add_some_flair

@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name + " Your age is: " + your_age
    return name_string

print(print_name_string("Jon","99"))

Which gives us the output…

Your name is: Jon Your age is: 99
******* Jon *******

Now that we’ve talked about the basics – lets talk about the long hand way of decorating a function. We’ve already said that a decorator is a function that takes another function as an argument. That being said, instead of using the decorator shorthand syntax we could probably just do something like this…

def a_decorator(a_function):
    def add_some_flair(a_name, a_age):
        print(a_function(a_name, a_age))
        return "******* " + a_name + " *******"
    return add_some_flair

#@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name + " Your age is: " + your_age
    return name_string

decorated_print_name_string = a_decorator(print_name_string)
print(decorated_print_name_string("Jon","99"))

You can see on line 12 that we are creating a decorated function called decorated_print_name_string. So it’s a function that takes another function as an argument, possibly does some modification to it, and then returns the new function. This will work just like the shorthand syntax did…

Your name is: Jon Your age is: 99
******* Jon *******

Now – you might be wondering if we could shorten this up like this…

def a_decorator(a_function):
    def add_some_flair(a_name, a_age):
        print(a_function(a_name, a_age))
        return "******* " + a_name + " *******"
    return add_some_flair

#@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name + " Your age is: " + your_age
    return name_string

print(a_decorator(print_name_string("Jon","99")))

Unfortunately, running this provides us this output…

<function a_decorator.<locals>.add_some_flair at 0x102268430>

Why is that? By calling the function including the () we’re telling Python to execute that function. So in this case, we’re passing the return value of print_name_string("Jon") to the function a_decorator. We can see this by just making the same call ourselves…

def a_decorator(a_function):
    def add_some_flair(a_name, a_age):
        print(a_function(a_name, a_age))
        return "******* " + a_name + " *******"
    return add_some_flair

#@a_decorator
def print_name_string(your_name, your_age):
    name_string = "Your name is: " + your_name + " Your age is: " + your_age
    return name_string

print(a_decorator(print_name_string("Jon","99")))
print(a_decorator("Your name is: Jon Your age is: 99"))

The output is now…

<function a_decorator.<locals>.add_some_flair at 0x103f8a430>
<function a_decorator.<locals>.add_some_flair at 0x103f8a430>

The key here is that we aren’t so much worried about the function input at this point. In fact, we’re not interested in it at all when we create decorated functions. The goal of decorating a function is to create a new function. This is something that’s done totally outside of the realm of consuming the function (aka, passing variables to it and expecting a result) and is done only once when Python imports the script.

It probably makes sense at this point to provide a couple of examples of how decorators might be used in real life. Here’s a quick example of one you might use to time a given function call…

import time

def timing(a_function):
    def time_func_call(x_counter):
        print("DECORATOR -> Doing a loop for " + str(x_counter) + " iterations")
        start_time = time.perf_counter()
        a_function(x_counter)
        end_time = time.perf_counter()
        total_run_time = end_time - start_time
        print("DECORATOR -> Total time was: " + str(total_run_time))
    return time_func_call

@timing
def print_name_string(x_counter):
    print("The function beginith....")
    while x_counter > 0:
    	time.sleep(.1)
    	x_counter = x_counter - 1
    print("The function endith....")

print_name_string(10)

The output would look something like this…

DECORATOR -> Doing a loop for 10 iterations
The function beginith....
The function endith....
DECORATOR -> Total time was: 1.034948312

Ideally you’d use something besides print statements – but I think it’s enough to make the point. Decorators can be extremely useful and there are lots of Python projects that make great use of them. There are also lots of other problems to solve in regards to decorators but I’ll save those for another blog post. Im hoping this is enough to at least get folks to a base level understanding of how they work. As always – comments are welcome and appreciated!