Creating your own Ansible filter plugins

      15 Comments on Creating your own Ansible filter plugins

Update: Ivan Pepelnjak reached out to me after this was published and suggested that it would make more sense to move the functions inside the defined class.  Doing this removes the functions from the global namespace and avoids the possibility of function names overlapping.  I think that’s a great idea so I’ve updated the examples below to align with this thinking.  Thanks Ivan!

I’ve been playing around with Ansible quite a bit lately.  One of the issues I’ve started to run into is that Ansible can’t always do what you want by default.  Put more plainly, Ansible can’t always easily do what you want.    Many times I found myself writing tasks to manipulate variables and thinking to myself, “Man – if I could just run some Python code on this it would be way easier”.  As luck would have it, you can!  Ansible supports a whole slew of plugins but the type I want to talk about today are called filter plugins.  Filter plugins, in my opinion, are one of the easiest ways to manipulate your variables with native Python.  And once you know how to do it you’ll see that it opens up a whole realm of possibilities in your playbooks.  Some of the most popular filters that exist today were once custom filters that someone wrote and then contributed to Ansible.  The IP address (ipaddr) filter set is a great example of filters that can be used to manipulate IP related information.

When I first looked into writing a custom filter I found the Ansible documentation not very helpful.  It essentially points you to their GitHub repo where you can look at the current filters.  Since I learn best by example, let’s write a quick filter so you can see how easy it is to do…

#!/usr/bin/python
class FilterModule(object):
    def filters(self):
        return {
            'a_filter': self.a_filter,
            'another_filter': self.b_filter
        }

    def a_filter(self, a_variable):
        a_new_variable = a_variable + ' CRAZY NEW FILTER'
        return a_new_variable

Nothing to it right?  This file defines a single filter called ‘a_filter’.  When called it receives the variable being passed into it (the variable to the left of the pipe (|)), appends the string ‘ CRAZY NEW FILTER’ to it, and then returns the new variable.  Now the trick is where to put this file.  For now, let’s create a folder called ‘filter_plugins’ in the same location as your playbook.  So in my case, the file structure would look like this…

/home/user/my_playbook.yml
/home/user/filter_plugins/my_filters.py

So let’s go ahead and create a quick little test playbook too…

---
- hosts: localhost
  tasks:
    - name: Print a message
      debug:
        msg: "{{'test'|a_filter}}"

This is a pretty simply playbook.  All it does is use the debug module to output a variable.  However, note that instead of just outputting the word ‘test’, we’re wrapping it in double curly braces like we do for any normal Ansible variable and we’re also piping it to ‘a_filter’.   The piping piece is what’s typically used to pass the variable to any of the predefined, or built-in, filters.  In this case, we’re piping the variable to our own custom filter.

This playbook assumes that you’ve told Ansible to use a local connection when talking to the locahost.  To do this, you need to set the ‘ansible_connection’ variable for the localhost to ‘local’ in your Ansible host file…

user@ansibletest:~$ more /etc/ansible/hosts
#Localhost
localhost ansible_connection=local
user@ansibletest:~$

Once this is set, and you have both the playbook and the filter files in place, we can try running the playbook…

user@ansibletest:~$ ansible-playbook my_playbook.yml

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [Print a message] *********************************************************
ok: [localhost] => {
    "msg": "test CRAZY NEW FILTER"
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0

user@ansibletest:~$

As you can see, the filter worked as expected.  The variable ‘test’ was passed to our custom filter where it was then modified and returned.  Pretty slick huh?  This is a uber simple example but it shows just how easy it is to inject custom Python functionality into your playbooks.  You’ll notice that in this example, there was only one variable passed to our function.  In this case, it was the variable to the left of the pipe.  In the case of filters, that will always be your first variable however, you can always add more.  For instance, let’s add a new filter to our function like this…

#!/usr/bin/python
class FilterModule(object):
    def filters(self):
        return {
            'a_filter': self.a_filter,
            'another_filter': self.b_filter
        }

    def a_filter(self, a_variable):
        a_new_variable = a_variable + ' CRAZY NEW FILTER'
        return a_new_variable

    def b_filter(self, a_variable, another_variable, yet_another_variable):
        a_new_variable = a_variable + ' - ' + another_variable + ' - ' + yet_another_variable
        return a_new_variable

There are a couple of interesting things to point out in our new my_filters.py file.  First off – you’ll notice that we added another Python function called ‘b_filter’.  Its worthwhile to point out that your filter names don’t need to match your function names.  Down in the filters function at the bottom you’ll notice that we map the filter name ‘another_filter’ to the Python function ‘b_filter’.  You’ll also notice that the function b_filter takes 3 arguments.  The first will be the variable to the left of the pipe and the remaining need to be passed directly to the function as we’d normally pass variables.  For example…

---
- hosts: localhost
  tasks:
    - name: Print a message
      debug:
        msg: "{{'test'|another_filter('the','filters')}}"

Here you can see that we pass the second and third variables to the filter just like we’d normally pass variables to a function.  And while these examples only show doing this with strings, you can pass many other Python data types such as lists and dicts as well.

Lastly – I want to talk about the location of the filters.  By default, Ansible will look in the directory your playbook is located for a folder called ‘filter_plugins’ and load any filters it finds in that folder.  While this works, I don’t particularly care for this as I find it confusing for when you’re moving around playbooks.  I prefer to tell Ansible to look elsewhere for the filters.  To do this, we can edit the /etc/ansible/ansible.cfg file and uncomment and update the ‘filter_plugins’ parameter with your new location.

As you can see, filter plugins make it ridiculously easy to get your variables into native Python for manipulation.  Keep in mind that there are LOTS of default filter plugins so before you go crazy search the documentation to see if what you want already exists.

15 thoughts on “Creating your own Ansible filter plugins

  1. Alain Chiasson

    Excelent article – it simplifies alot.

    The first pass failed – a simple issue – you would need to remove the “b_filter” reference.


    def filters(self):
    return {
    ‘a_filter’: self.a_filter,
    ‘another_filter’: self.b_filter
    }

    b_filter is undefiend the first time through.

    I have been playing with Molecule for testing ansible roles, so I wrapped the role up in a test_module :https://github.com/alainchiasson/jinja2-ansible-test

    Hope that it helps ! Thanks again. Feel free to send comments if I miss-attributed.

    Reply
    1. Srinivasa

      That’s awesome

      Quick question, you know that filter plugins are executed on control node, how can we achieve this to make it run on other host? This is because my filter plugins accesses NFS path which not able to access by ansible awx container

      Reply
  2. Tom Bartsch

    Hi there,

    very good explanation. Thank you for that.
    Now I have one more question:

    You have described the plugin call like this
    {{‘test’|another_filter(‘the’,’filters’)}}

    Now what if the strings ‘the’ and ‘filters’ would be values of 2 ansible vars?

    For example
    firstParam: “the”
    secondParam: “filters”

    How can I do the plugin call with this 2 ansible vars instead of passing the 2 strings directly?

    Reply
  3. Chris F

    Thank you! The article helped me create my first plugin, which does nothing but splits a semantic version in its major, minor, and patch parts, and returns the desired part accordingly.

    Reply
  4. keltik85

    I get this Error:

    fatal: [192.168.178.201]: FAILED! => {“msg”: “template error while templating string: no filter named ‘a_filter’. String: stdout of the stupid mongodb is = {{ stupid_mongo_status.stdout | a_filter }}”}

    It cant find the filter.

    I have placed it like you told me to? Any hints on why it does not understand where the file is?

    Reply
  5. Doug Byrd

    Awesome. Thank you so much! This allowed me to write a custom filter to convert between MAC address formats since ACI and NXOS prefer the opposite!

    Reply
  6. Steve Rodrigue

    Very nice tuturial to get us started. Really appreciated.
    The only thing I would like to see is a more cleaner template. You’re example makes pylint not happy.

    If you could update my_filters.py with a bullletproof/clean version that would be awesome. If you want to integrates to pipelines (CI/CD), this example will be rejected for sure. 😀

    Reply
  7. Scott S

    First, this is an excellent tutorial. I was able to create a custom filter and it worked on the first try. That never happens for me, so thank you.

    Second, Steve, I also like to make pylint happy. So I’m sharing the filter I wrote. It is used to add access-list entries in conjunction with the Cisco ASA collection for Ansible. But that’s not important, what is important is that pylint gives it 10/10 (Python 3.8). Hopefully that helps you.

    $ cat acl_line_filter.py
    “””
    acl_line_filter

    Takes a list (one or more ace entries in the format
    used by the asa_acls module) and adds an incrementing
    line number to each entry.
    “””

    class FilterModule():
    “””Container for custom filter plugins.”””
    def filters(self):
    “””Return list of filter names and their associated functions.”””
    return {
    ‘acl_filter’: self.acl_filter,
    }

    @staticmethod
    def acl_filter(ace_list):
    “””Add incrementing line numbers to ACL entries.”””
    current_line = 1
    new_ace = []
    for ace_entry in ace_list:
    ace_entry[“line”] = current_line
    new_ace.append(ace_entry)
    current_line += 1
    return new_ace

    Reply
  8. Ed McGuigan

    I used this article to get started and thanks for that. What I soon found is that I was writing the same filters in different playbooks and wondering how I could modularize my filters so that I wasn’t writing the same snippets of code over and over again.
    There are a few approaches but the one I currently favor is creating a collection and putting the filters into that.

    I work a lot with the Bluecat IP Address Management solution so I found that there were tasks I was wanting execute in many different plays so I created a collection and placed theses task in the collections as roles. The collection has a name sdopbc9049.bluecat_ipam_roles .

    Working with this collection I came upon the need for a filter to perform a standard function on a list of properties and wanted to make that available from the collection as well. Details of the actual data manipulation are not that important but I needed to strip one field from a string containing a lot of object properties.

    The actual filter code is:
    ============================================================
    class FilterModule(object):
    def filters(self):
    return {
    ‘strip_cidr_from_properties’: self.strip_cidr_from_properties
    }

    def strip_cidr_from_properties(self, properties):
    return ‘|’.join([ item for item in properties.split(‘|’) if item[:len(“CIDR”)] != “CIDR” ])
    ======================================================================

    The way you do this is to place your filters.py file in the collection under sdopbc9049/bluecat_ipam_roles/plugins/filter/ ( NOTE the singular filter ).

    With the collection available to your playbooks you can now invoke the filter with something like:

    ===========================================================
    – name: Call a filter from the IPAM collection
    set_fact:
    new_object_properties: “{{ properties_with_cidr_property | sdopbc9049.bluecat_ipam_roles.strip_cidr_from_properties }}”
    ===========================================================

    So you have to give the full path of the collection and then the filter name. This is very powerful IMO in terms of v=being able to apply the DRY ( Don’t Repeat Yourself ) principle to your ansible coding.

    Reply

Leave a Reply to keltik85 Cancel reply

Your email address will not be published. Required fields are marked *