Simulating latency and packet loss on a Linux host

Every once and a great while there is a need to simulate bad network behavior.  Simulating things like packet loss and high latency are often harder to do than you’d expect.  However – if you’re dealing with a Linux host – there’s a simple solution.  The ‘tc’ command which comes along with the ‘iproute2’ toolset can help you simulate symptoms of a bad network quite easily.

The tc command offers a lot of functionality but in this post we’re just going to walk through a couple of quick examples of using it in conjunction with the netem (network emulator) included in the Linux kernal .  To do this, we’ll use just use two hosts…

To start with – let’s make sure that ‘tc’ is installed and that it’s working…

So what did we just do here? Well we used the tc command to return the current qdisc configuration for our servers physical network interface named ‘ens32’.  So what’s a qdisc?  Qdisc is shorthand for ‘Queue discipline’ and defines the queuing method assigned to the interface. In our case we can see that this interface is using the ‘pfifo_fast’ queuing methodology. This is a default configuration and you should see something similar on your Linux host.

Side Note: Im doing all of this in a lab.  Make sure you know what you’re changing before you change anything.  For instance, configuring a high rate of packet loss on an interface you’re using to SSH into the host is likely not a great idea.  Common sense rules apply.

Let’s first start a constant ping on our host ubuntu-5 heading toward ubuntu-1. Now let’s head back to ubuntu-1 and configure the following command…

So let’s break this command down. We’re telling tc that we want to work with the interface ens32 and add a delay of 200ms to the root qdisc. The root qdisc is an egress queue and where all the packets will inherently get queued by default. If we go and check the ping on the other server we should see the latency has increased…

Great! Now let’s modify the rule with this command…

This command tells the rule to include a random deviation of up to 50ms. Once the rule is in, you should start seeing latency number in between 150ms and 250ms…

Perfect! Having that random deviation helps make it look like ‘real’ latency. Now let’s delete the rule like so…

Its important to keep track of the ‘add|change|del’ keywords. If you try to change a rule that doesnt exist or something similar you’re going to start getting weird errors when you try to work with the rules.

Next up – let’s introduce some packet loss!

The command is similar to the latency commands but now we’re specifying ‘loss’ instead of ‘delay’. If we send 10 ICMP pings to ubuntu-1 from ubuntu-5 we should see some packet loss…

Yep, so 20% packet loss as we expected. When you’re done testing, dont forget to delete the rule…

One interesting thought to point out here is that you could quite easily build a server that acted as a sort of ‘WAN simulator’ to use in your lab.  In that case, if the server was inline with two interface you could enact policy in both flow directions by applying specific policy to each interface.  Lots of possibilities there!

This is literally just the tip of the iceberg. There are so many more things you can do with tc and netem.  If you’re looking for more info here’s a good starting point.  Be aware that the in-document hyperlinks don’t appear to work but you can just scroll down to see all of the examples.

Update: Heres another link with better info about TC and NetEm.  Thanks John!

  1. John Holmes’s avatar

    Here’s a better example / reference page. You don’t have to mess around with token bucket filters, the rate can be set directly with the same command as the rest of the conditions.

    http://man7.org/linux/man-pages/man8/tc-netem.8.html

    Reply

    1. Jon Langemak’s avatar

      Thanks, added a link to the post!

      Reply

Reply

Your email address will not be published. Required fields are marked *