Docker Networking 101 – Host mode

In our last post we covered what docker does with container networking in a default configuration.  In this post, I’d like to start covering the remaining non-default network configuration modes.  There are really 4 docker ‘provided’ network modes in which you can run containers…

Bridge mode – This is the default, we saw how this worked in the last post with the containers being attached to the docker0 bridge.

Host mode – The docker documentation claims that this mode does ‘not containerize the containers networking!’.  That being said, what this really does is just put the container in the hosts network stack.  That is, all of the network interfaces defined on the host will be accessible to the container.  This one is sort of interesting and has some caveats but we’ll talk about those in greater detail below.

Mapped Container mode – This mode essentially maps a new container into an existing containers network stack.  This means that while other resources (processes, filesystem, etc) will be kept separate, the network resources such as port mappings and IP addresses of the first container will be shared by the second container.

None – This one is pretty straight forward.  It tells docker to put the container in its own network stack but not to do configure any of the containers network interfaces.  This allows for you to create custom network configuration which we’ll talk about more in a later post.

Keep in mind that all these modes area applied at the container level so we can certainly have a mix of different network modes on the same docker host.

For this post, I’m going to use the same lab I used in the first post but with a minor tweak…

Note that my docker1 host now has two IP interfaces.   Other than that, everything is the same.

Host Mode
Host mode seems pretty straight forward but there are some items you need to keep in mind when deploying it.  Namely, some of the configuration I thought might happen automagically doesn’t actually happen.  Let’s look at an example so you can see what I’m talking about.  Let’s start a basic web container on the docker2 host.  I’ll do so with this command…

docker run -d --name web1 --net=host jonlangemak/docker:webinstance1

Note: All of the containers I use in these labs are available in my public repo so feel free to download them for testing.  There are 4 images I use in this lab all of which are running CentOS with Apache.  The only difference is slight configuration changes in the index.html page so we can see which one is which as well as some Apache config which I talk about more below.

Note that I’m passing the ‘–net=host’ flag in the docker run command.  Also note that I’m not specifying any port mappings.  Once the image is downloaded docker will run the image as a container called ‘web1’.  Since we told docker to run this container as a daemon let’s connect to a bash shell on the container using this command…

docker exec –ti web1 /bin/bash

Once connected, let’s check and see what network interfaces we have in the container…

Note that we don’t have an IP address in the 172.17.0.0/16 address space.  Rather, we actually have all of the interfaces from the docker2 host.  For comparison sake, let’s see them side by side…

I hope that makes this obvious, but they’re totally identical.  This brings up some interesting possibilities.  But before we get carried away, let’s check and see what’s going on with our Apache server.  Since the IP of the container is the IP of the host one would assume that we should be able to hit our index.html on 10.20.30.101.  Let’s give it a try…

No dice.  So what’s going on?  Recall that docker makes rather extensive use of iptables for it’s bridging mode.  Let’s take a look at the iptables rule set and see what it has…

No rule to allow http.  One might suggest that since we didn’t use the ‘-p’ flag docker didn’t know to make a rule in iptables.  While that seems to be a possible fix it really isn’t.  Starting the container with a port mapping yields the same result.  That being said, it’s safe to say that you’re on your own when it comes to host mode networking.  Docker will expose the hosts network stack to the container but its then up to you to make the appropriate firewall rules.  So let’s add a rule that allows port 80 traffic through iptables…

iptables -I INPUT 5 -p tcp -m tcp --dport 80 -j ACCEPT

Note: Interestingly enough you could actually make this rule from the container itself if you were to pass the ‘privileged=true’ flag in the docker run command.  While this may seem appealing from a automation perspective it seems unnecessary and possibly a bad idea.

With the iptables rule in place we should be able to browse to the web page through the host IP address…

Cool, so now we’re up and running in host mode.  One thing to keep in mind is that this mode of operation severely limits the services you can run on a single host.  AKA, we don’t get port mapping anymore.  If we try to run another container that also wants to use port 80 we’re going to run into issues.  Let’s try so you can see what I’m talking about.  Let’s spin up a second container called webinstance2 on docker2…

docker run -d --name web2 --net=host jonlangemak/docker:webinstance2

If we check we can see that both containers are now running…

At this point I can still get to my web1 index page but what happened with web2?  Let’s log into web2 and see what’s going on…

docker exec –it web2 /bin/bash

Let’s check the service status…

Alright, that looks bad.  Let’s try and start httpd…

That looks even worse.  So this isn’t a docker problem, it just the fact that the web instance 1 container is already bound to port 80 on the interfaces of the docker2 host.  We can see this binding by checking out netstat from inside the web instance 2 container…

We don’t get the PID info since this container’s processes are different than web instance 1’s but if we head back to the host and run the command again we can see that port is being used by httpd…

So that all sort of makes sense.  You can’t use the same IP address for the same service on different containers.  So at this point, I’d argue that our diagram looks a lot more like this…

The docker2 host is still there but the container is really right up front on the physical edge sine it’s sharing the same network stack as the host.  So this all seems very limiting.  Fortunately, we do have an option for running multiple identical services on the same docker host.  Recall that docker1 now has two IP address, .100 and .200.  If we run two Apache instances in host network mode one should be able to use .100 and the other .200.

Again – This isn’t a docker configuration problem.  Just as if this was a physical server running Apache we need to tell Apache where to listen and on what port.  This is done by modifying the apache config (in my case /etc/httpd/conf/httpd.conf) and setting the ‘Listen’ command.  By default, Apache will listen on port 80 on every interface.  The default config would look something like ‘Listen 80’.  To make this work we need to change the config to something like what is shown below on each respective container…

Fortunately for you, I already have two containers pre-configured with this configuration.  Testweb1 is setup to listen on 10.20.30.100:80 and testweb2 is listening on 10.20.30.200:80.  Let’s run them on docker1 and see the result…

docker run -d --name web1 --net=host jonlangemak/docker:testweb1
docker run -d --name web2 --net=host jonlangemak/docker:testweb2

Once both are downloaded let’s ensure they are both running…

Now let’s test and see what we get on each IP address…

Note: Same rules apply here, add a rule in iptables to allow http on the host

Looks good, let’s check the docker host to see what it thinks is going on…

Nice, so the docker host sees two Apache processes one listening on each of it’s interfaces.  Our final diagram shows each container almost as its own distinct host…

So as you can see, host mode networking is a little bit different than bridge mode and requires some additional config to get it working properly.  However, the tradeoff is performance.  Containers running in the hosts network stack should see a higher level of performance than those traversing the docker0 bridge and iptables port mappings.

Next up we’ll cover the container in container mode of docker networking, stay tuned!