Load balancing – The Missing Piece of the Container World: part 2

Feng 2

In my previous blog post, I described how easy it is to run a load balancer using tutum/haproxy image. However, the real world use case requires more controls on how the load balancer behaves. I am going to talk about some advanced topics in this article, but before starting, I would like to introduce the concept of a “service”, which serves as the basic build block of our load balancer tutum/haproxy.

Service vs Container

What is a service?

A service is a group of containers that run with the same parameters on the same image. For example, if you run docker run -d tutum/hello-world 3 times, you could say the 3 containers created belong to the same service.

Why service?

The concept of a service perfectly matches the function of a load balancer — a load balancer dispatches requests to the same application server, which is an application container in the docker world. For instance, if we link service A (containing 3 containers) and service B (containing 2 containers) to a load balancer, the load balancer will balance the traffic on 3 containers when accessing service A, and on 2 containers when accessing service B respectively.

How to setup services?

  1. Just as with tutum/haproxy, the basic building block of Tutum are services too. This means if you run your application using Tutum, the service of your application has been setup by Tutum natively.
  2. If you run tutum/haproxy outside of tutum, say using docker only, the link alias of your application container matters. Any link alias has the same prefix followed by “-” and an integer is considered from the same service. For instance, web-1 and web-2 are from service web, but web1 and web2 are from two different services web1 and web2.

Virtual host and virtual path

Virtual host

When you link multiple web application services to tutum/haproxy, you can specify an environment variable VIRTUAL_HOST in your web application services, so that when you access the load balancer with a different host name, you can still access different services. Here is an example:

docker run -d --name web1-1 -e VIRTUAL_HOST="www.example.com" <your_app_1>
docker run -d --name web1-2 -e VIRTUAL_HOST="www.example.com" <your_app_1>
docker run -d --name web2 -e VIRTUAL_HOST="app.example.com" <your_app_2>
docker run -d --link web1-1:web1-1 --link web1-2:web1-2 --link web2:web2 -p 80:80 tutum/haproxy

When you access http://www.example.com, tutum/haproxy takes you to your first application balancing on two instances, and when you access app.example.com, you are brought to your second web application.

Virtual path

Apart from the domain name, you can also tell haproxy to select services based on the path of the url you are accessing. For example, if your application is set with -e 'VIRTUAL_HOST=*/static/, */static/*, all the urls whose path starts with static will go to that service. Similarly, if you specify -e 'VIRTUAL_HOST=*/*.php, all the requests to an url that ends with .php will be directed to your php application service.

For more information on the usage of VIRTUAL_HOST, please see Github: tutum/haproxy.

Affinity and session stickiness

There are three environment variables you can use to set affinity and session stickiness in your application services: BALANCE, APPSESSION and COOKIE:

  1. Set BALANCE=source. When it is set, HAProxy will hash the IP address of the visitor. It makes sure that the visitor with the same IP address can alway be dispatched to the same application container. It works for both tcp mode and http mode.
  2. Set APPSESSION=<appsession>. HAProxy uses the application session to determine which application container a visitor should be directed to. It works only for http mode. A possible value of <appsession> could be JSESSIONID len 52 timeout 3h.
  3. Set COOKIE=<cookie>. Similar to appsession, it uses cookies to determine which application container a visitor should connect to. A possible value of <cookie> could be SRV insert indirect nocache.

Check HAProxy:appsession and HAProxy:cookie for more information.

Multiple SSL certs termination

As mentioned in the previous article, you can activate SSL termination by simply adding SSL_CERT in tutum/haproxy. But in many cases, you may have multiple SSL certs bound with different domains. For example, you have cert A with common name prod.example.com and cert B with staging.example.com. What you expect is that when a user accesses prod.example.com, HAProxy terminates SSL with cert A, and SSL of staging.example.com is terminated by cert B. To achieve this, you only need to set two environment variables SSL_CERT and VIRTUAL_HOST settings on your application services:

docker run -d --name prod -e SSL_CERT="<cert_A>" -e VIRTUAL_HOST="https://prod.example.com" <prod_app>
docker run -d --name staging -e SSL_CERT="<cert_B>" -e VIRTUAL_HOST="https://staging.example.com" <staging_app>
docker run -d --link prod:prod --link staging:staging -p 443:443 tutum/haproxy

TCP Loading balancing

tutum/haproxy runs in http mode by default, but it also has the ability to load balance TCP connections by using environment variables TCP_PORTS set in your application service. Below is an example:

docker run -d --name web -e VIRTUAL_HOST=www.example.com --expose 80 <web_app>
docker run -d --name git -e VIRTUAL_HOST="https://git.example.com" -e SSL_CERT="<cert>" -e TCP_PORTS=22 --expose 443 --expose 22 <git_app>
docker run -d --link web:web --link git:git -p 443:443 -p 22:22 -p 80:80 tutum/haproxy

In the example above, when you access http://www.example.com, you will visit your <web_app>; when you access https://git.example.com, you will go to <git_app> with SSL termination. In addition, port 22 is accessible by TCP connection.

tutum/haproxy also supports SSL termination on TCP. To enable it, instead of setting TCP_PORTS=22, simply set TCP_PORTS=22/ssl together with a SSL_CERT.


In the above sections, we introduced some basic examples of the advanced functions of tutum/haproxy. Using these functions in combination with one another can be very powerful. To find more information, please visit:

Load balancing – The Missing Piece of the Container World: part 1

Github: tutum/haproxy

Tagged with: , ,
Posted in Features, Tutorial
10 comments on “Load balancing – The Missing Piece of the Container World: part 2
  1. jotermoter says:

    This is so much awesome. Huge win, thanks!

  2. Great goods from you, man. I have understand your stuff previous
    to and you are just extremely wonderful. I actually like what you have acquired here,
    really like what you’re saying and the way in which you say it.

    You make it enjoyable and you still care for
    to keep it sensible. I can’t wait to read much more from you.
    This is actually a great site.

  3. I am regular reader, how are you everybody? This paragraph posted at this site is actually pleasant.

  4. Appreciation to my father who informed me about this
    website, this blog is truly awesome.

  5. […] Load balancing – The Missing Piece of the Container World: part 2 […]

  6. I don’t even know the way I ended up here, but I assumed this put
    up was good. I don’t understand who you are but certainly
    you are going to a famous blogger if you aren’t already.

  7. hello!,I love your writing so a lot! proportion we communicate extra about your article
    on AOL? I need a specialist on this area to solve my problem.
    May bee that is you! Looking aheadd to see you.

  8. Greetings from California! I’m bored to death at work so I decided to check out yoour blog on my iphone during lunch break.
    I love the infco you provide here and can’t wait to take a look when I get home.

    I’m shocked at how fast your blog loaded onn my phone ..
    I’m not even using WIFI, just 3G .. Anyhow, great blog!

  9. plan cul says:

    whoah this weblog is magnificebt i really like reading your posts.
    Keep up the good work! You already know, a lot of persons are searching around for
    this information, you can help them greatly.

  10. Heya i’m for the prinary time here. I found this board and I find It tduly helpful & it helped me out
    much. I’m hoping to give one thing back and aid others like you
    aided me.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: