Easy Docker Deployments Using Tutum API

Tutum is a container platform that enables easy management of docker-backed services on node clusters running on platforms such as Digital Ocean, AWS,  and Azure. You can also use Bring your own node to use any Linux host as a node to deploy containers to. In this tutorial, we will be deploying a simple Node.js application on Digital Ocean. The application, Tweedapp.io, is a simple RSS feeds and tweets aggregator for popular tech sites.

tweed

You can checkout the source code at github which you should go ahead and grab a local copy by running:

git clone git@github.com:kelonye/tutum-tweed.git

Assuming that we have already created a Tutum account, we first need to login into our private Tutum docker registry with:

$ docker login tutum.co
Username
Password
Email
Login Succeeded

We will be using the Tutum REST API to easily deploy the application to a single node cluster composed of the following services:

  • A Web service built in Node and Ember.js
  • Redis cache service that stores the updates
  • Tweets worker that subscribes to matching tweets
  • Feeds worker that periodically fetches feeds

You can see that each of the services is located in its own directory under the lib directory:

Screen Shot 2014-12-30 at 10.29.46 PM

In order to realize a successful deployment, we need to carry out the following tasks:

  • Create the node cluster if non existent
  • Build and push the 4 service images to the registry
  • Create the 4 services if not available
  • Start the services if not running
  • Update the services configuration
  • Redeploy the services

Deploying

To deploy the application, assuming that you have linked your Digital Ocean account to Tutum, create a twitter developer application at apps.twitter.com and go to the application’s Keys and Access Tokens tab.

Screen_Shot_2014-12-30_at_10_53_57_PM

Kelonye___Twitter_Application_Management

Run the following in your favorite shell emulator using the four provided credentials:

export TUTUM_TWEED_TWITTER_CONSUMER_KEY=""
export TUTUM_TWEED_TWITTER_CONSUMER_SECRET=""
export TUTUM_TWEED_TWITTER_ACCESS_TOKEN=""
export TUTUM_TWEED_TWITTER_ACCESS_TOKEN_SECRET=""

This set’s environment variables that will be used by the tweets service to access Twitter’s streaming APIs. We also need to define two more settings to be used during deployment:

export TUTUM_USER=""
export TUTUM_APIKEY""
export TUTUM_TWEED_CLUSTER="" # the desired name of your new cluster

The Tutum API key can be obtained by logging into Tutum, clicking on the menu on the upper right corner of the screen, selecting Account info and then lastly selecting Api Key. Use your Tutum username to define the TUTUM_USER setting. All the settings defined above can alternatively be set in either the .bashrc or .bash_profile files located in your home directory for convenience.

We will be using tutum-deploy which enables a fig like management of node clusters using the Tutum API. So go ahead and install Node.js. The simplest installation step is to use your platforms package manager to install nodejs. For example, on MAC OSX, you’d use brew to install it as:

brew install node

This also installs npm, the Node.js package manager which in turn can be used to install the utility with:

npm install -g tutum-deploy

This adds the utility executable name, td, to your shell’s PATH so that it can be executed globally. You can the use the utility to first build and push the service images to your provided private Tutum registry with:

td build
td push

Once the images have been pushed, we can let Tutum use the images to deploy the services to the cluster with:

td up

For convenience, all these commands have been defined in the Makefile at the root for the project.

deploy:
  @$(MAKE) build push up

build:
@td build

push:
@td push

up:
@td up

We can then just deploy the application once by simply running:

make up

We can then hopefully visit the deployed app using the hostname provided in the the web interface!

tl;dr

How did that work .. really, what happened?

As mentioned earlier, tutum-deploy utility, when run, checks and ensures that services in a given cluster or custom node are running as required. We simply needed to define the expected state of our cluster in a tutum.yaml configuration file as:

---
clusters:

- name: "{{TWEED_CLUSTER}}"
 region: ams2
 type: 512mb
 nodes: 1
 tags:
 - tweed

services:

- name: redis
 image: "tutum.co/{{USER}}/redis"
 build: lib/redis
 containers: 1
 tags:
 - tweed

- name: tweets
 image: "tutum.co/{{USER}}/tweets"
 build: lib/tweets
 containers: 1
 env:
 TWEED_TWITTER_CONSUMER_KEY: "{{TWEED_TWITTER_CONSUMER_KEY}}"
 TWEED_TWITTER_CONSUMER_SECRET: "{{TWEED_TWITTER_CONSUMER_SECRET}}"
 TWEED_TWITTER_ACCESS_TOKEN: "{{TWEED_TWITTER_ACCESS_TOKEN}}"
 TWEED_TWITTER_ACCESS_TOKEN_SECRET: "{{TWEED_TWITTER_ACCESS_TOKEN_SECRET}}"
 require:
 - redis
 tags:
 - tweed

- name: feeds
 image: "tutum.co/{{USER}}/feeds"
 build: lib/feeds
 containers: 1
 require:
 - redis
 tags:
 - tweed

- name: web
 image: "tutum.co/{{USER}}/web"
 build: lib/web
 containers: 1
 ports:
 - inner_port: 80
 outer_port: 80
 port_name: http
 protocol: tcp
 published: true
 env:
 NODE_ENV: production
 MINIFY: 1
 PORT: 80
 require:
 - redis
 tags:
 - tweed

Reading from the top, we specified that we wished to have a deployed cluster on Digital Ocean with a single 512mb sized node. The configuration also defined the 4 services that used docker images that were pushed to our private registry. This configuration file, as shown, may contain variables which get passed in through environment variables. For example, the {{TWEED_CLUSTER}} variable got replaced with TUTUM_TWEED_CLUSTER. Notice that the environment variables have to be prefixed with TUTUM_.

When the deployment command, make up was run, we first built the service images using the corresponding Dockerfiles whose locations were specified by the build attribute, and then pushed them to the registry. Note that the image names are namespaced as tutum.co/my-tutum-username/my-image since we are using the Tutum-provided private registry.

Next, the td up began the actual deployment step where we first ensured that our cluster was created and deployed as expected. If no cluster existed, the new cluster was created and deployed appropriately.

Just like in the case of the node cluster, each of the services were created, started and deployed if necessary. A service is a group of containers that provide a specific resource, for example, database storage. Services are run in multiple containers that can span multiple nodes and even clusters. Services make themselves available to each other using service links. Keep in mind that three of our services need to connect to the Redis service. Fortunately, Tutum makes docker container linking easy. A service’s linked_to_service property contains all services that it depends on. With tutum-deploy, we only need to define the links via the require property.

In the example above, through linking, the web service, it contained the REDIS_PORT environment variable which was then used to connect to the redis store. Notice that with service links, we did not need to expose the Redis service port.

Lastly, the web service exposes a tcp port that serves the traffic on port 80. We simply needed to specify the node and container ports as:

ports:
- guest: 80
  host: 80
  name: http
  protocol: tcp
  published: true

Re-deploying

Tutum supports hot redeployment of services. If we needed to deploy a new version of the application, running tutum-deploy would redeploy the application using the newly built images by replacing the old services. This ensures that our servers remains immutable and hence less likely to experience problems.

There’s more that can be done with Tutum’s API and so have fun experimenting with it.

Tagged with: , , , , ,
Posted in Tutorial
3 comments on “Easy Docker Deployments Using Tutum API
  1. krishnasp says:

    Does tutum provide api to scale application? In the above example if I want to scale my web container to 2 instances does tutum provide api for that?

  2. kelonye says:

    Yes. You can scale several resources .e.g. :

    – services to multiple nodes/clusters
    – number of containers in a service (see `target_num_containers` at https://docs.tutum.co/v2/api/#update-an-existing-service)

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories
%d bloggers like this: