Why Stacks are HUGE for Devs Using Docker

Stacks

Developing stacks for Tutum has been a bit challenging, but we are happy to see that our users are making heavy use of them. This is clearly a big step forward in our goal of providing powerful but yet flexible tools to develop, test and deploy Docker applications.

A Little Bit of Context

A primary benefit of the Docker revolution is the ability to deploy development environments even for complex microservice architectures, minimizing the amount of unexpected problems when code goes to production. Docker achieves this goal with a very small footprint, making it possible to run development environments on your local machine.

In this context, Fig (now Docker Compose) is a great piece of work that provides a YAML file specification where users are able to define relations between the different services such as links, environment variables, volumes and more. This was really convenient since YAML files follow a declarative approach of defining development environments that spin up with a single command. Soon, people started to enjoy these benefits and spread the use of the tool.

However, Fig was designed for developers and isn’t able to deploy applications in a multi-host environment. This problem only grows more complex when people want a tool to simplify the deployment of applications in different environments like staging, pre-production and production.

Implementation Challenges

At Tutum, we created stacks to address these problems. They allow you to define YAML files with needed extensions for a multi-host environment, such as deployment tags to select the target deployment host, or deployment strategies to optimize your resources. You can find more information about how to use stacks here.

A stack sits at the highest abstraction level for managing Docker containers and has strong dependencies on other capabilities that make it easy to go from your local environment to a multi-host environment. This is why our stack support was delayed until Tutum built solutions for networking, volumes and service discovery.

The implementation is based on a new stack endpoint that processes YAML files and automates Tutum Service API calls to schedule the deployment of the stack services in the right order. The stack endpoint translates the YAML syntax of a classic Fig file into the Tutum internal JSON representation, which allows us to build a dependency graph based on the links and volumes-from relations. The dependency graph lets us process the deployment of the stack in parallel whenever possible. For example, imagine the following dependency graph:

70784213

The stack endpoint would only trigger the deployment of A and B in parallel since they don’t depend on each other, but C is put on hold until A is deployed. Also, D must wait for B to be deployed, but E might be deployed in parallel with D when C finishes its deployment. These are complex task relationships that we manage by running a trigger at the end of each service deploy task. This in turn re-schedules other service deploy tasks that depend on the service that has just finished its deployment.

Using Stacks for Configuration Management

Stacks are the perfect fit for configuration management. You might need Puppet, Chef, Ansible or Salt for complex configurations, but a YAML file should suffice for the majority of use cases by just defining the links and environment variables.

We recommend the use of fig.yml for local environments, and to have different tutum.yml files for your testing, validation and staging environments. tutum.yml keeps your configuration simple, readable and versioned in your repository.

In order to facilitate the adoption of stacks in more complex production environments, Tutum is considering the following features:

  • External links: define services in your YAML files that are not running in Tutum. For example, you can define an external service to a RDS instance and Tutum links will resolve to the RDS endpoint. This makes it easy to substitute a mysql container in a local environment with a RDS instance in a production environment.

  • Modular YAML files: extend the use of a base YAML file to reduce duplication of work for different environments. i.e. having one tutum.yml YAML file that serves as the base for three different environments, and including something like import tutum.yml at the beginning of each specific YAML file for each environment.

I’d love to hear your feedback about these features and stay tuned for what is coming!

Posted in Features, General
3 comments on “Why Stacks are HUGE for Devs Using Docker
  1. Rob says:

    Definitely plus one for both of those features. Being able to link to another service like RDS would be great. Also having Yaml files split out and including a parent YAML would make like a lot easier for production/staging etc…

  2. mickdelaney says:

    +1 for external links, pretty important for common setups

  3. […] Why Stacks are HUGE for Devs Using Docker. […]

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories
%d bloggers like this: