Developing stacks for Tutum has been a bit challenging, but we are happy to see that our users are making heavy use of them. This is clearly a big step forward in our goal of providing powerful but yet flexible tools to develop, test and deploy Docker applications.
A Little Bit of Context
A primary benefit of the Docker revolution is the ability to deploy development environments even for complex microservice architectures, minimizing the amount of unexpected problems when code goes to production. Docker achieves this goal with a very small footprint, making it possible to run development environments on your local machine.
In this context, Fig (now Docker Compose) is a great piece of work that provides a YAML file specification where users are able to define relations between the different services such as links, environment variables, volumes and more. This was really convenient since YAML files follow a declarative approach of defining development environments that spin up with a single command. Soon, people started to enjoy these benefits and spread the use of the tool.
However, Fig was designed for developers and isn’t able to deploy applications in a multi-host environment. This problem only grows more complex when people want a tool to simplify the deployment of applications in different environments like staging, pre-production and production.
At Tutum, we created stacks to address these problems. They allow you to define YAML files with needed extensions for a multi-host environment, such as deployment tags to select the target deployment host, or deployment strategies to optimize your resources. You can find more information about how to use stacks here.
A stack sits at the highest abstraction level for managing Docker containers and has strong dependencies on other capabilities that make it easy to go from your local environment to a multi-host environment. This is why our stack support was delayed until Tutum built solutions for networking, volumes and service discovery.
The implementation is based on a new stack endpoint that processes YAML files and automates Tutum Service API calls to schedule the deployment of the stack services in the right order. The stack endpoint translates the YAML syntax of a classic Fig file into the Tutum internal JSON representation, which allows us to build a dependency graph based on the
volumes-from relations. The dependency graph lets us process the deployment of the stack in parallel whenever possible. For example, imagine the following dependency graph:
The stack endpoint would only trigger the deployment of
B in parallel since they don’t depend on each other, but
C is put on hold until
A is deployed. Also,
D must wait for
B to be deployed, but
E might be deployed in parallel with
C finishes its deployment. These are complex task relationships that we manage by running a trigger at the end of each service deploy task. This in turn re-schedules other service deploy tasks that depend on the service that has just finished its deployment.
Using Stacks for Configuration Management
Stacks are the perfect fit for configuration management. You might need Puppet, Chef, Ansible or Salt for complex configurations, but a YAML file should suffice for the majority of use cases by just defining the links and environment variables.
We recommend the use of
fig.yml for local environments, and to have different
tutum.yml files for your testing, validation and staging environments.
tutum.yml keeps your configuration simple, readable and versioned in your repository.
In order to facilitate the adoption of stacks in more complex production environments, Tutum is considering the following features:
- External links: define services in your YAML files that are not running in Tutum. For example, you can define an external service to a RDS instance and Tutum links will resolve to the RDS endpoint. This makes it easy to substitute a mysql container in a local environment with a RDS instance in a production environment.
Modular YAML files: extend the use of a base YAML file to reduce duplication of work for different environments. i.e. having one
tutum.ymlYAML file that serves as the base for three different environments, and including something like
import tutum.ymlat the beginning of each specific YAML file for each environment.
I’d love to hear your feedback about these features and stay tuned for what is coming!