Node.js and Continuous Deployment with CircleCI, Docker Hub and Tutum

Node js, circleCI, docker, tutum

This post follows a live-demo given by @kytwb during Beercamp Chiang Mai on May 20th, 2015.

In this post, we’re going to see how to get your next web project up and running quickly and easily, using a Node.js starter kit and hosted services to avoid the struggle of self-hosted and self-managed solutions. This workflow is specifically suitable to ship minimum viable products fast with a small team. You need to get yourself a GitHub plan to be able to make your repository private, apart from that we will be using free plans.

Table of Contents

  1. Getting started: Node.js Boilerplate
  2. Continuous Integration using CircleCI
  3. Continuous Delivery using Docker Hub
  4. Continuous Deployment using Tutum
  5. Going further

Getting started: Node.js Boilerplate

First things first, we need a code base. We’re going to use the open source Node.js boilerplate sahat/hackathon-starter as a base for our project. Let’s create the repository that will host our code and process to a clone – you usually can’t just fork a public repository into a private one, that’s why we’re cloning it:

$ git clone --bare git@github.com:sahat/hackathon-starter.git
$ cd hackathon-starter.git
$ git push --mirror git@github.com:kytwb/project-name.git
$ cd - && rm -rf hackathon-starter.git
$ git clone git@github.com:kytwb/project-name.git

The code base being open source, it might be subject to a lot of changes. Add the original repository as a remote, here called upstream, then make a pull from upstream to master every time you want to get those updates. The pull generates a merge commit on master.

$ cd project-name
$ git remote add upstream
git@github.com:sahat/hackathon-starter.git
$ git pull upstream master
$ git push origin master

Out of the box, if you don’t have MongoDB installed, configured and running, you won’t be able to run the app. We will use Docker Compose for a more efficient development workflow and go around this problem. I invite you to read the linked blog post to understand how Docker Compose works and helps in this case; it’ll be useful as well later on when deploying our stack to production using Tutum.

Meanwhile, here’s the docker-compose.yml file for our stack:

app:
  image: node
  volumes:
    - ./:/root/
  working_dir: /root/
  command: node app.js
  ports:
    - "3000:3000"
  links:
    - database
database:
  image: mongo

We now need to slightly edit our config/secrets.js file for it to properly resolve the database:

db: process.env.MONGODB || 'mongodb://' + (process.env.DATABASE_1_PORT_27017_TCP_ADDR || 'localhost') + ':' + (process.env.DATABASE_1_PORT_27017_TCP_PORT || '27017') + '/test',

You can then run docker-compose up -d and visit http://localhost:3000.
Don’t forget to push your new changes:

$ git add docker-compose.yml config/secrets.js
$ git commit -m "Setup docker-based development workflow"
$ git push

Continuous Integration using CircleCI

The original code base comes with a basic battery of tests, so we can directly go to setting up our continuous integration.

  1. Create an account on CircleCI
  2. From the sidebar, click the Add Projects icon
  3. Choose your account then press Build project next to your repository name

And that’s it. From here, you will see your tests running and hopefully showing a successful build. You can trigger a build yourself by pushing some changes and, by the same occasion, remove the now deprecated .travis.yml present in the original code base.

$ git rm .travis.yml
$ git commit -m "Remove deprecated .travis.yml";
$ git push origin master

CircleCI works seamlessly by default in our case and doesn’t require you to explicitly define your stack or the services required by your project – node.js and mongo in our case. It automatically provisions the container running your tests with many default services and tries to auto-detect how to run your tests.

Continuous Delivery using Docker Hub

Now we have a tested code base that goes through continuous integration, we can start thinking about deploying our application, first by continuously delivering a deployable version of our project. What we will deliver and deploy in production will be a Docker image, so we need to « Dockerize » our application by creating a Dockerfile.

FROM node:0.12
COPY . /root/
RUN cd /root/; npm install --production
EXPOSE 3000
CMD ["node", "/root/app.js"]

You can find more details on « Dockerizing a Node.js web app » here.
Now that we have a Dockerfile, we can start setting up our continuous delivery.

  1. Create an account on Docker Hub and sign in
  2. Click Add Repository > Automated Build
  3. Select the repository previously created on GitHub
  4. Uncheck « When active we will build when new pushes occur »
  5. Click Create Repository

We unchecked the option on step 4 so we can trigger a new build when we want it, and not every time new pushes occur. We want to create a new Docker image for our application when new pushes occur on master and when continuous integration tests have been successfully run. The Docker Hub build should then be triggered when CircleCI succeeds.

  1. From the Docker Hub repository page, click on SettingsBuild Triggers
  2. Toggle on the Trigger Status to be able to see your Trigger URL
  3. Create the following circle.yml file at the root of your repository
    deployment:
      dockerhub:
        branch: master
        commands:
          - $TRIGGER_DOCKER_HUB_BUILD
    
  4. Go back to your project’s CircleCI details page
  5. Click on Project settingsEnvironment variables
  6. Create a new variable named TRIGGER_DOCKER_HUB_BUILD with value set to:
    curl -H "Content-Type: application/json" --data '{"build": true}' -X POST <YOUR_TRIGGER_URL>
    

Make sure to replace YOUR_TRIGGER_URL with your actual Trigger URL in that last step.
We now push the newly created files to our repository:

$ git add Dockerfile circle.yml
$ git commit -m "Add Dockerfile and circle.yml to trigger Docker image build on CI success"
$ git push origin master

And you’re good to go! This last push should normally trigger a green build on CircleCI, which will then trigger a Docker image build on Docker Hub. Check for yourself.

Continuous Deployment using Tutum

So far, we haven’t talked about server and production environment.
For this part, we’re going to use Tutum and get everything up and running from there.

Create a node cluster ready for Docker deployment

  1. Create an account on Tutum and sign in
  2. Go to Account info > Cloud providers and connect the provider(s) of your choice
    You might want to check AWS Free Tier if you don’t have any cloud provider yet
  3. Under the Nodes tab, click Launch new node cluster
  4. Select a cluster name (e.g. project-name-production), server type and cluster size (one should be enough to begin with, you can scale later on) then click Launch node cluster

At this point, you should have your server(s) ready for production and reachable on a node.tutum.io sub-domain. Feel free to point your custom domain to your server.

Deploy your application stack

Time to use the production-ready version of our application: the Docker image built by Docker Hub. Remember I told you the docker-compose.yml will come handy? To deploy our application, we’re gonna use Tutum Stacks, which are YAML files very similar to Docker Compose files.
Here the tutum.yml we’re going to use:

app:
  image: kytwb/project-name
  ports:
    - "80:3000"
  links:
    - database
  tags:
    - project-name-production
database:
  image: mongo

Differences with our docker-compose.yml file: we use our application image, so no more shared volume; we specify (deploy) tags to let Tutum know on which node to deploy the service, and we bind our app port to the port 80 of the host so our app can be directly accessible via our custom domain name or our node.tutum.io subdomain.

Before importing our stack file in Tutum, we need to give Tutum access to our private image:

  1. Under the Services tab, click on Create service
  2. Go to the Private images tab then Add image
  3. Fill the form with your image name (e.g. kytwb/project-name) and your Docker Hub credentials
  4. Press Add Image

Now we have our tutum.yml and our private image added to Tutum, let’s deploy:

  1. Under the Stacks tab, click on Create your first stack
  2. Set a name for your stack (e.g project-name)
  3. Paste the content of your tutum.yml in the Stack file textarea
  4. Click on Create and deploy

Congratulations! You just deployed your application.

Continuously deploy your application

We are a few extra steps from achieving continuous deployment. We now want to automatically redeploy our application service whenever there is a new image available on Docker Hub. The same way we triggered the Docker Hub build from CircleCI, we are going to trigger a Tutum Redeploy from Docker Hub when a new image is successfully published.

  1. From the Services tab, click on your application service
  2. Go to the Triggers tab and add a redeploy trigger named Docker Hub
  3. Refresh the page (this is needed at the time of writing) and copy the trigger URL
  4. Go back to your Docker Hub repository page, then click on Settings > Webhooks
  5. Add a webhook calling the trigger URL you just copied

Now, push some edits to your repository:

$ echo "# project-name" > README.md && git add README.md
$ git commit -m "Update README.md"
$ git push origin master

And follow your application as it goes through your deployment pipeline:
Development→ GitHub → CircleCI → Docker Hub → Tutum → Production

Going Further

I can imagine I’ve been going pretty fast on some points, so feel free to ask me your questions in the comments. Some steps could definitely be enhanced or done differently, but I just tried to show a straight-forward way to reach continuous deployment.

Meanwhile, here are some things you could do to go further with this stack:

Tagged with: , , , , ,
Posted in Tutorial
Categories
%d bloggers like this: