Docker Compose, AngularJS and Tutum — Part 2

tutorial angular 2

Level: Intermediate

This is part 2 of a 3 post series, where we will cover topics related to development, testing, Docker, continuous integration and continuous delivery.

It is highly recommended to first read Docker, AngularJS and Tutum – Part 1

In this tutorial you will learn how to:

  • Improve your development workflow with Gulp
  • Create a Development, Test and Production environment
  • Create a simple API with Express
  • Run MongoDB on its own docker container
  • Service orchestration with Docker Compose
  • Continuous Deployment with Tutum

Requirements

To follow the tutorial you will need these tools installed in your system:

Other tools will be mentioned and explained throughout the tutorial.

All the code of this tutorial is available on github

Let’s get started.

Clone the repo and install dependencies

$ git clone https://github.com/dciccale/docker-compose-angular-tutum.git
$ cd docker-compose-angular-tutum
$ [sudo] npm install -g gulp bower
$ npm install && bower install

Clean the repo history and push the code to your Github account:

(You will need this repo later to deploy with tutum)

$ rm -rf .git
$ git init
$ git add -A
$ git commit -m 'initial commit'
$ git remote -a origin https://github.com/<your-name>/<your-repo>.git
$ git push -u origin master

Using Gulp

We are going to automate some tasks using Gulp to speed up development and to generate a production-ready distribution of our web app.

If you haven’t used gulp before you should go the official Getting started guide and this Basic Gulp tutorial

If you have, then here we will review some useful tasks to include in your gulpfile.js

What to automate with gulp?

For this tutorial, we will focus on two main categories:

  • Improve the development workflow
  • Generate a production-ready distribution

By no means, all the tasks mentioned here are strictly necessary, and some others may be missing, but are good enough for the purpose of this tutorial.
Please feel free to leave suggestions in the comments at the end of the post.

To quickly go over the tasks I will link to the line in the gulpfile.js and make a brief explanation on what it does. I also encourage you to read the file which has plenty of comments

Improve development workflow

index task gulpfile.js:27 and gulpfile.js:154

This task will inject all the css and javascript files needed for our application to run (our own and third party) using these plugins wiredep and gulp-inject.
It will automatically add/remove a script or link tag in the index.html for every dependency on our app. These dependencies will be injected in between the special inject comments you see in index.html.

build and default tasks gulpfile.js:30 and gulpfile.js:33

build task is just a shortcut so we can execute it with gulp build.
default task is defined for convenience. If we execute gulp without any other argument, then the default task will be run.

serve task gulpfile.js:36

The serve task will run our server defined in server/app.js using gulp-nodemon and start browserSync. both modules will take care of restarting/refreshing server/browser respectively when files are changed.

Generate a production-ready distribution

vendors-js task gulpfile.js:55

This task will grab all js paths from bower.json file using main-bower-files and run the dist function defined in line gulpfile.js:181 to concatenate, minify, rename and version those assets.
This will create .tmp/vendors.min.js file.

vendors-js task gulpfile.js:61

This task will grab all css paths from bower.json file using main-bower-files and run the dist function defined in line gulpfile.js:181 to concatenate, minify, rename and version those assets.
This will create .tmp/vendors.min.css file.

vendors task gulpfile.js:67

This task will run vendors-js and vendors-css tasks.

templates task gulpfile.js:70

This task will minify all html views (gulp-htmlmin), compile them into an angular module (gulp-ng-html2js) and finally concatenate ([gulp-concat])(https://www.npmjs.com/package/gulp-concat) everything into a temporary file called .tmp/templates.js. This will be later concatenated with the final app script file.
By using this task, we prevent loading templates through xhr and cache them into angular $templateCache.

scripts and styles tasks gulpfile.js:79 and gulpfile.js:84

This will grab all our app js/css files (include previously generated template.js) and run a series of tasks to prepare them for distribution, the same process we made for vendors task creating .tmp/app.min.js and .tmp/app.min.css files.

inject task gulpfile.js:89

This will inject .vendors.min.js, vendors.min.css, app.min.js and app.min.css into the index.html where special comments are defined and output it to dist/public directory.

rev task gulpfile.js:102

On the process of generating the previous minified files the task gulp-rev was executed to generate the versioned files and create a json file in the .tmp directory that we can use to replace the paths on our index.html
So this task will use gulp-rev-collector to read that json file and replace the previous paths to the versioned paths. The json would look something like this:

{
  "app.min.js": "app.min-dc50bc920b.js"
}

copy-server task gulpfile.js:109

This will just copy all our server code into the dist directory.

copy-favicon task gulpfile.js:115

This will copy our favicon into the dist directory

copy-glyphicons task gulpfile.js:121

Since we are using bootstrap and we want the icons to be available we will copy those also into our dist directory.

clean-dist task gulpfile.js:127

This will remove the dist directory, useful for making sure that before creating the production version there are no old files there.

build-all task gulpfile.js:132

This task will call all the previous production only tasks and as a last step, minify the index.html and put it into dist/client directory.

dist task gulpfile.js:139

This task is a shortcut that will run clean-dist task first and then run build-all.

serve-dist task gulpfile.js:144

This task will run the server in production mode serving the dist directory so we can test or just view the production app in our local environment, you could also use it to run the server in whatever machine you will be running your app later.

Environments

Our server will have 3 different environment configurations where it can run.

development, test and production.

The configuration files are located at

server/config/environment

There you’ll see the following files

Each of these files will allow us define different configuration for each environment.

In index.js:37 we require the configuration file for the current environment defined by NODE_ENV and we merge the common configuration with the environment configuration.

This full configuration file is being required at the main file of the server server/app.js:8

API

Let’s create a very simple API using Express and Mongoose. Remember you’ll need mongodb installed in your system.

The connection to the MongoDB database is defined in server/config/db.js

For this tutorial we will create only 1 endpoint which will be used to handle a collection of fruits on our app.

Since our server will be also serving static files, we want to make a specific path for our api to have an easy way to make api requests.

On file server/routes.js:12 we define that every request going to /api/fruit will pass through the fruit router.

app.use('/api/fruit', require('./api/fruit'));

Let’s create the fruit router.

server/api/fruit/index.js

'use strict';

var express = require('express');
var controller = require('./fruit-controller');

var router = express.Router();

router.get('/', controller.index);
router.get('/:id', controller.show);
router.post('/', controller.create);
router.put('/:id', controller.update);
router.patch('/:id', controller.update);
router.delete('/:id', controller.destroy);

module.exports = router;

Ok, all urls are defined and also what HTTP method will be used on each handler.

Now let’s define the server/api/fruit/fruit-controller.js

/*
* GET /fruits -> index
* POST /fruits -> create
* GET /fruits/:id -> show
* PUT /fruits/:id -> update
* DELETE /fruits/:id -> destroy
*/

'use strict';

var _ = require('lodash');
var Fruit = require('./fruit-model');

// Get list of fruits
exports.index = function (req, res) {
  Fruit.find(function (err, fruits) {
  if (err) { return handleError(res, err); }
    return res.status(200).json(fruits);
  });
};

// Get a single fruit by id
exports.show = function (req, res) {
  Fruit.findById(req.params.id, function (err, fruit) {
    if (err) { return handleError(res, err); }
    if (!fruit) { return res.sendStatus(404); }
    return res.json(fruit);
  });
};

// Creates a new fruit
exports.create = function (req, res) {
  Fruit.create(req.body, function(err, fruit) {
    if (err) { return handleError(res, err); }
    return res.status(201).json(fruit);
  });
};

// Updates an existing fruit
exports.update = function (req, res) {
  if (req.body._id) { delete req.body._id; }
  Fruit.findById(req.params.id, function (err, fruit) {
    if (err) { return handleError(res, err); }
    if (!fruit) { return res.sendStatus(404); }
    var updated = _.merge(fruit, req.body);
    updated.save(function (err) {
      if (err) { return handleError(res, err); }
      return res.status(200).json(fruit);
    });
  });
};

// Deletes a fruit
exports.destroy = function (req, res) {
  Fruit.findById(req.params.id, function (err, fruit) {
    if (err) { return handleError(res, err); }
    if (!fruit) { return res.sendStatus(404); }
    fruit.remove(function (err) {
      if (err) { return handleError(res, err); }
      return res.sendStatus(204);
    });
  });
};

function handleError(res, err) {
  return res.status(500).send(err);
}

Now the mongoose model which will define the Schema of this collection.

server/api/fruit/fruit-model.js

'use strict';

var mongoose = require('mongoose');
var Schema = mongoose.Schema;

var FruitSchema = new Schema({
  name: {type: String, required: true}
});

module.exports = mongoose.model('Fruit', FruitSchema);

Run MongoDB

Before running mongo on a docker container we can try out our app and api just by running mongo daemon on a terminal window, let’s do that first.

$ mongod

Now on another terminal window in the project root directory we will start our app

$ gulp serve

This should run our server in development mode as defined in server/app.js:4 which defaults NODE_ENV to development if none is defined.

A browser tab should be opened by browsersync showing the app with a pre-populated list of fruit.

You can:
– add fruits (POST to /api/fruit)
– update a fruit name (PUT to /api/fruit/:id) with double click
– delete a fruit (DELETE to /api/fruit/:id)

If everything is working let’s continue to the next interesting part, docker-compose.

Docker Compose

Docker Compose is a tool from Docker that allows us to define a set of services that will be run by docker and also link containers to communicate with each other.

To tell docker-compose how to build and run our containers we will define it in a yml file.

But first we need to define a Dockerfile that will have all the necessary steps to build and run the main container which is our app.

Dockerfile

# This image will be based on the official nodejs docker image
FROM node:latest

# Commands will run in this directory
WORKDIR /home/app

# Add all our code inside that directory that lives in the container
ADD . /home/app

# Install dependencies and generate production dist
RUN npm update -g npm
RUN
npm install -g bower gulp &&
npm install &&
bower install --config.interactive=false --allow-root &&
gulp dist

# Tell Docker we are going to use this port
EXPOSE 8080

# The command to run our app when the container is run
CMD ["npm", "run", "start-prod"]

When docker builds an image from this Dockerfile, all necessary dependencies will be installed on the image and a production distribution will be generated using our task gulp dist task.
When the image is run, will start with the command npm start-prod which is defined in package.json:11 that executes our server with NODE_ENV=production.

We do not need a Dockerfile for the MongoDB container, because there already exists an official image.

Next we need to create the docker-compose.yml file.

docker-compose.yml

web:
build: .
ports:
  - "80:8080"
environment:
  NODE_ENV: production
links:
  - mongo
mongo:
  image: mongo
  command: --smallfiles
  ports:
    - "27017:27017"

Let’s explain what line by line.

1) web: defines a service with that name
2) build: . tells docker-compose to build the current directory into an images (has to have a Dockerfile)
3) ports: a list of ports that this service is going to use
4) - "80:8080" tells docker that our service will run on port 8080 but we want to map it to port 80
5) environment: a list of environment variables that will be defined when this service is run
6) NODE_ENV: production a key value pair of environment variable
7) links: a list of services linked to this service
8) - mongo tells docker-compose to link a service named mongo to this service
8) mongo: defines a service with that name
9) image: mongo tells docker-compose to pull the official mongo image
10) command: --smallfiles is a command that will run when this service is run
11) ports: a list of ports that this service is going to use, in this case default mongodb ports
12) - "27017:27017" default mongo ports, not mapping use as is

Go to the official website for the full docker-compose documentation.

docker-compose cli

On a terminal window in the root of the project we will use docker-compose cli tool to build and run all our services.

Create and start containers

This will build and run the defined services in detached mode.

$ docker-compose up -d

Open in your browser the docker ip to access the app, to know that ip just run

$ boot2docker ip

Enter the ip on the browser, and you should see the app up and running.

In this case without any fruit, you can add some into the list. Will be added to the production database and saved into the mongodb instance running on its own container.

So how does our app knows how to connect to the mongodb running container? Remember we have linked our web container with the mongo container in the docker-compose.yml file, and because of this, some environment variables will be exposed in our web container.

Open the file server/config/environment/production.js and in line 5 and 6 you will see that the connection address and port comes from the environment variables that docker exposed.

var MONGO_ADDR = process.env.MONGO_PORT_27017_TCP_ADDR || 'localhost';
var MONGO_PORT = process.env.MONGO_PORT_27017_TCP_PORT || 27017;

This will give us the ip and port of the mongodb instance running container.

And just for making the app a bit more robust, if for some reason in production we are not running in a container, try to connect to a running instance of mongo in the host machine with localhost and default port.

List containers

You can list the running containers and see some basic information.

$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------------------
dockercomposeangulartutum_mongo_1 /entrypoint.sh --smallfiles Up 0.0.0.0:27017->27017/tcp
dockercomposeangulartutum_web_1 node ./dist/server/app.js Up 0.0.0.0:80->8080/tcp

View output from containers

You can also see the logs of the running containers

$ docker-compose logs

Or if you want to see only the logs of specific service:

$ docker-compose logs web

Now try adding/removing fruit in the app you’ll see the requests log in the terminal.
To detach from the log output press CTRL-C

Stop services

To stop all services run:

$ docker-compose stop

Remove stopped containers

You can removed all stopped containers.

$ docker-compose rm

Enter a container

If you need to enter the running mongodb container you can.

Let’s run our app again (if not already running):

$ docker-compose up -d

See the running containers

$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------------------
dockercomposeangulartutum_mongo_1 /entrypoint.sh --smallfiles Up 0.0.0.0:27017->27017/tcp
dockercomposeangulartutum_web_1 npm run start-prod Up 0.0.0.0:80->8080/tcp

To enter the mongo container we will use docker exec -it [container_id_or_name] bash command passing the id of the running mongo container and the command to execute on the container, in this case bash.

$ docker exec -it dockercomposeangulartutum_mongo_1 bash

Now you are in the mongo container, where you can use the mongo cli, for example for querying the database.

$ mongo
$ show dbs
$ use docker-angular-tutum
$ db.fruit.find({})
$ exit

Continuous Deployment using Tutum

Deploy your application stack

1) Push your code to github/bitbucket

Make sure you created a repo in your github/bitbucket account and pushed the code you cloned at the beginning of the post.

2) Add an automated build in DockerHub https://registry.hub.docker.com/builds/add/

Choose the repository from your account and that’s it.

3) Create a Node in Tutum

4) Create a tutum.yml stack file.

mongo:
image: 'mongo:latest'
autorestart: always
ports:
  - '27017:27017'
web:
  image: 'dciccale/docker-compose-angular-tutum:latest'
  autorestart: always
  links:
    - 'mongo:mongo'
  ports:
    - '80:8080'

This file is very similar to our docker-compose but with the difference that here the web service is going to use our DockerHub image we created in step 2.

4) Create a Tutum Stack and drag&drop the stack file or copy and paste the content there and fill in a name for your stack. I will use docker-compose-angular-tutum

Click on Create and Deploy. This will deploy your container in the emptiest node (that means, the one you created in step 3).

Done! your application is deployed

Continuously deploy your application

As we’ve seen in part I here are the steps to redeploy your app every time a successful build is made in Dockerhub.

1) Go to Tutum dashboard services list and click on the web service
2) Click on Webhooks tab and create a new webhook named something like redeploy
3) Copy the URL, go to your DockerHub repository, on the right sidebar click Webhooks then click on Add Webhook, put a Short name like redeploy-tutum, paste the URL there and save.

Now every time a successful build of your Docker image is done in DockerHub, Tutum will re-deploy the container with the latest image.

The End!

Next

The next and last post of the series will be advanced and will cover:

  • Separating the API into it’s own repository and including a git submodule to the App repo
  • Create a full API using hapijs with auth token bearer strategy and user roles
  • Create a Dockerfile and docker-compose.yml for API
  • API automated testing, coverage and continuous integration with CircleCI
  • Upgrade the App to manage login and private routes
  • App automated testing, linting, coverage and continuous integration with CircleCI
  • Create an API and App Tutum stack and deploy each service on it’s own Docker image in Dockerhub
  • Continuous deployment of all services using Tutum
Tagged with: , , , , , , ,
Posted in Tutorial
8 comments on “Docker Compose, AngularJS and Tutum — Part 2
  1. deminetix says:

    Great tutorial, but is there a reason why you run gulp and npm install on the host machine?

    Docker is meant to solve the issue of running code anywhere, but each developer on a project similar to this tutorial would run the risk of developing with entirely different versions of node, gulp, bower and mongodb

    • Denis Ciccale says:

      better define the “good” versions in the host/s machine/s instead of making every dev match the same.

      now is true that even if across host machines you’ve defined a specific version of node,gulp,bower,mongo,etc (which is recommended), developers may have something different in their locals.

      basically what you would do is create and run a docker-compose-dev.yml which will solve that issue as you could run in development mode same stack that would be on the host.

      the part 3 of the tutorial will cover this, is coming soon

  2. Aksel Gresvig says:

    Great article, thanks! When will the follow up come? I’m really keen on learning how to deal with private submodules in repos that build on Tutum – I can’t find any pointers or info on how to do this anywhere! And no-one in the Slack channel will help me. This blocks our app from running on tutum.

    Thanks!

  3. Connor Ross says:

    Great article! Love the detail and explanations makes it super easy to follow.

    Saw a copy and past error with: “vendors-js task gulpfile.js:61” should be “vendors-css task gulpfile.js:61”

    Look forward to the 3rd installment.

  4. Seeing as it is much past Jan 1 2016, seems Part 3 is sort of vaporblog 🙂

  5. […] Docker Compose, AngularJS and Tutum — Part 2 […]

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories
%d bloggers like this: