Presenting Tutum Stream API

Tutum_blog_streamAPI

We have had a REST API for a long time. It’s useful to create new objects (eg. service, node, etc.), to get their current state, modify them, or terminate them. We have been using it for our dashboard, our CLI, and our images. Also, a lot of users are using this REST API to create awesome things on top of Tutum. Today we are releasing a new element in our API called Stream API. We hope this addition will allow you to build even more powerful applications.

A Little History

For some time, our dashboard has had an advantage over the apps that can be developed on top of Tutum. We had a socket.io server that sent notifications to every user when one of their objects was created, updated or deleted. For example, every time a new service is created or a stack is redeployed, notifications for the different changes happening are sent out. This allows Tutum to give real-time updates to the Dashboard UI, making it responsive and more interactive.

We started to see the need to have this same power in a couple of our images. One such example is our HAProxy load balancer. The load balancer polls every 30 seconds to check if the containers in a service have changed. If it has changed, the configuration of the proxy needs to be updated to handle these new containers. This generates a lot of unnecessary requests to our API as most services do not change every 30 seconds. Furthermore, when services do change, it can take up to 29 seconds for HAProxy to update and reflect these changes in its configuration.

This past month we have been changing a lot of our code to allow our images and every one of our users to be able to listen to these events by creating a new Stream API. We exchanged socket.io with a standard implementation of web sockets. We did this to ensure cross compatibility with your language of choice. Now our users are able to track in real time every container, service, node, node cluster and action in their Tutum account without needing to poll.

How to Use it

The first step is to get a websocket client; for python we recommend this one and for nodejs we recommend this one. Please let us know your suggestions for other languages in the comments section!

Once you have the library installed, you need to open a connection to the server and set the handlers for open connection, error, close connection and every message.

token = 'apikey';
username = 'username';

ws = websocket.WebSocketApp('wss://stream.tutum.co/v1/events?token={}&user={}'.format(token, username),
     on_message = on_message,
     on_error = on_error,
     on_close = on_close,
     on_open = on_open)

The most important one is, arguably, the on_message one, that’s where most of the logic of your code should revolve around. In this example, we will parse the message into a dictionary and then check if it’s an auth completed message that occurred after the connection was complete. Then we check when the socket is ready to receive messages.

After we print the type of object that was modified, all following messages will contain the new state of the object. The resource url can be requested to obtain any information from the object.

def on_message(ws, message):
  msg_as_JSON = json.loads(message)
  type = msg_as_JSON.get("type")
  if type:
    if type == "auth":
      print("Auth completed")
    else:
      print("{}:{}:{}".format(type, msg_as_JSON.get("state"), msg_as_JSON.get("resource_uri")))

This handler will generate a log like this one below:

Auth completed
action:In progress:/api/v1/action/d1c3e851-e025-433f-a29f-a44aa482fb14/
action:In progress:/api/v1/action/d1c3e851-e025-433f-a29f-a44aa482fb14/
container:Stopping:/api/v1/container/f1e1042f-de66-402f-8f64-8ba40bee5c2c/
action:In progress:/api/v1/action/d1c3e851-e025-433f-a29f-a44aa482fb14/
action:In progress:/api/v1/action/d1c3e851-e025-433f-a29f-a44aa482fb14/
container:Stopped:/api/v1/container/f1e1042f-de66-402f-8f64-8ba40bee5c2c/
action:Success:/api/v1/action/d1c3e851-e025-433f-a29f-a44aa482fb14/
action:Pending:/api/v1/action/d770dbad-0bca-4a56-818f-fe56b95e3cfb/
action:Pending:/api/v1/action/d770dbad-0bca-4a56-818f-fe56b95e3cfb/
action:In progress:/api/v1/action/d770dbad-0bca-4a56-818f-fe56b95e3cfb/
service:Stopped:/api/v1/service/ba4d5bec-8bc4-443e-b032-64f39cb5a9d8/

For examples using python and node js, check out this gist.

Using the Stream API, doors open up to some really simple but super interesting use cases. A custom notifications system, such as email updates every time a service is stopped is an obvious one. But this can also be leveraged for more complex scenarios dealing with infrastructure and/or container auto-scaling, custom application triggers, slack notifications, if-this-then-that systems, etc. Your creativity is the only limit, so let us know what are you building with the Stream API and any way in which we can make it even simpler, more flexible, and ultimately more useful to you.

What’s Next?

In the coming weeks, the Stream API will handle logs per container, docker events per node, and docker exec to a particular container.

We hope that you are as excited as we are about this new feature, and how it will evolve over the upcoming weeks. Please do not hesitate to send your feedback!

Thank you.

Tagged with: , ,
Posted in Features
5 comments on “Presenting Tutum Stream API
  1. […] recently introduced its new Tutum Stream API. This is a great new feature that allows you to use WebSockets to monitor Events from your Tutum […]

  2. […] recently introduced its new Tutum Stream API. This is a great new feature that allows you to use WebSockets to monitor Events from your Tutum […]

  3. […] open-source container load-balancer that has been developed to take full advantage of Tutum’s Stream API – dynamically updating as containers are […]

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories
%d bloggers like this: