Background

After opening my blog, I am quite lazy to write and update blog posts. Part of the reason is that I set up and hosted my own blog on my own server. But all my post are saved in the Github repo. So every time I wrote a blog, I need commit it and push it to the Git repo, then login to my server, and manually do a git pull to fetch the newly created post to display them. This is quite tedious so I really want to get rid of this step. Luckily, I found that Github provide a wonderful service called Webhook that can help me to solve the headache.

What is Webhook?

First, we should understand what is Webhook. Wikipedia gave us a well-defined definition:

Webhooks are “user-defined HTTP callbacks”. They are usually triggered by some event, such as pushing code to a repository or a comment being posted to a blog.

From Wikipedia

Basically, if we have configured the Github webhook for a repo, every time we do something to that Github repo (trigger an event), a HTTP request will be sent to the designated endpoints with the event data. So the user on the endpoint could make use of the event data and do some meaningful things. With the help of Webhook, developers could integrate or subscribe their services for these special Github events, and use it in things like a CI builds, a code backup, or (what I will do) deploy the code to the server.
For example, Slack provide integration with Github. It you push come commits or send a PR, Slack could display some message in the channel and notify users, which is quite helpful for a team to track the progress.

Solution design

So how could I make use of the Webhook for my blog? Let’s say if I add a new post, and write some agreed keywords in the commit message (such as “NEW POST”). If I push this commit to the Github, Github will send the payload to my server’s designated endpoint. Once I receive this payload, I could check the commit message contained in the payload data. If it has the keyword, I will execute git pull on my server. And hence the blog is in sync with Github.

Express Middleware

As my blog is integrated in my website, and my website is built with Express framework. I probably have two choices: either use a middleware or create a router for it. As I wish to make it a more general service that could be used by other people who is using Express framework, I choose to build the service as a middleware. Express middleware, as its name suggests, is basically a service that is specifically designed for a purpose, and it has full access to the request and response object, as well as a special next callback. It has the ability to verify request object and add more data to it, and it could respond to the request if the request matches its functionality. If could then call next() to pass the modified request and let next middleware to handle it.

Github Signature

It is important to protect our webhook handler, otherwise anyone could fake such request and send to our endpoint. Then our system may wrongly believed the request is from Github and hence execute some tasks.
Github will ask us to set up a secret token. When the secret token is set, GitHub uses it to create a hash signature with each payload. So we will also verify the signature for the incoming payload to make sure it really comes from Github.
The Signature could be found in header as X-Hub-Signature.

Implementation

The code could be found here
As Express app by default will use two body-parser middlewares for parsing JSON data as well as form data in POST request (since the body in POST request is sent as Stream in Node.js, this two middlewares help to concatenate the buffer to a complete object, and put it to req.body for the later usage). In my first implementation I will assume these two middlewares are already being used.

Some implementation highlights:

  1. I make my middleware works like a EventEmitter. This is achieved by
    1
    2
    Object.setPrototypeOf(githookHandler, EventEmitter.prototype);
    EventEmitter.call(githookHandler);

The benefit for making it a EventEmitter (kind of) is we don’t need to write the complex handling logic when in the middleware. By emitting events, we could write the event handler in a centralized place, which makes the code simpler and more manageable.

  1. I use another package called buffer-equal-constant-time to verify if the signature matches my hash. This library is quite simple. It makes uses of the bit-wise comparison of Buffer object to achieve constant time equality check (xor two buffer object and see if result is zero). This will improve the comparison performance.

Outcome

  1. I learned about how to set up Webhook on Github.
  2. Gained experience developing node package, and have a deeper insight in Node API.
  3. Gained experience with server-end JavaScript, like how to handle POST data in the request.
  4. The package could be used in my future projects. (Currently 124 downloads without any promotion, which is quite good in my mind :D)
  5. This blog post is auto-deployed on my blog with the help of this package :)