Our original goal for this hackathon this year was to speed up the web builds.

But, as you will see, this year is special, and we have not tackled one, but two subjects in the same hackathon!

Speed up web builds using esbuild

Before the hackathon, our web build system was basically composed of webpack , with ts-loader to transpile TypeScript into JavaScript, fork-ts-checker-webpack-plugin to enable TypeScript type checking, and eslint for additional static analysis.

The issue is that all these tools are developed in JavaScript, and to put it simply, they are slow and memory hungry!

It takes around 1m30s and 1456.5 MiB of memory to compile the web part of one of our products.

To be faster, simply do less!

At Intersec, we don’t always do some web development. In those cases, we don’t need to do the type checking and static analysis.

So a quick win was to add the handling of a new environment variable NO_WWW_CHECK=1 to our build system to skip ts-checker and eslint .

We don’t sacrifice quality for speed here as our CI system still launches ts-checker and eslint on every commit.

It now takes around 40s and 1342.4 MiB of memory to compile while hardly changing anything!

Replacing ts-loader

The next step was then to replace ts-loader as our TypeScript transpiler.

There are two candidates that can be used with webpack without changing our entire web build system, swc with swc-loader , and esbuild with esbuild-loader .

Trying swc

swc is a TypeScript and JavaScript builder written in Rust for fast building. Unfortunately, it is still very young, and as you will see it was not the final choice.

Replacing ts-loader by swc-loader was very easy. There were only two minor TypeScript issues and with some webpack configuration, at 11:00 the first day of the hackathon, swc was compiling our code base!

Unfortunately, our joy was short-lived. When actually browsing to the web page of our product, there were multiple JavaScript errors that required some deep modifications in our code base. The main issue was that the way we use wildcard imports is not properly supported by swc .

Well, swc is promising, but we don’t want to spend all our hackathon deeply modifying our code base just to use it.

Unexpected success with esbuild

So let’s try the other option, esbuild with esbuild-loader !

es-build is a TypeScript and JavaScript builder written in Go. It is older than swc with similar performances.

The configuration of esbuild-loader is quite similar to swc-loader , so two minutes later, we ran our first build with esbuild …

…and it compiled successfully directly!

Okay, so now, let’s fire the browser and see what amount of work still needs to done…

…some JavaScript errors ‘React is not defined’. This is not a big deal, and was expected. We just need to configure the jsxFactory in order to use VTYX and VueJS .

Let’s start the build again and see what else has to be fixed…

…well, the web page is working perfectly. No more JavaScript errors in the console, everything is responsive.

Is the product web page really built with esbuild ? Let’s clean everything and try again…

…well, everything is working perfectly. Neat!

Let’s push our modifications and see what is actually broken…

…all our tests are green in our CI system, no regressions!

Let’s try the compilation time and memory usage now.

Not surprisingly, if we still use ts-checker and eslint , we don’t have much improvements, it takes around 1m23s and 1454.2 MiB of memory to compile.

But with NO_WWW_CHECK=1, it’s another story! It now takes 18s and 925.3 MiB of memory to compile the product!

Bonus point, we now also use esbuild to transpile next-gen JavaScript to old JavaScript for browser compatibility. It was something that we never did before as we didn’t use babel .

Conclusion and improvements

As you can see on the graph above, the speed-up values are very significant.

In just a bit more than two hours we were done with our hackathon project.

One thing that we could do in the future is completely replace webpack with esbuild . However, this requires some changes in our build system that might need more than a hackathon.

Healthcheck in Grafana

So, it is noon on the first day of the hackathon, and we have already completed our project… what do we do now? Well, we have decided to pick another subject, and start working on it 💪.

Context

More than two years ago, during the Hackathon 0x09, we worked on the introduction of the monitoring of our products using Prometheus and Grafana (see this article ). Since then, Prometheus/Grafana have been properly integrated in our products and their usage is growing, especially with the creation of our new DevOps team. By the way, we have written our own Prometheus client in our open-source C library, the lib-common . Use it, it’s great!

The purpose of using Grafana is to easily find out if a product is working fine, exploiting metrics stored in Prometheus. However, we already have a feature in our products allowing to answer the question “Is everything OK” in real-time that is called the “Healthcheck”. It is a report, computed internally based on a lot of various indicators, and displayed on the web interface of our products, as in this example:

This is not something that can be computed based on Prometheus metrics, so there is no way to display it on Grafana currently. Which is a shame because a platform administrator cannot use Grafana only to get a full status of a platform.

So the purpose of our (second) subject is to be able to display the Healthcheck of our products on Grafana dashboards…

What was done

The first step was to read some documentation about Grafana, to understand how this goal could be achieved. We realized pretty quickly that we would have to write a panel Grafana plugin, following this tutorial .

Bootstraping it was quite easy, and luckily the builtin Grafana plugins are open-sourced and a great source of inspiration. They are coded in typescript, using the React framework. At Intersec, we use typescript as a language for our web interfaces, and the Vue.js framework, which is an equivalent of React, so it was not too difficult for us to adapt.

There were two main things to do in our plugin:

• Get the Healthcheck data from our product: we first considered creating a Grafana datasource plugin for that, but we soon realized that it was too much work for a hackathon, and that it was not needed (and probably not even a good idea). So we then decided that the panel plugin itself would fetch the data, just like the builtin news panel does. For that, we first had to expose in HTTP/REST the RPC allowing to get the Healthcheck status from our products, and then call it when the panel loads, is modified, or when the dashboard is refreshed. The URL and credentials to use can be configured in the panel options.

• Render the data: rendering the data is “just” a matter of writing a React component to display it. As usual with web development, the main challenge was to make it “pretty”, fighting with CSS rules, especially because the Healthcheck is a tree structure. But nothing is impossible…

Results

Better than a long speech, here is a screen capture showing how to add a Healthcheck panel to a Grafana dashboard:

It is even possible to have several Healthcheck panels on the same dashboard, to have the status of multiple platforms at one glance, in real-time (which is not possible from the web interface of our products):

Nice, isn’t it? 😎