Winning a hackathon with kepler.gl

On the 23rd and 24th of January, an internal hackathon took place at Intersec. Our team “Laws of the Universe” took part in this hackathon, with the ambition of “testing” kepler.gl, an open-source solution of geodata viz and analysis. More precisely, what we meant by “test” was a twofold objective: See if we could build nice viz based on the type of data commonly processed by our solutions Ideally, integrate them directly in our products, to demonstrate the feasibility of an industrialized solution based on this technology To be honest, before the hackathon, our knowledge of kepler.

Read more

Hackathon 0x09 – Monitoring with Prometheus/Grafana

Objective In our products, we use a home-made technology called QRRD (for Quick Round Robin Database) to store monitoring metrics (system CPU/memory monitoring, incoming event flows, …). QRRD (which is written in C) was actively developed between 2009 and 2013, but we have not been investing in it since, so it has not evolved anymore. And even if this is a really great technology (especially in terms of scaling and performances), it has the following drawbacks:

Read more

Introducing auto-formatting in an existing codebase

At Intersec, we aim at maintaining a consistent style throughout all the code, depending on the programming language. For example, the C codebase is the most consistent, with our coding rules being enforced on code review. This enforcement, however, can lead to significant time loss and frustrations when patches must be repushed to be adapted to specific rules. In the past, there has been several attempts at configuring auto-formatting tools. They were never fully satisfactory because several of our coding rules did not fit into the limited configuration options of these tools.

Read more

Hackathon 0x09 – lib-common benchmarks

The goal was to develop benchmarks on a few of our core technologies in lib-common, in order to: Be able to compare the performances of our custom implementations with standard implementations. Be able to add automated tests on performance (e.g. adding non-regression tests to ensure that changes which seem to be harmless do not worsen performance). Benchmark library The first step was to develop a benchmark library; the success criteria we established were the following (compared to the already existing benchmarks in our code base):

Read more

Hackathon 0x09 – eBPF

At Intersec, we love new technologies that can improve our working tasks, our code, and because it is fun! During Hackathon 0x09, I tested the possibility to use BPF for tracing and debugging our C codebase. What is BPF? In the beginning, BPF was a technology used for packet filtering 1. For example, when using the command tcpdump -i lo arp, BPF is used to filter ARP packets on the loopback interface.

Read more

Hackathon – 5 years later

During summer 2014 we organized our first hackathon. The rules are simple and are still up to date: The subjects are suggested by anyone in the company and there is no defined framework nor limit on what they can be although they often fall in the same categories, I’ll come back to that. Anyone can show interest in any of the proposed subjects and then work on it. In practice though, participants are mostly people from technical teams.

Read more

Improved debugging with rr

Introduction To investigate a bug, people will often resort to two methods once the bug has been reproduced: Add traces in the code. Use a debugger to instrument the execution of the bug. The second solution is much more powerful but also quite limited in the types of bug it can handle: If the instrumentation of the execution goes too far, it is impossible to go in reverse.

Read more

Intersec Object Packer Part 1 : the basics

This post is an introduction to a useful tool here at Intersec, a tool that we call IOP: the Intersec Object Packer. IOP is our take on the IDL approach. It is a method to serialize structured data to use in our communication protocols or data storage technologies. It is used to transmit data over the network in a safe manner, to exchange data between different programming languages or to provide a generic interface to store (and load) C data on disk.

Read more

Middleware (or how do our processes speak to one-another)

About multi-process programming In modern software engineering, you quickly reach the point where one process cannot handle all the tasks by itself. For performance, maintainability or reliability reasons you do have to write multi-process programs. One can also reach the point where he wants its softwares to speak to each-other. This situation raises the question: how will my processes “talk” to each other? If you already have written such programs in C, you are probably familiar with the network sockets concept.

Read more

C Modules

We do write complex software, and like everyone doing so, we need a way to structure our source code. Our choice was to go modular. A module is a singleton that defines a feature or a set of related feature and exposes some APIs in order for other modules to access these features. Everything else, including the details of the implementation, is private. Examples of modules are the RPC layer, the threading library, the query engine of a database, the authentication engine, … Then, we can compose the various daemons of our application by including the corresponding modules and their dependencies.

Read more