Introducing auto-formatting in an existing codebase

At Intersec, we aim at maintaining a consistent style throughout all the code, depending on the programming language. For example, the C codebase is the most consistent, with our coding rules being enforced on code review. This enforcement, however, can lead to significant time loss and frustrations when patches must be repushed to be adapted to specific rules.

In the past, there has been several attempts at configuring auto-formatting tools. They were never fully satisfactory because several of our coding rules did not fit into the limited configuration options of these tools. This subject was born out of these attempts. However, instead of focusing on adapting the tools to our coding rules, we considered doing the opposite. What about adapting our coding rules (in particular, some of those peculiar rules) so that auto-formatting tools could be applied easily? After all, if some of our rules are too specific to exist in popular tools, maybe those rules cause more harm than improvement to our code.

Read more

Hackathon 0x09 – lib-common benchmarks

The goal was to develop benchmarks on a few of our core technologies in lib-common, in order to:

  • Be able to compare the performances of our custom implementations with standard implementations.
  • Be able to add automated tests on performance (e.g. adding non-regression tests to ensure that changes which seem to be harmless do not worsen performance).

Benchmark library

The first step was to develop a benchmark library; the success criteria we established were the following (compared to the already existing benchmarks in our code base):

Read more

Hackathon 0x09 – eBPF

At Intersec, we love new technologies that can improve our working tasks, our code, and because it is fun! During Hackathon 0x09, I tested the possibility to use BPF for tracing and debugging our C codebase.

What is BPF?

In the beginning, BPF was a technology used for packet filtering 1. For example, when using the command tcpdump -i lo arp, BPF is used to filter ARP packets on the loopback interface. BPF has since been enhanced. The BPF instructions set was extended in Linux kernel 3.15 and was called “extended BPF” or eBPF. Today “BPF” could be defined as the technology that uses a set of tools to write and execute eBPF code.

Read more

Hackathon – 5 years later

During summer 2014 we organized our first hackathon.
The rules are simple and are still up to date: The subjects are suggested by anyone in the company and there is no defined framework nor limit on what they can be although they often fall in the same categories, I’ll come back to that.

Anyone can show interest in any of the proposed subjects and then work on it. In practice though, participants are mostly people from technical teams. Teams are formed and work on their topic from Thursday morning until Friday 4pm. They can even work late or at night if they wish to.

Read more

Improved debugging with rr

Introduction

To investigate a bug, people will often resort to two methods once the bug has been reproduced:

  • Add traces in the code.
  • Use a debugger to instrument the execution of the bug.

The second solution is much more powerful but also quite limited in the types of bug it can handle:

  • If the instrumentation of the execution goes too far, it is impossible to go in reverse. A new execution is needed, with new breakpoints or watches, but also new addresses, variable values…
  • If the reproduction is random, having to restart the process and reproduce the bug again can be very time consuming.
  • Instrumenting the bug with breakpoints can render the bug non-reproducible.

For those complex bugs, the investigation will thus often end as a combination of traces + placed assertion to generate core files and being able to investigate with a debugger the state of the program.

Read more

Intersec Object Packer Part 1 : the basics

This post is an introduction to a useful tool here at Intersec, a tool that we call IOP: the Intersec Object Packer.

IOP is our take on the IDL approach. It is a method to serialize structured data to use in our communication protocols or data storage technologies. It is used to transmit data over the network in a safe manner, to exchange data between different programming languages or to provide a generic interface to store (and load) C data on disk. IOP provides data integrity checking and backward-compatibility.

Read more

Middleware (or how do our processes speak to one-another)

About multi-process programming

In modern software engineering, you quickly reach the point where one process cannot handle all the tasks by itself. For performance, maintainability or reliability reasons you do have to write multi-process programs. One can also reach the point where he wants its softwares to speak to each-other. This situation raises the question: how will my processes “talk” to each other?

If you already have written such programs in C, you are probably familiar with the network sockets concept. Those are handy (at least compared to dealing with the TCP/IP layer yourself): it offers an abstraction layer and lets you have endpoints for sending and receiving data from a process to another. But quickly some issues arise:

Read more

C Modules

We do write complex software, and like everyone doing so, we need a way to structure our source code. Our choice was to go modular. A module is a singleton that defines a feature or a set of related feature and exposes some APIs in order for other modules to access these features. Everything else, including the details of the implementation, is private. Examples of modules are the RPC layer, the threading library, the query engine of a database, the authentication engine, … Then, we can compose the various daemons of our application by including the corresponding modules and their dependencies.

Read more

One year at Intersec (actually two)

Jeez! It’s been a year already! A year working at Intersec… Time goes by so quickly! A year ago, I was thinking about giving up the idea of appending “PhD” to my name: I quit the graduate program I was in after only 10 months. Before that I had been working for a year at this other company called… wait! Was it Intersec already? Oh my God! It was!

Intersec? Really? Is that even a name? Could it be possible that some guy picked such an unsexy name for his company? This is the first train of thought that went through my brain after ending my nth phone conversation with a head hunter. An interview was scheduled a couple of days later.

Read more

Blocks rewriting with clang

Introduction

Back in 2009, Snow Leopard was quite an exciting OS X release. It didn’t focus on new user-visible features but instead introduced a handful of low level technologies. Two of those technologies Grand Central Dispatch (a.k.a. GCD) and OpenCL were designed to help developers benefit from the new computing power of modern computer architectures: multicore processors for the former and GPUs for the latter.

Alongside the GCD engine came a C language extension called blocks. Blocks are the C-based flavor of what is commonly called a closure: a callable object that captures the context in which it was created. The syntax for blocks is very similar to the one used for functions, with the exception that the pointer star is * replaced by a caret ^. This allows inline definition of callbacks which often can help improving the readability of the code.

Read more