More about QPOD

The previous article on QPOD was mainly about intercepting commands from the shell. The handling of these commands was not really described. This is the purpose of this article.

QPOD environment and template

Before taking a look at how QPOD handles the execution of various programs, we need to introduce the QPOD environment. An environment defines a directory in which QPOD will handle program execution.

As QPOD is designed for building and running programs, it uses the git repository to define an environment location. This makes it easy to find the environment from any sub-directory with some simple git commands.

An environment is associated with an environment template. The template defines most of the parameters like the container image to use, environment variables, ...

A simple template definition could look like this:

{
    "name": "qpod",
    "image": "localhost/centos7-qpod:latest",
    "mounts": [{
        "source": "/tmp"
    }],
    "binaries": [{
        "name": "waf"
    }, {
        "name": "gdb"
    }],
    "defaultRunArgs": [
        "-it",
        "--rm",
        "--privileged"
    ]
}

This is actually the template used to build QPOD using qpod! It just forces `waf` and `gdb` commands to be run in a container where `/tmp` will be mounted.

Running intercepted commands

The main task of QPOD is obviously to execute the intercepted command. But the way to do so can vary depending on the command. While we want to use a container when building or launching our product, most commands do not need that. For instance git commands can run on the host. So QPOD has to decide how the command will be run:

  • Where the binary is located: If the binary is located inside the environment, it must be run in a container.
  • In any other case, QPOD defaults to run the command normally. But it checks a list of binaries declared in the template. If the binary is in this list, the command runs in a container.

As we do not have many different cases, this list is usually small (about a dozen entries at most). And it does not change much from one template to another.

It's also possible to force QPOD to use a container by simply setting an environment variable.

Building the podman command line

Once QPOD has determined that the command should run in a container, it needs to create the podman command line to do so.

To run succesfully, the program must have access to the needed resources:

  • Filesystem mount
  • Network
  • PID of other program

At the same time, each container which is run from one environment must be independent from containers in other environments.

To this end, QPOD creates one pod per environment and all containers of the environment are attached to it.

Filesystem mount is handled at the container level. QPOD automatically adds the git workdir mount. This mount point is also added as `/qpod`. Doing so allows to easily reference the environment directory from the template.

Other mount points can be added by the template of the environment.

The network is shared among containers in the pod, but it's not shared with the host system. So any port that needs to be accessed from outside the container environment needs to be mapped when creating the pod.

Services

As QPOD is mainly used to build and run software, containers are usually run in foreground. But at one point, we usually depend on some background containers that run as services. For instance, to access our GUI, we need a web server.

To this end, QPOD supports services. The difference with regular commands is that a service does not need to exist in the user PATH. When referenced, QPOD runs the command declared in the service declaration. A service can be started in 3 different ways:

  • It can be set to be started automatically. In this case, QPOD checks the service state before running any container.
  • It can be set to be a dependency of a binary. QPOD ensures that the container is launched before launching the container binary.
  • Finally, it can be started manually.

Example:

$ qpod service list
services:
  web (stopped, web: 8000)
  zookeeper (stopped)
  grafana (stopped, grafana: 3000)
$ qpod service start grafana
bf9402023babe753e5944a4363b125d1beac031f248433b3ab15da8c09e6439d
$ qpod service list
services:
  web (stopped, web: 8000)
  zookeeper (stopped)
  grafana (running, grafana: 3000)
$ # Run a command that will use a container. It will automatically start web and zookeeper containers.
$ waf
35ef4d5fc000506bc34fe00c74dcdfbaa08c0a59adf4bb0e60b8be6352fac4e1
bb305d159db2d841ee1aaa04baa6930336ca0609686f4a56a307e94df27af343
$ qpod service list
services:
  web (running, web: 8000)
  zookeeper (running)
  grafana (running, grafana: 3000)

As we can see, the `web` and `zookeeper` services were automatically started.

Services can also be set to depend on the repository content. In that case, if during the execution of a container, QPOD detects that one dependency has changed, the container of the service is automatically restarted. This allows to always have a working service if the container depends on a plugin that is built by the environment.

Guillaume Chevallereau