This is the first in a series of posts where we implement a continuous integration and deployment server as a micro-webservice using OCaml, Webmachine and Rashell. We codename this project “cid” in the memory of El Cid, formerly known as Rodrigo Díaz de Vivar, the Count of Bivar and Prince of Continuous Integration and Deployment Systems, immortalized by the French author Pierre de Corneille.

Here is the list of features we want for cid and its components:

  • A cid system communicates over HTTPS, using its own key and certificate.
  • A cid system can be provided several git repositories and discover products provided by these repositories.
  • A cid system can discover how the artefcats associated to these products can be built, tested, published and deployed.
  • A cid system maintains a database holding the description of these products, their artefacts as they are produced, the results of the tests and deployments, and can present reports based on the contents of this database.
  • A cid system can be commanded to build, test, publish and deploy artefacts associated to the products it knows.
  • A cid system manages identities and policies to allow or forbid its users to perform the actions they request.
  • A cid system can be used as a an application/json interface but also offers a minimal web interface based on Bootstrap or a similar framework, and a command-line interface as well.

The program cid should therefore be useful to to achieve a smooth transition from “the one big VCS repository for all projects” organisation model to the “many small VCS repositories” model?.

Definition of micro-services

There is no formal definition of a micro-service, and one can find several attempts to characterize them. These characterizations are usually not confined to the pure technical level, the well-known “If you build it, run it” underpins this. A compact yet satisfactory way to summarize all these charaterizations can be the formulation

“A micro-service is a project unit that allows the team responsible for the project to function like a start-up.“

This means that the team responsible for the project has to work with other teams — its “customers” — to define the goals of its project. It has also absolute freedom in its choice of tools to implement the solution they want to deliver and are also responsible to manage it and to provide assistance to other teams, analogous to “customer support”.

Identification of functional subsystems

We know identify our applications subsystems, that we will implement in this series of posts.

The first generic yet central subsystem we identify is the identity and access policies management. Access policies combine subjects — the “users” of the system — with verbs and objects in rules that determine if a given subject is allowed to perform the given verb on an object. The system is also responsible to protocol all access attempts, if access has been granted or not, and to justify its decision.

The repository book holds the list of repositories holding the products we are working with. Repositories can be added, updated, removed and listed. When a repository is added, a list of branches, tags and commits, where we want to discover products and their artifacts. Since products are decoupled from repositories, we also need to work with branch or version vectors, keeping track of version information for several repositories. We call such a version vector a release.

The shipyard is responsible for building artifacts, like source tarballs and their signatures, binary packages or docker images. The working copy is a further form of artifact, which is usually not published but used for testing by code-validation tools or code-metrics. The list of artifacts that can be produced for each product at a given version is obtained by exploring the repositories.

The sandbox tests the artefacts. The list of available tests for each product is obtained by exploring the repositories. Tests are performed for a given release, each test advertises the artifacts it requires to run and the artifacts it validates.

The customs is responsible to publish artifacts after ensuring they were properly validated by tests. Publishing can mean as little as storing the artifact on an appropriate place on the filesystem, bluntly discarding it, or upload it to some public or corporate repository.

The admirality is responsible to deploy artifacts after publication, so that they arrive on production systems.

Project canvas

A canvas for this project, which is by now essentially an empty shell, can be found on GitHub under https://github.com/michipili/cid. The next post will present the implementation of a minimal server with webmachine, together with the preparation of certificates and private keys.

Your questions and remarks are welcome!