Install, configure and run Gitlab Runner with the Docker executor

Gitlab makes continuous integration possible with Gitlab Runners. In this short blog, we will go through all the steps required in order to have Gitlab Runner ready to run and test your code immediately after a change is made to your code base.

1. Installing Gitlab Runner
2. Configuring Gitlab Runner
3. Running Gitlab Runner
   
   

1. Installing Gitlab Runner

There are multiple ways for installing Gitlab Runner. Gitlab docs are very (overly) instructive and can be found at https://docs.gitlab.com/runner/install/linux-repository.html.

For our purposes, we will use the packaged version:

sudo apt-get install gitlab-ci-multi-runner

2. Configuring Gitlab Runner

Gitlab Runner offers multiples environments (executors) for running and testing your code. I highly recommend using the Docker executor. It has the following advantages:

1. Ability to test your code in an environment close to your production environment.
2. Running untested (since this will the first time running it!) code in an isolated environment.
3. Reproducible results since a docker image is immutable.

Configuration is comprised of multiple steps through a set of prompts. To start configuring, run the following command:

sudo gitlab-ci-multi-runner register

The first prompt will be:

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):

 If the gitlab instance is running on http://mygitlabinstance.com then the registration address should be: http://mygitlabinstance.com/ci

The following prompt will be:

Please enter the gitlab-ci token for this runner:

You can find the token at http://mygitlabinstance.com/admin/runners

Please enter the gitlab-ci description for this runner:

Enter a human readable description which will appear in the admin page.

Please enter the gitlab-ci tags for this runner (comma separated):

Imagine there are java and c++ projects. Each may have their own set of OS particularities based on your production setup. You could have a separate docker runner for your Java and C++ apps which we will specify in the prompts below. More to the point, there may be multiple Java or C++ apps working with specific versions of the compiler.

In this particular example, enter “java,v1.8” as a tag.

Whether to run untagged builds [true/false]:

If true, the gitlab runner will run for changes made to the repository, tagged or untagged. Setting it to true is my preference as it allows error detection earlier in the development cycle.

Please enter the executor: docker-ssh, virtualbox, ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, shell:

We are focusing on docker executors so let’s select ‘docker’.

Please enter the default Docker image (e.g. ruby:2.1):

This is key and this is where tags become very useful. If you want this runner to be used to compile and test Java 1.8 programs then you would select an image which has a Java 1.8 compiler. If your project is built using Maven, that should be in the image as well.

The docker registry used will be whichever is set in the docker engine. By default, the Gitlab runner is setup to always pull images during each run. That can be changed.

Please refer to the next post on common errors/troubleshooting.

3. Running Gitlab Runner

In order for the project to trigger the runner, a .gitlab-ci.yaml must be created at the root of the project.

In order to build and test a java project using maven, the docker image used must have java and maven installed.

The .gitlab-ci.yaml can be as simple as:

image: ubuntu_runner:latest

job:
  script:
   - mvn install -q

There may be several Gitlab runners already configured for different purposes, different languages, different versions and so on so forth. To ensure we use the runner we have just configured, we need to specify the runner through tags in the yaml file.

Early, we had specified “java” and “v1.8” as tags and those can set in the yaml file.

image: ubuntu_runner:latest

job:
  tags:
   - java
   - v1.8
  script:
   - mvn install -q 

The next post focuses on Gitlab Runner troubleshooting.

Common errors setting up Gitlab runner using the Docker executor

When you first register your Gitlab runner to use the Docker executor, you may run into the following issues:

• Problem1: You get a host unknown error when the runner attempts to checkout code: fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@my.server.com/my-projects/my-project.git/': Couldn't resolve host 'my.server.com'. Chances are you are not running a local DNS server that your Docker container can use to resolve my.server.com. 
• Problem 2: Everything is setup correctly but Gitlab runner always attempts to pull Docker images from Docker Hub.
• Problem 3: Gitlab runner may complain that you are checking out code in the root directory.

Below is a solution to each of the 3 stated problems above.

We will make the following assumptions for the purpose of this example:

•  Gitlab runner can access Gitlab server.

•  Local Docker image: my_runner_image

•  Hostname of machine hosting my Gitlab server: my.server.com

•  IP address of machine hosting my Gitlab server: 192.168.1.144

•  Default Gitlab runner configuration file: /etc/gitlab-runner/config.toml

•  If you have a home network and lack a DNS Server, it is preferable to set a fixed IP address for your machines.

Solutions:

1. Open the default Gitlab runner configuration file (/etc/gitlab-runner/config.toml) with your favorite editor which will look something like this:

concurrent = 1

check_interval = 0

[[runners]]

 name = "myRunner"

 url = "http://192.168.1.144/ci"

 token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"

 executor = "docker"

   [runners.docker]

   tls_verify = false

   image = "my_runner_image:latest"

   privileged = false

   disable_cache = false

   volumes = ["/cache"]

 [runners.cache]

The goal here is tell the container how to resolve my.server.com which the Gitlab runner will inject in the Docker container. Add “extra_hosts” as an extra variable in the configuration:

[[runners]]

 name = "myRunner"

 url = "http://192.168.1.144/ci"

 token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"

 executor = "docker"

   [runners.docker]

   extra_hosts = ["my.server.com:192.168.1.144"]

   tls_verify = false

   image = "my_runner_image:latest"

   privileged = false

   disable_cache = false

   volumes = ["/cache"]

 [runners.cache]

2. Gitlab runner has the concept of a pull policy of either always attempting to download the image from the registry, download if not available locally or never download. By default, it always attempts to download images so if you are trying to use an image you have built locally, it will fail. The best option is to download when not available:

[[runners]]

 name = "myRunner"

 url = "http://192.168.1.144/ci"

 token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"

 executor = "docker"

   [runners.docker]

   extra_hosts = ["my.server.com:192.168.1.144"]

   pull_policy = "if-not-present"

   tls_verify = false

   image = "my_runner_image:latest"

   privileged = false

   disable_cache = false

   volumes = ["/cache"]

 [runners.cache]

3. Create a directory in your container which you set via your Dockerfile:

RUN mkdir -p /my_build_directory

         Specify in the Gitlab runner configuration file the build directory:

           [[runners]]

 name = "myRunner"

 url = "http://192.168.1.144/ci"

 token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"

 executor = "docker"

 builds_dir = "/my_build_directory"

   [runners.docker]

   extra_hosts = ["my.server.com:192.168.1.144"]

   pull_policy = "if-not-present"

   tls_verify = false

   image = "my_runner_image:latest"

               privileged = false

   disable_cache = false

   volumes = ["/cache"]

 [runners.cache]

The code will now be checked out in /my_build_directory

If you have any questions or feedback, please do not hesitate to drop a note!