GitLab CI template: YAML anchor vs. ‘extends’ keyword

Creating a CI pipeline can get tedious quite quickly. Fortunately, there are a couple of ways to avoid creating multiple similar jobs. In case of GitLab’s .gitlab-ci.yml files, we can use job templates. Based on templates, we can easily define the actual jobs. The two most common methods to use a GitLab CI template are YAML anchors and ‘extends’ keyword. These are very similar, but there is at least one not obvious difference.

CI job template

A GitLab’s documentation defines how to create hidden jobs. It also specifies that these can be used “as templates for reusable configuration” with YAML anchors or ‘extends’ keyword. In my previous post about a CI for LaTeX documents I’ve used and example of hidden job template. It greatly simplifies the definition of multiple similar jobs.

Imagine that we want to create a CI stage with multiple unit-tests and a second stage with multiple documentations files created. We can define each job independently, but that would result in a lot of repeated definition. What’s even a lot worse, in case of any change, we’d need to update each job individually. That can easily become a big headache.

Defining a job template

Creating a hidden job template is very simple. Simply start a job’s name with a dot (.) and define all of the common jobs’ parameters. You can even use variables that are not defined in a job template but will be created later in actual jobs. Just like in the example from my previous post.

Defining an actual job based on template

The GitLab’s documentation mentions two ways to use a hidden job template to define an actual job. We can use GitLab CI template with YAML anchors or ‘extends’ keyword. In most cases, these two can be used interchangeably. However, there is at least one import difference that I’d like to highlight.

yaml-list:
  - value1
  - value2
  - value3

yaml-dictionary:
  keyX: valueX
  keyY: valueY
  keyZ: valueZ

The YAML anchor is a standard YAML feature. The ‘extends’ keyword is GitLab CI-specific keyword. Both can be used to merge the YAML arrays. But with an important difference. In both cases, a list (with elements denoted with starting dash -) with the same name is defined in both a template job and in the actual job (using a template job), the list defined in the template is overridden. But in case of dictionaries (with elements not denoted with starting dash -), the two methods behave differently. If YAML anchors are used, the dictionaries behave like lists (they are overridden). But if ‘extends’ keyword method is used, the dictionaries from a template job and an actual job (using a template job) are merged (and only if the individual keys match, they are overridden like with lists). It sounds complicated, but it becomes clear if look at the example:

A sample .gitlab-ci.yml file:

stages:
   - test

########################################
# hidden jobs used as templates

.job_template: &job_anchor
   stage: test
   tags:
      - default_tag1
      - default_tag2
   variables:
      TEST_VAR1: "test1"
      TEST_VAR2: 42
   script:
      - echo "TEST_VAR1 = $TEST_VAR1"
      - echo "TEST_VAR2 = $TEST_VAR2"
   artifacts:
      when: on_failure
      expire_in: 1 week
      paths:
         - build/

########################################
job with extends keyword:
   extends: .job_template
   tags:
      - aaaa
   variables:
      TEST_VAR1: 11
      TEST_VAR3: 314
   script:
      - echo "TEST_VAR3 = $TEST_VAR3"
   artifacts:
      expire_in: 2 week

job with YAML anchor:
   <<: *job_anchor
   tags:
      - bbbb
   variables:
      TEST_VAR1: 11
      TEST_VAR3: 314
   script:
      - echo "TEST_VAR3 = $TEST_VAR3"
   artifacts:
      expire_in: 5 day

We defined a hidden job template called job_template, with anchor job_anchor. It defines a stage, two tags, two variables two lines of script and artifacts configuration. We also defined two separate actual jobs: one that uses a template job by ‘extends’ keyword (called job with extends keyword) and a second one that uses a template job by YAML anchor (called job with YAML anchor). Both of them define a different tag, two variables (one with the same name as a variable in template job), one line of script and a different value of one artifact configuration key.

A final merged jobs look like this:

---
stages:
- test

########################################
# hidden jobs used as templates

.job_template:
  stage: test
  tags:
  - default_tag1
  - default_tag2
  variables:
    TEST_VAR1: test1
    TEST_VAR2: 42
  script:
  - echo "TEST_VAR1 = $TEST_VAR1"
  - echo "TEST_VAR2 = $TEST_VAR2"
  artifacts:
    when: on_failure
    expire_in: 1 week
    paths:
    - build/

########################################
job with extends:
  stage: test
  tags:
  - aaaa
  variables:
    TEST_VAR1: 11
    TEST_VAR2: 42
    TEST_VAR3: 314
  script:
  - echo "TEST_VAR3 = $TEST_VAR3"
  artifacts:
    when: on_failure
    expire_in: 2 week
    paths:
    - build/
  extends: ".job_template"

job with anchor:
  stage: test
  tags:
  - bbbb
  variables:
    TEST_VAR1: 11
    TEST_VAR3: 314
  script:
  - echo "TEST_VAR3 = $TEST_VAR3"
  artifacts:
    expire_in: 5 day

We can see that in both cases the tags and script lines are overridden. In a job using YAML anchors, all of the variables from a template jobs are removed, and only the variables from the actual jobs are present in the final (merged) CI configuration. Also the artifacts configuration from a template job is removed, and only the configuration from the actual job definition are present.

In a job using ‘extends’ keyword, three variables are present: two from the job’s definition and one from the template. One of the template’s variables was overridden by a variable with the same name from the actual job definition. Similar thing happened to artifacts configuration. All configuration from the template stayed intact, except the one parameter that was overridden in the actual job definition.