GitLab CI Yocto Build - How to use SSTATE and DL_DIR

How to configure GitLab CI to store the SSTATE_DIR and the DL_DIR between several jobs? Currently, bitbake rebuilds the complete project every time, which is very time consuming. So i would like to use the sstage again. I tried caching, but building time increases effectively, due to the big zip/unzip overhead.

SSTATE_DIR

DL_DIR

I don't even need a shared sstate between several projects, just a method to store the output between jobs.

I'm using Gitlab 11.2.3 with a shell executor as runner.

Thanks a lot!

2 Answers
2

If you're only using one runner for this, you could potentially use GIT_STRATEGY: none, which would re-use the project workspace for the following job; relevant documentation. However, this wouldn't be extremely accurate in case you have multiple jobs running which requires the runner, as it could dilute the repository, if jobs are started from across different pipelines.

GIT_STRATEGY: none

Another way, if you're still using one runner, is you could just copy the directories out and back into the job you require.

Otherwise, you may potentially be out of luck, and have to wait for the sticky runners issue.

GIT_STRAGETY: none

You can reuse a shared-state cache between jobs simply as follows:

Specify the path to the sstate-cache directory in the .yml file of your
gitlab-ci pipeline. An example fragment from one of mine:

.yml

myrepo.yml

stages:
...
...

variables:
...
TCM_MACHINE: buzby2
...
SSTATE_CACHE: /sstate-cache/$TCM_MACHINE/PLAT3/
PURGE_SSTATE_CACHE: N
...


In my case /sstate-cache/$TCM_MACHINE/PLAT3/ is a directory in the docker container
that runs the build. This path is mounted in the docker container from a
permanent sstate cache directory on the build-server's filesystem, /var/bitbake/sstate-cache/<machine-id>/PLAT3.

/sstate-cache/$TCM_MACHINE/PLAT3/

/var/bitbake/sstate-cache/<machine-id>/PLAT3

PURGE_SSTATE_CACHE is overridable by a private variable
in the pipeline schedule settings so that I can optionally delete the cache for a squeaky clean
build.

PURGE_SSTATE_CACHE

Ensure that the setting of SSTATE_CACHE is appended to the bitbake conf/local.conf
file of the build, e.g.

SSTATE_CACHE

conf/local.conf

.build_image: &build_image
stage: build
tags:
...
before_script:
...
script:
- echo "SSTATE_DIR ?= "$SSTATE_CACHE"" >> conf/local.conf
...


Apply the same pattern to DL_DIR if you use it.

DL_DIR

Variables you use in the .yml file can be overriden by gitlab-ci trigger
or schedule variables. See Priority of variables

.yml

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.