神刀安全网

Evolution of our Makefile for a Docker based project

It’s hard to move to GitLab and resist the temptation of its integrated GitLab CI . And with GitLab CI, it’s just natural to run all CI jobs in Docker containers. Yet, to avoid vendor lock of its integrated Docker support, we choosed to keep our .gitlab-ci.yml configurations minimal and do all Docker calls with GNU make instead. This also ensured, that all CI tasks remain locally reproducible. In addition, we wanted to use official upstream Docker images from the official hub as far as possible.

Yet, as always with make, it’s a danger that Makefiles themselves become projects of their own. So, let’s begin with a completely hypothetical Makefile :

all: test  test:      karma test  .PHONY: all test

Separation of concerns

At first, we want to keep all Docker related commands separate from the actual project specific commands. This lead us to have two separate Makefiles. A traditional one, which expects all the build tools and other dependencies to exist, and a Docker specific one. We named them Makefile (as already seen above) and Makefile.docker (below):

all: test  test:      docker run --rm -v $PWD:/build -w /build node:5 make test  .PHONY: all test

So, we simply run a Docker container of required upstream language image (here Node 5.x), mount our project into the container and run make for the default Makefile inside the container.

Of course, the logical next step is to abstract that Docker call into a function to make it trivial to wrap also other make targets to be run in Docker:

make = docker run --rm -v $PWD:/build -w /build node:5 make $1  all: test  test:      $(call make,test)  .PHONY: all test

Docker specific steps in the main Makefile

In the beginning, I mentioned, that we try to use the official upstream Docker images when possible. Yet, what if we need just minor modifications to them, like installation of a couple of extra packages.

Because our Makefile.docker mostly just wraps the make call for the default Makefile into a auto-removed Docker container run ( docker run --rm ), we cannot easily install extra packages into the container in Makefile.docker . This is the exception, when we add Docker-related commands into the default Makefile .

There are probably many ways to detect the run in Docker container, but my favourite is testing the existence of /.dockerenv file. So, any Docker container specific command in Makefile is wrapped with test for that file, as in:

all: test  test:      @[ -f /.dockerenv ] && npm -g i karma || true      karma test  .PHONY: all test

Getting rid of side-effects

Unfortunately, one does not simply mount a source directory from host into container and run arbitrary commands with arbitrary users with that mount in place.

To avoid all issues related to Docker creating files into mounted file system, we may run Docker without host mount, by piping project sources into container:

make = git archive HEAD | /        docker run -i --rm -v /build -w /build node:5 /        bash -c "tar x --warning=all && make $1"  all: test  test: bin/test      $(call make,test)  .PHONY: all test
  • git archive HEAD writes tarball of the project git repository HEAD (latest commit) into stdout.
  • -i in docker run enables stdin in Docker.
  • -v /build in docker run ensures /build to exist in container (as a temporary volume).
  • bash -c "tar x --warning=all && make $1" is the single command run in the container ( bash with arguments), which extracts the piped tarball from stdin into the current working directory in container ( /build ) and then executes given make target from the extracted tarball contents.

Caching dependencies

One well known issue with Docker based builds is the amount of language specific dependencies required by your project on top of the official language image. We’ve solved this by creating a persistent data volume for those dependencies, and share that volume from build to build.

For example, defining a persistent NPM cache in our Makefile.docker would look this:

CACHE_VOLUME = npm-cache  make = git archive HEAD | /        docker run -i --rm -v $(CACHE_VOLUME):/cache /        -v /build -w /build node:5 /        bash -c "tar x --warning=all && make /        NPM_INSTALL_ARGS='--cache /cache --cache-min 604800' $1"  all: test  test: bin/test      $(INIT_CACHE)      $(call make,test)  .PHONY: all test  INIT_CACHE = /     docker volume ls | grep $(CACHE_VOLUME) || /     docker create --name $(CACHE_VOLUME) -v $(CACHE_VOLUME):/cache node:5
  • CACHE_VOLUME variable holds the fixed name for the shared volume and the dummy container keeping the volume from being garbage collected by docker run --rm .
  • INIT_CACHE ensures that the cache volume is always present (so that it can simply be removed if its state goes bad).
  • -v $(CACHE_VOLUME:/cache in docker run mounts the cache volume into test container.
  • NPM_INSTALL_ARGS='--cache /cache --cache-min 604800' in docker run sets a make variable NPM_INSTALL_ARGS with arguments to configure cache location for NPM. That variable, of course, should be explicitly defined and used in the default Makefile :
NPM_INSTALL_ARGS =  all: test  test:      @[ -f /.dockerenv ] && npm -g $(NPM_INSTALL_ARGS) i karma || true      karma test  .PHONY: all test

Cache volume, of course, adds state between the builds and may cause issues that require resetting the cache containers to solve the issue. Still, most of the time, these have been working very well for us, significantly reducing the build time.

Retrieving the build artifacts

The downside of running Docker without mounting anything from the host is that it’s a bit harder to get build artifacts (e.g. test reports) out of the container. We’ve used both stdout and docker cp for this. At the end we ended up using dedicated build data volume and docker cp in Makefile.docker :

CACHE_VOLUME = npm-cache DOCKER_RUN_ARGS =  make = git archive HEAD | /        docker run -i --rm -v $(CACHE_VOLUME):/cache /        -v /build -w /build $(DOCKER_RUN_ARGS) node:5 /        bash -c "tar x --warning=all && make /        NPM_INSTALL_ARGS='--cache /cache --cache-min 604800' $1"  all: test  test: DOCKER_RUN_ARGS = --volumes-from=$(BUILD) test: bin/test      $(INIT_CACHE)      $(call make,test); /        status=$$?; /        docker cp $(BUILD):/build .; /        docker rm -f -v $(BUILD); /        exit $$status  .PHONY: all test  INIT_CACHE = /     docker volume ls | grep $(CACHE_VOLUME) || /     docker create --name $(CACHE_VOLUME) -v $(CACHE_VOLUME):/cache node:5  # http://cakoose.com/wiki/gnu_make_thunks BUILD_GEN = $(shell docker create -v /build node:5 BUILD = $(eval BUILD := $(BUILD_GEN))$(BUILD)

A few powerful make patterns here:

  • DOCKER_RUN_ARGS = sets a placeholder variable for injecting make target specific options into docker run .
  • test: DOCKER_RUN_ARGS = --volumes-from=$(BUILD) sets a make target local value for DOCKER_RUN_ARGS . Here it adds volumes from a container defined by variable BUILD .
  • BUILD is a lazily evaluated Make variable (created with GNU make thunk -pattern ). It gets its value when it’s used for the first time. Here it is set to a id of a new container with a shareable volume at /build so that docker run ends up writing all its build artifacts into that volume.
  • Because make would stop its execution after the first failing command we must wrap the make test call of docker run so that we 1) capture the original return value with status=$$? , 2) copy the artifacts to host using docker cp , 3) delete the build container, and 4) finally return the captured status with exit $$status .

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » Evolution of our Makefile for a Docker based project

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址