Build Docker image with cache on build server?

Question

We've got a Jenkins CI server that fetches our code from Git, builds it, makes a Docker image, and then ships it off to some production servers.

Our project is primarily written in Python, so "building" involves running

pip install -r requirements.txt

That works fine, except it's kind of slow. It has to fetch packages over the network, plus it has to build C libraries for a few of them (and 'lxml' is not small!).

In development, I've had success using pip-accel to speed up this process. It has the same interface as pip but it caches both the Python downloads and the built C code, so

pip-accel install -r requirements.txt

is fast.

I want to do this for our production builds, but I'm running into some obstacles.

Obviously, pip-accel needs a directory in which to store the cache. Since our CI server is what runs the builds, that's the logical place to put it. But the pip install command runs inside a fresh Docker container, so it can't just access a common directory on that server.

Docker "volumes" seem like they're designed for sharing directories with containers, but our build happens (surprise surprise) inside docker build, and only docker run lets you attach volumes. You can't attach volumes with docker build.

Is there something I'm missing? How can I run a docker build and share a cache folder with my host, outside of the container I'm in?

Answer 1

I have few reputations to comment on your question, so my answer has some questions as well.

I tried creating the same setup as yours but minified (based on your explanation above) and it seems to have improvements with the docker's own caching mechanisms

My example Dockerfile looks like this:

FROM ubuntu:14.04   

RUN apt-get update \
  && apt-get install -y python-pip python-dev build-essential \
  && pip install pip-accel

COPY requirements.txt /requirements.txt

RUN pip-accel install -r /requirements.txt

CMD tail -f /dev/null

And requirements.txt looked like this:

Flask==0.8
Jinja2==2.6
Werkzeug==0.8.3

After building this for first time (look a longer time), I added a new library "chardet==1.0.1" and now my requirements.txt looked like:

Flask==0.8
Jinja2==2.6
Werkzeug==0.8.3
chardet==1.0.1

After running the docker build, it did use all the docker's own cache which had also the old pip libraries

anovil@anovil-Latitude-E6440:~/tmp/serverfault/docker$ time docker build --rm .
Sending build context to Docker daemon 3.072 kB
Step 1 : FROM ubuntu:14.04
 ---> 89d5d8e8bafb
...
...
Removing intermediate container 337c23340e7a
Step 5 : CMD tail -f /dev/null
 ---> Running in 5cb25bc75bbe
 ---> d3dfe184934b
Removing intermediate container 5cb25bc75bbe
Successfully built d3dfe184934b

real    0m6.325s
user    0m0.024s
sys 0m0.012s

Because, docker build by default has '--force-rm=false', '--no-cache=false'.
It might be a different story if your Jenkins CI runs this build as different users or in different hosts. Otherwise, its a matter of ordering the commands in Dockerfile.
If you still have questions, can you share your sample Dockerfile and also inform here how much/often your requirements.txt changes every jenkins build.