search
Ctrlk

Protecting your keys in public Docker containers

Access to the mirrors and caches in StableBuild are limited through API keys which are listed directly in your Dockerfile. For example, here your API key is both used in FROM and set as an ARG which is used by sb-apt.sh to authenticate with the package registry.

FROM stablebuild-my-secret-api-key.dockermirror.stablebuild.com/ubuntu:focal-20231003

ARG SB_API_KEY=stablebuild-my-secret-api-key
ARG APT_PIN_DATE=2023-11-10T10:40:01Z

COPY ./sb-apt.sh /opt/sb-apt.sh
RUN bash /opt/sb-apt.sh load-apt-sources ubuntu

RUN apt update && apt install -y curl

It's important to keep these keys private, as these keys are used to bill your account for traffic. If you see weird traffic spikes, check the raw usage logs to see if a key might be leaked. You can rotate keys via Dashboard > Keys.

hashtag
Finding these keys in your Docker image

The approach to list keys in your Dockerfile is fine if you build Docker containers for internal use, but problematic if you want to publish your Docker containers publicly (in Docker Hub, or another public registry). Anyone pulling the container can see the layers that make up the container, and your API key will be listed in there. In addition, the build can write files to your container containing your API key. For instance: in the example above the key can be seen both when you inspect the layers, and its persisted on file system.

To view this, first build the container above via:

docker build -t unsecure-demo .

And then inspect the layers via divearrow-up-right to see that the key is present:

dive unsecure-demo
Key leaked through the layers of the container (inspected with dive)

You can also find the key on the file system. Run the container with an interactive shell via:

And you can find your key persisted on disk via:

Key leaked through files in the container

hashtag
Protecting your keys

Let's protect these keys. First, our base image. The FROM line:

Is not leaked to any layers (visible via dive) or on the file system; but is in the image metadata. To fix this you can create a multi-stage Docker image instead:

hashtag
Prevent key leakage through layers

Next, let's tackle the key leaking through the layers:

Instead of hard-coding the key you can use Docker build secrets. Secrets are mounted in at build time, and are not included in the final image. Here you can use:

If you build this container via:

And then inspect the image again via dive:

The hard-coded key is gone:

No longer leaking the key in the Docker image layers

hashtag
Prevent key leakage on file system

To avoid leaking the keys through the file system, you'll need to clean up any file where the key is persisted in the same step as where you use the key - otherwise someone can inspect the file system at a specific layer and recover your key that way.

From inspecting the file system earlier we found that the key is listed in /etc/apt/sources.list (written by sb-apt.sh) and that the key is present in the file names of files at /var/lib/apt/lists/ (apt cache files). We can rewrite our Dockerfile to:

When you build this container via:

And then run the container and search the file system, there are no results anymore:

hashtag
Final check, inspecting the exported image

As a final check we can export the full image (including all layers and metadata) to disk, and do a final scan for our API key:

Great 🎉! Your container now no longer contains your key, and you can safely push this container to Docker Hub or another public registry.

The above works for the Ubuntu package registry, but if you use other mirrors or caches this will require some manual work in inspecting the layers and file system to ensure no keys are leaked. Both dive and just opening an interactive shell into the container should make this relatively painless though.

hashtag
Using build secrets with Kaniko

If you use Kaniko to build your containers then build secrets are not available. However, you can use the fact that the /kaniko folder on your file system is shared between the Kaniko container and the build process. So you can use this in your Dockerfile:

Here, the target /kaniko/sb-api-key.txt is on a shared file system, so you can just write the key to this location inside the Kaniko container before calling /kaniko/executor:

You can do this f.e. by setting the entrypoint of the gcr.io/kaniko-project/executor container to sh, and the arguments to [ "-c", "echo -n 'stablebuild-my-secret-api-key' > /kaniko/sb-api-key.txt && /kaniko/executor --dockerfile=./Dockerfile" ].

PreviousBilling

Last updated