# Protecting your keys in public Docker containers Access to the mirrors and caches in StableBuild are limited through API keys which are listed directly in your Dockerfile. For example, here your API key is both used in `FROM` and set as an `ARG` which is used by `sb-apt.sh` to authenticate with the package registry. ```docker FROM stablebuild-my-secret-api-key.dockermirror.stablebuild.com/ubuntu:focal-20231003 ARG SB_API_KEY=stablebuild-my-secret-api-key ARG APT_PIN_DATE=2023-11-10T10:40:01Z COPY ./sb-apt.sh /opt/sb-apt.sh RUN bash /opt/sb-apt.sh load-apt-sources ubuntu RUN apt update && apt install -y curl ``` It's important to keep these keys private, as these keys are used to bill your account for traffic. If you see weird traffic spikes, check the raw usage logs to see if a key might be leaked. You can rotate keys via **Dashboard > Keys**. ### Finding these keys in your Docker image The approach to list keys in your Dockerfile is fine if you build Docker containers for internal use, but problematic if you want to publish your Docker containers publicly (in Docker Hub, or another public registry). Anyone pulling the container can see the layers that make up the container, and your API key will be listed in there. In addition, the build can write files to your container containing your API key. For instance: in the example above the key can be seen both when you inspect the layers, and its persisted on file system. To view this, first build the container above via: ``` docker build -t unsecure-demo . ``` And then inspect the layers via [dive](https://github.com/wagoodman/dive) to see that the key is present: ``` dive unsecure-demo ```

Key leaked through the layers of the container (inspected with dive)

You can also find the key on the file system. Run the container with an interactive shell via: ``` docker run --rm -it unsecure-demo bash ``` And you can find your key persisted on disk via: ``` grep -rio --exclude-dir={ece,pytorch,sys,proc} 'stablebuild-my-secret-api-key' / find / -name "*stablebuild-my-secret-api-key*" ```

Key leaked through files in the container

### Protecting your keys Let's protect these keys. First, our base image. The `FROM` line: ```docker FROM stablebuild-my-secret-api-key.dockermirror.stablebuild.com/ubuntu:focal-20231003 ``` Is not leaked to any layers (visible via dive) or on the file system; but is in the image metadata. To fix this you can create a multi-stage Docker image instead: ```docker FROM stablebuild-my-secret-api-key.dockermirror.stablebuild.com/ubuntu:focal-20231003 AS base FROM scratch COPY --from=base / / ``` #### Prevent key leakage through layers Next, let's tackle the key leaking through the layers: ``` ARG SB_API_KEY=stablebuild-my-secret-api-key ``` Instead of hard-coding the key you can use [Docker build secrets](/en/mirrors-and-caches/docker-mirror.md). Secrets are mounted in at build time, and are not included in the final image. Here you can use: ```docker FROM stablebuild-my-secret-api-key.dockermirror.stablebuild.com/ubuntu:focal-20231003 AS base FROM scratch COPY --from=base / / ARG APT_PIN_DATE=2023-11-10T10:40:01Z COPY ./sb-apt.sh /opt/sb-apt.sh # We mount secret "sb-api-key" to file "/kaniko/sb-api-key.txt" # And then we load the content of the file into the env variable SB_API_KEY RUN --mount=type=secret,id=sb-api-key,target=/kaniko/sb-api-key.txt \ SB_API_KEY=$(cat /kaniko/sb-api-key.txt) bash /opt/sb-apt.sh load-apt-sources ubuntu RUN apt update && apt install -y curl ``` If you build this container via: ``` export SB_API_KEY=stablebuild-my-secret-api-key docker build -t demo-build-secrets --secret id=sb-api-key,env=SB_API_KEY . ``` And then inspect the image again via dive: ``` dive demo-build-secrets ``` The hard-coded key is gone:

No longer leaking the key in the Docker image layers

#### Prevent key leakage on file system To avoid leaking the keys through the file system, you'll need to clean up any file where the key is persisted **in the same step** as where you use the key - otherwise someone can inspect the file system at a specific layer and recover your key that way. From inspecting the file system earlier we found that the key is listed in `/etc/apt/sources.list` (written by `sb-apt.sh`) and that the key is present in the file names of files at `/var/lib/apt/lists/` (apt cache files). We can rewrite our Dockerfile to: ```docker FROM stablebuild-my-secret-api-key.dockermirror.stablebuild.com/ubuntu:focal-20231003 AS base FROM scratch COPY --from=base / / ARG APT_PIN_DATE=2023-11-10T10:40:01Z COPY ./sb-apt.sh /opt/sb-apt.sh # Backup the original /etc/apt/sources.list (will restore later) RUN cp /etc/apt/sources.list /etc/apt/sources.list.bak # 1. We mount secret "sb-api-key" to file "/kaniko/sb-api-key.txt" # 2. Load the content of the file into the env variable SB_API_KEY, and run `sb-apt.sh` to load the sources - # this creates '/etc/apt/sources.list' with the key on FS # 3. Do the apt update && apt install in one run command (so /etc/apt/sources.list is not persisted in a layer) # 4. Copy the original sources.list back # 5. Remove cached files from apt which have our key in /var/lib/apt/lists/ RUN --mount=type=secret,id=sb-api-key,target=/kaniko/sb-api-key.txt \ SB_API_KEY=$(cat /kaniko/sb-api-key.txt) bash /opt/sb-apt.sh load-apt-sources ubuntu && \ apt update && \ apt install -y curl && \ cp /etc/apt/sources.list.bak /etc/apt/sources.list && \ rm -r /var/lib/apt/lists/ ``` When you build this container via: ``` export SB_API_KEY=stablebuild-my-secret-api-key docker build -t demo-clean-fs --secret id=sb-api-key,env=SB_API_KEY . ``` And then run the container and search the file system, there are no results anymore: ```bash $ docker run --rm -it demo-clean-fs bash root@012e063d3d45:/# grep -rio --exclude-dir={ece,pytorch,sys,proc} 'stablebuild-my-secret-api-key' / # no results root@012e063d3d45:/# find / -name "*stablebuild-my-secret-api-key*" # no results ``` #### Final check, inspecting the exported image As a final check we can export the full image (including all layers and metadata) to disk, and do a final scan for our API key: ```bash $ mkdir -p out && \ docker save -o out.tar demo-clean-fs && \ mkdir -p out && \ tar -xvf out.tar -C out/ $ cd out $ grep stablebuild-my-secret-api-key -R . # no results! ``` Great 🎉! Your container now no longer contains your key, and you can safely push this container to Docker Hub or another public registry. The above works for the Ubuntu package registry, but if you use other mirrors or caches this will require some manual work in inspecting the layers and file system to ensure no keys are leaked. Both dive and just opening an interactive shell into the container should make this relatively painless though. ### Using build secrets with Kaniko If you use Kaniko to build your containers then build secrets are not available. However, you can use the fact that the `/kaniko` folder on your file system is shared between the Kaniko container and the build process. So you can use this in your Dockerfile: ```docker RUN --mount=type=secret,id=sb-api-key,target=/kaniko/sb-api-key.txt \ SB_API_KEY=$(cat /kaniko/sb-api-key.txt) bash /opt/sb-apt.sh load-apt-sources ubuntu ``` Here, the target `/kaniko/sb-api-key.txt` is on a shared file system, so you can just write the key to this location inside the Kaniko container before calling `/kaniko/executor`: ```bash echo -n "stablebuild-my-secret-api-key" > /kaniko/sb-api-key.txt && \ /kaniko/executor --dockerfile=./Dockerfile ... ``` You can do this f.e. by setting the entrypoint of the `gcr.io/kaniko-project/executor` container to `sh`, and the arguments to `[ "-c", "echo -n 'stablebuild-my-secret-api-key' > /kaniko/sb-api-key.txt && /kaniko/executor --dockerfile=./Dockerfile" ]`. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://stablebuild.gitbook.io/en/tutorials/protecting-your-keys-in-public-docker-containers.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.