docker
Set up a registry mirror
Once the registry is running and either exposed on the local network with HTTP or via the Internet with HTTPS, you'll need to configure Docker and potentially buildx too.
You can see how we do this within a Firecracker VM, to access the registry over the local Ethernet bridge: https://github.com/self-actuated/hub-mirror/blob/master/action.yml
For the Docker daemon, edit /etc/docker/daemon.json
.
{
"insecure-registries" : ["192.168.128.1:5000" ],
"registry-mirrors": ["http://192.168.128.1:5000"]
}
- Give each mirror under
registry-mirrors
and include the URL scheme - If you're using HTTP, without TLS, you need to specify
insecure-registries
Then make sure you reload Docker:
(
sudo systemctl daemon-reload && \
sudo systemctl restart docker
)
To try it out, run docker run -ti alpine:latest
, you should see the images when you run sudo find /var/lib/registry/
Buildx is a little more complicated to configure.
Create a buildkit.toml
[registry."docker.io"]
mirrors = ["192.168.128.1:5000"]
http = true
insecure = true
[registry."192.168.128.1:5000"]
http = true
insecure = true
You can omit http
and insecure
if you're using TLS and HTTPS.
Then, create a new buildx builder and tell Docker to use it:
docker buildx create --config ~/buildkitd.toml --name mirrored
docker buildx use mirrored
Finally, the buildx command will reference buildkit's configuration instead of Docker's and any base images will be pulled through the mirror.
docker buildx build -f Dockerfile .
We have a custom GitHub Action that makes all of the above just one line:
jobs:
build:
runs-on: actuated
steps:
- uses: self-actuated/hub-mirror@master
- name: Pull image using cache
run: |
docker pull alpine:latest
TLS is better
We used HTTP for the registry as it's accessed over a kind of loopback device between the VM and the server, however I'd recommend always using TLS where you can.
Perhaps you could even setup your registry on the Internet and use free Let's Encrypt certificates. Caddy or Nginx are simple enough to configure for that.
Then, if you're worried about bandwidth charges - Linode, DigitalOcean and Hetzner all have generous amounts included with 5-10 USD / mo VMs.
And you could also set up an IP allow list, so only your servers or build machines can consume your bandwidth allowance.
Setting up multiple mirrors
ou may want multiple mirrors if you pull images from both docker.io and another registry like gcr.io, ecr.io, ghcr.io or quay.io.
The Docker documentation says that dockerd
itself can only support a mirror of the Docker Hub itself. And any information that I found about multiple mirrors only applied to Kubernetes or to buildx.
Each registry mirror needs to run on its own HTTP port and if you're using TLS, will require its own distinct TLS certificate.
For instance, here are the things to change for a second registry mirroring ghcr.io:
storage:
filesystem:
- rootdirectory: /var/lib/registry
+ rootdirectory: /var/lib/registry-ghcr
proxy:
- remoteurl: https://registry-1.docker.io
+ remoteurl: https://ghcr.io
- username: $USERNAME
http:
- addr: 192.168.128.1:5000
+ addr: 192.168.128.1:5001
So then, buildx or cri (when using Kubernetes) need to be configured to pull from either of these endpoints.
- 192.168.128.1:5000 mirrors docker.io
- 192.168.128.1:5001 mirrors ghcr.io
dockerd itself, can have two mirrors defined, but in my experience it was unable to pull from the mirror for ghcr.io.
So let's look at buildx:
[registry."docker.io"]
mirrors = ["192.168.128.1:5000"]
http = true
insecure = true
[registry."192.168.128.1:5000"]
http = true
insecure = true
[registry."ghcr.io"]
mirrors = ["192.168.128.1:5001"]
http = true
insecure = true
[registry."192.168.128.1:5001"]
http = true
insecure = true
There's two ways to know if the cache is being used:
- Check the filesystem for the path set under
rootdirectory
- Enable the access logs for the registry itself
To enable access logs change
log:
accesslog:
- disabled: true
+ disabled: false
- level: warn
+ level: debug
formatter: text
In my testing, after running buildx create
and buildx use
, I then needed a Dockerfile that used both the Docker Hub and GHCR:
FROM alpine:3.17 as alpine
FROM ghcr.io/openfaasltd/figlet as figlet
RUN echo -n "Mirror" | figlet
Running the build with docker buildx build -t mirror-test .
gave me access logs on both registries and files under the respective /var/lib/
folders.
For Kubernetes configuration, you need to update the CRI plugin in containerd's toml file: Configure Image Registry.
Beware that CRI is an abstraction layer that sits between containerd and the kubelet, configuring this will not affect buildx, containerd or dockerd.
camunda
docker pull camunda/camunda-bpm-platform:latest
docker run -d --name camunda -p 8080:8080 camunda/camunda-bpm-platform:latest
# open browser with url: http://localhost:8080/camunda-welcome/index.html