Set up Secure Docker Registry Container

现在想想，可以使用cert-manager 以及 Let’ encrypt去做，这样就不用再去配置cert trust in OS以及可以自动更新certificate~ 唉，当时完全不知道!

This post is about configuring your own secure docker registry in the form of docker container, check this to set up a secured docker registry in K8s.

More about SSL please check my blog SSL Demystify. It contains the theory, workflow and practice.

Securing access to your docker images is paramount, the docker registry natively supports TLS and basic authentication, let’s do it.

Generate Self-signed Certificate

See document from docker.

mkdir -p /root/certs

## generate domain.key and self-signed domain.crt
## I use -days 3650
openssl req \
      -newkey rsa:4096 -nodes -x509 -sha256\
      -keyout certs/domain.key -out certs/domain.crt -days 3650 \
      -subj "/C=US/ST=CA/L=San Jose/O=<Company Name>/OU=Org/CN=chengdol.registry.com"

Notice that the CN=chengdol.registry.com must be the registry access url, no port number suffix needed.

Parameters explanation from here:

openssl req: 
  The req command primarily creates and processes certificate requests in PKCS#10 format. It can additionally create self signed certificates for use as root CAs for example.
-newkey: 
   this option creates a new certificate request and a new private key. 
  rsa:nbits: 
    where nbits is the number of bits, generates an RSA key nbits in size.
-nodes:
  if this option is specified then if a private key is created it will not be encrypted.
-x509: 
  this option outputs a self signed certificate instead of a certificate request. This is typically used to generate a test certificate or a self signed root CA .
-[digest]:
  this specifies the message digest to sign the request with (such as -md5, -sha1, -sha256)
-keyout:
  this gives the filename to write the newly created private key to.
-out:
  this specifies the output filename to write to or standard output by default.
-days:
  when the -x509 option is being used this specifies the number of days to certify the certificate for. The default is 30 days.
-subj:
  replaces subject field of input request with specified data and outputs modified request. The arg must be formatted as /type0=value0/type1=value1/type2=..., characters may be escaped by \ (backslash), no spaces are skipped.

There are mutli-way to do the same thing，一步一步的构造self-signed certificate: OpenSSL Essentials: Working with SSL Certificates, Private Keys and CSRs

Setup Secure Docker Registry

see document from docker.

We have the certs folder with crt and key created by openssl. Start the docker registry container, using TLS certificate:

docker run -d \
--restart=always \
--name registry \
-v /root/certs:/certs \
-e REGISTRY_HTTP_ADDR=0.0.0.0:443 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
-p 443:443 \
registry:2

Here we overwrite some env variables to change the default configuration.

Also follow the instruction in docker web, instruct every docker daemon to trust that certificate. The way to do this depends on your OS, for Linux:

1
2
3

mkdir -p /etc/docker/certs.d/<docker registry domain>/
## copy domain.crt (generate by openssl) to this folder on every Docker host
cp /root/certs/domain.crt /etc/docker/certs.d/<docker registry domain>/

注意，Docker官方文档中用的是<docker registry domain>:5000/文件夹名，但如果你配置的是443端口,则会出错，通过Docker daemon中的log，发现对于443端口，这里不需要:5000. 但如果设置了basic authentication且用的是5000端口，则需要了。

当时还发生了奇怪的事情，我发现不需要这个trust操作居然也能进行push，后来才发现原来是旧配置在docker daemon json 文件中设置了insecure registry，这样一来根本就不会检查证书了。

If you don’t do this, when run docker push you will get this error:

1	Error response from daemon: Get https://chengdol.registry.com/v2/: x509: certificate signed by unknown authority

If docker still complains about the certificate when using authentication? When using authentication, some versions of Docker also require you to trust the certificate at the OS level.

For RedHat, do:

1 2	cp certs/domain.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt update-ca-trust

Now you can push and pull like below, no need to specify port number, it will use 443 port:

docker pull ubuntu
docker tag ubuntu chengdol.registry.com/ubuntu:v1
docker push chengdol.registry.com/ubuntu:v1
docker pull chengdol.registry.com/ubuntu:v1

So far, secure configuration is done, now the docker registry will use HTTPS in 443 port to communciate with docker client. If you want to setup basic authentication, see below:

Setup Basic Authrntication

Warning: You cannot enable authentication that send credentials as clear text. You must configure TLS first for authentication to work.

Use htpasswd to create the user info:

1 2	mkdir -p /root/auth htpasswd -Bbn demo demo > /root/auth/htpasswd

Then, we switch back to 5000 port: (注意这里没用443端口)

docker run -d \
-p 5000:5000 \
--restart=always \
--name registry \
-v /root/auth:/auth \
-e "REGISTRY_AUTH=htpasswd" \
-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
-e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
-v /root/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
registry:2

Do the same trust thing in every docker host, under /etc/docker/certs.d/ directory, we create a folder <docker registry domain>:5000 and put domain.crt in it:

1
2
3

mkdir -p /etc/docker/certs.d/<docker registry domain>:5000/
## copy domain.crt (generate by openssl) to this folder on every Docker host
cp domain.crt /etc/docker/certs.d/<docker registry domain>:5000/

Then, you need to first login to push or pull:

1	docker login <docker registry domain>:5000 -u demo -p demo

Conclusion

OK, now a secure docker registry container with basic authentication is up and running. You can push and pull after docker login.