The step to set up secure docker registry service in K8s is different from docker. There are some adjustments and changes to apply.
Toolkits we need to achieve our goal:
openssl
htpasswd
skopeo
Create SSL/TLS Certificate and Key
Use openssl command to generate certificate and private key for setup secure connection:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 mkdir -p /root/registry-certsDOCKER_REGISTRY_URL=blair1.fyre.com openssl req \ -newkey rsa:4096 -nodes -x509 -sha256 \ -keyout /root/registry-certs/tls.key \ -out /root/registry-certs/tls.cert \ -days 3650 \ -subj "/C=US/ST=CA/L=San Jose/O=IBM/OU=Org/CN=${DOCKER_REGISTRY_URL} " cp /root/registry-certs/tls.cert /root/registry-certs/tls.crt
Then copy the crt file to every host under /etc/docker/certs.d/<${DOCKER_REGISTRY_URL}>:5000
folder for self-signed certificate trust.
Notice that if the docker daemon json file has enabled the insecure registry, it will not verify the ssl/tls cert! You get docker user account and password, then you can login without certs!
Create Docker User Info
1 2 3 4 5 6 DOCKER_USER=demo DOCKER_PASSWORD=demo mkdir -p /tmp/registry-authhtpasswd -Bbn ${DOCKER_USER} ${DOCKER_PASSWORD} > /tmp/registry-auth/htpasswd
Generate Secret
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 kubectl create secret tls docker-registry-tls \ --key=/root/registry-certs/tls.key \ --cert=/root/registry-certs/tls.cert \ -n default kubetctl create secret generic docker-registry-auth \ --from-file=htpasswd=/tmp/registry-auth/htpasswd \ -n default WORKING_NAME_SPACE=test-1 DOCKER_REGISTRY_SERVER="${DOCKER_REGISTRY_URL} :5000" kubectl create namespace ${NAME_SPACE} kubectl create secret docker-registry docker-registry-creds \ --docker-server=$DOCKER_REGISTRY_SERVER \ --docker-username=$DOCKER_USER \ --docker-password=$DOCKER_PASSWORD \ -n ${WORKING_NAME_SPACE}
Bind Image Pull Secret to Service Account
see document in K8s.
1 2 3 4 5 kubectl patch serviceaccount default \ -p '{"imagePullSecrets": [{"name": "docker-registry-creds"}]}' \ -n ${WORKING_NAME_SPACE}
Or you can specify imagePullSecrets in yaml explicitly, for example:
1 2 3 4 5 6 7 8 9 10 apiVersion: v1 kind: Pod metadata: name: private-reg spec: containers: - name: private-reg-container image: <your-private-image> imagePullSecrets: - name: <secret name>
Create Secure Docker Registry
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 --- apiVersion: apps/v1 kind: Deployment metadata: name: docker-registry labels: app: docker-registry spec: replicas: 1 selector: matchLabels: app: docker-registry template: metadata: labels: app: docker-registry spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - {key: docker-registry , operator: In , values: ["true" ]} hostNetwork: true tolerations: - key: "node-role.kubernetes.io/master" operator: "Exists" effect: "NoSchedule" containers: - name: docker-registry image: localhost:5000/registry:2.7.1 imagePullPolicy: IfNotPresent env: - name: REGISTRY_STORAGE_DELETE_ENABLED value: "true" - name: REGISTRY_AUTH value: "htpasswd" - name: REGISTRY_AUTH_HTPASSWD_REALM value: "Registry Realm" - name: REGISTRY_AUTH_HTPASSWD_PATH value: "/auth/htpasswd" - name: REGISTRY_HTTP_TLS_CERTIFICATE value: /certs/tls.crt - name: REGISTRY_HTTP_TLS_KEY value: /certs/tls.key ports: - name: registry containerPort: 5000 hostPort: 5000 volumeMounts: - name: docker-data mountPath: /var/lib/registry - name: docker-tls mountPath: /certs readOnly: true - name: docker-auth mountPath: /auth readOnly: true volumes: - name: docker-data persistentVolumeClaim: claimName: registry-pv-claim - name: docker-tls secret: secretName: docker-registry-tls - name: docker-auth secret: secretName: docker-registry-auth
So far the secure docker registry in K8s is up and running in default namespace, it’s host network true so can be accessed from remote. Later can expose it by ingress.
Update Docker User Info
See this post .
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 DOCKER_USER=demonew DOCKER_PASSWORD=demonew mkdir -p /tmp/registry-authhtpasswd -Bbn ${DOCKER_USER} ${DOCKER_PASSWORD} > /tmp/registry-auth/htpasswd AUTH_BASE64=$(cat /tmp/registry-auth/htpasswd | base64 -w 0) kubectl get secret docker-registry-auth -o yaml -n default \ | sed -e "/htpasswd/c\ htpasswd: ${AUTH_BASE64} " \ | kubectl replace -f - NEW_REGISTRY_CREDS=$(kubectl create secret docker-registry docker-registry-creds \ --docker-server=$DOCKER_REGISTRY_SERVER \ --docker-username=$DOCKER_USER \ --docker-password=$DOCKER_PASSWORD \ -n default \ -o yaml --dry-run \ | grep "\.dockerconfigjson" | cut -d":" -f2) kubectl get secret docker-registry-creds -o yaml -n ${WORKING_NAME_SPACE} \ | sed -e "/\.dockerconfigjson/c\ .dockerconfigjson: ${NEW_REGISTRY_CREDS} " \ | kubectl replace -f -
Skopeo Operation
Please refer my skopeo blog for more details.
1 2 3 4 5 6 7 8 9 10 skopeo copy \ --dest-creds ${DOCKER_USER} :${DOCKER_PASSWORD} \ --dest-cert-dir /root/registry-certs \ docker-archive:/root/busybox.tar.gz \ docker://${DOCKER_REGISTRY_SERVER} /busybox:latest skopeo inspect \ --creds ${DOCKER_USER} :${DOCKER_PASSWORD} \ --cert-dir /root/registry-certs \ docker://${DOCKER_REGISTRY_SERVER} /busybox:latest