Kubernetes Cert-manager

Posted on | Edited on | In

Cert-manager git repo, it is recommended to read through the document.

Install

Using helm chart to deploy cert-manager in K8s cluster in cert-manager namespace(customizing chart if needed), verifying deployment is good, see here

Highlight Concepts

Issuers and ClusterIssuers: https://cert-manager.io/docs/concepts/issuer/

Issuers configuration, especially ACME protocol as we use tarsier CA: https://cert-manager.io/docs/configuration/

Certficiate resources: https://cert-manager.io/docs/usage/certificate/

Usage Case

We use Google tarsier CA and cert-manager to manage, renew certificate for ingresses, see secure ingress resources. Although it cannot directly work on Anthos MCI(multi-cluster ingress) but the workaround is simple by manually creating certificate to associate with target tls secret.

After deployment of cert-manager is done and run correctly, add cert-manager supported annotations to target ingress, for example:

1
2
3
4
5
6
7
kind: Ingress
metadata:
  annotations:
    # cluster issuer is cluster scope and deployed by yourself
    cert-manager.io/cluster-issuer: example-cert
    cert-manager.io/duration: 2160h
    cert-manager.io/renew-before: 72h

Then cert-manager will automatically create certificate resource and start to issue certificate. Note that you can manually create this resource for some scenarios: tls secret is used by multiple ingresses(no need to add annotations to each ingress) or Anthos MCI(does not support cert-manager):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
apiVersion: cert-manager.io/v1
kind: Certificate
  name: example
  namespace: default
spec:
  dnsNames:
  # this is from ingress host name
  - '*.service.example.google'
  duration: 2160h0m0s
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: example-cert
  renewBefore: 72h0m0s
  # this is from ingress tls name
  secretName: example-tls

The spec.dnsNames is Subject Alternative Name(SAN) that can have multiples, Common Name(CN) is derived from the first item. Note CN is discouraged from being used and deprecated, see here

So, this actually is a SAN certificate, not CN(common name) certificate(as mentioned here CN field is deprecated).

Describing it you can examine the Dns names, condition and validation of the new certificate:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Spec:
  Dns Names:
    *.service.example.google
  Duration:  2160h0m0s
  Issuer Ref:
    Group:       cert-manager.io
    Kind:        ClusterIssuer
    Name:        example-cert
  Renew Before:  72h0m0s
  Secret Name:   example-tls
Status:
  Conditions:
    Last Transition Time:  2021-10-08T19:18:51Z
    Message:               Certificate is up to date and has not expired
    Observed Generation:   1
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2022-01-06T18:18:43Z
  Not Before:              2021-10-08T18:18:44Z
  Renewal Time:            2022-01-03T18:18:43Z
  Revision:                1

The certificate and key are stored in K8s secret resource, once certificate issuing is done, cert-manager will manage this secret by adding specific annotations, for example:

1
2
3
4
5
6
7
8
9
10
kind: Secret
metadata:
  annotations:
    cert-manager.io/alt-names: '*.service.example.google'
    cert-manager.io/certificate-name: example
    cert-manager.io/common-name: '*.service.example.google'
    cert-manager.io/ip-sans: ""
    cert-manager.io/issuer-group: cert-manager.io
    cert-manager.io/issuer-kind: ClusterIssuer
    cert-manager.io/issuer-name: example-cert

If no secret exists then the cert-manager will create the secret for you. Note that the old secret will be overridden every time new certificate is issued.

Also note that deletion of certificate will not delete associated secret.

To decode the certificate content, using base64 and openssl, usually there are multiple-level certificates placed, from bottom to top(root CA, intermediate CA to certificate), select one to decode:

1
2
3
echo <secret encode block> | base64 -d
# select one CERTIFICATE block to decode
openssl x509 -in certificate.crt -text -noout

Or decoding online: https://www.sslshopper.com/certificate-decoder.html

In reverse, to encode certificate and used in secret:

1
2
# -w 0: git rid of new line, there may be a % chart at end, drop it.
cat certificate.crt | base64 -w 0
0%