K8s NFS Mount Volume Permission

Posted on | Edited on | In

Kubernetes version 1.13.2

This is an interesting issue which involves 4 topices: Volume, Security Context, NFS and initContainer.

The issue comes from the permission denied error. The process fail to create the file under the mount path, I check the owner and group of that path, they are both root.

In the yaml file, I specify the fsGroup field as id 9092, from the official document here (The example use id 2000):

1
Since fsGroup field is specified, all processes of the container are also part of the supplementary group ID 2000. The owner for volume /data/demo and any files created in that volume will be Group ID 2000.

so the owner of the volume should be 9092, but they don’t.

I searched online and met the same issue from others: https://github.com/kubernetes/examples/issues/260

It seems fsGroup securityContext does not apply to nfs mount especially we run the containers as non-root user we cannot access the mount. This issue may be solved in later version, need to take care.

!!! Why this happens? Because we use hostPath, it by default will create root owned path if path does not exist. Here the NFS is not the NFS way kubernetes use, we use hostPath then manually nfs the nodes externally, not by the setting of K8s(need to do experiment).

The workaround is using initContainers with busybox run as root and chown to the nfs mount with expected id, for example:

1
2
3
4
5
6
7
8
9
10
initContainers:
  - name: busybox
    image: xxx.com:5000/busybox:latest
    imagePullPolicy: IfNotPresent
    command: ["sh", "-c", "chown 9092:9092 /mnt"]
    securityContext:
      runAsUser: 0
    volumeMounts:
    - name: <volume name from Volumes>
      mountPath: /mnt

then we are good.

0%