Security container access to resources using built-in Linux security features

In this article, you learn how to secure container access to resources for your Azure Kubernetes Service (AKS) workloads.

Overview

In the same way that you should grant users or groups the minimum privileges required, you should also limit containers to only necessary actions and processes. To minimize the risk of attack, avoid configuring applications and containers that require escalated privileges or root access.

You can use built-in Kubernetes pod security contexts to define more permissions, such as the user or group to run as, the Linux capabilities to expose, or setting allowPrivilegeEscalation: false in the pod manifest. For more best practices, see Secure pod access to resources.

For even more granular control of container actions, you can use built-in Linux security features such as AppArmor and seccomp.

1. Define Linux security features at the node level.
2. Implement features through a pod manifest.

Built-in Linux security features are only available on Linux nodes and pods.

App Armor

To limit container actions, you can use the AppArmor Linux kernel security module. AppArmor is available as part of the underlying AKS node OS and is enabled by default. You create AppArmor profiles that restrict read, write, or execute actions, or system functions like mounting filesystems. Default AppArmor profiles restrict access to various /proc and /sys locations and provide a means to logically isolate containers from the underlying node. AppArmor works for any application that runs on Linux, not just Kubernetes pods.

To see AppArmor in action, the following example creates a profile that prevents writing to files.

1. SSH to an AKS node.

2. Create a file named deny-write.profile.

3. Copy and paste the following content:

   #include <tunables/global>
   profile k8s-apparmor-example-deny-write flags=(attach_disconnected) {
     #include <abstractions/base>
   
     file,
     # Deny all file writes.
     deny /** w,
   }
   

AppArmor profiles are added using the apparmor_parser command.

1. Add the profile to AppArmor.

2. Specify the name of the profile created in the previous step:

   sudo apparmor_parser deny-write.profile
   

   If the profile is correctly parsed and applied to AppArmor, you won't see any output and you'll return to the command prompt.

3. From your local machine, create a pod manifest named aks-apparmor.yaml. This manifest:

   • Defines an annotation for container.apparmor.security.beta.kubernetes.
   • References the deny-write profile created in the previous steps.

   apiVersion: v1
   kind: Pod
   metadata:
     name: hello-apparmor
     annotations:
       container.apparmor.security.beta.kubernetes.io/hello: localhost/k8s-apparmor-example-deny-write
   spec:
     containers:
     - name: hello
       image: mcr.microsoft.com/dotnet/runtime-deps:6.0
       command: [ "sh", "-c", "echo 'Hello AppArmor!' && sleep 1h" ]
   

4. With the pod deployed, run the following command and verify the hello-apparmor pod shows a Running status:

   kubectl get pods
   
   NAME             READY   STATUS    RESTARTS   AGE
   aks-ssh          1/1     Running   0          4m2s
   hello-apparmor   0/1     Running   0          50s
   

For more information about AppArmor, see AppArmor profiles in Kubernetes.

Secure computing (seccomp)

While AppArmor works for any Linux application, seccomp (secure computing) works at the process level. Seccomp is also a Linux kernel security module and is natively supported by the containerd runtime used by AKS nodes. With seccomp, you can limit a container's system calls. Seccomp establishes an extra layer of protection against common system call vulnerabilities exploited by malicious actors and allows you to specify a default profile for all workloads in the node.

Configure a default seccomp profile (preview)

You can apply default seccomp profiles using custom node configurations when creating a new Linux node pool. There are two values supported on AKS: RuntimeDefault and Unconfined. Some workloads might require a lower number of syscall restrictions than others. This means that they can fail during runtime with the 'RuntimeDefault' profile. To mitigate such a failure, you can specify the Unconfined profile. If your workload requires a custom profile, see Configure a custom seccomp profile.

Limitations

• SeccompDefault is not a supported parameter for windows node pools.
• SeccompDefault is available starting in 2024-09-02-preview API.

Register the KubeletDefaultSeccompProfilePreview feature flag

1. Register the KubeletDefaultSeccompProfilePreview feature flag using the az feature register command.

   az feature register --namespace "Microsoft.ContainerService" --name "KubeletDefaultSeccompProfilePreview"
   

   It takes a few minutes for the status to show Registered.

2. Verify the registration status using the az feature show command.

   az feature show --namespace "Microsoft.ContainerService" --name "KubeletDefaultSeccompProfilePreview"
   

3. When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider using the az provider register command.

   az provider register --namespace Microsoft.ContainerService
   

Restrict your container's system calls with seccomp

1. Follow steps to apply a seccomp profile in your kubelet configuration by specifying "seccompDefualt": "RuntimeDefault".

RuntimeDefault uses containerd's default seccomp profile, restricting certain system calls to enhance security. Restricted syscalls will fail. For more information, see the containerD default seccomp profile.

2. Check that the configuration was applied.

You can confirm the settings are applied to the nodes by connecting to the host and verifying configuration changes have been made on the filesystem.

3. Troubleshoot workload failures.

When SeccompDefault is enabled, the container runtime default seccomp profile is used by default for all workloads scheduled on the node. This might cause workloads to fail due to blocked syscalls. If a workload failure has occurred, you might see errors such as:

• Workload is existing unexpectedly after the feature is enabled, with "permission denied" error.
• Seccomp error messages can also be seen in auditd or syslog by replacing SCMP_ACT_ERRNO with SCMP_ACT_LOG in the default profile.

If you experience the above errors, we recommend that you change your seccomp profile to Unconfined. Unconfined places no restrictions on syscalls, allowing all system calls, which reduces security.

Configure a custom seccomp profile

With a custom seccomp profile, you can have more granular control over restricted syscalls. Align to the best practice of granting the container minimal permission only to run by:

• Defining with filters what actions to allow or deny.
• Annotating within a pod YAML manifest to associate with the seccomp filter.

To see seccomp in action, create a filter that prevents changing permissions on a file.

1. SSH to an AKS node.

2. Create a seccomp filter named /var/lib/kubelet/seccomp/prevent-chmod.

3. Copy and paste the following content:

   {
     "defaultAction": "SCMP_ACT_ALLOW",
     "syscalls": [
       {
         "name": "chmod",
         "action": "SCMP_ACT_ERRNO"
       },
       {
         "name": "fchmodat",
         "action": "SCMP_ACT_ERRNO"
       },
       {
         "name": "chmodat",
         "action": "SCMP_ACT_ERRNO"
       }
     ]
   }
   

   In version 1.19 and later, you need to configure:

   {
     "defaultAction": "SCMP_ACT_ALLOW",
     "syscalls": [
       {
         "names": ["chmod","fchmodat","chmodat"],
         "action": "SCMP_ACT_ERRNO"
       }
     ]
   }
   

4. From your local machine, create a pod manifest named aks-seccomp.yaml and paste the following content. This manifest:

   • Defines an annotation for seccomp.security.alpha.kubernetes.io.
   • References the prevent-chmod filter created in the previous step.

   apiVersion: v1
   kind: Pod
   metadata:
     name: chmod-prevented
     annotations:
       seccomp.security.alpha.kubernetes.io/pod: localhost/prevent-chmod
   spec:
     containers:
     - name: chmod
       image: mcr.microsoft.com/dotnet/runtime-deps:6.0
       command:
         - "chmod"
       args:
        - "777"
        - /etc/hostname
     restartPolicy: Never
   

   In version 1.19 and later, you need to configure:

   apiVersion: v1
   kind: Pod
   metadata:
     name: chmod-prevented
   spec:
     securityContext:
       seccompProfile:
         type: Localhost
         localhostProfile: prevent-chmod
     containers:
     - name: chmod
       image: mcr.microsoft.com/dotnet/runtime-deps:6.0
       command:
         - "chmod"
       args:
        - "777"
        - /etc/hostname
     restartPolicy: Never
   

5. Deploy the sample pod using the kubectl apply command:

   kubectl apply -f ./aks-seccomp.yaml
   

6. View pod status using the kubectl get pods command.

   • The pod reports an error.
   • The chmod command is prevented from running by the seccomp filter, as shown in the example output:

   kubectl get pods
   
   NAME                      READY     STATUS    RESTARTS   AGE
   chmod-prevented           0/1       Error     0          7s
   

Seccomp security profile options

Seccomp security profiles are a set of defined syscalls that are allowed or restricted. Most container runtimes have a default seccomp profile that is similar if not the same as the one Docker uses. For more information about available profiles, see Docker or containerD default seccomp profiles.

AKS uses the containerD default seccomp profile for our RuntimeDefault when you configure seccomp using custom node configuration.

Significant syscalls blocked by default profile

Both Docker and containerD maintain allowlists of safe syscalls. This table lists the significant (but not all) syscalls that are effectively blocked because they aren't on the allowlist. If any of the blocked syscalls are required by your workload, don't use the RuntimeDefault seccomp profile.

When changes are made to Docker and containerD, AKS updates their default configuration to match. Updates to this list may cause workload failure. For release updates, see AKS release notes.

Next steps

For associated best practices, see Best practices for cluster security and upgrades in AKS and Best practices for pod security in AKS.