Securing GKE Workloads

Introduction to GKE Security

As Kubernetes adoption grows, securing containerized workloads becomes increasingly critical. Google Kubernetes Engine (GKE) provides robust security features out of the box, but properly configuring these features is essential for protecting your applications and data.

In this comprehensive guide, we'll explore three fundamental aspects of GKE security: Pod Security Standards, Role-Based Access Control (RBAC), and Workload Identity. Understanding and implementing these security measures will help you build a defense-in-depth strategy for your GKE workloads.

Pod Security in GKE

Pod security controls what pods can do within your cluster and restricts potentially dangerous behaviors. GKE supports the Kubernetes Pod Security Standards, which define three different policies to restrict pod behavior.

Pod Security Standards Levels:

• Privileged: Unrestricted policy providing the widest possible level of permissions
• Baseline: Minimally restrictive policy that prevents known privilege escalations
• Restricted: Highly restrictive policy following current pod hardening best practices

Enforcing Pod Security Standards:

GKE allows you to enforce these standards at the namespace level using labels:

# namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: my-restricted-namespace
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/audit: restricted

Example of a Secure Pod Specification:

# secure-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: secured-app
  namespace: my-restricted-namespace
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  containers:
  - name: app-container
    image: my-app:1.0.0
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop: ["ALL"]
      readOnlyRootFilesystem: true
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

Role-Based Access Control (RBAC)

RBAC is a critical security mechanism that controls access to Kubernetes resources based on roles assigned to users or service accounts.

RBAC Components:

• ServiceAccount: An identity for processes running in pods
• Role/ClusterRole: Defines permissions within a namespace or cluster-wide
• RoleBinding/ClusterRoleBinding: Grants the permissions defined in a role to a user or service account

Creating a Least-Privilege Service Account:

# service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-serviceaccount
  namespace: my-app-namespace

Defining a Role with Minimal Permissions:

# app-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: my-app-namespace
  name: app-role
rules:
- apiGroups: [""]
  resources: ["pods", "services", "configmaps"]
  verbs: ["get", "list", "watch", "create", "update"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update"]

Binding the Role to the Service Account:

# role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-role-binding
  namespace: my-app-namespace
subjects:
- kind: ServiceAccount
  name: my-app-serviceaccount
  namespace: my-app-namespace
roleRef:
  kind: Role
  name: app-role
  apiGroup: rbac.authorization.k8s.io

Using the Service Account in a Deployment:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-app-namespace
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      serviceAccountName: my-app-serviceaccount
      containers:
      - name: my-app
        image: gcr.io/my-project/my-app:v1.0.0
        ports:
        - containerPort: 8080

Workload Identity

Workload Identity is the recommended way for your GKE applications to access Google Cloud services securely. It allows Kubernetes service accounts to impersonate Google Cloud service accounts.

Why Use Workload Identity?

• Eliminates the need to distribute and manage Google Cloud service account keys
• Provides fine-grained access control to Google Cloud resources
• Automatically rotates credentials
• Auditable through Cloud Audit Logs

Setting Up Workload Identity:

1. Enable Workload Identity on Your Cluster:

gcloud container clusters update my-cluster \
    --zone us-central1-a \
    --workload-pool=my-project.svc.id.goog

2. Create a Google Cloud Service Account:

gcloud iam service-accounts create my-app-gsa \
    --description="Service account for my app" \
    --display-name="My App GSA"

3. Grant Permissions to the Google Service Account:

gcloud projects add-iam-policy-binding my-project \
    --member="serviceAccount:my-app-gsa@my-project.iam.gserviceaccount.com" \
    --role="roles/cloudsql.client"

4. Allow Kubernetes Service Account to Impersonate Google Service Account:

gcloud iam service-accounts add-iam-policy-binding my-app-gsa@my-project.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:my-project.svc.id.goog[my-app-namespace/my-app-serviceaccount]"

5. Annotate the Kubernetes Service Account:

# service-account-with-wi.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-serviceaccount
  namespace: my-app-namespace
  annotations:
    iam.gke.io/gcp-service-account: my-app-gsa@my-project.iam.gserviceaccount.com

Implementing Network Policies

Network Policies act as a firewall for your pods, controlling traffic flow between them.

Example Network Policy:

# network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-allow-only-frontend
  namespace: my-app-namespace
spec:
  podSelector:
    matchLabels:
      app: api-server
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080

Security Best Practices Checklist

• ✅ Enable Pod Security Admission with restrictive policies
• ✅ Use non-root users for container processes
• ✅ Implement least-privilege RBAC principles
• ✅ Use Workload Identity instead of service account keys
• ✅ Regularly scan container images for vulnerabilities
• ✅ Implement network policies to control pod-to-pod communication
• ✅ Enable GKE's built-in security features (Binary Authorization, Shielded Nodes)
• ✅ Regularly rotate credentials and review access permissions
• ✅ Use namespaces to isolate environments and applications
• ✅ Monitor and audit cluster activities with Cloud Audit Logs

Monitoring and Auditing

Continuous monitoring and auditing are essential for maintaining a secure GKE environment:

Enable GKE Audit Logging:

gcloud container clusters update my-cluster \
    --zone us-central1-a \
    --logging=SYSTEM,WORKLOAD,AUDIT \
    --monitoring=SYSTEM,WORKLOAD

Example Cloud Monitoring Alert for Security Events:

# security-alert.yaml
name: projects/my-project/alertPolicies/12345
displayName: Privileged Pod Created
combiner: OR
conditions:
- displayName: Privileged pod creation
  conditionThreshold:
    filter: resource.type="k8s_container" AND protoPayload.methodName="io.k8s.core.v1.pods.create" AND jsonPayload.request.spec.containers.securityContext.privileged=true
    aggregations:
    - alignmentPeriod: 60s
      perSeriesAligner: ALIGN_COUNT
    comparison: COMPARISON_GT
    thresholdValue: 0
    duration: 0s
    trigger:
      count: 1

Conclusion

Securing GKE workloads requires a multi-layered approach that incorporates pod security, RBAC, Workload Identity, and other defense mechanisms. By implementing the practices outlined in this guide, you can significantly enhance the security posture of your Kubernetes applications.

Remember that security is an ongoing process, not a one-time configuration. Regularly review your security settings, stay informed about new GKE security features, and continuously monitor your clusters for potential vulnerabilities. With proper implementation of pod security standards, RBAC, and Workload Identity, you can build a robust security foundation for your GKE workloads.