Scaling and Load Balancing

One of Kubernetes' most powerful features is its ability to automatically scale applications and distribute traffic across them. This ensures your applications can handle varying loads while maintaining performance and availability.

Horizontal and Vertical Pod Autoscaling

Kubernetes provides two primary approaches to scaling applications: horizontal scaling (adding more pod instances) and vertical scaling (increasing resources for existing pods).

Horizontal Pod Autoscaler (HPA)

The Horizontal Pod Autoscaler automatically scales the number of pods in a deployment, replica set, or stateful set based on observed CPU utilization or custom metrics.

How HPA Works:

1. Monitors metrics from pods or external sources
2. Compares current metrics to target values
3. Adjusts the replica count to maintain the target metrics
4. Repeats this process continuously

Example HPA Definition:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 70
    

Creating HPA with kubectl:

kubectl autoscale deployment my-app --cpu-percent=50 --min=2 --max=10
    

Vertical Pod Autoscaler (VPA)

The Vertical Pod Autoscaler automatically adjusts the CPU and memory requests and limits of pods based on usage history. This helps ensure pods have appropriate resources without over-provisioning.

VPA Components:

• Recommender: Suggests resource values
• Updater: Evicts pods that need new resource limits
• Admission Controller: Sets resource requests when pods are created

Example VPA Definition:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-app-vpa
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
    - containerName: "*"
      minAllowed:
        cpu: 100m
        memory: 50Mi
      maxAllowed:
        cpu: 1
        memory: 500Mi
      controlledResources: ["cpu", "memory"]
    

Ingress Controllers and Load Balancers

Kubernetes provides several ways to expose your applications and distribute incoming traffic across your pods.

Service Types for Load Balancing

ClusterIP (Default)

Exposes the service on a cluster-internal IP, providing basic load balancing between pods.

NodePort

Exposes the service on each node's IP at a static port, allowing external access.

LoadBalancer

Creates an external load balancer in cloud environments, distributing traffic to nodes.

Example LoadBalancer Service:

apiVersion: v1
kind: Service
metadata:
  name: my-loadbalancer-service
spec:
  selector:
    app: my-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376
  type: LoadBalancer
  externalTrafficPolicy: Local
    

Ingress Resources and Controllers

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. An Ingress controller is required to fulfill the Ingress rules.

Popular Ingress Controllers:

• NGINX Ingress Controller
• Traefik
• HAProxy
• Istio Ingress Gateway
• AWS Application Load Balancer (ALB) Controller

Example Ingress Resource:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-app-service
            port:
              number: 80
  - host: api.example.com
    http:
      paths:
      - path: /v1
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080
    

TLS Termination with Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-ingress
spec:
  tls:
  - hosts:
      - myapp.example.com
    secretName: myapp-tls-secret
  rules:
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-app-service
            port:
              number: 80
    

Advanced Load Balancing Strategies

Service Mesh Integration

Service meshes like Istio or Linkerd provide advanced traffic management capabilities:

• Fine-grained traffic routing (canary releases, blue-green deployments)
• Circuit breaking and fault injection
• Advanced load balancing algorithms
• Observability and monitoring

Custom Metrics for Autoscaling

HPA can scale based on custom metrics from applications or external systems:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: custom-metric-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metric:
        name: requests_per_second
      target:
        type: AverageValue
        averageValue: 1k
    

Best Practices

• Set appropriate resource requests and limits for reliable autoscaling
• Use readiness probes to ensure traffic only goes to healthy pods
• Configure pod disruption budgets to maintain availability during disruptions
• Monitor autoscaling behavior and adjust targets as needed
• Use multiple metrics for autoscaling to handle different types of load
• Consider using service meshes for complex traffic management scenarios

Kubernetes provides a comprehensive set of tools for scaling applications and managing traffic, from simple load balancing to sophisticated autoscaling based on multiple metrics. Understanding these capabilities allows you to build highly available and responsive applications.