Google Cloud Operations: Logging & Monitoring Essentials

Google Cloud Operations (formerly Stackdriver) provides powerful tools for monitoring, logging, and alerting across your cloud infrastructure. Let's explore how to gain visibility into your applications using these services.

Logs Explorer: Your Centralized Logging Hub

Key Features

• Aggregates logs from all GCP services and applications
• Powerful query language for log analysis
• Log-based metrics creation
• Integration with Cloud Audit Logs

Basic Log Query Examples

-- Find high severity errors
severity >= ERROR

-- Filter Compute Engine logs
resource.type="gce_instance"

-- Search for specific text
textPayload:"Connection timeout"

-- Combine conditions
severity=ERROR AND resource.type="cloud_function"

Creating Log-Based Metrics

Turn important log patterns into trackable metrics:

1. Open Logs Explorer
2. Run your query to isolate the log entries
3. Click "Create Metric"
4. Name your metric (e.g., "high_latency_requests")
5. Set the metric type (counter, distribution, etc.)

Cloud Monitoring: Metrics & Dashboards

Core Components

• Metrics Explorer: Visualize and analyze metric data
• Dashboards: Custom monitoring views
• Uptime Checks: Verify service availability
• Service Monitoring: Track SLIs/SLOs

Creating a Custom Dashboard

Build a dashboard to monitor key metrics:

# Using gcloud to create a dashboard
gcloud monitoring dashboards create --config-from-file=dashboard.json

Example dashboard.json:

{
  "displayName": "App Service Dashboard",
  "gridLayout": {
    "widgets": [
      {
        "title": "CPU Utilization",
        "xyChart": {
          "dataSets": [{
            "timeSeriesQuery": {
              "timeSeriesFilter": {
                "filter": "resource.type=\"gce_instance\" metric.type=\"compute.googleapis.com/instance/cpu/utilization\"",
                "aggregation": {
                  "alignmentPeriod": "60s",
                  "perSeriesAligner": "ALIGN_MEAN"
                }
              }
            }
          }]
        }
      }
    ]
  }
}

Alerting: Proactive Incident Management

Creating Alert Policies

Set up alerts for critical conditions:

Step 1: Define the Condition

# Example condition
metric.type="compute.googleapis.com/instance/cpu/utilization"
resource.type="gce_instance"
condition: utilization > 0.8 for 5 minutes

Step 2: Configure Notification Channels

• Email
• SMS
• PagerDuty
• Slack
• Webhooks

Step 3: Set Documentation

Include runbook links and troubleshooting steps

Alert Policy Example via API

gcloud alpha monitoring policies create \
  --policy-from-file=alert-policy.json

Best Practices for Cloud Operations

1. Structured Logging

Use JSON format for application logs:

{
  "severity": "WARNING",
  "message": "High latency detected",
  "component": "payment-service",
  "latency_ms": 1250,
  "request_id": "abc123"
}

2. Meaningful Metrics

• Track business metrics alongside infrastructure metrics
• Use labels for dimensionality
• Set appropriate aggregation periods

3. Alert Design

• Focus on symptoms, not causes
• Use multi-condition alerts to reduce noise
• Implement escalation policies

Getting Started with Cloud Operations

To begin implementing these tools:

1. Enable Cloud Operations API in your project
2. Install the Cloud Monitoring agent on your VMs
3. Instrument your applications with client libraries
4. Start with basic dashboards and refine as needed

Remember: Effective observability requires both proper tool configuration and good instrumentation practices in your applications.