Claude Skills Guide

Claude Skills for Site Reliability Engineers SRE

Site reliability engineers need tools that handle incident response, log analysis, monitoring, and system debugging. Claude Code provides skills that integrate with common SRE tooling to accelerate these workflows. This guide covers practical applications for SRE teams.

Incident Response Automation

When a production incident occurs, speed matters Claude Code helps you build incident response runbooks that execute directly in your terminal. Describe your alerting setup and Claude generates bash scripts for common remediation steps.

For example, a high-memory alert response might look like this:

#!/bin/bash
# Incident response: high memory remediation
HOST=$1
THRESHOLD=90

# Check current memory usage
MEM_USAGE=$(ssh $HOST "free | grep Mem | awk '{printf \"%.0f\", \$3/\$2 * 100}'")

if [ "$MEM_USAGE" -gt "$THRESHOLD" ]; then
  # Find top memory consumers
  ssh $HOST "ps aux --sort=-%mem | head -10"
  
  # Restart largest consumer if it's a known service
  ssh $HOST "systemctl restart $(ssh $HOST "ps aux --sort=-%mem | head -1 | awk '{print \$11}'" | xargs basename)"
  
  # Log the action
  echo "$(date): Restarted process on $HOST due to memory pressure" >> /var/log/incident.log
fi

Claude can also help you structure incident post-mortems. Paste your incident timeline and ask Claude to format it using the standard industry format: summary, impact, root cause, trigger, resolution, and action items.

Log Analysis and Pattern Detection

SREs spend significant time grepping through logs. Claude Code enhances this workflow by helping you construct precise log queries and recognize patterns across multiple log sources.

When analyzing application logs, describe the error patterns you’re seeing:

I'm seeing timeout errors across three services. Help me construct a grep command that finds all timeout exceptions in /var/log/app/ from the last hour, grouped by service name.

Claude generates commands like:

# Find timeout errors across services
for log in /var/log/app/*.log; do
  service=$(basename $log .log)
  timeout_count=$(grep -c "timeout" "$log" 2>/dev/null)
  if [ "$timeout_count" -gt 0 ]; then
    echo "$service: $timeout_count timeouts"
  fi
done

For structured logs in JSON format, Claude helps you use jq effectively:

# Extract error rates from JSON logs
cat /var/log/app.json | jq -c 'select(.level=="error") | {timestamp, service, message}' | \
  jq -s 'group_by(.service) | map({service: .[0].service, count: length})'

Monitoring Dashboard Construction

Building Prometheus alerts or Grafana dashboards becomes faster with Claude’s assistance. Describe your metrics and desired visualization, and Claude generates the configuration.

For Prometheus alerting rules:

groups:
  - name: service-health
    rules:
      - alert: HighErrorRate
        expr: sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) > 0.05
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "High error rate on {{ $labels.service }}"
          description: "{{ $labels.service }} error rate is {{ $value | humanizePercentage }}"
      
      - alert: HighLatency
        expr: histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service)) > 2
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High p95 latency on {{ $labels.service }}"

Claude also helps you write Grafana panel JSON by describing your visualization needs. Specify the metric, aggregation, and visual style, and receive ready-to-paste dashboard configurations.

On-Call Workflow Enhancement

Managing on-call rotations and escalations requires clear runbooks and automation. Claude helps you build scripts that integrate with PagerDuty, OpsGenie, or similar tools.

A basic escalation script might look like:

#!/bin/bash
# Automated escalation check
INCIDENT_ID=$1
CURRENT_ESCALATION=$2

# Get incident details via PagerDuty API
INCIDENT=$(curl -s -H "Authorization: Token token=$PAGERDUTY_API_KEY" \
  "https://api.pagerduty.com/incidents/$INCIDENT_ID")

# Check if incident is acknowledged
STATUS=$(echo "$INCIDENT" | jq -r '.incident.status')

if [ "$STATUS" == "triggered" ]; then
  # Calculate time since trigger
  CREATED_AT=$(echo "$INCIDENT" | jq -r '.incident.created_at')
  NOW=$(date -u +%Y-%m-%dT%H:%M:%SZ)
  SECONDS_SINCE=$(( $(date -d "$NOW" +%s) - $(date -d "$CREATED_AT" +%s) ))
  
  # Escalate if unacknowledged for more than 15 minutes
  if [ "$SECONDS_SINCE" -gt 900 ]; then
    curl -s -X PUT -H "Authorization: Token token=$PAGERDUTY_API_KEY" \
      -H "Content-Type: application/json" \
      -d "{\"incident\": {\"type\": \"incident_reference\", \"escalation_policy\": \"$NEXT_ESCALATION_POLICY\"}}" \
      "https://api.pagerduty.com/incidents/$INCIDENT_ID"
    echo "Incident escalated after $(($SECONDS_SINCE / 60)) minutes"
  fi
fi

Chaos Engineering and Testing

SRE teams increasingly practice chaos engineering. Claude helps you write chaos scripts that safely inject failures to test system resilience.

#!/usr/bin/env python3
# Simple chaos monkey: randomly terminate containers
import subprocess
import random
import time

def terminate_random_container():
    # List running containers
    result = subprocess.run(
        ["docker", "ps", "--format", "{{.Names}}"],
        capture_output=True, text=True
    )
    containers = result.stdout.strip().split('\n')
    
    if containers and containers[0]:
        target = random.choice(containers)
        print(f"Terminating {target} for chaos testing")
        subprocess.run(["docker", "kill", "--signal", "SIGTERM", target])

# Run every 30 minutes during business hours
while True:
    hour = int(time.strftime("%H"))
    if 9 <= hour <= 17:  # Business hours only
        if random.random() < 0.1:  # 10% chance each interval
            terminate_random_container()
    time.sleep(1800)

Capacity Planning and Resource Analysis

Claude assists with analyzing resource utilization data and generating capacity reports. Feed it your Prometheus metrics and ask for projections:

Our database CPU averages 70% with 5000 connections. Generate a capacity projection for 2x traffic growth assuming linear scaling.

Claude helps you build the analysis queries:

# Get CPU utilization percentiles via Prometheus HTTP API
curl -g 'http://localhost:9090/api/v1/query?query=histogram_quantile(0.50,rate(node_cpu_seconds_total{mode="idle"}[5m]))by(instance)'
curl -g 'http://localhost:9090/api/v1/query?query=histogram_quantile(0.95,rate(node_cpu_seconds_total{mode="idle"}[5m]))by(instance)'
curl -g 'http://localhost:9090/api/v1/query?query=histogram_quantile(0.99,rate(node_cpu_seconds_total{mode="idle"}[5m]))by(instance)'

Key Takeaways

Claude Code skills accelerate SRE workflows across multiple domains: incident response automation, log analysis, monitoring configuration, on-call management, chaos engineering, and capacity planning. The key is describing your infrastructure and goals clearly, then iterating on the generated code.

Start by integrating Claude into your most frequent SRE tasks. Build reusable scripts for common incidents, standardize your log queries, and create templates for monitoring dashboards. Over time, these scripts become institutional knowledge that your entire team can share and build on.

Built by theluckystrike — More at zovo.one