Reducing AWS CloudWatch and Logging Costs

Why CloudWatch Costs Spiral

CloudWatch is essential for monitoring, but its logging costs can surprise you. At $0.50/GB for log ingestion and $0.03/GB/month for storage, a verbose application writing 100 GB/day of logs costs $1,500/month in ingestion alone. Here's how to fix it.

Top CloudWatch Cost Drivers

1. Log ingestion volume — The #1 cost. Every byte written to CloudWatch Logs is charged at $0.50/GB.
2. Log retention — Default retention is "Never expire". Old logs accumulate and cost $0.03/GB/month in storage.
3. Custom metrics — Each custom metric costs $0.30/month. High-cardinality metrics (per-user, per-request) multiply rapidly.
4. Dashboard widgets — Each widget costs $3/month after the first 3 dashboards.
5. Contributor Insights — $0.02 per matching log event. Can be expensive on high-volume log groups.

Fix 1: Set Log Retention Policies

# Set 30-day retention on all log groups
aws logs describe-log-groups --query 'logGroups[].logGroupName' --output text | \
  tr '\t' '\n' | while read lg; do
    aws logs put-retention-policy --log-group-name "$lg" --retention-in-days 30
  done

Fix 2: Use Subscription Filters to Route Selectively

Instead of sending all logs to CloudWatch, use subscription filters to route only important logs there. Stream the rest to S3 (at $0.023/GB) for long-term storage.

Fix 3: Reduce Log Verbosity

• Set production log levels to WARN or ERROR, not DEBUG
• Exclude health check logs from ALB access logging
• Sample verbose logs (log 1 in 100 requests instead of all)
• Remove duplicate logging (application + sidecar logging the same events)

Fix 4: Use Metric Filters Instead of Insights

CloudWatch Metric Filters extract numeric values from log lines and create CloudWatch Metrics — for free. This is much cheaper than running CloudWatch Insights queries on raw logs.

Fix 5: Consider Alternatives for Log Aggregation

For large-scale logging, consider routing logs to S3 + Athena (query on demand) or a self-managed ELK/OpenSearch stack. The cost per GB is 10-20x lower than CloudWatch Logs.

Real-world result: By setting 14-day retention, reducing log verbosity, and routing bulk logs to S3, we cut a client's CloudWatch bill from $4,200/month to $680/month.