CloudWatch Logs and Logs Insights

Logs written on a server disappear with the server. Send them to CloudWatch Logs.

Add a log section to the agent configuration:

{ "logs": { "logs_collected": { "files": { "collect_list": [
    { "file_path": "/var/log/nginx/error.log",
      "log_group_name": "web/nginx-error",
      "retention_in_days": 30 } ] } } } }

Always set a retention period; the default is to keep logs forever and be billed for it.

Logs Insights runs queries across a log group. Errors from the last hour:

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 20

Count by status code:

parse @message "HTTP/1.1\" * " as status
| stats count(*) by status

Create a metric filter on a pattern such as ERROR to turn log lines into a metric you can alarm on.