3

I would like to view all my logs in cloudwatch. I currently have a multi-docker Elastic beanstalk environment.

I have selected for the logs to push to cloudwatch by going to

Elastic Beanstalk > App > Env > Configuration > Software Configuration > CloudWatch Logs

This is enabled.

When I look in cloudwatch I see the following bits..

/aws/elasticbeanstalk/myapp-staging/var/log/docker-events.log
/aws/elasticbeanstalk/myapp-staging/var/log/eb-activity.log
/aws/elasticbeanstalk/myapp-staging/var/log/eb-ecs-mgr.log
/aws/elasticbeanstalk/myapp-staging/var/log/ecs/ecs-agent.log
/aws/elasticbeanstalk/myapp-staging/var/log/ecs/ecs-init.log

But I do not see the nginx access/error logs.

I have this in my Dockerrun.aws.json

"mountPoints":[
        {
          "sourceVolume": "awseb-logs-nginx",
          "containerPath": "/var/log/nginx"
        }

If i SSH to the instance with this contain on I can see these logs (when I hit URLS) being generated (as expected) on the local path of /var/log/containers/nginx/access.log etc.

Also If i go to EBS > Logs and request recent logs I can also see them what is in the access log, it just not being sent to CloudWatch?

I wondered if I had to setup a Log Group with the correct path, i tried this but it did not get populated?

I am sure that I am missing something to push this on to Cloudwatch, many thanks in advance!

Update: I've since added the below, which was suggested as what is required. This file sits in .ebextensions folder named nginx_logs.conf.

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/config/nginx_logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/containers/nginx/access.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/nginx/access.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/nginx/access.log*
      [/var/log//containers/nginx/error.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/nginx/error.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/nginx/error.log*

commands:
  "01":
    command: chkconfig awslogs on
  "02":
    command: service awslogs restart

Still no joy, as again I can still see them when I hit request logs or request last 100 logs. But nothing in cloudwatch...

2
1

According to the official AWS documentation, the following files are collected out of the box for Docker Multicontainer EB environments:

  • /var/log/eb-activity.log
  • /var/log/ecs/ecs-init.log
  • /var/log/eb-ecs-mgr.log
  • /var/log/ecs/ecs-agent.log
  • /var/log/docker-events.log

Since these paths don't include the directory which contains the nginx logs, it makes sense they won't be streamed to CloudWatch.

To stream your logs, looks like you will have to configure the CloudWatch Logs agent to collect the files in the containers directory. There are sample configurations here. Frankly it's a little strange the default config for Docker Multicontainer doesn't include container logs, but apparently this is how AWS has implemented this.

3
  • I updated my question with more information, however still not working? any ideas? – Matt The Ninja Sep 4 '17 at 12:53
  • According to the docs, files in .ebextentions should be with .config suffix. After changing this, I would check the EB logs on the instance to verify that the tasks in the file are indeed executed. Then it's just a matter of troubleshooting according to the errors you might get in the logs. – Johannes Liebermann Sep 4 '17 at 14:41
  • Sorry, I had it as a .config. I'll look if I can check they are executed then. Thanks! – Matt The Ninja Sep 4 '17 at 15:37
0

We use the following ebextensions to stream application (nginx/ror) logs to cloudwatch

$ cat .ebextensions/cw-logs.config
files:
  "/etc/awslogs/config/nginx-access-log.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
        [nginx-access.log]
        datetime_format = %Y-%m-%dT%H:%M:%S%f
        file = /var/log/containers/project1-staging/nginx_access.log
        buffer_duration = 5000
        log_stream_name = project1_staging_{instance_id}
        initial_position = start_of_file
        log_group_name = PROJECT1_STAGING_NGINX

  "/etc/awslogs/config/app-log.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
        [app-access.log]
        datetime_format = %Y-%m-%dT%H:%M:%S
        file = /var/log/containers/project1-staging/app.log
        buffer_duration = 5000
        log_stream_name = project1_staging_{instance_id}
        initial_position = start_of_file
        log_group_name = PROJECT1_STAGING_APP

commands:

    00-cmd:
        command: chkconfig --level 35 awslogs on
        test: "[ -s /usr/sbin/awslogsd ]"

container_commands:
    00-cmd:
        command: service awslogs restart
        test: "[ -s /etc/awslogs/config/nginx-access-log.conf -a -s /etc/awslogs/config/app-log.conf ]"

And rotation to S3 policy

$ cat .ebextensions/cw-logs-s3-rotation.config
files:
  "/tmp/logrotate-project1-staging.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
        /var/log/containers/project1-staging/*.log {
            size 512M
            weekly
            rotate 0
            missingok
            compress
            notifempty
            copytruncate
            dateext
            dateformat -%Y-%m-%d_%s
            olddir /var/log/containers/project1-staging/rotated
            lastaction
                aws s3 mv --recursive /var/log/containers/project1-staging/rotated/ \
                s3://app-logs-archive-s3-bucket/project1-staging/$(date '+%Y')/
            endscript
        }

container_commands:
    00-cmd:
        command: mv /tmp/logrotate-project1-staging.conf /etc/logrotate.elasticbeanstalk.hourly/logrotate.elasticbeanstalk.applogs-project1-staging.conf
        test: "[ -f /tmp/logrotate-project1-staging.conf ]"

Our Dockerrun.aws.json looks like

$ cat Dockerrun.aws.json 
{
    "AWSEBDockerrunVersion": 2,

    "containerDefinitions": [{
        "name": "project1-staging",
        "image": "0123456789.dkr.ecr.us-east-1.amazonaws.com/project1:staging_v.1234567",
        "cpu": {{ env['DOCKER_CPU'] }},
        "memory": {{ env['DOCKER_MEMORY'] }},
        "essential": true,

        "portMappings": [{
            "hostPort": 80,
            "containerPort": 80
        },
        {
            "hostPort": 443,
            "containerPort": 443
        }],

        "mountPoints": [{
            "sourceVolume": "awseb-logs-project1-staging",
            "containerPath": "/home/app/project1/log"
        }]
    }]
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.