Transferring EC2 Logs to CloudWatch and Setting up CPU Utilization Alarm with SNS Email Notification

I have created a blog to share my experience with dockerizing a Node.js application, monitoring logs with CloudWatch, and setting up a CPU utilization alarm with SNS email notification. As a fresher and learner in the field, I wanted to document my journey and provide a concise overview of the steps involved in this process.

In the blog, I walk you through the process of dockerizing a Node.js app using Docker Compose. We explore how to create a Dockerfile and define the service dependencies. Once the app is containerized, I guide you in installing the CloudWatch agent on your EC2 instance to capture logs from Nginx and Docker. We then dive into setting up a CloudWatch alarm that triggers an SNS email notification when CPU utilization exceeds 80%.

Throughout the blog, I strive to provide clear and easy-to-follow instructions, focusing on the essential aspects of monitoring on AWS. I hope that my blog serves as a helpful resource for fellow beginners and learners in this domain. By sharing my own learning journey, I aim to encourage others to embark on their own learning adventures and continue exploring the vast possibilities that cloud computing has to offer.

Launch EC2 Machine

sudo su
apt update

Cloning the project into the machine

apt intsall git
git init
git clone https://github.com/BroDevOps/node-todo-cicd.git

Creating a Dockerfile and docker-compose.yml file

Dockerfile

FROM node:latest
WORKDIR /app
COPY . .
RUN npm install
EXPOSE 3000
CMD ["npm", "start"]

docker-compose.yml

version: '3.5'
services:
  myproject:
    image: node-app-image
    restart: always
    build: 
      context: .
      dockerfile: .Dockerfile
    container_name: node-app-container
    ports:
      - "3000:3000"

Our container is working now

Installing nginx

sudo apt install nginx -y

Now I am going to install CloudWatch Agent, first, we need to create a Role for my EC2

Once it's created now I am attaching this role with EC2 Instance

Now time to install the Cloudwatch agent

Link

https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb

sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

Link

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

After this, we will get popups which we have to answer

Here you have mentioned the path of the nginx logs and also you have given the group and stream name. In this, I used the common group name for my nginx logs and another group name for my docker logs

Nginx Logs Path
/var/log/nginx/access.log
/var/log/nginx/error.log

Docker Logs Path
/var/lib/docker/containers/<container_id>/<container_id>-json.log

Copy this path

/opt/aws/amazon-cloudwatch-agent/bin/config.json

Now time to install the collectd

sudo apt install collectd

I am starting my Cloudwatch agent, in the configuration file path we have to pass the above path

Link

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:configuration-file-path

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file://opt/aws/amazon-cloudwatch-agent/bin/config.json

After that, we will check the status

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status

Cloudwatch

Here are 2 logs groups are showing

Inside Nginx, we have 2 streams

Our Docker Logs

CloudWatch Alarm and SNS Mail Notification

Click on Alarm and clicked to metrics and searched for the EC2

Set the custom time, I have selected 1 min just for testing purposes

Select the machine and click to Select Metric

After selecting this screen will be shown and you have to reselect the time again

Mentioned the condition here

Creating an SNS Topic

Mentioned the name and mail ID

After that click to create a topic, we can also add ASG and EC2 actions

Gave the name and description of the Alarm and click on next

Finally, preview and create

Clicked to create

On mail, we have to confirm it first

I am going to test this by using the stress command

sudo apt update
sudo apt install stress
sudo stress --version

htop

stress --cpu 8 --timeout 300s

As we can see it starts working

Testing the Process

We have received the mail

We have received another mail

Thank you for reading my blog, and I hope you find it informative and inspiring. If you have any feedback or questions, please feel free to reach out. Happy monitoring!

Thank you for reading my blog.

LinkedIn