End-to-End DevOps Project: Building, Deploying, and Monitoring a Full-Stack Application

Introduction

This guide will walk you through setting up a full-stack application using DevOps practices. We'll cover infrastructure setup, CI/CD pipelines, containerization, deployment, monitoring, security, and more.

Project Overview

We’ll build a full-stack application, deploy it to AWS using Kubernetes, and implement continuous monitoring. The application will be containerized using Docker, and we’ll set up a CI/CD pipeline with Jenkins.

Prerequisites

• Basic knowledge of AWS, Docker, Kubernetes, and CI/CD concepts.

• AWS account and CLI configured.

• Docker installed.

• Jenkins and Terraform installed.

• GitHub repository for the project.

Step 1: Infrastructure Setup on AWS

1.1 Setting Up the VPC and Networking

Create a Virtual Private Cloud (VPC) to isolate your network. Use the AWS Management Console or CLI.

AWS CLI Command:

bashCopy codeaws ec2 create-vpc --cidr-block 10.0.0.0/16

1.2 Provisioning EC2 Instances

Provision EC2 instances to run your application.

AWS CLI Command:

bashCopy codeaws ec2 run-instances --image-id ami-0abcdef1234567890 --count 2 --instance-type t2.micro --key-name MyKeyPair

1.3 Setting Up an RDS Database

Launch an RDS instance for your database needs.

AWS CLI Command:

bashCopy codeaws rds create-db-instance --db-instance-identifier mydbinstance --db-instance-class db.t2.micro --engine mysql --allocated-storage 20 --master-username admin --master-user-password password

Step 2: Installing and Configuring Jenkins

2.1 Jenkins Installation

Install Jenkins on your EC2 instance or use a managed Jenkins service.

Installation Command for Ubuntu:

bashCopy codesudo apt update
sudo apt install openjdk-11-jdk
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins

2.2 Configuring Jenkins for GitHub Integration

Configure Jenkins to pull code from GitHub by setting up a GitHub plugin and creating a webhook.

Steps:

1. Install the GitHub plugin from Jenkins Plugin Manager.

2. Configure GitHub repository credentials in Jenkins.

3. Create a webhook in GitHub to trigger Jenkins builds.

2.3 Setting Up Jenkins Pipelines

Create a Jenkins pipeline for continuous integration and deployment.

Jenkinsfile Example:

groovyCopy codepipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                script {
                    docker.build('my-app')
                }
            }
        }
        stage('Deploy') {
            steps {
                script {
                    // Deployment scripts or commands
                }
            }
        }
    }
}

Step 3: Containerizing the Application with Docker

3.1 Writing a Dockerfile

Create a Dockerfile to define the environment for your application.

Dockerfile Example:

dockerfileCopy codeFROM node:14
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
CMD ["npm", "start"]

3.2 Building and Pushing Docker Images

Build your Docker image and push it to a container registry.

Build Command:

bashCopy codedocker build -t my-app .

Push Command:

bashCopy codedocker tag my-app myregistry/my-app
docker push myregistry/my-app

3.3 Docker Compose for Local Development

Use Docker Compose to manage multi-container applications.

docker-compose.yml Example:

yamlCopy codeversion: '3'
services:
  app:
    image: my-app
    ports:
      - "8080:8080"
  db:
    image: postgres
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password

Step 4: Deploying to Kubernetes (Amazon EKS)

4.1 Setting Up the EKS Cluster

Create an EKS cluster for deploying your containers.

AWS CLI Command:

bashCopy codeaws eks create-cluster --name my-cluster --role-arn arn:aws:iam::123456789012:role/EKS-Role --resources-vpc-config subnetIds=subnet-12345678,subnet-abcdef12,securityGroupIds=sg-12345678

4.2 Creating Kubernetes Manifests

Write Kubernetes manifests to define deployments and services.

Deployment Manifest Example:

yamlCopy codeapiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: myregistry/my-app
        ports:
        - containerPort: 8080

4.3 Deploying the Application on EKS

Apply your Kubernetes manifests to deploy the application.

Kubectl Command:

bashCopy codekubectl apply -f deployment.yml

Step 5: Implementing Continuous Monitoring with Prometheus and Grafana

5.1 Installing Prometheus

Set up Prometheus for monitoring metrics.

Helm Command:

bashCopy codehelm install prometheus prometheus-community/prometheus

5.2 Configuring Grafana Dashboards

Use Grafana to visualize metrics from Prometheus.

Grafana Command:

bashCopy codehelm install grafana grafana/grafana

5.3 Setting Up Alerts

Configure Prometheus Alertmanager to notify on issues.

Alertmanager Configuration Example:

yamlCopy codereceivers:
- name: 'email'
  email_configs:
  - to: 'your-email@example.com'

Step 6: Securing the CI/CD Pipeline

6.1 Scanning for Vulnerabilities with Trivy

Use Trivy to scan Docker images for vulnerabilities.

Trivy Command:

bashCopy codetrivy image myregistry/my-app

6.2 Integrating SonarQube for Code Quality

Analyze code quality with SonarQube.

SonarQube Scanner Command:

bashCopy codesonar-scanner -Dsonar.projectKey=my-project -Dsonar.sources=. -Dsonar.host.url=http://localhost:9000 -Dsonar.login=my-token

Step 7: Automating Infrastructure with Terraform

7.1 Writing Terraform Scripts

Define infrastructure using Terraform.

Terraform Script Example:

hclCopy codeprovider "aws" {
  region = "us-west-2"
}

resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
}

7.2 Managing Infrastructure as Code

Apply Terraform configurations to manage infrastructure.

Terraform Commands:

bashCopy codeterraform init
terraform apply

7.3 Terraform State Management

Manage Terraform state files for consistency.

Terraform Command:

bashCopy codeterraform state list

Step 8: Implementing Blue-Green Deployments

8.1 Setting Up Blue-Green Deployments

Configure blue-green deployments for zero-downtime updates.

Deployment Manifest Example:

yamlCopy codeapiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-blue
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: my-app
        color: blue

8.2 Automating Traffic Shifts

Use Kubernetes services to shift traffic between blue and green environments.

Service Manifest Example:

yamlCopy codeapiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080

8.3 Rollback Strategies

Implement rollback strategies in case of failures.

Kubectl Command for Rollback:

bashCopy codekubectl rollout undo deployment/my-app

Conclusion

In this guide, we covered the entire DevOps Consulting Services process from infrastructure setup to monitoring. Following these steps will help you build, deploy, and manage a full-stack application efficiently, leveraging modern DevOps practices for optimal results.