From bab3363ba326abdbde15a7fc20ff2303d184fe12 Mon Sep 17 00:00:00 2001 From: Ashot Date: Wed, 24 Jul 2024 20:09:39 +0400 Subject: [PATCH 1/4] terraform files added --- terraform/README.md | 150 ++++++++++++++++++++++++++++++ terraform/ec2_instance/main.tf | 9 ++ terraform/ec2_instance/outputs.tf | 9 ++ terraform/main.tf | 82 ++++++++++++++++ terraform/networking/main.tf | 32 +++++++ terraform/networking/outputs.tf | 16 ++++ terraform/provider.tf | 3 + terraform/remote_backend_s3.tf | 7 ++ 8 files changed, 308 insertions(+) create mode 100644 terraform/README.md create mode 100644 terraform/ec2_instance/main.tf create mode 100644 terraform/ec2_instance/outputs.tf create mode 100644 terraform/main.tf create mode 100644 terraform/networking/main.tf create mode 100644 terraform/networking/outputs.tf create mode 100644 terraform/provider.tf create mode 100644 terraform/remote_backend_s3.tf diff --git a/terraform/README.md b/terraform/README.md new file mode 100644 index 0000000..05f3cb7 --- /dev/null +++ b/terraform/README.md @@ -0,0 +1,150 @@ +# DevOps Project README + +Welcome to the DevOps project README. This document provides an overview of the project and detailed instructions for setting up and deploying the infrastructure and applications. + +## Project Overview +This project aims to deploy a scalable web application infrastructure on AWS using Terraform, Packer, Ansible, GitHub Actions, CloudFront, and Amazon EKS. Below are the main tasks and their respective setups: + +## Tasks +#### 1. Database Creation with Packer + Ansible and Deployment on EC2 +Description: Create a database using Packer and Ansible and deploy it on EC2 instances. + +#### 2. Remote Backend on S3 Bucket +Description: Configure Terraform to use S3 as a remote backend for state storage. + +#### 3. Infrastructure Creation with Terraform on AWS +Description: Use Terraform to provision infrastructure on AWS. + +#### 4. Building React Files with GitHub Actions and Uploading to S3 Bucket +Description: Automate the build process of React files using GitHub Actions and deploy to an S3 bucket. + +#### 5. Frontend Serving with CloudFront, SSL Setup, and HTTPS Redirections +Description: Serve the frontend application using CloudFront with SSL setup and HTTPS redirections. + +#### 6. Backend Deployment on EKS with Minimum 3 Replicas +Description: Deploy the backend application on Amazon EKS with a minimum of 3 replicas for scalability and high availability. + +## Setup Instructions + +### Prerequisites +Ensure you have the following installed and configured: + +AWS CLI +Terraform +Packer +Ansible +GitHub Actions setup with appropriate secrets and workflows + +## Steps + +### Automating PostgreSQL AMI Creation with Packer, Ansible, and GitHub Actions + +Overview +This guide shows you how to automate creating an AWS AMI with PostgreSQL using Packer, Ansible, and GitHub Actions. + +##### Step 1: Create the Packer Template + +Make a Packer template to set up the AMI build process. This template specifies the base AMI, instance type, region, and the Ansible playbook for provisioning. + +##### Step 2: Create the Ansible Playbook + +Write an Ansible playbook to install and set up PostgreSQL on the instance. The playbook handles installing PostgreSQL, starting the service, and creating a PostgreSQL user and database. + +##### Step 3: Set Up GitHub Actions Workflow + +Set up a GitHub Actions workflow to automate the AMI creation. The workflow includes steps to: + +• Check out the repository + +• Set up Python and Ansible + +• Install Packer and AWS CLI + +• Configure AWS credentials + +• Build the AMI + +• Capture and output the AMI ID + +##### Step 4: Add Secrets to GitHub + +Securely store your AWS credentials in GitHub Secrets. This allows the workflow to access the needed credentials to build the AMI. + +##### Step 5: Using the AMI with Terraform + +After the AMI is created, use the AMI ID in your Terraform configuration to launch an EC2 instance. This allows you to deploy infrastructure using the newly created AMI. + +##### Conclusion + +Following these steps, you can automate the creation of an AWS AMI with PostgreSQL using Packer, Ansible, and GitHub Actions. This makes it easier to manage and deploy your infrastructure. + +### Remote Backend on S3 Bucket + +• Create an S3 bucket manually. + +• Update backend.tf to configure Terraform to use the S3 bucket as the backend. + + +### Infrastructure Setup with Terraform + +This guide outlines the setup of a complete infrastructure on AWS using Terraform. The infrastructure includes a VPC, an EC2 instance with a PostgreSQL image created using Packer and Ansible, an EKS cluster for the backend, a load balancer, S3 for the frontend, and CloudFront for serving the frontend. + +##### Step 1: Set Up a VPC + +Create a Virtual Private Cloud (VPC) to isolate your resources. Configure subnets, route tables, and internet gateways to allow communication within the VPC and to the internet. + +##### Step 2: Launch an EC2 Instance + +Use the AMI created by Packer and Ansible to launch an EC2 instance. This instance will have PostgreSQL installed and configured, ready for your applications to use. + +##### Step 3: Set Up an EKS Cluster + +Create an Amazon Elastic Kubernetes Service (EKS) cluster to manage your backend services. This cluster will run your applications in a highly available and scalable environment. + +##### Step 4: Configure a Load Balancer + +Set up a load balancer using Terraform to distribute traffic to your backend services running in the EKS cluster. This ensures that your applications can handle varying loads and remain accessible. + +##### Step 5: Set Up S3 for Frontend + +Create an S3 bucket using Terraform to store your frontend files. These files can be static assets like HTML, CSS, and JavaScript, which will be served to users. Automate this setup with GitHub Actions. + +##### Step 6: Configure CloudFront + +Use CloudFront to serve your frontend files from S3. CloudFront will cache your files at edge locations worldwide, providing faster access to users and reducing load on your S3 bucket. Automate this configuration with GitHub Actions. + +##### Step 7: Create a GitHub Actions Workflow +Create a new YAML file (e.g., .github/workflows/deploy-infrastructure.yml) in your repository under the .github/workflows/ directory. This file will define the GitHub Actions workflow for deploying your Terraform infrastructure. + + +### Building React Files with GitHub Actions and Uploading to S3 Bucket + +• Configure GitHub Actions workflows in .github/workflows/ to build React files. + +• Ensure the workflow uploads the built files to the designated S3 bucket. + +### Frontend Serving with CloudFront, SSL Setup, and HTTPS Redirections + +• Create a CloudFront distribution with the S3 bucket as the origin. + +• Obtain or import an SSL certificate from AWS ACM. + +• Configure CloudFront to use HTTPS and set up HTTP to HTTPS redirections. + +### 6.Backend Deployment on EKS with Minimum 3 Replicas + +• Define Kubernetes manifests (deployment.yaml, service.yaml) for the backend application. + +• Create a new YAML file (e.g., .github/workflows/deploy-backend-eks.yml) in your repository under the .github/workflows/ directory. This file will define the GitHub Actions workflow for deploying your backend application on EKS with a minimum of 3 replicas. + +## Usage +Provide instructions on accessing and using the deployed application and infrastructure. + +Contributing +If you wish to contribute to this project, please fork the repository and submit pull requests. + +Troubleshooting +Include troubleshooting tips and solutions for common issues. + +License +This project is licensed under the [License Name] - see the LICENSE file for details. diff --git a/terraform/ec2_instance/main.tf b/terraform/ec2_instance/main.tf new file mode 100644 index 0000000..2a05b05 --- /dev/null +++ b/terraform/ec2_instance/main.tf @@ -0,0 +1,9 @@ +resource "aws_instance" "postgresql" { + ami = var.ami_id + instance_type = var.instance_type + subnet_id = var.subnet_id + + tags = { + Name = var.instance_name + } +} diff --git a/terraform/ec2_instance/outputs.tf b/terraform/ec2_instance/outputs.tf new file mode 100644 index 0000000..96814fc --- /dev/null +++ b/terraform/ec2_instance/outputs.tf @@ -0,0 +1,9 @@ +output "instance_id" { + description = "ID of the EC2 instance" + value = aws_instance.postgresql.id +} + +output "private_ip" { + description = "Public IP address of the EC2 instance" + value = aws_instance.postgresql.private_ip +} diff --git a/terraform/main.tf b/terraform/main.tf new file mode 100644 index 0000000..b7aa30c --- /dev/null +++ b/terraform/main.tf @@ -0,0 +1,82 @@ +# vpc module + +module "networking" { + source = "./networking" + vpc_cidr = var.vpc_cidr + vpc_name = var.vpc_name + us_availability_zone_1 = var.us_availability_zone_1 + us_availability_zone_2 = var.us_availability_zone_2 + cidr_private_subnet_1 = var.cidr_private_subnet_1 + cidr_private_subnet_2 = var.cidr_private_subnet_2 +} + +output "vpc_id" { + value = module.networking.vpc_id +} + +output "private_subnet_1" { + value = module.networking.private_subnet_1_ids +} + + +output "private_subnet_2" { + value = module.networking.private_subnet_2_ids +} + + +# ec2_instance module + +module "ec2_instance" { + source = "./ec2_instance" + ami_id = var.ami_id + instance_type = var.instance_type + subnet_id = module.networking.private_subnet_1_ids + instance_name = var.instance_name +} + + + +output "ec2_instance_id" { + value = module.ec2_instance.instance_id +} + + +output "ec2_instance_private_ip" { + value = module.ec2_instance.private_ip +} + + + +module "eks_cluster" { + source = "terraform-aws-modules/eks/aws" + + cluster_name = var.cluster_name + cluster_version = "1.21" + + vpc_id = module.networking.vpc_id + subnet_ids = [ + module.networking.private_subnet_1_ids, + module.networking.private_subnet_2_ids + ] + control_plane_subnet_ids = module.networking.private_subnet_2_ids + + +eks_managed_node_groups = { + example = { + + instance_types = ["t2.micro"] + + min_size = 3 + max_size = 3 + desired_size = 3 + } +} + + + tags = { + Environment = "Production" + # Add more tags as needed + } +} + + diff --git a/terraform/networking/main.tf b/terraform/networking/main.tf new file mode 100644 index 0000000..1bcf698 --- /dev/null +++ b/terraform/networking/main.tf @@ -0,0 +1,32 @@ +resource "aws_vpc" "dev-proj-us-west-vpc-1" { + cidr_block = var.vpc_cidr // Replace with your desired CIDR block + enable_dns_support = true + enable_dns_hostnames = true + + tags = { + Name = "main-vpc" + } +} + +// Subnet 1 in Availability Zone b +resource "aws_subnet" "private_subnet_1" { + vpc_id = aws_vpc.dev-proj-us-west-vpc-1.id + cidr_block = var.cidr_private_subnet_1 // Replace with your desired CIDR block + availability_zone = var.us_availability_zone_1 // Replace with your desired AZ + + tags = { + Name = "dev-proj-private-subnet-1" + } +} + + +// Subnet 2 in Availability Zone c +resource "aws_subnet" "private_subnet_2" { + vpc_id = aws_vpc.dev-proj-us-west-vpc-1.id + cidr_block = var.cidr_private_subnet_2 // Replace with your desired CIDR block + availability_zone = var.us_availability_zone_2 // Replace with your desired AZ + + tags = { + Name = "dev-proj-private-subnet-2" + } +} diff --git a/terraform/networking/outputs.tf b/terraform/networking/outputs.tf new file mode 100644 index 0000000..d22be42 --- /dev/null +++ b/terraform/networking/outputs.tf @@ -0,0 +1,16 @@ +output "vpc_id" { + description = "The ID of the VPC" + value = aws_vpc.dev-proj-us-west-vpc-1.id +} + +output "private_subnet_1_ids" { + description = "The IDs of the private subnets" + value = aws_subnet.private_subnet_1.id +} + + + +output "private_subnet_2_ids" { + description = "The IDs of the private subnets" + value = aws_subnet.private_subnet_2 +} \ No newline at end of file diff --git a/terraform/provider.tf b/terraform/provider.tf new file mode 100644 index 0000000..b63750c --- /dev/null +++ b/terraform/provider.tf @@ -0,0 +1,3 @@ +provider "aws" { + region = "us-west-1" +} \ No newline at end of file diff --git a/terraform/remote_backend_s3.tf b/terraform/remote_backend_s3.tf new file mode 100644 index 0000000..866c1c4 --- /dev/null +++ b/terraform/remote_backend_s3.tf @@ -0,0 +1,7 @@ +terraform { + backend "s3" { + bucket = "devops-proj-1-remote-state-bucket-11" + key = "devops-project-1/dev_project/terraform.tfstate" + region = "us-west-1" + } +} \ No newline at end of file From 031c41b23efa0a5e4355d66505d8e4de18c80cc2 Mon Sep 17 00:00:00 2001 From: Ashot Date: Wed, 24 Jul 2024 20:13:58 +0400 Subject: [PATCH 2/4] kube service added --- terraform/kub_lb_service.tf | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 terraform/kub_lb_service.tf diff --git a/terraform/kub_lb_service.tf b/terraform/kub_lb_service.tf new file mode 100644 index 0000000..3684452 --- /dev/null +++ b/terraform/kub_lb_service.tf @@ -0,0 +1,28 @@ +provider "kubernetes" { + config_path = "~/.kube/config" +} +resource "kubernetes_service" "backend" { + metadata { + name = "backend" + } + spec { + selector = { + app = "backend" + } + port { + port = 80 + target_port = 8080 + } + type = "LoadBalancer" + } +} +# Wait for the Load Balancer to be provisioned and get its DNS name +data "external" "get_lb_dns" { + program = ["sh", "-c", < Date: Fri, 26 Jul 2024 16:11:22 +0400 Subject: [PATCH 3/4] building and pushing Docker image --- .github/workflows/build-and-push-backend.yml | 34 ++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 .github/workflows/build-and-push-backend.yml diff --git a/.github/workflows/build-and-push-backend.yml b/.github/workflows/build-and-push-backend.yml new file mode 100644 index 0000000..3392fb0 --- /dev/null +++ b/.github/workflows/build-and-push-backend.yml @@ -0,0 +1,34 @@ +name: Build and Push Docker Image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v2 + + - name: Clone the awesome-compose repository + run: git clone https://github.com/docker/awe some-compose.git + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v2 + + - name: Log in to Docker Hub + uses: docker/login-action@v2 + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKER_PASSWORD }} + + - name: Build and push Docker image + uses: docker/build-push-action@v4 + with: + context: ./awesome-compose/react-rust-postgres/backend + file: ./awesome-compose/react-rust-postgres/backend/Dockerfile + push: true + tags: ashotmartirosyan02/awesome-compose-react-rust-backend:latest \ No newline at end of file From cf07d8f0053ab3c767ac39b819a6e3110dd03167 Mon Sep 17 00:00:00 2001 From: Ashot Date: Fri, 26 Jul 2024 16:23:01 +0400 Subject: [PATCH 4/4] added managed node groups size --- terraform/main.tf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/terraform/main.tf b/terraform/main.tf index b7aa30c..b581332 100644 --- a/terraform/main.tf +++ b/terraform/main.tf @@ -66,9 +66,9 @@ eks_managed_node_groups = { instance_types = ["t2.micro"] - min_size = 3 + min_size = 1 max_size = 3 - desired_size = 3 + desired_size = 2 } }