Project Overview
The Yelcamp Application is a comprehensive platform designed to manage campground locations, user registrations, reviews, and multimedia content. This project aims to implement a robust 3-tier architecture, encompassing the frontend, backend, and database layers, with an automated CI/CD pipeline to streamline development, testing, and deployment processes.
Architecture Diagram
Introduction
Context and Background
- Business Challenge: Need for a scalable, secure, and automated deployment infrastructure to support a multi-tier application.
- Pain Points: Manual provisioning delays, inconsistent configurations, and lack of monitoring and scalability.
- Strategic Objectives: Enable infrastructure automation, enhance deployment efficiency, and implement robust monitoring.
Personal Role and Approach
- Role: Architected the infrastructure and automated deployment pipeline.
- Initial Assessment: Identified inefficiencies in current practices and gaps in CI/CD workflows.
- Strategy: Leveraged Terraform for IaC, Trivy for Container image scanning, Sonarqube for Static code analysis, Jenkins for CI/CD, and Azure AKS for scalability.
Technical Journey
Problem Definition
- Challenge: Automating a multi-tier architecture deployment on Azure while ensuring scalability and monitoring.
- Limitations: Legacy manual workflows, lack of observability, and deployment inconsistencies.
- Constraints: Resource optimization, uptime assurance, and seamless integration.
Solution Design
Technology Selection Rationale
- Terraform: For consistent and repeatable infrastructure provisioning.
- Jenkins: To automate build, test, and deployment workflows.
- Azure AKS: Chosen for container orchestration with native cloud features.
- Comparative Analysis: Alternatives like AWS EKS were evaluated, but Azure AKS aligned with existing stack and budget.
Architectural Design
- Conceptual Approach: Designed a modular, reusable Terraform structure to support different environments. Also utilised Terraform workspace to provision infra for local, dev and production env.
- Principles Applied: Scalability, automation, and cost-efficiency.
- Innovative Strategies: Integrated Trivy for container vulnerability scanning.
Implementation Challenges
- Integration Complexity: Syncing Terraform modules with Jenkins pipelines and AKS configurations.
- Performance Bottlenecks: Managed through optimized AKS node scaling and resource allocation.
- Security: Implemented secure secrets management using Azure Key Vault.
Detailed Implementation Walkthrough
- Infrastructure Provisioning: Deployed using Terraform modules (resource groups, networks, AKS, key Vault, etc).
- Pipeline Configuration: Built CI/CD pipelines in Jenkins with static code analysis using SonarQube.
- Key Snippets:
terraform { required_providers { azuread = "~> 2.52.0" random = "~> 3.1" azurerm = "~> 3.110.0" } } provider "azurerm" { # Configuration options features { key_vault { purge_soft_delete_on_destroy = true recover_soft_deleted_key_vaults = true } } } terraform { backend "azurerm" { resource_group_name = "backend-rg" storage_account_name = "backendsa4tfvijaysingh" container_name = "tfstate" key = "DeployTrio.terraform.tfstate" } } module "resourcegroup" { source = "./modules/resourcegroup" project = var.project env = var.env location = var.location } module "ServicePrincipal" { source = "./modules/ServicePrincipal" project = var.project env = var.env spn = var.spn subscription_id = var.subscription_id depends_on = [ module.resourcegroup ] } module "keyvault" { source = "./modules/keyvault" project = var.project env = var.env location = var.location rg_name = module.resourcegroup.rg_name spn_name = module.ServicePrincipal.spn_name client_id = module.ServicePrincipal.client_id client_secret = module.ServicePrincipal.client_secret ssh_public_key_name = var.ssh_public_key_name ssh_public_key_path = var.ssh_public_key_path depends_on = [ module.ServicePrincipal, module.resourcegroup ] } ----------------- ----------------- ----------------- module "compute" { source = "./modules/compute" env = var.env location = var.location rg_name = module.resourcegroup.rg_name subnet_id = module.network.snet_id ssh_public_key = module.keyvault.ssh_public_key # Constructing dynamic names nic_map = { for k, v in var.nic_map : k => { nic_name = "${v.base_nic_name}-${k}-${var.project}-${var.env}" ip_config_name = "${v.base_ip_config_name}-${k}-${var.project}-${var.env}" pip_name = "${v.base_pip_name}-${k}-${var.project}-${var.env}" } } vm_map = { for k, v in var.vm_map : k => { vm_name = "${v.base_vm_name}-${k}-${var.project}-${var.env}" ip_name = "${v.base_ip_name}-${k}-${var.project}-${var.env}" vm_size = v.vm_size os_disk = "${v.base_os_disk_name}-${k}-${var.project}-${var.env}" admin_ssh_key_user = v.admin_ssh_key_user custom_data_script = v.custom_data_script } } depends_on = [ module.resourcegroup, module.network, module.keyvault ] } module "aks" { source = "./modules/aks" project = var.project env = var.env rg_name = module.resourcegroup.rg_name aks_location = var.aks_location client_id = module.ServicePrincipal.client_id client_secret = module.ServicePrincipal.client_secret ssh_public_key = module.keyvault.ssh_public_key service_principal_name = module.ServicePrincipal.spn_name }
Outcomes and Impact
Quantifiable Results
- Efficiency Gains: 60% reduction in infrastructure provisioning time.
- Cost Savings: Optimized AKS scaling reduced costs by 30%.
- Scalability: Supported seamless application scaling during peak loads.
Technical Achievements
- DevOps Practices: End-to-end automation of provisioning and deployment.
- Monitoring: Enhanced visibility with integrated dashboards.
- Innovation: Introduced modular Terraform design for reusable configurations.
Learning and Reflection
- Insights: Importance of modular design and continuous monitoring.
- Unexpected Challenges: Overcame resource contention in AKS clusters.
- Future Improvements: Explore advanced automation with GitOps practices.
Conclusion
- Significance: Demonstrates expertise in DevOps automation and cloud infrastructure.
- Lessons Learned: Value of consistent configuration and proactive monitoring.
- Future Scope: Enhanced monitoring capabilities, Advanced auto-scaling mechanisms,Experimenting with hybrid cloud models for enhanced flexibility.
Technical Appendix
- Stack: Terraform, Jenkins, Azure AKS, Docker, Trivy, SonarQube.
- Configurations: Included in GitHub Repository.
- Resources: Detailed guide available on Blog Link.