In this blog post, I’ll guide you through the step-by-step process of setting up a Continuous Integration and Continuous Deployment (CI/CD) pipeline for microservices using Azure DevOps and ArgoCD. This comprehensive guide will help you automate the building, testing, and deployment of your applications, ensuring a seamless and efficient development workflow.
Introduction #
Modern software development demands rapid iterations and seamless deployments. Continuous Integration and Continuous Deployment (CI/CD) practices enable developers to merge code frequently, automate tests, and deploy applications efficiently. In this blog, we’ll leverage Azure DevOps for Continuous Integration (CI) and ArgoCD for Continuous Deployment (CD) to build a robust pipeline that handles everything from code commits to deploying applications in Azure Kubernetes cluster(AKS).
Architecture #
Prerequisites #
Before we dive in, make sure you have the following:
- An Azure account
- An SSH client (e.g., terminal for Mac/Linux, PuTTY for Windows)
- Git
- Basic knowledge of Kubernetes
Getting Started with Azure #
1. Sign Up for an Azure Account #
If you don’t already have an Azure account, sign up at Azure.
2. Sign In to Azure Portal #
Once you have an account, sign in to the Azure Portal.
3. Provision Azure Resources #
You can either use Terraform scripts to automate the provisioning of resources or manually create the necessary resources on the Azure portal. For simplicity, we’ll outline the manual steps:
- Create a Linux VM: This VM will serve as your agent pool for Azure DevOps.
- Create an Azure Container Registry (ACR): This will store your Docker images.
- Create an Azure Kubernetes Service (AKS) Cluster: This will host your microservices.
Detailed Steps for Provisioning Resources #
Approach. 1: Manual creation
- Create a Linux VM:
- Go to the Azure Portal, click on “Create a resource,” and select “Virtual Machine.”
- Follow the wizard to set up the VM.
- Create an Azure Container Registry (ACR):
- Navigate to “Create a resource” and select “Container Registry.”
- Fill in the required details and create the registry.
- Create an Azure Kubernetes Service (AKS) Cluster:
- Go to “Create a resource” and select “Kubernetes Service.”
- Follow the wizard to set up the cluster. Note that if you are using a free tier, you may need to choose a different region to avoid usage quota issues.
Note: Make sure all the resources are in the same resource group it will be easy to delete them.
Approach. 1: Using IaC [terraform] creation
- Install Azure CLI You can visit the blog and install Az CLI it will hardly take 5 mins.
- Login to Azure: Open your terminal and type the following command This will open a new browser window for you to sign in to your Azure account. If the CLI can open your default browser, it will do so and load an Azure sign-in page. Otherwise, you need to open a browser page and follow the instructions on the command line to enter an authorization code.
- Set your subscription (optional):
If you have multiple Azure subscriptions, and the one you want to use isn’t your default, you can set the subscription you want to use with this command:
Replace
az account set --subscription "your-subscription-id"
"your-subscription-id"
with your actual subscription ID. - Install Terraform: If you haven’t installed Terraform, you can download it from the official Terraform website. Unzip the package and move the binary to your PATH. Terraform Download
- Provision Azure Resources:
git clone https://github.com/vsingh55/AzureDevOps-CI-CD.git cd AzureDevOps-CI-CD/Terraform terraform init terraform plan terraform apply
Setting Up Azure DevOps #
1. Create an Azure DevOps Project #
- Sign in to Azure DevOps.
- Create a new project by clicking on “New Project.”
- Give your project a name and description and select the visibility (private or public).
2. Export the Repository to Azure #
- Export the voting-app repository to your azure devops portal because we are going to use microservices of voting-app.
- https://github.com/dockersamples/example-voting-app.git
3. Obtain Personal Access Tokens (PAT) #
You’ll need two personal access tokens: one for the Azure agent and one for ArgoCD. Follow these steps to generate a PAT:
- Go to your profile in Azure DevOps.
- Navigate to “Personal Access Tokens (PAT).”
- Generate new tokens with the required scopes.
Configuring the CI Pipeline #
1. Set Up the Agent Pool #
Adding VM to Agent Pool #
- Add the Created VM to the Agent Pool:
- In Azure DevOps, go to “Organization settings” and select “Agent pools.”
- Create a new agent pool and register your Linux VM.
- Install the Azure DevOps Agent: SSH into your Linux VM and run the following commands:
During configuration, provide your Azure DevOps server URL (
wget https://vstsagentpackage.azureedge.net/agent/3.239.1/vsts-agent-linux-x64-3.239.1.tar.gz sudo apt update sudo apt install docker.io mkdir myagent && cd myagent tar zxvf vsts-agent-linux-x64-3.239.1.tar.gz ./config.sh
https://dev.azure.com/{your-organization}
) and the personal access token. - Start the Agent:
Ensure the agent is running and listed as online in the Azure DevOps portal.
./run.sh
2. Configure Pipelines #
Creating and Configuring Pipelines #
- Create a New Pipeline:
- Go to the Pipelines section in Azure DevOps.
- Create a new pipeline and select “Azure Repos Git” as the source.
- Add YAML Configuration: Use the provided YAML files for each microservice. These files define the steps to build and push Docker images to the ACR. For example, the
voting-service.yml
file might look like this:# Docker # Build and push an image to Azure Container Registry # https://docs.microsoft.com/azure/devops/pipelines/languages/docker trigger: paths: include: - vote/* resources: - repo: self variables: # Container registry service connection established during pipeline creation dockerRegistryServiceConnection: 'this Will be automatically generated' imageRepository: 'voteapp' containerRegistry: 'vijayazurecicd.azurecr.io' dockerfilePath: '$(Build.SourcesDirectory)/vote/Dockerfile' tag: '$(Build.BuildId)' pool: name: 'azureagent' stages: - stage: Build displayName: Build stage jobs: - job: Build displayName: Build steps: - task: Docker@2 displayName: Build an image to Azure container registry inputs: containerRegistry: '$(dockerRegistryServiceConnection)' repository: '$(imageRepository)' command: 'build' Dockerfile: 'vote/Dockerfile' tags: '$(tag)' - stage: Push displayName: Push stage jobs: - job: Push displayName: Push steps: - task: Docker@2 displayName: Push an image to Azure container registry inputs: containerRegistry: '$(dockerRegistryServiceConnection)' repository: '$(imageRepository)' command: 'push' tags: '$(tag)'
- Run the Pipeline: Trigger the pipeline to ensure everything is set up correctly. The pipeline should build the Docker image and push it to the ACR.
For all the services visit Pipelines folder in Git Repo.
3. Continuous Integration with Azure DevOps #
Azure DevOps will handle the building and pushing of Docker images to the Azure Container Registry. Every time a change is pushed to the repository, the pipeline will automatically run, ensuring that the latest version of the code is built and ready for deployment.
Setting up Continuous Delivery: #
Azure Kubernetes Services (AKS) #
If you have provisioned resources using terraform then skip the step and run the cmd to connect from terminal. It is for those who wants to do it manually.
Run cmd on terminal
az aks get-credentials --resource-group <resource-group-name> --name <K8's cluster name> --overwrite-existing
Deploying with ArgoCD #
1. Install ArgoCD #
Installing ArgoCD on Kubernetes Cluster #
- Create a Namespace:
kubectl create namespace argocd
- Install ArgoCD:
Ensure all ArgoCD pods are running.
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml kubectl get pods -n argocd
- Access ArgoCD:
Change the
kubectl get svc -n argocd kubectl edit svc argocd-server -n argocd
type
fromClusterIP
toNodePort
to expose the ArgoCD server.
2. Configure ArgoCD #
Configuring ArgoCD for Continuous Deployment #
- Retrieve Initial Admin Password:
Within the file find password and copy it.
kubectl get secrets -n argocd kubectl edit secret argocd-initial-admin-secret -n argocd
This will give you the password for the ArgoCD admin user.echo <password> | base64 --decode
- Access ArgoCD UI: In your browser, navigate to
http://<node-external_ip>:<nodeport>
. Useadmin
as the username and the decoded password to log in. - Allow inbound rule for ArgoCD: Add inbound rule for argoCD nodeport in AKS networking settings.
- Connect to Azure Git Repo:
- In the ArgoCD UI, go to Settings and connect your Azure Git repository.
- Use the repository URL format:
https://<personal_access_token>@dev.azure.com/<organization_name>/<project_name>/_git/<project_name>
- Create an Application in ArgoCD:
- In ArgoCD, create a new application.
- Fill in the details such as application name, project, sync policy, repository URL, and path.
- Automate Image Updates:
- Write a script to update Kubernetes manifests with the new image name from the ACR.
- Create a folder in your Azure repo for the scripts and integrate them into your pipelines lets call it update stage.
- Enable AKS to Pull Images from ACR:
Get the service principal ID and password from the Azure portal:
kubectl create secret docker-registry <secret-name> \ --namespace <namespace> \ --docker-server=<container-registry-name>.azurecr.io \ --docker-username=<service-principal-ID> \ --docker-password=<service-principal-password>
- Go to the Azure portal, navigate to your ACR, and enable admin access.
- Copy the service principal ID and password.
Verifying the Deployment #
- Verify ArgoCD Sync:
- Make changes to the application code and push to the repository.
- ArgoCD will detect the changes and sync the application automatically.
- Access Deployed Applications:
- Get the external IP and node ports for your applications:
kubectl get nodes -o wide //To know node external ip kubectl get svc -n <namespace> //
- Access the applications using the external IP and ports.
- Add Inbound rule for htttp port.
- Here are some screenshots before and after the implementing CI/CD
- Get the external IP and node ports for your applications:
Conclusion #
Congratulations! You’ve successfully set up a CI/CD pipeline using Azure DevOps, AKS and ArgoCD. This pipeline automates the building, testing, and deployment of your microservices, ensuring a seamless and efficient workflow. By leveraging these powerful tools, you can focus on writing code and delivering features while the pipeline takes care of the rest.
Acknowledgements #
Thanks to Azure, Azure DevOps, ArgoCD for providing the platform and tools to build this CI/CD pipeline.
Reply by Email