AWS EKS Country-Based Routing & Deployment

Published in

Now.gg Tech

6 min readAug 8, 2022

Every organization with mass-adopted products often go through rigorous testing before large-scale deployments. Large-scale deployments can be complex when dealing with multi-cloud, multi-region deployments.

Implementing a phased deployment of an internet-facing application can be a vital piece of the puzzle and mitigate significant complexities that often lead to failed deployments. This article showcased how you can work with phased deployments to achieve better control over your application exposure and testing in real-world scenarios.

For the scope of this article, we will utilize AWS ELB (Elastic Application Load Balancer with EKS (Elastic Kubernetes Service) and on top is CloudFront. AWS ELB is an application layer load balancer with several features that help with the manipulation and routing of the incoming requests based on URL, headers, host, path, query params and even cookies.

To use an application load balancer for an Internet-facing application running on Elastic Kubernetes Services (AWS EKS), we need to configure the load balancer Ingress Controller to add a load balancer or to expose our service running on EKS to the internet.

Problem Statement

One of our applications runs on AWS EKS workloads. With this application, we already configured ingress and ELB to run as an Internet-Facing application exposed to users through CloudFront.

Based on the requirements, we wanted to use a canary-type deployment to test this new release on the production system. We considered using a weightage-centric deployment strategy of the load balancer for this canary use case. However, that required us to add a layer or some other functionalities to identify and control requests making it complex to test.

We decided to run down the entire control structure and get this in control in a non-complex and easy manner. Our team’s brainstorming session resulted in an innovative solution to test the application in a production environment without jeopardizing the entire deployment after all large-scale deployments must be handled with utmost caution.

The idea was to deploy the new release version to a specific part of the world, say a particular country, where only a few requests get initiated. This strategy effectively enabled us to run two different deployment versions of the same application in different global regions. It made the lives of our developers and testers much more manageable.

Easy how? You might ask.

Here is the answer.

Let us assume we will release a new version of our application for India.
The rest of the world is still being served with our previous version.
Using a VPN, our developers and testers can easily access the new version that was rolled for the India region, enabling a more accessible approach to test the functionalities on the production environment and also measure the performance impacts of the new version before it is rolled out globally.

SOLUTION

After intense brainstorming around this, the research uncovered no viable or direct solutions to facilitate this scenario. However, we did come across some hints to use Application ELB features to route requests based on the components, such as:

Header
Query String
Host
Body
Port
Cookie

AWS CloudFront enables us to identify a viewer’s location, more precisely the viewer’s country, as it appends the CloudFront-Viewer-Country header in the incoming request and forwards it to the origin. Using this header, we can manipulate the target groups from which we want to complete the requests. AWS ELB created by the ingress controller of EKS has the capability to route requests based on given conditions/annotations.

While we use the ALB ingress controller for our EKS, the specific features of an application load balancers can be triggered using:

alb.ingress.kubernetes.io/actions.<UNIQUE_RULE_NAME>

Annotations/conditions can be further used inside the .spec.rules array within the ingress controller. The following is a sample JSON for annotation:

{
 “type”:”forward”,
 “ForwardConfig”:{
 “TargetGroups”:[
 {
 “ServiceName”:”Service-INDIA”,
 “ServicePort”:”80"
 }
 ]
 }
}

We were able to create this annotation by referencing these docs on GitHub.

WORKFLOW

Let us go through the workflow using the following visual representation:

The user will initiate a request, wherein the DNS will be resolved based on the CloudFront configurations.
CloudFront will then add the CloudFront-Viewer-Country Header along with its specified value and forward the request to the origin, which is our Ingress Load Balancer.
Now, We have created four Deployment K8S workloads and four Corresponding services:

Deployments:

Deployment-INDIA
Deployment-SINGAPORE
Deployment-USA
Deployment-RestOfTheWorld

Service/Endpoints:

Service-INDIA
Service-SINGAPORE
Service-USA
Service-RestOfTheWorld

We use Jenkins as a CI/CD Deployment tool to handle release and code for these deployments. Based on the requirement, let us assume that all four deployments currently serve the same code version, meaning that all the current code is live worldwide. However, we want to deploy and test a new version.

To test a new version using Jenkins:

Select the new tag from GIT
Select the countries where you wish to deploy the new code, as in our case, we will select the deployment name which represents the country.
Select the Deployment-INDIA and Deployment-SINGAPORE with the new release tag.

Fig. 2 is a screengrab from the build with parameters page in our Jenkins Job, where:

Global is for the worldwide deployment of all four versions.
Others are for ‘Deployment-RestOfTheWorld’. In our case, this will deploy to all regions other than India, Singapore, and the US.

What happens now?

As soon as we trigger the build, a new docker image will be created.
Deployment YAML files of ‘Deployment-INDIA and Deployment-SINGAPORE’ will be attached with this docker image and applied using kubectl or helm.
Now, Deployment-USA and Deployment-RestOfTheWorld are serving older versions.
While Deployment-INDIA & Deployment-SINGAPORE are serving newer versions.
Remember, this implementation was through the Ingress controller. In this case, if the header value for CloudFront-Viewer-Country is:

US -The request will be routed to service Service-USA which is the endpoint service for Deployment-USA.IN — The request will be routed to service Service-INDIA.SG — The request will be routed to service Service-SINGAPOREOthers — If the header value doesn’t match any of these three, then the request will be routed to Service-RestOfTheWorld.

IMPLEMENTATION

Combining the strategies, we can implement this by first creating an ingress file having the conditions for routing requests to the respective backend. The following is a sample ingress file for your reference:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
 namespace: default
 name: <NAME>
 annotations:
 kubernetes.io/ingress.class: alb
 alb.ingress.kubernetes.io/scheme: internet-facing
 alb.ingress.kubernetes.io/target-type: ip
 alb.ingress.kubernetes.io/subnets: <SUBNETID>
 alb.ingress.kubernetes.io/listen-ports: ‘[{“HTTP”:80}]’
 alb.ingress.kubernetes.io/actions.rule-path3: ‘{“type”:”forward”,”ForwardConfig”:{“TargetGroups”:[{“ServiceName”:”Service-INDIA”,”ServicePort”:”80"}]}}’
 alb.ingress.kubernetes.io/conditions.rule-path3: ‘[{“field”:”http-header”,”httpHeaderConfig”:{“httpHeaderName”: “cloudfront-viewer-country”, “values”:[“IN”]}}]’
 alb.ingress.kubernetes.io/actions.rule-path4: ‘{“type”:”forward”,”ForwardConfig”:{“TargetGroups”:[{“ServiceName”:”Service-USA”,”ServicePort”:”80"}]}}’
 alb.ingress.kubernetes.io/conditions.rule-path4: ‘[{“field”:”http-header”,”httpHeaderConfig”:{“httpHeaderName”: “cloudfront-viewer-country”, “values”:[“US”]}}]’
 alb.ingress.kubernetes.io/actions.rule-path5: ‘{“type”:”forward”,”ForwardConfig”:{“TargetGroups”:[{“ServiceName”:”Service-SINGAPORE”,”ServicePort”:”80"}]}}’
 alb.ingress.kubernetes.io/conditions.rule-path5: ‘[{“field”:”http-header”,”httpHeaderConfig”:{“httpHeaderName”: “cloudfront-viewer-country”, “values”:[“SG”]}}]’
 alb.ingress.kubernetes.io/actions.rule-path6: ‘{“type”:”forward”,”ForwardConfig”:{“TargetGroups”:[{“ServiceName”:”Service-RestOfTheWorld”,”ServicePort”:”80"}]}}’spec:
 backend:
 serviceName: rule-path6
 servicePort: use-annotation
 rules:
 — http:
 paths:
 — backend:
 serviceName: rule-path3
 servicePort: use-annotation
 — backend:
 serviceName: rule-path4
 servicePort: use-annotation
 — backend:
 serviceName: rule-path5
 servicePort: use-annotation

Save the above-illustrated ingress file and then apply it.

kubectl apply -f ingress.yml

Now, get the ingress address using kubectl.

kubectl get ingress -n default

Copy the ingress address from the output and create a CloudFront distribution using AWS CloudFront Console. To do this, paste the copied address into the Origin domain section of the CloudFront distribution creation section, as illustrated below: