Who Is This For?
This is an advanced topic for software developers who are looking to migrate their existing x86 containerized applications to Arm
What Will You Learn?
Upon completion of this learning path, you will be able to:
- Add Arm-based nodes to an existing x86-based GKE cluster
- Rebuild an x86-based application to make it multi-arch and run on Arm
- Learn how to add taints and tolerations to GKE clusters to schedule application pods on architecture specific nodes
- Run a multi-arch application across multiple architectures on a single GKE cluster
Prerequisites
Before starting, you will need the following:
Migrate an Existing x86-Based Application to Run on Arm-Based Nodes in a Single GKE Cluster
Google Kubernetes Engine (GKE) supports hybrid clusters with x86 and Arm based nodes. The Arm-based nodes can be deployed on the Tau T2A family of virtual machines. The Tau T2A virtual machines are powered by Ampere Altra Arm-based processors.
Before You Begin
On your local machine, install the following tools.
You will need a Google Cloud account. Create an account if needed.
Three tools are required on your local machine. Follow the links to install the required tools.
Create a Docker Repository in Google Artifact Registry
This section assumes that you have a GKE cluster with 3 x86-based nodes running in your environment. If you do not have the cluster, then follow the official google docs to create one.
Setup the following environment variables:
export PROJECT_ID=<your-project-id>
export ZONE=<zone id - us-central1-c>
export CLUSTER_NAME=<your-cluster-name>
The github project repository listed below contains all the required files to follow this learning path. Clone it to your local machine:
git clone https://github.com/pbk8s/gke-arm
Create a docker repository in the Google Artifact Registry with the following command:
gcloud artifacts repositories create docker-repo \
--repository-format=docker \
--location=<your-region> \
--description="Docker repository for multi-arch images"
Replace <your-region> in the command above with the location where you want to create the repository storage.
Configure the cli to authenticate the docker repository in the Artifact Registry:
gcloud auth configure-docker us-central1-docker.pkg.dev
Build the docker image for the existing x86-based version of application:
docker build -t us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/x86-hello:v0.0.1 .
Push the docker image you created to the docker repository:
docker push us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/x86-hello:v0.0.1
Connect to Your Existing GKE Cluster (with x86-based nodes) and Deploy the Application
Retrieve the GKE cluster credentials:
gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE --project $PROJECT_ID
Update the docker image with a tool called Kustomize. This tool allows you to customize kubernetes objects.
$(cd k8s/overlays/x86 && kustomize edit set image hello=us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/x86-hello:v0.0.1)
kubectl apply -k k8s/overlays/x86
To access the application from outside your cluster, deploy the following kubernetes service:
kubectl apply -f k8s/hello-service.yaml
After an external IP is assigned to this service, open a browser and access the webpage as shown:
http://$external_ip
Alternatively, you can also use the curl command to access the webpage:
curl -w '\n' http://$external_ip
You should see output similar to what is shown below:
Hello from NODE:gke-multi-arch-cluster-default-pool-45537239-q83v,
POD:x86-hello-deployment-9e7b823ed8-xutvf, CPU PLATFORM:linux/amd64
Add Arm-based Nodes to your GKE Cluster
Use the following command to add an Arm-based node pool with VM type t2a-standard-2 to your GKE cluster:
gcloud container node-pools create arm-pool \
--cluster $CLUSTER_NAME \
--zone $ZONE \
--machine-type=t2a-standard-2 \
--num-nodes=3
After the Arm-nodes are successfully added to the cluster, run the following command to check if both types of nodes show up in the cluster:
kubectl get nodes -o wide
The output should show both x86 and Arm-based nodes.
You have now successfully setup a hybrid cluster with both x86 and Arm64 architecture nodes.
Taints and Tolerations
In a hybrid cluster setup with nodes from different architectures (x86 and Arm64), GKE adds a taint on the nodes to avoid the possibility of scheduling pods on wrong architecture. A node taint lets the kubernetes scheduler know that a particular node is designated for one architecture only. A toleration lets you designate pods that can be used on tainted nodes.
In the github repository, view the following yaml file:
cat k8s/overlays/arm/add_arm_support.yaml
In this file, refer to the section shown below:
nodeSelector:
kubernetes.io/arch: arm64
This field specifies that the application should only run on Arm-based nodes. After applying this file, GKE adds a toleration that matches the taint on Arm nodes, so that it can schedule the Arm-based application pods on these nodes.
Modify Application to Run on Arm-Based Nodes
The application used in this learning path is developed in Go language. Check the contents of following file:
cat Dockerfile_arm
In this file, the architecture flag is set to GOARCH=arm64. This way, the application built will be compatible with Arm. You can now use the following set of commands to build the docker image and push it to registry:
docker build -t us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/arm-hello:v0.0.1 -f Dockerfile_arm .
docker push us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/arm-hello:v0.0.1
Now, using taints and tolerations, deploy this application in the cluster:
$(cd k8s/overlays/arm && kustomize edit set image hello=us-central1-docker.pkg.dev/$PROJECT_ID/docker-repo/arm-hello:v0.0.1)
kubectl apply -k k8s/overlays/arm
After the application gets deployed, check the status of pods with the following command:
kubectl get pods
Open a web browser and hit the external IP URL or use curl command as shown below:
curl -w '\n' http://$external_ip
Refresh the browser a couple of times and you should see the output from both x86 and Arm compatible versions of the application.
The output will be similar to what is shown below:
Hello from NODE:gke-multi-arch-cluster-default-pool-45537239-q83v, POD:x86-hello-deployment-9e7b823ed8-xutvf, CPU PLATFORM:linux/amd64
Hello from NODE:gke-multi-arch-cluster-arm-pool-n381qvv-bqcr, POD:arm-hello-deployment-21b8d2exfc-o8q33, CPU PLATFORM:linux/arm64
You have now migrated your existing x86-based application to run on an Arm-based GKE cluster and made it multi-arch.
Want to continue learning? Visit learn.arm.com to find more tutorials designed to help you develop quality Arm software faster.
