Azure Arc Enabled Kubernetes Part 3: Deploying Containers

In the third and final part of this series, we’ll explore how to deploy containers onto our Azure Arc-enabled Kubernetes cluster.

Prerequisites

For this tutorial, you need the following:

• An active Azure subscription
• An Azure Arc-enabled Kubernetes cluster deployed to an Azure Arc subscription (as described in the first article of this series)
• An active GitHub account
• A code editor to modify the code and commit it to GitHub, like Visual Studio Code

Azure Setup

In the previous articles, we set up our Kubernetes cluster. We added security and monitoring. Now we are ready to deploy our application to our cluster. Now we need a service principal that we can configure in GitHub and an Azure Container Registry that communicates with our cluster and deploys the application from the container registry to the Kubernetes cluster.

To simplify the setup, we’ll use a few variables throughout this article. To get started, set the following variables:

SERVICE_PRINCIPLE="K8ARCSP"
SUBSCRIPTION_ID="<Your Subscription ID>"
RESOURCE_GROUP="AzureARCRG"
ACR_NAME="arckubeacr"
CLUSTER_NAME="do-ams3-azure-arc"

Service Principle

Now, we need to create a new service principal that we use to authenticate to Azure using the following code:

az ad sp create-for-rbac --name $SERVICE_PRINCIPLE --role contributor --scopes /subscriptions/$SUBSCRIPTION_ID --sdk-auth

Be sure to save the details in the output to use later.

Azure Container Registry

To create our Azure Container Registry, we run the following command on our Azure CLI or our local terminal:

az acr create --resource-group $RESOURCE_GROUP --name $ACR_NAME --sku Basic

This command registers a new Azure Container Registry under the same resource group as our Kubernetes cluster. We use the basic SKU as it is cost-optimized for development purposes.

Verify Azure Arc Services

Before we set up the deployment, we must ensure that our Azure Arc services are running by running the following command in our terminal:

kubectl get pods -n azure-arc

Enable Cluster Connect

To deploy to our Kubernetes cluster, we need to enable the cluster to connect so that it accepts external connections using the service principle we created earlier. To enable the Cluster-connect feature, run this command:

az connectedk8s enable-features --features cluster-connect -n $CLUSTER_NAME -g $RESOURCE_GROUP

We need to wait for the operation to complete, as it will take a while. You'll receive a confirmation once the cluster-connect feature is enabled.

Grant Permissions to the Service Principle

Next, we need to grant permission to the service principal that we created earlier so that we can deploy it to our Kubernetes cluster. To do so, we need the user ID, which we can retrieve using the ClientId from our service principle.

Let's assign our client ID to a variable, CLIENT_ID, and then use that in our query to retrieve the USER_ID:

CLIENT_ID=<YourClientId>
USER_ID=$(az ad sp show --id $CLIENT_ID --query objectId -o tsv)

Be sure to check the value of the saved user ID using echo $USER_ID.

Then, create the role binding with the following command:

kubectl create clusterrolebinding admin-user-binding --clusterrole cluster-admin --user=$_USERID

The Web Application

The creation of the container is out of scope for this article, so for this demonstration, we’ll use an existing GitHub repo, gh-arc-app-deploy, which you can fork and clone to your local environment. This repo contains a voting app developed in Python with deployment manifests for a back-end and front-end, and a GitHub action workflow file that we edit with our details.

Setup Variables

In the repo, under the location .github\workflows\github-action-arc-app-deploy.yaml, we assign the following variables:

env:
  AZURE_CONTAINER_REGISTRY: "<ACR_NAME>"
  CONTAINER_NAME: "azure-voting-app"
  RESOURCE_GROUP: "<RESOURCE_GROUP>"
  CLUSTER_NAME: "<CLUSTER_NAME>"
  IMAGE_PULL_SECRET_NAME: "image-pull-secret-name"

Be sure to use the same value in the variables ACR_NAME, RESOURCE_GROUP, and CLUSTER_NAME.

We need to update the /manifest/azure-vote-frontend-deployment.yaml file with the Azure Container Registry name, which has the same value as the ACR_NAME variable:

spec:
      containers:
        - name: azure-vote-front
          image: <ACR_NAME>.azurecr.io/azure-voting-app
          ports:
            - containerPort: 80
          env:
            - name: REDIS
              value: "azure-vote-back"

The code is ready for deployment!

With your application, you add the workflow when you add GitHub actions to your code. The steps of the workflow remain the same, but the manifests will differ based on your requirements.

Create a GitHub Actions Pipeline

We can now push our code to our repository. If you’re working with your own code, you can push your code to GitHub. We won’t cover that in this article, but you can find instructions on how to get your code committed.

After we push our code, we navigate to our GitHub repository.

GitHub Secrets

We use GitHub Secrets to Authenticate Azure from GitHub using the service principal that we bound to our Azure Container Registry.

Let's add the details of the service principle as a GitHub Secret. To retrieve this from your repository, select Settings. Under options, then select Secrets to open the Actions secrets page.

On the Actions secrets page, select New repository secret.

On the New secret page, we must specify a name and a value. We’ll call our secret AZURE_CREDENTIALS, and for the value, we provide a JSON extract in the format of the service principle that we created earlier:

{
    "clientId": "<GUID>",
    "clientSecret": "<GUID>",
    "subscriptionId": "<GUID>",
    "tenantId": "<GUID>",
    (...)
  }

Copy the service principle into the Value field and select Add secret.

Our newly created secrets are under Action secrets.

Setup Workflow

From here, we can navigate to Actions. If you cloned the demo repo, you can skip this step.

When using your code, you create a new workflow and use the same steps as with the YAML file.

Be sure to select set up a workflow yourself.

Give your workflow a name and replace the boilerplate code with the deployments YAML. Be sure to edit the section according to your requirements.

# This workflow will build and push an application to a Azure Arc-enabled Kubernetes (Connected Cluster) Service cluster when you push your code
#
# This workflow assumes you have already created the target Connected cluster and have created an Azure Container Registry (ACR)
# For instructions see:
#   - https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal
#   - https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-portal
#   - https://github.com/Azure/aks-create-action
#
# To configure this workflow:
#
# 1. Set the following secrets in your repository
#    - AZURE_CREDENTIALS
#    
#
# 2. Set the following environment variables (or replace the values below):
#    - AZURE_CONTAINER_REGISTRY (name of your container registry / ACR)
#    - RESOURCE_GROUP (where your Arc-enabled cluster is deployed)
#    - CLUSTER_NAME (name of your Arc-enabled cluster)
#    - CONTAINER_NAME (name of the container image you would like to push up to your ACR)
#    - SECRET_NAME (name of the secret associated with pulling your ACR image)
#    - DEPLOYMENT_MANIFEST_PATH (path to the manifest yaml for your deployment)
#
# For more information on GitHub Actions for Azure, refer to https://github.com/Azure/Actions
# For more samples to get started with GitHub Action workflows to deploy to Azure, refer to https://github.com/Azure/actions-workflow-samples
# For more options with the actions used below please refer to https://github.com/Azure/login
 
name: Deploy to Connected Cluster
 
on:
  workflow_dispatch:
 
env:
  AZURE_CONTAINER_REGISTRY: "arckubeacr"
  CONTAINER_NAME: "azure-voting-app"
  RESOURCE_GROUP: "AzureARCRG"
  CLUSTER_NAME: "do-ams3-azure-arc"
  IMAGE_PULL_SECRET_NAME: "image-pull-secret-name"
 
jobs:
  build:
    permissions:
      actions: read
      contents: read
      id-token: write
 
    runs-on: ubuntu-latest
    steps:
      # Checks out the repository this file is in
      - uses: actions/checkout@master
 
      # Logs in with your Azure credentials
      - name: Azure login
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
 
      # Builds and pushes an image up to your Azure Container Registry
      - name: Build and push image to ACR
        run: |
          az acr build --image ${{ env.CONTAINER_NAME }}:${{ github.sha }} --registry ${{ env.AZURE_CONTAINER_REGISTRY }} -g ${{ env.RESOURCE_GROUP }} ./azure-vote
 
      # Retrieves your Azure Kubernetes Service cluster's kubeconfig file
      - name: Get K8s context
        uses: azure/k8s-set-context@v2
        with:
          method: service-principal
          cluster-type: arc
          cluster-name: ${{ env.CLUSTER_NAME }}
          resource-group: ${{ env.RESOURCE_GROUP }}
         
 
      # Retrieves the credentials for pulling images from your Azure Container Registry
      - name: Get ACR credentials
        run: |
          az acr update -n ${{ env.AZURE_CONTAINER_REGISTRY }} -g ${{ env.RESOURCE_GROUP }} --admin-enabled true
          ACR_USERNAME=$(az acr credential show -g ${{ env.RESOURCE_GROUP }} -n ${{ env.AZURE_CONTAINER_REGISTRY }} --query username -o tsv)
          ACR_PASSWORD=$(az acr credential show -g ${{ env.RESOURCE_GROUP }} -n ${{ env.AZURE_CONTAINER_REGISTRY }} --query passwords[0].value -o tsv)
          echo "::add-mask::${ACR_USERNAME}"
          echo "::set-output name=username::${ACR_USERNAME}"
          echo "::add-mask::${ACR_PASSWORD}"
          echo "::set-output name=password::${ACR_PASSWORD}"
        id: get-acr-creds
 
      # Creates a kubernetes secret on your Azure Kubernetes Service cluster that matches up to the credentials from the last step
      - name: Create K8s secret for pulling image from ACR
        uses: Azure/k8s-create-secret@v1.1
        with:
          container-registry-url: ${{ env.AZURE_CONTAINER_REGISTRY }}.azurecr.io
          container-registry-username: ${{ steps.get-acr-creds.outputs.username }}
          container-registry-password: ${{ steps.get-acr-creds.outputs.password }}
          secret-name: ${{ env.IMAGE_PULL_SECRET_NAME }}

      # Deploys application based on given manifest file
      - name: Deploys application
        uses: Azure/k8s-deploy@v3.0
        with:
          action: deploy
          manifests: |
            manifests/azure-vote-backend-deployment.yaml
            manifests/azure-vote-backend-service.yaml
            manifests/azure-vote-frontend-deployment.yaml
            manifests/azure-vote-frontend-service.yaml
          images: |
            ${{ env.AZURE_CONTAINER_REGISTRY }}.azurecr.io/${{ env.CONTAINER_NAME }}:${{ github.sha }}
          imagepullsecrets: |
            ${{ env.IMAGE_PULL_SECRET_NAME }}

Select Start commit, provide your comments, and select Commit new file.

Run Workflow

Once the workflow is committed, we can manually run the workflow by going to Actions, selecting the workflow that we created in the previous step, then selecting Run workflow.

This displays the branch on which you want to run the workflow. For this demonstration, we run it on the primary branch.

Once it starts to run, we’ll see the status as in progress. We can select the running workflow to see the details.

This brings up the build details, which we can select to see the progress step by step.

We can expand the steps further to see exactly what’s executing.

The workflow takes a while to run through all the steps, so be patient and don’t be alarmed if it takes some time. Should it fail, you’ll be given the exact error message in detail so you can proceed to troubleshoot the problem.

Note that in this example, the Kubernetes cluster doesn’t have enough resources to create a replica of the application. Be sure to check your Kubernetes dashboard for more details about the deployment. In the event of this kind of error, you’ll see something like the above image.

What we want to end up with is the green success notification about our workflow, like the one below:

Run and Verify within Azure Arc Kubernetes

After we get a successful deployment, we can verify the pods and services from our terminal with the following commands:

kubectl get po

kubectltl get svc

The second command gives us the external IP and port to use for the application. Depending on your Kubernetes cluster, you might need to forward your ports or create a tunnel to access the application.

Now, we simply need to type the external IP and port in our browser to access the app.

If you followed the steps correctly, you can access the application and interact with the application to test the functionality.

Conclusion

Throughout this series, we’ve taken a hands-on look at how to set up and manage non-Azure-hosted Kubernetes clusters via Azure Arc. In this article, we have covered how to create a GitHub Actions pipeline that can deploy the image onto our existing Arc-enabled Kubernetes cluster, how to run the pipeline, and then ensure that the container image we chose is up and running in our cluster as expected. This marks the final article in this three-part series!

For more information on how to deploy to an Azure Arc-enabled Kubernetes cluster, be sure to check out the article Deploy apps to Azure-Arc enabled Kubernetes cluster using Cluster Connect and GitHub Actions.

To learn more about how to get started with Azure Arc-enabled Kubernetes or how Azure Arc-enabled Kubernetes allows you to attach and configure Kubernetes clusters running anywhere, check out our resource page on Azure Arc Enabled Kubernetes.