Migrating VMs to Azure DevTest Labs

So here’s something I’ve been running into a lot, and at the heart of it is cost management. No one wants to spend more money than they need to and that is especially true if you are running your solutions in the cloud. And for most organizations, a big cost factor can be running lower environments.

So the situation is this, you have a virtual machine, be it a developer machine, or some form of lower environment (Dev, test, QA, etc.). And the simple fact is that you have to pay for this environment to be able to run testing and ensure that everything works when you deploy to production. So it's a necessary evil, and extremely important if you are adhering to DevOps principals. Now the unfortunate part of this, is that you likely to only really use these lower environments during work hours. They probably aren’t exposed to the outside world, and they probably aren’t being hit by customers in any fashion.

So ultimately, that means that you are paying for this environment for 24/7 but only using it for 12 hours a day. Which means you are basically paying for double of what you need.

Enter DevTest Labs, which allows you to create VMs and artifacts to make it easy to spin-up and spin-down environments without any effort on your part so that you can save money with regard to running these lower environments.

Now the biggest problem then becomes, “That’s great, but how do I move my existing VMs into new DevTest lab, how do I do that?” The answer is it can be done, via script, and I’ve written a script to that here.

#The resource group information, the source and destination
resourceGroupName="<Original Resource Group>"
newResourceGroupName="<Destination Resource Group>"

#The name of the VM you wish to migrate
vmName="<VM Name>"
newVmName="<New VM Name>"

#The OS image of the VM
imageName="Windows Server 2016 Datacenter"
osType="Windows"

#The size of the VM
vmsize="<vm size>"
#The location of the VM
location="<location>"
#The admin username for the newly created vm
adminusername="<username>"

#The suffix to add to the end of the VM
osDiskSuffix="_lab.vhd"
#The type of storage for the data disks
storageType="Premium_LRS"

#The VNET information for the VM that is being migrated
vnet="<vnet name>"
subnet="<Subnet name>"

#The name of the lab to be migrated to
labName="<Lab Name>"

#The information about the storage account associated with the lab
newstorageAccountName="<Lab account name>"
storageAccountKey="<Storage account key>"
diskcontainer="uploads"

echo "Set to Government Cloud"
sudo az cloud set --name AzureUSGovernment

#echo "Login to Azure"
#sudo az login

echo "Get updated access token"
sudo az account get-access-token

echo "Create new Resource Group"
az group create -l $location -n $newResourceGroupName

echo "Deallocate current machine"
az vm deallocate --resource-group $resourceGroupName --name $vmName

echo "Create container"
az storage container create -n $diskcontainer 
--account-name $newstorageAccountName --account-key $storageAccountKey

osDisks=$(az vm show -d -g $resourceGroupName -n $vmName 
--query "storageProfile.osDisk.name") 
echo ""
echo "Copy OS Disks"
echo "--------------"
echo "Get OS Disk List"

osDisks=$(echo "$osDisks" | tr -d '"')

for osDisk in $(echo $osDisks | tr "[" "\n" | 
tr "," "\n" | tr "]" "\n" )
do
   echo "Copying OS Disk $osDisk"

   echo "Get url with token"
   sas=$(az disk grant-access --resource-group $resourceGroupName 
   --name $osDisk --duration-in-seconds 3600 --query [accessSas] -o tsv)

   newOsDisk="$osDisk$osDiskSuffix"
   echo "New OS Disk Name = $newOsDisk"

   echo "Start copying $newOsDisk disk to blob storage"
   az storage blob copy start --destination-blob $newOsDisk 
   --destination-container $diskcontainer --account-name $newstorageAccountName 
   --account-key $storageAccountKey --source-uri $sas

   echo "Get $newOsDisk copy status"
   while [ "$status"=="pending" ]
   do
      status=$(az storage blob show --container-name $diskcontainer 
      --name $newOsDisk --account-name $newstorageAccountName --account-key $storageAccountKey 
      --output json | jq '.properties.copy.status')
      status=$(echo "$status" | tr -d '"')
      echo "$newOsDisk Disk - Current Status = $status"

      progress=$(az storage blob show --container-name $diskcontainer 
      --name $newOsDisk --account-name $newstorageAccountName --account-key $storageAccountKey 
      --output json | jq '.properties.copy.progress')
      echo "$newOsDisk Disk - Current Progress = $progress"
      sleep 10s
      echo ""

      if [ "$status" != "pending" ]; then
      echo "$newOsDisk Disk Copy Complete"
      break
      fi
   done

   echo "Get blob url"
   blobSas=$(az storage blob generate-sas --account-name $newstorageAccountName 
   --account-key $storageAccountKey -c $diskcontainer -n $newOsDisk 
   --permissions r --expiry "2019-02-26" --https-only)
   blobSas=$(echo "$blobSas" | tr -d '"')
   blobUri=$(az storage blob url -c $diskcontainer -n $newOsDisk 
   --account-name $newstorageAccountName --account-key $storageAccountKey)
   blobUri=$(echo "$blobUri" | tr -d '"')

   echo $blobUri

   blobUrl=$(echo "$blobUri")

   echo "Create image from $newOsDisk vhd in blob storage"
   az group deployment create --name "LabMigrationv1" 
   --resource-group $newResourceGroupName --template-file customImage.json 
   --parameters existingVhdUri=$blobUrl --verbose

   echo "Create Lab VM - $newVmName"
   az lab vm create --lab-name $labName -g $newResourceGroupName 
   --name $newVmName --image "$imageName" --image-type gallery 
   --size $vmsize --admin-username $adminusername --vnet-name $vnet --subnet $subnet
done 

dataDisks=$(az vm show -d -g $resourceGroupName -n $vmName 
--query "storageProfile.dataDisks[].name") 
echo ""
echo "Copy Data Disks"
echo "--------------"
echo "Get Data Disk List"

dataDisks=$(echo "$dataDisks" | tr -d '"')

for dataDisk in $(echo $dataDisks | tr "[" "\n" | 
tr "," "\n" | tr "]" "\n" )
do
   echo "Copying Data Disk $dataDisk"

   echo "Get url with token"
   sas=$(az disk grant-access --resource-group $resourceGroupName 
   --name $dataDisk --duration-in-seconds 3600 --query [accessSas] -o tsv)

   newDataDisk="$dataDisk$osDiskSuffix"
   echo "New OS Disk Name = $newDataDisk"

   echo "Start copying disk to blob storage"
   az storage blob copy start --destination-blob $newDataDisk 
   --destination-container $diskcontainer --account-name $newstorageAccountName 
   --account-key $storageAccountKey --source-uri $sas

   echo "Get copy status"
   while [ "$status"=="pending" ]
   do
      status=$(az storage blob show --container-name $diskcontainer 
      --name $newDataDisk --account-name $newstorageAccountName 
      --account-key $storageAccountKey --output json | jq '.properties.copy.status')
      status=$(echo "$status" | tr -d '"')
      echo "Current Status = $status"

      progress=$(az storage blob show --container-name $diskcontainer 
      --name $newDataDisk --account-name $newstorageAccountName 
      --account-key $storageAccountKey --output json | jq '.properties.copy.progress')
      echo "Current Progress = $progress"
      sleep 10s
      echo ""

      if [ "$status" != "pending" ]; then
      echo "Disk Copy Complete"
      break
      fi
   done
done 

echo "Script Completed"

So the above script breaks out into a couple of key pieces. The first part is the following parts needs to happen:

• Create the destination resource group
• Deallocate the machine
• Create a container for the disks to be migrated to

echo "Create new Resource Group"
az group create -l $location -n $newResourceGroupName

echo "Deallocate current machine"
az vm deallocate --resource-group $resourceGroupName --name $vmName

echo "Create container"
az storage container create -n $diskcontainer 
   --account-name $newstorageAccountName --account-key $storageAccountKey

The next process is to identify the OS disk for the VM, and copy the disk from its current location, over to the storage account associated with DevTest lab. The next key part is to create a custom image based on that VM and then create a VM based on that image.

osDisks=$(az vm show -d -g $resourceGroupName -n $vmName --query "storageProfile.osDisk.name") 
echo ""
echo "Copy OS Disks"
echo "--------------"
echo "Get OS Disk List"

osDisks=$(echo "$osDisks" | tr -d '"')

for osDisk in $(echo $osDisks | tr "[" "\n" | 
tr "," "\n" | tr "]" "\n" )
do
   echo "Copying OS Disk $osDisk"

   echo "Get url with token"
   sas=$(az disk grant-access --resource-group $resourceGroupName 
   --name $osDisk --duration-in-seconds 3600 --query [accessSas] -o tsv)

   newOsDisk="$osDisk$osDiskSuffix"
   echo "New OS Disk Name = $newOsDisk"

   echo "Start copying $newOsDisk disk to blob storage"
   az storage blob copy start --destination-blob $newOsDisk 
   --destination-container $diskcontainer --account-name $newstorageAccountName 
   --account-key $storageAccountKey --source-uri $sas

   echo "Get $newOsDisk copy status"
   while [ "$status"=="pending" ]
   do
      status=$(az storage blob show --container-name $diskcontainer 
      --name $newOsDisk --account-name $newstorageAccountName 
      --account-key $storageAccountKey --output json | jq '.properties.copy.status')
      status=$(echo "$status" | tr -d '"')
      echo "$newOsDisk Disk - Current Status = $status"

      progress=$(az storage blob show --container-name $diskcontainer 
      --name $newOsDisk --account-name $newstorageAccountName 
      --account-key $storageAccountKey --output json | jq '.properties.copy.progress')
      echo "$newOsDisk Disk - Current Progress = $progress"
      sleep 10s
      echo ""

      if [ "$status" != "pending" ]; then
      echo "$newOsDisk Disk Copy Complete"
      break
      fi
   done

   echo "Get blob url"
   blobSas=$(az storage blob generate-sas --account-name $newstorageAccountName 
   --account-key $storageAccountKey -c $diskcontainer -n $newOsDisk 
   --permissions r --expiry "2019-02-26" --https-only)
   blobSas=$(echo "$blobSas" | tr -d '"')
   blobUri=$(az storage blob url -c $diskcontainer -n $newOsDisk 
   --account-name $newstorageAccountName --account-key $storageAccountKey)
   blobUri=$(echo "$blobUri" | tr -d '"')

   echo $blobUri

   blobUrl=$(echo "$blobUri")

   echo "Create image from $newOsDisk vhd in blob storage"
   az group deployment create --name "LabMigrationv1" 
   --resource-group $newResourceGroupName --template-file customImage.json 
   --parameters existingVhdUri=$blobUrl --verbose

   echo "Create Lab VM - $newVmName"
   az lab vm create --lab-name $labName -g $newResourceGroupName 
   --name $newVmName --image "$imageName" --image-type gallery 
   --size $vmsize --admin-username $adminusername --vnet-name $vnet --subnet $subnet
done 

Now part of the above is to use a ARM template to create the image. The template is below:

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "existingLabName": {
      "type": "string",
      "defaultValue":"KevinLab",
      "metadata": {
        "description": "Name of an existing lab where the custom image will be created or updated."
      }
    },
    "existingVhdUri": {
      "type": "string",
      "metadata": {
        "description": "URI of an existing VHD from which the custom image will be created or updated."
      }
    },
    "imageOsType": {
      "type": "string",
      "defaultValue": "windows",
      "metadata": {
        "description": "Specifies the OS type of the VHD. 
        Currently 'Windows' and 'Linux' are the only supported values."
      }
    },
    "isVhdSysPrepped": {
      "type": "bool",
      "defaultValue": true,
      "metadata": {
        "description": "If the existing VHD is a Windows VHD, 
        then specifies whether the VHD is sysprepped (Note: If the existing VHD is NOT a Windows VHD, 
        then please specify 'false')."
      }
    },
    "imageName": {
      "type": "string",
      "defaultValue":"LabVMMigration",
      "metadata": {
        "description": "Name of the custom image being created or updated."
      }
    },
    "imageDescription": {
      "type": "string",
      "defaultValue": "",
      "metadata": {
        "description": "Details about the custom image being created or updated."
      }
    }
  },
  "variables": {
    "resourceName": "[concat(parameters('existingLabName'), 
    '/', parameters('imageName'))]",
    "resourceType": "Microsoft.DevTestLab/labs/customimages"
  },
  "resources": [
    {
      "apiVersion": "2018-10-15-preview",
      "name": "[variables('resourceName')]",
      "type": "Microsoft.DevTestLab/labs/customimages",
      "properties": {
        "vhd": {
          "imageName": "[parameters('existingVhdUri')]",
          "sysPrep": "[parameters('isVhdSysPrepped')]",
          "osType": "[parameters('imageOsType')]"
        },
        "description": "[parameters('imageDescription')]"
      }
    }
  ],
  "outputs": {
    "customImageId": {
      "type": "string",
      "value": "[resourceId(variables('resourceType'), 
      parameters('existingLabName'), parameters('imageName'))]"
    }
  }
}

So if you run the above, it will create the new VM under the DevTest lab.