I have a question. How to deploy an image connected to Azure Arc unattended. Everything should be unattended. If something needs attention, it disturbs you from playing games on Xbox. And as you can expect, because we are working with Microsoft technology, it is a Linux image. So, no surprise here that it is an Ubuntu. It is still the most loved one by Microsoft, even over their breed (Mariner and Flatcar).

For some reason, the choice of the Hypervisor was vSpehere (ESXi), but as a poor man, choice without the comfort of the vCenter.

The workflow is simple - through the management network from our management workstation, we deploy an image with NIC pointed to the port group connected to the internet. So after the virtual machine boots, it will board into Azure Arc.

Architecture drawing

For that procedure, we utilize cloud-init. So we deploy the Virtual Machine with the additional metadata. When it boots, it reads metadata configures itself and adds local users and boards to Azure Arc. For cloud init deployment, we need to have:

  • cloud platform must support cloud-init
  • The Linux distribution must support cloud-init

The first requirement can be surpassed by creating an iso file - we do this, for example, in vagrant boxes for Virtual Box or Hyper-V. VMware supports cloud-init as a property of VM since 7.03. Our goal is to deploy Azure Arc without any user interaction, therefore we need to have a security principal with a valid secret.

The procedure:

  1. Download the cloud image from a known location (as vmdk)
  2. Execute PowerShell container and install PowerCLI (Optional)
  3. Get Azure Subscription and service principal properties (Optional)
  4. Prepare cloud-init configuration as a string with some variables evaluated
  5. Transform cloud init multi-line string to Base64 encoded string
  6. Copy downloaded vmdk to ESXi
  7. Create the virtual machine on the ESXi
  8. Add Base64 encoded string to the VM and start the VM
  9. Check the results

1. Download the image

Ova is useful with vCenter, for ESXi vmdk is enough. The image size is around 600MB.

# Temp folder on local computer
$tempPath = "C:\temp\vmImage"
$ubntCode = "focal"
# $ovaUrl = "http://cloud-images.ubuntu.com/$ubntCode/current/$ubntCode-server-cloudimg-amd64.ova"
$vmdkUrl = "http://cloud-images.ubuntu.com/$ubntCode/current/$ubntCode-server-cloudimg-amd64.vmdk"

[System.IO.Directory]::CreateDirectory($tempPath)
#Invoke-Webrequest -URI $ovaUrl -OutFile "$tempPath\ubuntu-$ubntCode.ova"
Invoke-Webrequest -URI $vmdkUrl -OutFile "$tempPath\ubuntu-$ubntCode.vmdk"

2. Container for PowerCLI

Use this step to isolate your local PowerShell from VMware stuff. Especially if you have Hyper-V installed, there is a name clash between commandlets. I did not resolve the issue with a prefix, so the easiest way was to use a purpose build container.

docker run -it -v ${tempPath}:/img mcr.microsoft.com/powershell

Inside the docker container run the next command

Install-Module vmWare.PowerCLI

3. Azure Arc

You need to have a service principal and service principal secret. You can use UI or the command line. See Microsoft docs.

Necessary variables:

  • $servicePrincipalClientId = “<YOUR SERVICE PRINCIPAL ID>”
  • $servicePrincipalSecret = “<SERVICE PRINCIPAL SECRET>”
  • $subscriptionId = “<SUBSCRIPTION ID FOR ARC RESOURCE>”
  • $resourceGroup = “<RESOURCE GROUP NAME>”
  • $tenantId = “<YOUR TENANT ID>”
  • $location = “<DESIRED LOCATION>”

4. Prepare cloud-init

Now we need to prepare cloud-init. In general, there are multiple cloud-inits. We need to use user_config. All the packages and users are configured here. The cloud config file example I will add later to this post.

Everything execute inside the container.

We set our environment

#Ignore SSL cert errors
Set-PowerCLIConfiguration -InvalidCertificateAction Ignore

#Ubuntu version
$ubntCode = "focal"

Section to configure our connection to ESXi.

  • Credentials are credentials for a user with enough rights to copy to the datastore and create VMs.
  • IP address is management IP see drawing
  • OVA/VMDK file is path to the mounted volume inside the docker image (see -v in docker command) otherwise it is the path to the respective files
  • Virtual Machine name, local datastore name and portgroup with access to the internet are other variables.
#ESXi credentials and info
$cred = Get-Credential
$esxiIP = "192.168.100.40"
$ovaFile = "/img/ubuntu-${ubntCode}.ova"
$vmdkFile = "/img/ubuntu-${ubntCode}.vmdk"
$vmName = "vmwubntarc01"
$dsName = "datastore1"
$pgName = "sLAN"

Now we need to configure our credentials and variables to board the VM to Azure Arc

# Azure environment for Arc boarding
$servicePrincipalClientId = "<YOUR SERVICE PRINCIPAL ID>"
$servicePrincipalSecret = "<SERVICE PRINCIPAL SECRET>"
$subscriptionId = "<SUBSCRIPTION ID FOR ARC RESOURCE>"
$resourceGroup = "<RESOURCE GROUP NAME>"
$tenantId = "<YOUR TENANT ID>"
$location = "westeurope"
$authType = "principal"
$cloud = "AzureCloud"

And finally, we are ready to use all defined variables and use PowerShell expressions to create the content of user_config.

In the user config, we need to define the user, to be able to log in. The most difficult part inside Windows OS is to create SHA-512 password. The easiest way is to use OpenSSL. And if we are using the container, OpenSSL is already present.

So let’s create the password for our azuser.

openssl passwd -6

Copy the result into the following script. Escape all dollar signs with a reverse apostrophe `. We use the literal sign (@), but also double quotes " so variables will be interpreted in the string in PowerShell.

In user config we:

  • Add the azuser with the password created in the previous step
  • Change the hostname to vmName
  • Add Microsoft’s repos
  • Install Arc agent
  • Connect to azure
# Genereate cloud configs
# Sample password
# OldPickle963*
# `$6`$Z.TpQG4SDJAqFu6T`$pBGTWd.fEseoEG95xeNY4nyhJz1GeRa43JHxTVu4lvdkqTcN8vtaDduGmYysHWTsHh0eLFpgdTnJuFsap3pnx.
$user_config=@"
#cloud-config

users:
  - name: azuser
#    passwd: #### PASTE PASSWORD HERE ESCAPE `$ SIGN #####
    passwd: `$6`$Z.TpQG4SDJAqFu6T`$pBGTWd.fEseoEG95xeNY4nyhJz1GeRa43JHxTVu4lvdkqTcN8vtaDduGmYysHWTsHh0eLFpgdTnJuFsap3pnx.
    groups: sudo
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    lock_passwd: false

preserve_hostname: false
hostname: ${vmName}

runcmd:
  - apt install -y azcmagent
  - azcmagent connect --service-principal-id "$servicePrincipalClientId" --service-principal-secret "$servicePrincipalSecret" --resource-group "$resourceGroup" --tenant-id "$tenantId" --location "$location" --subscription-id "$subscriptionId" --cloud "$cloud" --correlation-id "$correlationId";

apt:
  preserve_sources_list: true
  sources:
    msft.list:
      source: "deb https://packages.microsoft.com/ubuntu/20.04/prod focal main"
      key: |
        -----BEGIN PGP PUBLIC KEY BLOCK-----
        Version: GnuPG v1.4.7 (GNU/Linux)

        mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT
        LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV
        7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag
        OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j
        H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr
        M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs
        ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC
        AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH
        /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe
        MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy
        7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV
        KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ
        XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+
        NdCFTW7wY0Fb1fWJ+/KTsC4=
        =J6gs
        -----END PGP PUBLIC KEY BLOCK----- 
"@

5. Transform to Base64 encoded string

And it is time to create Base64 encoded string.

$userDataBase64 = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($user_config))

6. Copy vmdk to datastore

Now we connect to ESXi server, we get the datastore object according to the datastore name. Then we can create a new PS drive and create a folder for the vmdk (or ova) file. At the end, we copy vmdk file (but we need to use the specialized commandlet because of type conversion).

Connect-VIServer -Server $esxiIP -Credential $cred -Protocol https
$ds = Get-Datastore -Name $dsName
New-PSDrive -Location $ds -Name ds -PSProvider VimDatastore -Root "\"
New-Item -Name "vmdkfiles" -ItemType Directory
Copy-DataStoreItem -Item $vmdkFile "DS:\vmdkfiles\ubuntu-${ubntCode}.vmdk"

7. Create the virtual machine on the ESXi


$dskToCopy = Get-HardDisk -Datastore $ds -DatastorePath "[${dsName}] vmdkfiles/ubuntu-${ubntCode}.vmdk"
$hddForVM = Copy-HardDisk -HardDisk $dskToCopy -DestinationPath "[${dsName}] ${vmName}/${vmName}.vmdk"

$vm = New-VM -Name $vmName -Datastore $dsName -MemoryGB 2 -NetworkName $pgName -VMHost $esxiIP -DiskPath $hddForVM.Filename

8. Add Base64 encoded string to the VM and start the VM

# Add userdata
$vm | New-AdvancedSetting -Name "guestinfo.userdata.encoding" -Value "base64" -Confirm:$false
$vm | New-AdvancedSetting -Name "guestinfo.userdata" -Value $userDataBase64 -Confirm:$false

$vm | Start-VM

9. Check the results

  1. Try to sign in through VM ware console. If it works, at least the user is here.

  2. Get advanced properties through VM tools

vmtoolsd --cmd "info-get guestinfo.userdata" |base64 --decode

Easies way how to test everything again is to kill the VM

$vm |Shutdown-VMGuest -Confirm:$false
Start-Sleep 20
$vm |Remove-VM -DeletePermanently -Confirm:$false