Save Money with Azure, intelligent shutdown of VMs

This blog post is based on 3 separate asks from 3 separate customers about adding value for their Azure subscriptions. They came to be and said “Marc, I would like to devise a scheme where at 7pm every night, all Dev/Test VMs can be shut down to save money and the business owners have the final say.” There’s one problem though, the business owners don’t have access to the Azure portal – it has to be done over email. What’s more, an option to force the shutdown of the VMs if the business owners don’t respond in a time frame of about 30 mins.

Enter Logic Apps & Azure Automation Runbooks:

This is what the entire Logic App looks like:

20181103 Logic Apps Shut down VMs.png

I’ll dive into each step bit by bit, starting with the top – prerequisites.

Prerequisites

Some prerequisites, you need to setup Azure Automation:

  • The Run As account
  • Two Runbooks (PowerShell) scripts.

An Azure Runbook is run using an Azure Automation runas account.

  • Think of a Run As account as a service account, it’s what the runbook utilises in order to gain privileges to the Azure subscriptions

20181101 RunAs account

Azure Automation Run As accounts can be created automatically by Azure Automation when you tell it to. All it is, is a Service Principal using a certificate for authentication and Azure Automation takes care of the entire logon process to Azure. This is all you need as the logon construct:

$ServicePrincipalConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'

$null = Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $ServicePrincipalConnection.TenantId `
-ApplicationId $ServicePrincipalConnection.ApplicationId `
-CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint

You can add this Run As account like a normal account and assign (Contributor/Owner) permission to it from other Azure subscriptions you have in your AAD tenant.

Get-All-VMs

This runbook (and below) gets all VMs across all Azure subscriptions in which the Run As account has access to. It loops through all Azure subscriptions the account has access to.

It grabs:

  • the VM name
  • both an email address and an environment tag (either Dev/Test)

… and converts all of this information to JSON.


$ServicePrincipalConnection = Get-AutomationConnection Name 'AzureRunAsConnection'
$null = Add-AzureRmAccount `
ServicePrincipal `
TenantId $ServicePrincipalConnection.TenantId `
ApplicationId $ServicePrincipalConnection.ApplicationId `
CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
##################################################################
##################################################################
##################################################################
##################################################################
##################################################################
$Subscriptions = Get-AzureRmSubscription
# Get a list of VMs which are powered on, has an email address tag and has an environment tag (Dev or Test)
$VMs = @()
foreach ($Sub in $Subscriptions) {
$null = Select-AzureRmSubscription Subscription $Sub
# Get VMs that are running and actually have a tags of some sort
$VMs += Get-AzureRmVM Status | where {$_.PowerState -eq 'VM Running' -and $_.Tags.values -match '.'}
$VMs_with_Tags = @()
foreach ($VM in $VMs) {
$a = [psCustomObject]@{
name = $VM.Name
CreatedBy = ($VM.Tags.Values | where {($_ -match '.@.') -and ($_ -match '.\..')}) # Find email address in a Tag
Environment = ($VM.Tags.Values | where {($_ -match 'dev') -or ($_ -match 'test')}) # Find environment, Dev/Test
}
$VMs_with_Tags += $a
}
}
# Group the VMs with tags by email address
$GroupedVMs_with_Tags = @()
($VMs_with_Tags | where {($_.CreatedBy) -or ($_.Environment)} | Group-Object CreatedBy) | % {
$a = [psCustomObject]@{
name = $_.Name
VMNames = $_.Group.Name -join ', '
Environment = $_.Group.Environment -join ', '
}
$GroupedVMs_with_Tags += $a
}
$GroupedVMs_with_Tags | ConvertTo-Json Depth 100

Stop-Azure-VMs-Parameter

This other Azure Automation runbook (and below) is kicked off (Stop-Azure-VMs-Parameter), although a parameter value is passed through to this runbook, the ‘VMNames‘ property is passed through to this runbook – more on this further below in the For each loop.


param
(
[parameter(Mandatory=$true)]
[String] $VMnames
)
$ServicePrincipalConnection = Get-AutomationConnection Name 'AzureRunAsConnection'
$null = Add-AzureRmAccount `
ServicePrincipal `
TenantId $ServicePrincipalConnection.TenantId `
ApplicationId $ServicePrincipalConnection.ApplicationId `
CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
$Subscriptions = Get-AzureRmSubscription
# Get a list of VMs which are powered on, has an email address tag and has an environment tag (Dev or Test)
foreach ($Sub in $Subscriptions) {
$null = Select-AzureRmSubscription Subscription $Sub
# Filter out the VMs for the current subscription
$SubVMs = @()
# Get all Azure VMs based ont eh current Azure subscription
$AzureVMs = Get-AzureRmVM
# Do a foreach against all the VMs provided in parameter – some aren't in the current subscription
foreach($a in ($VMnames -split ', ')){
# Check if the current VM in the parameter array exists in the current Azure subscription
if($a -in $AzureVMs.name){
$b = Get-AzureRmVM | ? {$_.Name -eq $a}
$c = [psCustomObject]@{
name = $b.Name
ResourceGroupName = $b.ResourceGroupName
}
$SubVMs += $c
}
}
Write-Output $SubVMs
ForEach ($VM in $SubVMs)
{
Stop-AzureRmVM Name $VM.Name ResourceGroupName $VM.ResourceGroupName Force
}
}

Tags

In Azure, you can have up to 15 tags per Azure resource. Tags are in a key/value format. For this solution to work, it doesn’t matter so much the key, however the value is where the focus is.

For the email address & environment tag, I am doing a regular expression match to find them.

  • {($_ -match ‘.@.’) -and ($_ -match ‘.\..’)}
    • this matches any value with the ‘@’ symbol as well as any single character either side with the RegEx single character operator ‘.’
    • … and any existence of SingleCharacter{dot}SingleCharacter, I am using the RegEx escape character ‘\’ and matching one ‘.’ literally as well as any single character either side using the ‘.’
  • {($_ -match ‘dev’) -or ($_ -match ‘test’)}
    • Doing a simple match for either ‘dev‘ or ‘test
$a = [psCustomObject]@{
	name        = $VM.Name
	CreatedBy   = ($VM.Tags.Values | where {($_ -match '.@.') -and ($_ -match '.\..')}) # Find email address in a Tag
	Environment = ($VM.Tags.Values | where {($_ -match 'dev') -or ($_ -match 'test')}) # Find environment, Dev/Test
	}

So it doesn’t matter how your tags are applied, they can be applied as any ‘key‘, they’ll be found based on the logic above.

It’s not perfect I know, but it’s a good start. You might have multiple email addresses assigned to a resource, or other environments like LT, Stage, UAT etc etc. Now you know how you can tweak this to suit your needs.

Recurrence

The Azure Automation Recurrence schedule action is set to daily at 7pm.

20181103 Logic Apps Recurrence.png

Create job / Get job output

The Get All VMs PowerShell script is run using:

  • Azure Automation Create job action
  • Azure Automation Get job output action

20181103 Azure Automation, Logic Apps

The output from the Get All VMs PowerShell script is JSON and looks like this:

2018-10-31_0819

Parse JSON

This information outputted from the ‘Azure Automation Create job action’ (above) is then used with a Logic Apps ‘Parse JSONData Operation action to parse the JSON from the Get VMs runbook.

This JSON needs to be parsed by Logic Apps in order to make sense of it.

20181103 Parse JSON

When adding the Logic Apps Parse JSON Data Operations action, there’s an option at the bottom to ‘Use sample payload to generate schema‘, select this option and paste in the JSON below – if you use the exact PowerShell script above.

20181103 JSON payload schema

[
    {
        "name": "marc@marckean.com",
        "VMNames": "SquidProxyUS, MSMarcFileSync",
        "Environment": "Test, Dev"
    },
    {
        "name": "marc@ejukebox.com.au",
        "VMNames": "MAP-Demo",
        "Environment": "Dev"
    }
]

For Each

Once the JSON is parsed, a Logic Apps For Each Control action cycles through each entry in the JSON payload/output. For each – an email is sent to the ‘Name‘ property – which – as you can see above – is the email address of the VM owner. The JSON example above, there’ll be two emails. As all VMs are already collated for each VM owner email address, (thanks to the PowerShell script) each person will get the email only once and includes all the VMs they are owners of.

The ‘Logic Apps For Each Control action’ takes the ‘Body’ output from the ‘Logic Apps Parse JSON Data Operations action’.

Send approval email

The ‘Logic Apps Send approval email Office 365 Outlook action’ takes the ‘name‘, ‘VMNames‘ and ‘Environment‘ property from the ‘Logic Apps Parse JSON Data Operations action’ as well, then sends email with user optionsApprove, Reject‘.

20181101 for each loop

You will need to go into the settings of the ‘Logic Apps Send approval email Office 365 Outlook action’, to set the Timeout.

20181103 OutlookApprovalSettings

I have here the timeout set to 30 minutes, this is set in ISO 8601 format.

  • P – indicates that the duration that follows is specified by the number of years, months, days, hours, minutes, and seconds
  • T – indicates that a time value follows. Any value with a time must begin with T.

20181103 OutlookApprovalSettings2

When the approval email is sent out, below is an example of what the email looks like.

20181101 Approval Email

Condition

If the VM owner clicks approve in the email, this goes back to the Logic Apps workflow and continues to the next stage by using a ‘Logic Apps Condition control action’.

If the VM owner doesn’t click anything or anything but Approve, e.g. Reject, the ‘Logic Apps Send approval email Office 365 Outlook action’ interprets this is ‘false‘ and does nothing.

20181101 Approve condition

If the ‘Logic Apps Condition control action’ receives nothing, or an Approve message, this is classed as ‘true‘ and another ‘Azure Automation Create job action’ is kicked off.

So that the ‘Logic Apps Send approval email Office 365 Outlook action’ Timeout setting of 30 minutes actually works, change the setting on the Condition, edit the ‘Configure run after‘:

20181103 Condition, configure run after

Tick the two boxes ‘is successful‘ and ‘has timed out‘.

20181103 Condition, configure run after2

Create job

Another Azure Automation runbook is kicked off (Stop-Azure-VMs-Parameter) via an Azure Automation Create job action, although this time a parameter value is passed through to this runbook, the ‘VMNames‘ property is passed through to this runbook.

The below screenshot shows you how to construct the JSON needed for the Runbook parameter.

20181103 Create Job 2

Remember at stage in the Logic Apps workflow we are still in a for each loop? Well looking at the screenshot further above, the VMNames property could be ‘SquidProxyUS, MSMarcFileSync‘.

FAQs

  • If the VM’s are already shut down (i.e. they were not restarted from yesterday) – does it send the email anyway?
    • No, the first Azure Automation PowerShell script runbook which is used to get the VMs, it gets running VMs only each time – this is the section in the script which ensures it only gets running VMs ‘$VMs += Get-AzureRmVM -Status | where {$_.PowerState -eq ‘VM Running’ -and $_.Tags.values -match ‘.’}

Below is the full JSON output of my Logic App:


{
"$connections": {
"value": {
"azureautomation": {
"connectionId": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/resourceGroups/OutlookApprovalLogic/providers/Microsoft.Web/connections/azureautomation",
"connectionName": "azureautomation",
"id": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/providers/Microsoft.Web/locations/australiaeast/managedApis/azureautomation"
},
"office365": {
"connectionId": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/resourceGroups/OutlookApprovalLogic/providers/Microsoft.Web/connections/office365",
"connectionName": "office365",
"id": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/providers/Microsoft.Web/locations/australiaeast/managedApis/office365"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Create_job": {
"inputs": {
"host": {
"connection": {
"name": "@parameters('$connections')['azureautomation']['connectionId']"
}
},
"method": "put",
"path": "/subscriptions/@{encodeURIComponent('6bb00255-5486-4db1-96ca-5baefc18b0b2')}/resourceGroups/@{encodeURIComponent('Marc-Automation')}/providers/Microsoft.Automation/automationAccounts/@{encodeURIComponent('Marc-Automation')}/jobs",
"queries": {
"runbookName": "Get-All-VMs",
"wait": true,
"x-ms-api-version": "2015-10-31"
}
},
"runAfter": {},
"type": "ApiConnection"
},
"For_each": {
"actions": {
"Condition": {
"actions": {
"Create_job_2": {
"inputs": {
"body": {
"properties": {
"parameters": {
"VMnames": "@{items('For_each')['VMNames']}"
}
}
},
"host": {
"connection": {
"name": "@parameters('$connections')['azureautomation']['connectionId']"
}
},
"method": "put",
"path": "/subscriptions/@{encodeURIComponent('6bb00255-5486-4db1-96ca-5baefc18b0b2')}/resourceGroups/@{encodeURIComponent('Marc-Automation')}/providers/Microsoft.Automation/automationAccounts/@{encodeURIComponent('Marc-Automation')}/jobs",
"queries": {
"runbookName": "Stop-Azure-VMs-Parameter",
"wait": false,
"x-ms-api-version": "2015-10-31"
}
},
"runAfter": {},
"type": "ApiConnection"
}
},
"expression": {
"or": [
{
"equals": [
"@body('Send_approval_email')?['SelectedOption']",
"Approve"
]
},
{
"equals": [
"@body('Send_approval_email')",
"@null "
]
}
]
},
"runAfter": {
"Send_approval_email": [
"Succeeded",
"TimedOut"
]
},
"type": "If"
},
"Send_approval_email": {
"inputs": {
"body": {
"Message": {
"Body": "Do you want to shut down @{items('For_each')['VMNames']} in @{items('For_each')['Environment']}?",
"Importance": "Normal",
"Options": "Approve, Reject",
"Subject": "Approval Request",
"To": "@items('For_each')['name']",
"UseOnlyHTMLMessage": false
},
"NotificationUrl": "@{listCallbackUrl()}"
},
"host": {
"connection": {
"name": "@parameters('$connections')['office365']['connectionId']"
}
},
"path": "/approvalmail/$subscriptions"
},
"limit": {
"timeout": "PT30M"
},
"runAfter": {},
"type": "ApiConnectionWebhook"
}
},
"foreach": "@body('Parse_JSON')",
"runAfter": {
"Parse_JSON": [
"Succeeded"
]
},
"type": "Foreach"
},
"Get_job_output": {
"inputs": {
"host": {
"connection": {
"name": "@parameters('$connections')['azureautomation']['connectionId']"
}
},
"method": "get",
"path": "/subscriptions/@{encodeURIComponent('6bb00255-5486-4db1-96ca-5baefc18b0b2')}/resourceGroups/@{encodeURIComponent('Marc-Automation')}/providers/Microsoft.Automation/automationAccounts/@{encodeURIComponent('Marc-Automation')}/jobs/@{encodeURIComponent(body('Create_job')?['properties']?['jobId'])}/output",
"queries": {
"x-ms-api-version": "2015-10-31"
}
},
"runAfter": {
"Create_job": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Parse_JSON": {
"inputs": {
"content": "@body('Get_job_output')",
"schema": {
"items": {
"properties": {
"Environment": {
"type": "string"
},
"VMNames": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"name",
"VMNames",
"Environment"
],
"type": "object"
},
"type": "array"
}
},
"runAfter": {
"Get_job_output": [
"Succeeded"
]
},
"type": "ParseJson"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Day",
"interval": 1,
"schedule": {
"hours": [
"19"
]
},
"timeZone": "AUS Eastern Standard Time"
},
"type": "Recurrence"
}
}
}
}

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s