This blog post is based on 3 separate asks from 3 separate customers about adding value for their Azure subscriptions. They came to be and said “Marc, I would like to devise a scheme where at 7pm every night, all Dev/Test VMs can be shut down to save money and the business owners have the final say.” There’s one problem though, the business owners don’t have access to the Azure portal – it has to be done over email. What’s more, an option to force the shutdown of the VMs if the business owners don’t respond in a time frame of about 30 mins.
Enter Logic Apps & Azure Automation Runbooks:
This is what the entire Logic App looks like:
I’ll dive into each step bit by bit, starting with the top – prerequisites.
Prerequisites
Some prerequisites, you need to setup Azure Automation:
- The Run As account
- Two Runbooks (PowerShell) scripts.
An Azure Runbook is run using an Azure Automation runas account.
- Think of a Run As account as a service account, it’s what the runbook utilises in order to gain privileges to the Azure subscriptions
Azure Automation Run As accounts can be created automatically by Azure Automation when you tell it to. All it is, is a Service Principal using a certificate for authentication and Azure Automation takes care of the entire logon process to Azure. This is all you need as the logon construct:
$ServicePrincipalConnection = Get-AutomationConnection -Name 'AzureRunAsConnection' $null = Add-AzureRmAccount ` -ServicePrincipal ` -TenantId $ServicePrincipalConnection.TenantId ` -ApplicationId $ServicePrincipalConnection.ApplicationId ` -CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
You can add this Run As account like a normal account and assign (Contributor/Owner) permission to it from other Azure subscriptions you have in your AAD tenant.
Get-All-VMs
This runbook (and below) gets all VMs across all Azure subscriptions in which the Run As account has access to. It loops through all Azure subscriptions the account has access to.
It grabs:
- the VM name…
- both an email address and an environment tag (either Dev/Test)
… and converts all of this information to JSON.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ServicePrincipalConnection = Get-AutomationConnection –Name 'AzureRunAsConnection' | |
$null = Add-AzureRmAccount ` | |
–ServicePrincipal ` | |
–TenantId $ServicePrincipalConnection.TenantId ` | |
–ApplicationId $ServicePrincipalConnection.ApplicationId ` | |
–CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint | |
################################################################## | |
################################################################## | |
################################################################## | |
################################################################## | |
################################################################## | |
$Subscriptions = Get-AzureRmSubscription | |
# Get a list of VMs which are powered on, has an email address tag and has an environment tag (Dev or Test) | |
$VMs = @() | |
foreach ($Sub in $Subscriptions) { | |
$null = Select-AzureRmSubscription –Subscription $Sub | |
# Get VMs that are running and actually have a tags of some sort | |
$VMs += Get-AzureRmVM –Status | where {$_.PowerState -eq 'VM Running' -and $_.Tags.values -match '.'} | |
$VMs_with_Tags = @() | |
foreach ($VM in $VMs) { | |
$a = [psCustomObject]@{ | |
name = $VM.Name | |
CreatedBy = ($VM.Tags.Values | where {($_ -match '.@.') -and ($_ -match '.\..')}) # Find email address in a Tag | |
Environment = ($VM.Tags.Values | where {($_ -match 'dev') -or ($_ -match 'test')}) # Find environment, Dev/Test | |
} | |
$VMs_with_Tags += $a | |
} | |
} | |
# Group the VMs with tags by email address | |
$GroupedVMs_with_Tags = @() | |
($VMs_with_Tags | where {($_.CreatedBy) -or ($_.Environment)} | Group-Object CreatedBy) | % { | |
$a = [psCustomObject]@{ | |
name = $_.Name | |
VMNames = $_.Group.Name -join ', ' | |
Environment = $_.Group.Environment -join ', ' | |
} | |
$GroupedVMs_with_Tags += $a | |
} | |
$GroupedVMs_with_Tags | ConvertTo-Json –Depth 100 |
Stop-Azure-VMs-Parameter
This other Azure Automation runbook (and below) is kicked off (Stop-Azure-VMs-Parameter), although a parameter value is passed through to this runbook, the ‘VMNames‘ property is passed through to this runbook – more on this further below in the For each loop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
param | |
( | |
[parameter(Mandatory=$true)] | |
[String] $VMnames | |
) | |
$ServicePrincipalConnection = Get-AutomationConnection –Name 'AzureRunAsConnection' | |
$null = Add-AzureRmAccount ` | |
–ServicePrincipal ` | |
–TenantId $ServicePrincipalConnection.TenantId ` | |
–ApplicationId $ServicePrincipalConnection.ApplicationId ` | |
–CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint | |
$Subscriptions = Get-AzureRmSubscription | |
# Get a list of VMs which are powered on, has an email address tag and has an environment tag (Dev or Test) | |
foreach ($Sub in $Subscriptions) { | |
$null = Select-AzureRmSubscription –Subscription $Sub | |
# Filter out the VMs for the current subscription | |
$SubVMs = @() | |
# Get all Azure VMs based ont eh current Azure subscription | |
$AzureVMs = Get-AzureRmVM | |
# Do a foreach against all the VMs provided in parameter – some aren't in the current subscription | |
foreach($a in ($VMnames -split ', ')){ | |
# Check if the current VM in the parameter array exists in the current Azure subscription | |
if($a -in $AzureVMs.name){ | |
$b = Get-AzureRmVM | ? {$_.Name -eq $a} | |
$c = [psCustomObject]@{ | |
name = $b.Name | |
ResourceGroupName = $b.ResourceGroupName | |
} | |
$SubVMs += $c | |
} | |
} | |
Write-Output $SubVMs | |
ForEach ($VM in $SubVMs) | |
{ | |
Stop-AzureRmVM –Name $VM.Name –ResourceGroupName $VM.ResourceGroupName –Force | |
} | |
} |
Tags
In Azure, you can have up to 15 tags per Azure resource. Tags are in a key/value format. For this solution to work, it doesn’t matter so much the key, however the value is where the focus is.
For the email address & environment tag, I am doing a regular expression match to find them.
- {($_ -match ‘.@.’) -and ($_ -match ‘.\..’)}
- this matches any value with the ‘@’ symbol as well as any single character either side with the RegEx single character operator ‘.’
- … and any existence of SingleCharacter{dot}SingleCharacter, I am using the RegEx escape character ‘\’ and matching one ‘.’ literally as well as any single character either side using the ‘.’
- {($_ -match ‘dev’) -or ($_ -match ‘test’)}
- Doing a simple match for either ‘dev‘ or ‘test‘
$a = [psCustomObject]@{ name = $VM.Name CreatedBy = ($VM.Tags.Values | where {($_ -match '.@.') -and ($_ -match '.\..')}) # Find email address in a Tag Environment = ($VM.Tags.Values | where {($_ -match 'dev') -or ($_ -match 'test')}) # Find environment, Dev/Test }
So it doesn’t matter how your tags are applied, they can be applied as any ‘key‘, they’ll be found based on the logic above.
It’s not perfect I know, but it’s a good start. You might have multiple email addresses assigned to a resource, or other environments like LT, Stage, UAT etc etc. Now you know how you can tweak this to suit your needs.
Recurrence
The Azure Automation Recurrence schedule action is set to daily at 7pm.
Create job / Get job output
The Get All VMs PowerShell script is run using:
- Azure Automation Create job action
- Azure Automation Get job output action
The output from the Get All VMs PowerShell script is JSON and looks like this:
Parse JSON
This information outputted from the ‘Azure Automation Create job action’ (above) is then used with a Logic Apps ‘Parse JSON‘ Data Operation action to parse the JSON from the Get VMs runbook.
This JSON needs to be parsed by Logic Apps in order to make sense of it.
When adding the Logic Apps Parse JSON Data Operations action, there’s an option at the bottom to ‘Use sample payload to generate schema‘, select this option and paste in the JSON below – if you use the exact PowerShell script above.
[ { "name": "marc@marckean.com", "VMNames": "SquidProxyUS, MSMarcFileSync", "Environment": "Test, Dev" }, { "name": "marc@ejukebox.com.au", "VMNames": "MAP-Demo", "Environment": "Dev" } ]
For Each
Once the JSON is parsed, a Logic Apps For Each Control action cycles through each entry in the JSON payload/output. For each – an email is sent to the ‘Name‘ property – which – as you can see above – is the email address of the VM owner. The JSON example above, there’ll be two emails. As all VMs are already collated for each VM owner email address, (thanks to the PowerShell script) each person will get the email only once and includes all the VMs they are owners of.
The ‘Logic Apps For Each Control action’ takes the ‘Body’ output from the ‘Logic Apps Parse JSON Data Operations action’.
Send approval email
The ‘Logic Apps Send approval email Office 365 Outlook action’ takes the ‘name‘, ‘VMNames‘ and ‘Environment‘ property from the ‘Logic Apps Parse JSON Data Operations action’ as well, then sends email with user options ‘Approve, Reject‘.
You will need to go into the settings of the ‘Logic Apps Send approval email Office 365 Outlook action’, to set the Timeout.
I have here the timeout set to 30 minutes, this is set in ISO 8601 format.
- P – indicates that the duration that follows is specified by the number of years, months, days, hours, minutes, and seconds
- T – indicates that a time value follows. Any value with a time must begin with T.
When the approval email is sent out, below is an example of what the email looks like.
Condition
If the VM owner clicks approve in the email, this goes back to the Logic Apps workflow and continues to the next stage by using a ‘Logic Apps Condition control action’.
If the VM owner doesn’t click anything or anything but Approve, e.g. Reject, the ‘Logic Apps Send approval email Office 365 Outlook action’ interprets this is ‘false‘ and does nothing.
If the ‘Logic Apps Condition control action’ receives nothing, or an Approve message, this is classed as ‘true‘ and another ‘Azure Automation Create job action’ is kicked off.
So that the ‘Logic Apps Send approval email Office 365 Outlook action’ Timeout setting of 30 minutes actually works, change the setting on the Condition, edit the ‘Configure run after‘:
Tick the two boxes ‘is successful‘ and ‘has timed out‘.
Create job
Another Azure Automation runbook is kicked off (Stop-Azure-VMs-Parameter) via an Azure Automation Create job action, although this time a parameter value is passed through to this runbook, the ‘VMNames‘ property is passed through to this runbook.
The below screenshot shows you how to construct the JSON needed for the Runbook parameter.
Remember at stage in the Logic Apps workflow we are still in a for each loop? Well looking at the screenshot further above, the VMNames property could be ‘SquidProxyUS, MSMarcFileSync‘.
FAQs
- If the VM’s are already shut down (i.e. they were not restarted from yesterday) – does it send the email anyway?
- No, the first Azure Automation PowerShell script runbook which is used to get the VMs, it gets running VMs only each time – this is the section in the script which ensures it only gets running VMs ‘$VMs += Get-AzureRmVM -Status | where {$_.PowerState -eq ‘VM Running’ -and $_.Tags.values -match ‘.’}‘
Below is the full JSON output of my Logic App:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"$connections": { | |
"value": { | |
"azureautomation": { | |
"connectionId": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/resourceGroups/OutlookApprovalLogic/providers/Microsoft.Web/connections/azureautomation", | |
"connectionName": "azureautomation", | |
"id": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/providers/Microsoft.Web/locations/australiaeast/managedApis/azureautomation" | |
}, | |
"office365": { | |
"connectionId": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/resourceGroups/OutlookApprovalLogic/providers/Microsoft.Web/connections/office365", | |
"connectionName": "office365", | |
"id": "/subscriptions/6bb00255-5486-4db1-96ca-5baefc18b0b2/providers/Microsoft.Web/locations/australiaeast/managedApis/office365" | |
} | |
} | |
}, | |
"definition": { | |
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#", | |
"actions": { | |
"Create_job": { | |
"inputs": { | |
"host": { | |
"connection": { | |
"name": "@parameters('$connections')['azureautomation']['connectionId']" | |
} | |
}, | |
"method": "put", | |
"path": "/subscriptions/@{encodeURIComponent('6bb00255-5486-4db1-96ca-5baefc18b0b2')}/resourceGroups/@{encodeURIComponent('Marc-Automation')}/providers/Microsoft.Automation/automationAccounts/@{encodeURIComponent('Marc-Automation')}/jobs", | |
"queries": { | |
"runbookName": "Get-All-VMs", | |
"wait": true, | |
"x-ms-api-version": "2015-10-31" | |
} | |
}, | |
"runAfter": {}, | |
"type": "ApiConnection" | |
}, | |
"For_each": { | |
"actions": { | |
"Condition": { | |
"actions": { | |
"Create_job_2": { | |
"inputs": { | |
"body": { | |
"properties": { | |
"parameters": { | |
"VMnames": "@{items('For_each')['VMNames']}" | |
} | |
} | |
}, | |
"host": { | |
"connection": { | |
"name": "@parameters('$connections')['azureautomation']['connectionId']" | |
} | |
}, | |
"method": "put", | |
"path": "/subscriptions/@{encodeURIComponent('6bb00255-5486-4db1-96ca-5baefc18b0b2')}/resourceGroups/@{encodeURIComponent('Marc-Automation')}/providers/Microsoft.Automation/automationAccounts/@{encodeURIComponent('Marc-Automation')}/jobs", | |
"queries": { | |
"runbookName": "Stop-Azure-VMs-Parameter", | |
"wait": false, | |
"x-ms-api-version": "2015-10-31" | |
} | |
}, | |
"runAfter": {}, | |
"type": "ApiConnection" | |
} | |
}, | |
"expression": { | |
"or": [ | |
{ | |
"equals": [ | |
"@body('Send_approval_email')?['SelectedOption']", | |
"Approve" | |
] | |
}, | |
{ | |
"equals": [ | |
"@body('Send_approval_email')", | |
"@null " | |
] | |
} | |
] | |
}, | |
"runAfter": { | |
"Send_approval_email": [ | |
"Succeeded", | |
"TimedOut" | |
] | |
}, | |
"type": "If" | |
}, | |
"Send_approval_email": { | |
"inputs": { | |
"body": { | |
"Message": { | |
"Body": "Do you want to shut down @{items('For_each')['VMNames']} in @{items('For_each')['Environment']}?", | |
"Importance": "Normal", | |
"Options": "Approve, Reject", | |
"Subject": "Approval Request", | |
"To": "@items('For_each')['name']", | |
"UseOnlyHTMLMessage": false | |
}, | |
"NotificationUrl": "@{listCallbackUrl()}" | |
}, | |
"host": { | |
"connection": { | |
"name": "@parameters('$connections')['office365']['connectionId']" | |
} | |
}, | |
"path": "/approvalmail/$subscriptions" | |
}, | |
"limit": { | |
"timeout": "PT30M" | |
}, | |
"runAfter": {}, | |
"type": "ApiConnectionWebhook" | |
} | |
}, | |
"foreach": "@body('Parse_JSON')", | |
"runAfter": { | |
"Parse_JSON": [ | |
"Succeeded" | |
] | |
}, | |
"type": "Foreach" | |
}, | |
"Get_job_output": { | |
"inputs": { | |
"host": { | |
"connection": { | |
"name": "@parameters('$connections')['azureautomation']['connectionId']" | |
} | |
}, | |
"method": "get", | |
"path": "/subscriptions/@{encodeURIComponent('6bb00255-5486-4db1-96ca-5baefc18b0b2')}/resourceGroups/@{encodeURIComponent('Marc-Automation')}/providers/Microsoft.Automation/automationAccounts/@{encodeURIComponent('Marc-Automation')}/jobs/@{encodeURIComponent(body('Create_job')?['properties']?['jobId'])}/output", | |
"queries": { | |
"x-ms-api-version": "2015-10-31" | |
} | |
}, | |
"runAfter": { | |
"Create_job": [ | |
"Succeeded" | |
] | |
}, | |
"type": "ApiConnection" | |
}, | |
"Parse_JSON": { | |
"inputs": { | |
"content": "@body('Get_job_output')", | |
"schema": { | |
"items": { | |
"properties": { | |
"Environment": { | |
"type": "string" | |
}, | |
"VMNames": { | |
"type": "string" | |
}, | |
"name": { | |
"type": "string" | |
} | |
}, | |
"required": [ | |
"name", | |
"VMNames", | |
"Environment" | |
], | |
"type": "object" | |
}, | |
"type": "array" | |
} | |
}, | |
"runAfter": { | |
"Get_job_output": [ | |
"Succeeded" | |
] | |
}, | |
"type": "ParseJson" | |
} | |
}, | |
"contentVersion": "1.0.0.0", | |
"outputs": {}, | |
"parameters": { | |
"$connections": { | |
"defaultValue": {}, | |
"type": "Object" | |
} | |
}, | |
"triggers": { | |
"Recurrence": { | |
"recurrence": { | |
"frequency": "Day", | |
"interval": 1, | |
"schedule": { | |
"hours": [ | |
"19" | |
] | |
}, | |
"timeZone": "AUS Eastern Standard Time" | |
}, | |
"type": "Recurrence" | |
} | |
} | |
} | |
} |