Instant Monitoring of Windows Services

Has there been a seriously critical Windows Service which you need to monitor in real-time, or more than one Windows Service? – i.e. as soon as the Windows Service stops, you need to be notified by getting an SMS text message to your phone – within 5 seconds? While this is slightly manual, once setup, it works perfectly well and is reliable.

This is similar to my other blog post which discusses Instant Monitoring of Windows Performance.

This blog post walks you through everything, I am using both Azure Functions and the Telstra SMS API in Australia to send instant SMS notifications should a Windows Service stop. While this Telstra service is hosted in Australia, it’s highly available and can be used to send messages overseas.

  • Q: Can I send SMS and MMS to all countries?
  • A: You can send SMS and MMS to all countries EXCEPT to countries which are subject to global sanctions namely: Burma, Côte d’Ivoire, Cuba, Iran, North Korea, Syria.

And yes, there’s a free SMS plan! (Maximum 1000 free SMS messages) to get you started.

This blog will walk you through the process of creating an Azure Function, along with a scheduled task using the Windows event log as the trigger. Why the event log? Pretty much everything is logged to the event log instantly as things happen – e.g. Windows Services stopping…

  1. You specify a Windows Service
  2. As soon as this Windows Service stops, this triggers off a scheduled task which uses a query of (ID: 7036) with the word ‘stopped‘ in the event log query.
  3. The scheduled task kicks off a PowerShell script. This PowerShell script has a whole bunch of parameter values pre-specified
  4. The PowerShell script fires off an Azure Function by using a ‘Route Path‘ based URL.
  5. The Azure Function takes these parameter values at the time it’s fired off, then uses the parameter values to:
    1. send an SMS
    2. log an entry in a Log Analytics workspace custom log.

The below walks you through setting it up. While this blog focuses on Windows Services, you can easily follow this methodology with literally anything and call the Azure Functions URL to send the SMS based on any trigger you like.

Backend Setup (Manual)

Done once only…..

Setup an account with Telstra DEV

  1. Setup an account with https://dev.telstra.com
  2. Setup an SMS API app in the ‘develop‘ section
  3. Once setting up the app, you’ll get a Client key and Client secret. Don’t loose these, these are like the username and password you need each time an SMS is sent

Setup a Log Analytics workspace

Follow this guide to setup a Log Analytics workspace, or you can us an existing Log Analytics workspace. While you are notified by SMS instantly when a Windows Service stops, these messages are also logged in Log Analytics to keep track of the history/trends – where you can create a dashboard etc.

Doing a Log Search in Log Analytics, here’s an example of the query you would need to query back on past data:

ServiceStopped_CL
| project Message, TimeGenerated
| sort by TimeGenerated desc

Create an Azure Function

The Azure Function is what you need as the engine to fire off the SMS & Log to your Log Analytics workspace.

  1. Create a new PowerShell based Azure Function. For this guide, I called mine ‘EventDrivenFunction‘.

    FunctionNewPowerShell

  2. Under the Integrate menu of your Azure Function, select the Advanced Editor and paste in the following:
    {
    "bindings":[
    {
    "name":"req",
    "type":"httpTrigger",
    "direction":"in",
    "authLevel":"anonymous",
    "route":"EventDrivenFunction/{LogAnalyticsCustomerID}/{LogAnalyticsPrimaryKey}/{LogType}/{Telstra_app_key}/{Telstra_app_secret}/{tel_numbers}/{Message}"
    },
    {
    "name":"res",
    "type":"http",
    "direction":"out"
    }
    ],
    "disabled":false
    }
  3. Then click on the actual function itself and paste in the following: (and hit Save)


# Replace with your Workspace ID
$CustomerID = $REQ_PARAMS_LogAnalyticsCustomerID
# Replace with your Log Analytics workspace Primary Key
$SharedKey = $REQ_PARAMS_LogAnalyticsPrimaryKey
#Specify the name of the record type that we'll be creating.
$LogType = $REQ_PARAMS_LogType # To be used to search for as a custom log e.g. Type=iTunesPopCharts2_CL
# Specify a time in the format YYYY-MM-DDThh:mm:ssZ to specify a created time for the records
$TimeStampField = ''
# Telstra DEV APIs – https://dev.telstra.com
$Telstra_app_key = $REQ_PARAMS_Telstra_app_key
$Telstra_app_secret = $REQ_PARAMS_Telstra_app_secret
$tel_numbers = $REQ_PARAMS_tel_numbers
$message = $REQ_PARAMS_Message
# Message to send
$LAmessage = [PSCustomObject]@{
Channel = 'Azure Automation Webhook'
message = $message
}
function Send-TelstraSMS ($tel_numbers, $body) {
################# Get Telstra API access – https://dev.telstra.com/
[System.Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$UriToken = "https://tapi.telstra.com/v2/oauth/token"
$body = @{
client_id = $Telstra_app_key
client_secret = $Telstra_app_secret
grant_type = 'client_credentials'
scope = 'NSMS'
}
$contentType = 'application/x-www-form-urlencoded'
$auth_values = Invoke-RestMethod Uri $UriToken body $body ContentType $contentType Method Post
################# Provisioning – a 30 day mobile number
$token = $auth_values.access_token
$UriProvisioning = "https://tapi.telstra.com/v2/messages/provisioning/subscriptions"
$headers = @{
'Authorization' = "Bearer $token"
'cache-control' = 'no-cache'
}
$JSON = @{
activeDays = 30
notifyURL = 'http://example.com/callback'
callbackData = @{"anything" = "some data"}
} | ConvertTo-Json
$JSON
$contentType = 'application/json'
$MobileNumberRaw = Invoke-RestMethod Uri $UriProvisioning ContentType $contentType Headers $headers Method Post body $JSON
################# Send SMS
$MobileNumber = $MobileNumberRaw.destinationAddress
$token = $auth_values.access_token
$UriSend = "https://tapi.telstra.com/v2/messages/sms"
$JSON = @{
to = @($tel_numbers)
body = $message
} | ConvertTo-Json
$JSON = $JSON -replace '\\u0027', '"' -replace '""', '"'
$JSON
$headers = @{
'Authorization' = "Bearer $token"
'cache-control' = 'no-cache'
}
$contentType = 'application/json'
$sent_message = Invoke-RestMethod Uri $UriSend ContentType $contentType Headers $headers Method Post Body $JSON
###########################################################
}
# Function to create the authorization signature.
Function New-Signature ($CustomerID, $SharedKey, $date, $contentLength, $method, $contentType, $resource)
{
$xHeaders = 'x-ms-date:' + $date
$stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource
$bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)
$keyBytes = [Convert]::FromBase64String($SharedKey)
$sha256 = New-Object TypeName System.Security.Cryptography.HMACSHA256
$sha256.Key = $keyBytes
$calculatedHash = $sha256.ComputeHash($bytesToHash)
$encodedHash = [Convert]::ToBase64String($calculatedHash)
$authorization = 'SharedKey {0}:{1}' -f $CustomerID, $encodedHash
return $authorization
}
# Function to create and post the request
Function Send-OMSData($CustomerID, $SharedKey, $body, $LogType)
{
$method = 'POST'
$contentType = 'application/json'
$resource = '/api/logs'
$rfc1123date = [DateTime]::UtcNow.ToString('r')
$contentLength = $body.Length
$signature = New-Signature `
customerId $CustomerID `
sharedKey $SharedKey `
date $rfc1123date `
contentLength $contentLength `
method $method `
contentType $contentType `
resource $resource
$omsuri = 'https://' + $CustomerID + '.ods.opinsights.azure.com' + $resource + '?api-version=2016-04-01'
$headers = @{
'Authorization' = $signature
'Log-Type' = $LogType
'x-ms-date' = $rfc1123date
'time-generated-field' = $TimeStampField
}
$response = Invoke-WebRequest Uri $omsuri Method $method ContentType $contentType Headers $headers Body $body UseBasicParsing
return $response.StatusCode
}
# Send to Log Analytics workspace
$json = $LAmessage | ConvertTo-Json
Write-Output InputObject $json
Send-OMSData customerId $customerId sharedKey $sharedKey body $json logType $logType
Send-TelstraSMS tel_numbers $tel_numbers body $message

Setup the Scheduled Task | Query

This is like an interim step to build out the query for the exact event log you are looking for.

In this step you build out a query in order to setup a Scheduled task with an Windows event as the trigger. The idea here is you need to find the event in which you want to monitor (a service if it stops). Windows Stopped services are logged under Event ID 7036 in the Information event log.

Pick one of the events and Copy Details as Text.

2018-08-20_2138

Paste into Notepad and have a look at the EventData. Below shows the EventData of the Windows Service Running, however you would most likely want to monitor for a ‘Stopped‘ service.

2018-08-20_2138_001

Take note of the EventData details and build a query using this information – as per this example:

<QueryList>
<QueryId="0"Path="System">
<SelectPath="System">*[System[Provider[@Name='Service Control Manager'] and (Level=4 or Level=0) and (EventID=7036)]] and *[EventData[Data[@Name='param1'] and (Data='SHOUTcast')]] and *[EventData[Data[@Name='param2'] and (Data='stopped')]]
</Select>
</Query>
</QueryList>

This query is what your Scheduled task will use. So you need to make sure it works…… If you need help to write your query, check out this other blog.

To test your query, create a custom view:

2018-08-20_2142

Paste the query into the XML tab and hit OK. Make sure you can see events in the ‘Custom Views‘ which match your query.

2018-08-20_2144

Once you’re happy that the query is what you are looking for, take the code below, edit line 34 where is has $EventLog_Query, change this to your own query. Copy and paste the query across into PowerShell removing the carriage returns, making sure it’s all on one single line.

Client Setup (Automatic)

Once the backend is all setup (the Azure Function, Log Analytics Workspace & Telstra DEV account), this client setup part is fully automatic and can be rolled out to many machines all at once.  To run this remotely, you could run this using Remote PowerShell or you could set this up using Group Policy.

Run PowerShell ISE as administrator if running this on the actual machine. Don’t forget to change all the variables at the top section of the below script to suit your environment.

  1. $LogAnalyticsCustomerID – Obtain workspace ID and key
  2. $LogAnalyticsPrimaryKey – Obtain workspace ID and key
  3. $Telstra_app_key – From https://dev.telstra.com/
  4. $Telstra_app_secret – From https://dev.telstra.com/
  5. $tel_numbers – One or many mobile numbers to send the SMS to
  6. $FunctionUri – Your own Azure Functions URI

Run this whole script as Administrator, this will setup the scheduled task & the PowerShell script which is called from the scheduled task, which in turn calls the Azure Function.


### RUN AS ADMINISTRATOR
# This script here string, will be copied to the local computer to be run locally
# Change the variables in this script, they are at the top in this here string
$script = @'
# Replace with your Workspace ID
$LogAnalyticsCustomerID = "Workspace ID"
# Replace with your Log Analytics workspace Primary Key
$LogAnalyticsPrimaryKey = "workspace Primary Key"
#Specify the name of the record type that we'll be creating.
$LogType = "ServiceStopped" # To be used to search for as a custom log e.g. Type=ServiceStopped_CL
# Telstra DEV API Key – https://dev.telstra.com
$Telstra_app_key = "Telstra DEV API Key"
# Telstra DEV App Secret – https://dev.telstra.com
$Telstra_app_secret = "Telstra DEV App Secret"
# Mobile numbers to send, comma separated (with space), each number enclosed in single quotes
$tel_numbers = "'+61412345678', '+61498765432'"
# Message to send and add to the Log Analytics Custom Log
$Message = "The SHOUTcast service has stopped on $env:COMPUTERNAME"
$FunctionUri = 'https://marcfunction1.azurewebsites.net/api/EventDrivenFunction/{0}/{1}/{2}/{3}/{4}/{5}/{6}' `
-f $LogAnalyticsCustomerID, $LogAnalyticsPrimaryKey, $LogType, $Telstra_app_key, $Telstra_app_secret, $tel_numbers, $Message
Invoke-WebRequest -Uri $FunctionUri
'@
# Custom Event Log Query
$EventLog_Query = "<QueryList><Query Id='0' Path='System'><Select Path='System'>*[System[Provider[@Name='Service Control Manager'] and (Level=4 or Level=0) and (EventID=7036)]] and *[EventData[Data[@Name='param1'] and (Data='SHOUTcast')]] and *[EventData[Data[@Name='param2'] and (Data='stopped')]]</Select></Query></QueryList>"
$date = $(Get-Date Format yyyyMMddhhmmss)
$ScriptFile = "$($env:SystemDrive)\Windows\System32\$date.ps1"
Set-Content Path $ScriptFile Value $Script
$taskName = "Event Driven Task $date"
$Path = 'PowerShell.exe'
$Arguments = "-ExecutionPolicy Unrestricted -File $ScriptFile"
# This removes empty last line at the end of the text file
$in = [System.IO.File]::OpenText($ScriptFile)
$text = ($in.readtoend()).trim("`r`n")
$in.close()
$stream = [System.IO.StreamWriter]$ScriptFile
$stream.write($text)
$stream.close()
$Service = new-object ComObject ("Schedule.Service")
$Service.Connect()
$RootFolder = $Service.GetFolder("\")
$TaskDefinition = $Service.NewTask(0) # TaskDefinition object https://msdn.microsoft.com/en-us/library/windows/desktop/aa382542(v=vs.85).aspx
$TaskDefinition.RegistrationInfo.Description = ''
$TaskDefinition.Settings.Enabled = $True
$TaskDefinition.Settings.AllowDemandStart = $True
$TaskDefinition.Settings.DisallowStartIfOnBatteries = $False
$Triggers = $TaskDefinition.Triggers
$Trigger = $Triggers.Create(0) ## 0 is an event trigger https://msdn.microsoft.com/en-us/library/windows/desktop/aa383898(v=vs.85).aspx
$Trigger.Enabled = $true
# Expiry time if needed # $TaskEndTime = [datetime]::Now.AddMinutes(30);$Trigger.EndBoundary = $TaskEndTime.ToString("yyyy-MM-dd'T'HH:mm:ss")
$Trigger.Id = '7036' # Event ID
<#
Advanced XML filtering in the Windows Event Viewer
https://blogs.technet.microsoft.com/askds/2011/09/26/advanced-xml-filtering-in-the-windows-event-viewer/
#>
$Trigger.Subscription = $EventLog_Query
$Action = $TaskDefinition.Actions.Create(0)
$Action.Path = $Path
$action.Arguments = $Arguments
$RootFolder.RegisterTaskDefinition($taskName, $TaskDefinition, 6, "System", $null, 5) | Out-Null

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s