Instant Monitoring of Windows Performance

Here’s an example similar to my other Blog post Instant Monitoring of Windows Services. Instead, this blog demonstrates monitoring Performance Counters and using alerts to fire off a scheduled task. This scheduled task fires off a PowerShell script which in turn fires off an Azure Function, which in turn fires off an SMS text message and records an entry in a Log Analytics custom log. All this happens sequentially in the space of about 5 seconds.

This blog demonstrates monitoring these 3 basic performance counters:

  • Processor(_Total)\% Processor Time
  • Memory\% Committed Bytes In Use
  • LogicalDisk(C:)\% Free Space

2018-08-26_1037

You could monitor more counters, another blog post later will let you know how you can do this.

Components

Here are the components of this solution, bearing in mind all this is set up fully automatically on each client by using the script below.

  • Performance Counters

    2018-08-26_1139

  • Scheduled Task

    2018-08-26_1140

    -ExecutionPolicy Unrestricted -File C:\Windows\System32\PerfmonDrivenTask.ps1 $(Arg0)
  • PowerShell script | Takes the Arguments from the Perfmon Alert and uses this to construct the message to be sent as an SMS and send to the Log Analytics custom log.
    # Message to send and add to the Log Analytics Custom Log
    $Message = ('{1}{0}{2}{0}{3}{0}{4}' -f " ", $Args[0], $Args[1], $Args[2], $env:computername) -replace '/', '-'

Backend Setup (Manual)

The backend is set up once only. As per my other blog, setup the Azure Function, Log Analytics Workspace & Telstra DEV account.

Once you have the Azure Function, Log Analytics Workspace & Telstra DEV account setup, The script at the very bottom sets up everything on the client automatically.

Client Setup (Automated)

Once the backend is set up, the Script further below fully automates the setup of each client. Good thing is, you can run this over and over again, it will only make sure everything is set in place again.

  1. Run PowerShell ISE as administrator if running this on the actual machine
  2. Within the top 33 lines, change the variables to suite your setup:
    1. $LogAnalyticsCustomerID – Obtain workspace ID and key
    2. $LogAnalyticsPrimaryKey – Obtain workspace ID and key
    3. $Telstra_app_key – From https://dev.telstra.com/
    4. $Telstra_app_secret – From https://dev.telstra.com/
    5. $tel_numbers – One or many mobile numbers to send the SMS to
    6. $FunctionUri – Your own Azure Functions URI
  3. To run this remotely, you could run this using Remote PowerShell.
  4. You could set this up using Group Policy

Testing

If you want to test that this is working, it’s pretty easy, use this tool to put strain on the CPU & Memory.

### RUN AS ADMINISTRATOR
# This script here string, will be copied to the local computer to be run locally
# Change the variables in this script, they are at the top in this here string
$script = @'
# Testing, $args is an array object ($args | Out-File "C:\Scripts\PerformanceAlert.txt" –Append)
#########################################
# Replace with your Workspace ID
$LogAnalyticsCustomerID = "Workspace ID"
# Replace with your Log Analytics workspace Primary Key
$LogAnalyticsPrimaryKey = "workspace Primary Key"
#Specify the name of the record type that we'll be creating.
$LogType = "LocalPerfMon" # To be used to search for as a custom log e.g. Type=LocalPerfMon_CL
# Telstra DEV API Key – https://dev.telstra.com
$Telstra_app_key = "Telstra DEV API Key"
# Telstra DEV App Secret – https://dev.telstra.com
$Telstra_app_secret = "Telstra DEV App Secret"
# Mobile numbers to send, comma separated (with space), each number enclosed in single quotes
$tel_numbers = "'+61412345678', '+61498765432'"
# Message to send and add to the Log Analytics Custom Log
$Message = ('{1}{0}{2}{0}{3}{0}{4}' -f " ", $Args[0], $Args[1], $Args[2], $env:computername) -replace '/', '-'
$FunctionUri = 'https://marcfunction1.azurewebsites.net/api/EventDrivenFunction/{0}/{1}/{2}/{3}/{4}/{5}/{6}' `
-f $LogAnalyticsCustomerID, $LogAnalyticsPrimaryKey, $LogType, $Telstra_app_key, $Telstra_app_secret, $tel_numbers, $Message
Invoke-RestMethod -Uri $FunctionUri
'@
# Create the script on the local computer
$ScriptFile = "$($env:SystemDrive)\Windows\System32\PerfmonDrivenTask.ps1"
Remove-Item Path $ScriptFile Force ErrorAction SilentlyContinue
Set-Content Path $ScriptFile Value $Script
#####################################################################################
#####################################################################################
<# Create the Scheduled Task – no trigger, a perfmon alert is the trigger
Runs the PowerShell script above
Takes the $args object array from the perfmon alert and feeds this to the above script
Several Args are utilised in the above script $Args[0], $Args[1], $Args[2]
#>
$taskName = "PerfmonDrivenTask"
$Path = 'PowerShell.exe'
$Arguments = '-ExecutionPolicy Unrestricted -File {0} {1}Arg0{2}' -f $ScriptFile, "`$(", ")"
# Quick Clean-up
Unregister-ScheduledTask TaskName $taskName Confirm:$false ErrorAction SilentlyContinue
# This removes empty last line at the end of the text file
$in = [System.IO.File]::OpenText($ScriptFile)
$text = ($in.readtoend()).trim("`r`n")
$in.close()
$stream = [System.IO.StreamWriter]$ScriptFile
$stream.write($text)
$stream.close()
$Service = new-object ComObject ("Schedule.Service")
$Service.Connect()
$RootFolder = $Service.GetFolder("\")
$TaskDefinition = $Service.NewTask(0) # TaskDefinition object https://msdn.microsoft.com/en-us/library/windows/desktop/aa382542(v=vs.85).aspx
$TaskDefinition.RegistrationInfo.Description = ''
$TaskDefinition.Settings.Enabled = $True
$TaskDefinition.Settings.AllowDemandStart = $True
$TaskDefinition.Settings.DisallowStartIfOnBatteries = $False
# Expiry time if needed # $TaskEndTime = [datetime]::Now.AddMinutes(30);$Trigger.EndBoundary = $TaskEndTime.ToString("yyyy-MM-dd'T'HH:mm:ss")
<#
Advanced XML filtering in the Windows Event Viewer
https://blogs.technet.microsoft.com/askds/2011/09/26/advanced-xml-filtering-in-the-windows-event-viewer/
#>
$Action = $TaskDefinition.Actions.Create(0)
$Action.Path = $Path
$action.Arguments = $Arguments
$RootFolder.RegisterTaskDefinition($taskName, $TaskDefinition, 6, "System", $null, 5) | Out-Null
#####################################################################################
#####################################################################################
# Create the Perfmon Alerts
$xml = @'
<?xml version="1.0" encoding="UTF-16"?>
<DataCollectorSet>
<Status>1</Status>
<Duration>0</Duration>
<SchedulesEnabled>1</SchedulesEnabled>
<LatestOutputLocation>%systemdrive%\PerfLogs\Admin\Alert\PerformanceAlerts</LatestOutputLocation>
<Name>Alert</Name>
<OutputLocation>%systemdrive%\PerfLogs\Admin\Alert\PerformanceAlerts</OutputLocation>
<RootPath>%systemdrive%\PerfLogs\Admin\PerformanceAlerts</RootPath>
<Segment>-1</Segment>
<SegmentMaxDuration>0</SegmentMaxDuration>
<SegmentMaxSize>50</SegmentMaxSize>
<SerialNumber>20</SerialNumber>
<SubdirectoryFormat>3</SubdirectoryFormat>
<SubdirectoryFormatPattern>yyyyMMdd\-NNNNNN</SubdirectoryFormatPattern>
<TaskRunAsSelf>0</TaskRunAsSelf>
<UserAccount>SYSTEM</UserAccount>
<StopOnCompletion>0</StopOnCompletion>
<AlertDataCollector>
<DataCollectorType>3</DataCollectorType>
<Name>CPU Percent</Name>
<Alert>\Processor(_Total)\% Processor Time&gt;90</Alert>
<AlertDisplayName>\Processor(_Total)\% Processor Time&gt;90</AlertDisplayName>
<EventLog>0</EventLog>
<SampleInterval>15</SampleInterval>
<Task>PerfmonDrivenTask</Task>
<TaskRunAsSelf>0</TaskRunAsSelf>
<TaskArguments>"{date}" "CPU Percent" "{value}"</TaskArguments>
</AlertDataCollector>
<AlertDataCollector>
<DataCollectorType>3</DataCollectorType>
<Name>Memory Percent</Name>
<Alert>\Memory\% Committed Bytes In Use&gt;90</Alert>
<AlertDisplayName>\Memory\% Committed Bytes In Use&gt;90</AlertDisplayName>
<EventLog>0</EventLog>
<SampleInterval>15</SampleInterval>
<Task>PerfmonDrivenTask</Task>
<TaskRunAsSelf>0</TaskRunAsSelf>
<TaskArguments>"{date}" "Memory Percent" "{value}"</TaskArguments>
</AlertDataCollector>
<AlertDataCollector>
<DataCollectorType>3</DataCollectorType>
<Name>Low Disk (C) Space</Name>
<Alert>\LogicalDisk(C:)\% Free Space&lt;10</Alert>
<AlertDisplayName>\LogicalDisk(C:)\% Free Space&lt;10</AlertDisplayName>
<EventLog>0</EventLog>
<SampleInterval>3600</SampleInterval>
<Task>PerfmonDrivenTask</Task>
<TaskRunAsSelf>0</TaskRunAsSelf>
<TaskArguments>"{date}" "C Drive Free Space Percent" "{value}"</TaskArguments>
</AlertDataCollector>
<DataManager>
<Enabled>0</Enabled>
<CheckBeforeRunning>0</CheckBeforeRunning>
<MinFreeDisk>0</MinFreeDisk>
<MaxSize>0</MaxSize>
<MaxFolderCount>0</MaxFolderCount>
<ResourcePolicy>0</ResourcePolicy>
<ReportFileName>report.html</ReportFileName>
<RuleTargetFileName>report.xml</RuleTargetFileName>
<EventsFileName>
</EventsFileName>
</DataManager>
</DataCollectorSet>
'@
$DataCollectorName = 'PerformanceAlerts'
$DataCollectorSet = New-Object COM Pla.DataCollectorSet
$DataCollectorSet.SetXml($xml)
# Quick Clean-up
$datacollectorset.Query("$DataCollectorName",$null)
$datacollectorset.stop($false)
Start-Sleep Seconds 5
$datacollectorset.Delete()
$DataCollectorSet.Commit("$DataCollectorName" , $null , 0x0003) | Out-Null
$DataCollectorSet.Start($false)
# Set the Trigger for the Data Collector to run at StartUp
$T = New-ScheduledTaskTrigger AtStartup
Set-ScheduledTask TaskPath \Microsoft\Windows\PLA\ TaskName PerformanceAlerts Trigger $T

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s