6

I would like to setup monitoring for system resources on my Windows servers. I've noticed a common configuration in Linux is to use collectd daemon to get system metrics information. From collectd data can read by logstash and ultimately put into Elastic Search to be viewed with Kibana.

This is nice and works well in the Linux world. However I'm stuck with Windows server and I need some advice to the best tools to achieve a similar workflow. As a side node I'm already using Nxlog to send IIS logs to logstash.

2

The 'Tools that works with Graphite' page http://graphite.readthedocs.org/en/latest/tools.html list several. I have tried the PowerShell script 'Graphite PowerShell Functions' https://github.com/MattHodge/Graphite-PowerShell-Functions and it works well.

edit I mis-read your question, you were talking only about Logstash and Kibana but not about Graphite. I don't use Logstash+Kibana for system metrics, but I use Statsd+Graphite. So not sure if my answer is valid to you but if you use the Graphite Logstash input http://logstash.net/docs/1.4.2/ you could use these tools.

  • Thanks, seems like this could be a valid and convenient solution. – angaran Dec 2 '14 at 9:42
  • Ok, so specifically we configure the Logstash wmi input after installing the contrib-plugins – CrazyPyro Apr 2 '15 at 5:35
  • And for version 1.4.2, you may need to manually comment-out a line per issue#123 if you get the error LoadError: no such file to load -- logstash/inputs/eventlog/racob_fix – CrazyPyro Apr 2 '15 at 5:51
  • @CrazyPyro if you use the specified Powershell script you don't need the wmi input of logstash. But running logstash directly on your server is in fact another solution, but it implies you need to run Java too (even if you can just copy Java binaries without installing it). – daks Apr 28 '15 at 11:54
4

Elastic now offers a tool called topbeat that does what you're looking for. It sends cpu, memory and disk stats directly into Elasticsearch or into Logstash.

Example metrics are on github at elastic/topbeat.

0

Im using Powershell 5 and Filebeat to resolve this issue. Note this has not been tested more then a few hours and its a proof of concept.

#Version 0.2

#Todo:
#* Notify on fail
#Function to get Sql Server Counters
function Get-SqlServerData()
{
    $Data = get-counter ($SqlServerCounterPrefix + ":Buffer Manager\Buffer cache hit ratio"), 
                        ($SqlServerCounterPrefix + ":Buffer Manager\Page life expectancy"), 
                        ($SqlServerCounterPrefix + ":Access Methods\Page splits/sec")
    #$Data
    $TransformedData = $Data.CounterSamples | Select-Object -Property Path, CookedValue

    $object = New-Object psobject

    $object | Add-Member -NotePropertyName 'Timestamp' -NotePropertyValue $Data.Timestamp
    foreach ($row in $TransformedData)
    {
        $path = $row.Path
        $name = $path.Substring($path.LastIndexOf("\") + 1)
        $object | Add-Member -NotePropertyName $name -NotePropertyValue $row.CookedValue
    }
    $object
}


#Parameters
$SqlServerCounterPrefix = '\MSSQL$MSSQL_2008'
$Type = "SQLServerStatistics"
$Data = Get-SqlServerData
$Path = "Z:\Temp\PowershellTest\"
$AddTimeStamp = $false
$NumberOfDaysToKeepFiles = 7
$FileExtension = "csv"

#Variables (do not change)
$Date = Get-Date -format yyyy-MM-dd
$Timestamp = Get-Date
$FilenameRegex = "^" + $Type + "_(?<Date>\d{4}-\d{2}-\d{2})(?:\(\d\))?\." + $FileExtension + "$"
$Suffix = ''
$Counter = 0
$Done = $false

if ($AddTimeStamp -eq $true)
{
    $Data | ForEach-Object {
        $_ | Add-Member -NotePropertyName 'Timestamp' -NotePropertyValue $Timestamp
    }
}

#Try to export file if it fails (the headers have changed) add a (number)
while($Done -eq $false -and $Counter -le 9)
{
    Try
    {
        $Filename = $Type + "_" + $Date + $Suffix + "." + $FileExtension
        Write-Host "Trying to write $Filename"
        $FilePath = $Path + $Filename
        $Data | Export-Csv -Path $FilePath -Delimiter ";" -NoTypeInformation -Append
        $Done = $true
    }
    Catch [Exception]
    {
        Write-Host "Failed to create file " + $_.Exception.GetType().FullName, $_.Exception.Message
        $Counter++
        $Suffix = '(' + $Counter + ')'
    }
}

#Notify if we failed
if ($Done -eq $false)
{
    #Todo: Notify that we failed
}

#Cleanup
$Files = Get-ChildItem $Path -Filter ("*." + $FileExtension)
$Files | Foreach-Object {
    $FilePath = $_.FullName
    $Filename = [System.IO.Path]::GetFileName($FilePath)

    $Match = [regex]::Match($Filename, $FilenameRegex)

    Write-Host $FilePath
    Write-Host $Filename

    if ($Match.Success -eq $true)
    {
        Write-Host $Match.Groups["Date"].Value
        $FileDate = [datetime]::ParseExact($Match.Groups["Date"].Value, "yyyy-MM-dd", $null)
        $TimeSince = New-TimeSpan -Start $FileDate -End (Get-Date).Date
        if ($TimeSince.TotalDays -ge $NumberOfDaysToKeepFiles)
        {
            Remove-Item $FilePath
        }
    }
}

This will create a csv file in the $Path directory, if you change the counters then it will create a new file with (1-9) in the name.

A updated version can be found at: https://gist.github.com/AnderssonPeter/8261e255630d6f672ba5bc80f51b017f

-1

There is now a logstash input plugin for Windows Performance Monitor.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.