Graphite with powershell

Question

I'm have an instance of graphite configured on one server, and i'm using it to monitor my environment which sadly consists of both linux and windows machines. I want monitor the well-being of my servers so i have opted for collectl on my linux machines which does a nice job collecting system stats.

Sadly for windows there don't seem to be so many solution for getting system stats and sending them to graphite, but i've manage to handle this situation with powershell. I'm using the a script which was suggested here http://josh.behrends.us/2012/07/windows-powershell-graphite-awesome/ for the graphite connection and also for getting metrics out of the computer i'm using get-counter comandlet which, surprisingly, can gather a lot of info.

My script looks like this:

while (1 -eq 1)
{
$carbonServer = "hostname"
$carbonServerPort = 2003 
$epochtime = [int][double]::Parse((Get-Date -UFormat %s))

$socket = New-Object System.Net.Sockets.TCPClient
$socket.connect($carbonServer, $carbonServerPort)
$stream = $socket.GetStream()
$writer = new-object System.IO.StreamWriter($stream)
#Write out metric to the stream.

$DiskQue =  [int]((get-counter -Counter "\PhysicalDisk(1 e: f:)\Current Disk Queue Length" ).countersamples | select -property cookedvalue).cookedvalue
$BytesReceived = [int]((get-counter -Counter "\Server\Bytes Received/sec" ).countersamples | select -property cookedvalue).cookedvalue
$BytesSent =  [int]((get-counter -Counter "\Server\Bytes Transmitted/sec").countersamples | select -property cookedvalue).cookedvalue
$MemAvail = ((get-counter -Counter "\Memory\Available Bytes").countersamples | select -property cookedvalue).cookedvalue
$MemCommit = ((get-counter -Counter "\Memory\Committed Bytes").countersamples | select -property cookedvalue).cookedvalue
$ReadOp =  [int]((get-counter -Counter "\System\File Read Operations/sec").countersamples | select -property cookedvalue).cookedvalue
$WriteOp = [int]((get-counter -Counter "\System\File Write Operations/sec").countersamples | select -property cookedvalue).cookedvalue

$metric7 = ("hostname.metrics.WriteOp " + $WriteOp + " " + $epochTime+"`n")
$metric7

$writer.WriteLine($metric7)
$writer.Flush() #Flush and write our metrics.
$writer.Close()
$stream.Close()
sleep 5
}

Now this script outputs what it's supposed to at first glance. The output has the format hostname.metrics.name $metric_value $epochTime but there are no graphs being graphed. They apear in the graphite dashboard but they are empty. I've inspected the output sent to the graphite server with wireshark. It seams that in windows the message gets appended with CRLF as opposed to linux where you only have LF. I have added \n by hand and for a small period of time it did the trick but now it stopped working.

My question is am i doing something wrong in the transmission because i've been analysing the trafic and the only difference between traffic coming form linux machines that gets graphed and from windows that doesn't get graphed is the line terminators. In linux its LF(0a) and in windows is CRLF(0d0a), but again i've tried to send from linux LFCRLF(0a0d0a) hoping the graphite server will read only until the first LF and than interepret the message but still i'm not geting graphs.

Also when I transmit from linux i have only one message, and when i transmit form powershell i have three messages. From what i've saw with strace on the carbon-cache process i have one recvfrom system call with the message i want, and the I have an other one which is empty and the the write syscall(when transmitting from powershell) as opposed yo anly one recvfrom with the message and a write(when transmitting with netcat on linux) ,

score 3 · Accepted Answer · answered Oct 30 '12 at 11:13

Seems that nobody provided any suggestions and as such i have followed another approach. I am still very curious as to why Graphite doesn't interpret my sent message. The solution to this problem was using a UDP socket instead of a TCP one and sending the metrics to statsd which in term sends them to Graphite. This seems to work without any complications, and in if you think about it it's better for your network because it minimizes traffic(if you keep statsd and Graphite on the same node).

I'm posting the script in case anyone encounters my problem and his environment resembles mine. Here is the script

[int] $Port = 8125
$IP = "here is your IP" 
$Address = [system.net.IPAddress]::Parse($IP) 
$End = New-Object System.Net.IPEndPoint $address, $port 
$Saddrf   = [System.Net.Sockets.AddressFamily]::InterNetwork 
$Stype    = [System.Net.Sockets.SocketType]::Dgram 
$Ptype    = [System.Net.Sockets.ProtocolType]::UDP 
$Sock     = New-Object System.Net.Sockets.Socket $saddrf, $stype, $ptype 
$sock.Connect($end) 

function Send_Graphite
{param ($Metric)

$Enc     = [System.Text.Encoding]::ASCII 
$Buffer  = $Enc.GetBytes($Metric) 
$Sent   = $Sock.Send($Buffer) 
"{0} characters sent to: {1} " -f $Sent,$IP 
"Message is:" 
$Metric
sleep 1

}  

while (1 -eq 1)
{
    $DiskQue =  [int]((get-counter -Counter "\PhysicalDisk(1 e: f:)\Current Disk Queue Length" ).countersamples | select -property cookedvalue).cookedvalue
    $BytesReceived = [int]((get-counter -Counter "\Server\Bytes Received/sec" ).countersamples | select -property cookedvalue).cookedvalue
    $BytesSent =  [int]((get-counter -Counter "\Server\Bytes Transmitted/sec").countersamples | select -property cookedvalue).cookedvalue
    $MemAvail = ((get-counter -Counter "\Memory\Available Bytes").countersamples | select -property cookedvalue).cookedvalue
    $MemCommit = ((get-counter -Counter "\Memory\Committed Bytes").countersamples | select -property cookedvalue).cookedvalue
    $ReadOp =  [int]((get-counter -Counter "\System\File Read Operations/sec").countersamples | select -property cookedvalue).cookedvalue
    $WriteOp = [int]((get-counter -Counter "\System\File Write Operations/sec").countersamples | select -property cookedvalue).cookedvalue

    $Message1 = "MAIN.OSmetrics.DiscQueue:"+$DiskQue+"|c"
    $Message2 = "MAIN.OSmetrics.BytesReceived:"+$BytesReceived+"|c"
    $Message3 = "MAIN.OSmetrics.BytesSent:"+$BytesSent+"|c"
    $Message4 = "MAIN.OSmetrics.MemAvail:"+$MemAvail+"|c"
    $Message5 = "MAIN.OSmetrics.MemCommit:"+$MemCommit+"|c"
    $Message6 = "MAIN.OSmetrics.ReadOp:"+$ReadOp+"|c"
    $Message7 = "MAIN.OSmetrics.WriteOp:"+$WriteOp+"|c"

    $Mesages = $Message1, $Message2, $Message3, $Message4, $Message5, $Message6, $Message7
    foreach($Message in $Mesages)
    {
        Send_Graphite $Message 
    }
}

The script is extensible and you can monitor a lot of things, just run get-counter -ListSet * and you will see. I've installed the script with TaskScheduler, and you can control the frequency with the while loop by inserting a sleep.

score 1 · Answer 2 · answered Feb 08 '14 at 21:09

I made a series of PowerShell functions which are useful for sending metrics to Graphite easily. You can configure all the Windows performance counters and how often you want them sent on to the Graphite server. Full source code is here: https://github.com/MattHodge/Graphite-PowerShell-Functions

score 0 · Answer 3 · answered May 31 '19 at 23:27

I mostly wanted to let you know that you are not alone. I ran into the same problem where charts showed up, but the data was null. I was however able to make it past the 0d0a vs 0a problem in this code. I tried UDP as well but in my case that didn't work either. Perhaps you will have some luck with TCP using this method. Cobbled this from various sources.

$dtmUTC = [int][double]::Parse((Get-Date -UFormat %s))
$CR=""
$carbonServer = "host.domain.com"
[int] $carbonServerPort = 2003

[int]$bufSize = 1472  #MTU on this network

$IP = [System.Net.Dns]::GetHostAddresses($carbonServer)
$Address = [system.net.IPAddress]::Parse($IP)
$EndPoint = New-Object System.Net.IPEndPoint $Address, $carbonServerPort
$Saddrf   = [System.Net.Sockets.AddressFamily]::InterNetwork 

#UDP
$Stype    = [System.Net.Sockets.SocketType]::Dgram 
$Ptype    = [System.Net.Sockets.ProtocolType]::UDP 

#TCP
#$Stype    = [System.Net.Sockets.SocketType]::Stream 
#$Ptype    = [System.Net.Sockets.ProtocolType]::TCP 

$Sock     = New-Object System.Net.Sockets.Socket $Saddrf, $Stype, $Ptype 
$Sock.Connect($EndPoint)

function Send_Graphite
{
    param ($thisFile)
    #encoder object
    $Enc = [System.Text.Encoding]::ASCII

    # Read the entire file
    $bytes = [System.IO.File]::ReadAllText($thisFile)

    $begin=0
    $remain=$bytes.Length
    while ( $begin -lt $bytes.Length)
    {

        if ( $remain -lt $bufsize)
        {
            $bufsize = $remain
        }

        $Chunk = $bytes.Substring($begin, $bufsize)
        $Buffer = $Enc.GetBytes($Chunk)
        $rc=$Sock.Send($Buffer)
        sleep 1
        $begin  += $bufsize
        $remain = $remain - $bufsize
    }   
} 

$M=""
foreach ($C in "server1","server2")
{   
    #echo "polling $C ===================="
    ForEach ( $Drives in Get-WmiObject Win32_LogicalDisk -ComputerName $c -Filter "DriveType=3"  )
    {
        ForEach ( $Drive in $Drives )
        {
            $D=$Drive.DeviceID.Replace(":","")
            $SZ=$Drive.Size
            $FS=$Drive.FreeSpace
            $N=$Drive.VolumeName
            $N=$N.Replace(" ","")

            $U=$SZ-$FS
            $S="itinf.stor.$C"

            if ( $FS -eq "" )
            {
                echo "$N FreeSpace was null"
            }
            if ( $N -eq "" )
            {
                $N="default"
                #echo "$N VolumeName was null"
            }
            if ($SZ -eq "")
            {
                echo "$N Size was null"
            }
            $M = $M + $CR
            $CR="`n"
            $M = $M + "$S.$N.Size $SZ $dtmUTC$CR"
            $M = $M + "$S.$N.Free $FS $dtmUTC$CR"
            $M = $M + "$S.$N.Used $U $dtmUTC$CR"
            $M = $M + "$S.$D.Size $SZ $dtmUTC$CR"
            $M = $M + "$S.$D.Free $FS $dtmUTC$CR"
            $M = $M + "$S.$D.Used $U $dtmUTC"
        }
    } 
}

Out-File -NoNewline -FilePath gph.txt -InputObject $M
Send_Graphite gph.txt
sleep .1

if ($Ptype -eq "TCP") { $Sock.Disconnect($EndPoint) }

Graphite with powershell

3 Answers3