Logstash on Windows– transformation of data

Part 1: http://msprogrammer.serviciipeweb.ro/2016/12/05/logstash-on-windowsinstallation-and-io-examples/ 

Part 2 : http://msprogrammer.serviciipeweb.ro/2016/12/12/logstash-on-windows-transformation-of-data/

Now we want to use logstash for transforming data. For this , we use filter plugins to modify the data.

The process is like this:  Logstash receive the data(input plugin) , then apply a filter plugin( to parse and make new fields of data) and then sends data to output ( with an output plugin)

Let’s say we have this data that comes in a csv form,  like this:

PCName, RAM

AndreiPC, 10

OtherPC, 5

But we want to collect also from local pc ( let’s say console ) and do not put the PC name. The configuration is

input {
tcp {
    port => 9000
    type => "tcpLog"
  }
  stdin {
    type=> "console"
  }
}
filter{
    if [type] == "tcpLog" {   
        csv {
            columns => [       
                "PCName",
                "RAM"
                         ]
            add_field=>{
                "Source" => "tcp"
                }
        }
    }
    if [type] == "console" {   
        csv {
            columns => [                           
                "RAM"
                         ]
            add_field=>{
                "PCName" => "%{host}%"
                }
        }
    }
    mutate {
         convert => { "RAM" => "integer" }
        }
   
   
}
  output {
stdout {codec => rubydebug}

}

I find the configuration easy to understand – the output is a detailed json( rubydebug) and the input can be either tcp, either console.

If type is console, than a field will be add ( PCName ) .

And , at the final of the filter , the RAM field will be mutated into integer.

You can find filter plugins at https://www.elastic.co/guide/en/logstash/current/filter-plugins.html