logstash tab separator not escaping

Question

I have tab separated data which I want to input into logstash. Here is my configuration file:

input {
    file {
        path => "/*.csv"
        type => "testSet"
        start_position => "beginning"
    }
}

filter {
    csv {
        separator => "\t"
    }
}

output {
    stdout {
        codec => rubydebug
    }
}

It simply looks for all .csv files and separates them using tabs. For an input like this:

col1    col2
data1    data2

logstash output is (for the two rows):

column1 => "col1\tcol2"
column1 => "data1\tdata2"

Obviously it is not correctly parsing it. I saw that this issue was brought up a while ago here but there was no solution. Does anyone know if this problem has been resolved or maybe there's another way to do it? Thanks!

score 35 · Accepted Answer · answered Jun 18 '15 at 16:19

35

Instead of using "\t" as the seperator, input an actual tab. like this:

filter {
   csv {
    separator => "  "
   }
}

answered Jun 18 '15 at 16:19

Roman

2,108
1
18
20

Thanks so much. Looking now, that's pretty obvious. – Noah Santacruz Jun 21 '15 at 09:09
2

Well that deserves a comment in the config file: `separator => " " # <- this white space is actually a tab, don't change it, it's already perfect` – robbmj Jan 17 '17 at 12:50

score 0 · Answer 2 · answered Jun 27 '19 at 09:42

https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-csv.html#plugins-filters-csv-separator

Define the column separator value. If this is not specified, the default is a comma ,. If you want to define a tabulation as a separator, you need to set the value to the actual tab character and not \t.

logstash tab separator not escaping

2 Answers2