how to handle special characters ( " ) in input file logstash

Question

i'm having a problem with my data when push to ELK using logstash. here is my input file

input {
        file {
                path => ["C:/Users/HoangHiep/Desktop/test17.txt"]
                type => "_doc"
                start_position => beginning
        }
}
filter {
    dissect {
        mapping => {
            "message" => "%{word}"
        }
    }
}
output {
        elasticsearch{
                hosts => ["localhost:9200"]
                index => "test01"
        }
        stdout { codec => rubydebug}
}

My data is

"day la text"

this is the output

{
          "host" => "DESKTOP-T41GENH",
          "path" => "C:/Users/HoangHiep/Desktop/test17.txt",
    "@timestamp" => 2020-01-15T10:04:52.746Z,
      "@version" => "1",
          "type" => "_doc",
       "message" => "\"day la text\"\r",
          "word" => "\"day la text\"\r"
}

Is there any way to handle the character ( " ). i want the "word" just be like "day la text \r" don't have character \"

Thanks all.

JBone · Accepted Answer · 2020-01-16T01:24:14.077

I can explain more about this if this change works for you. The reason I say is I have newest mac so I don't see the trailing \r in my message.

the input just like you have it "day la text"

    filter {
        mutate {
            gsub => [
                 "message","(\")", ""  
        ]   
        }   
}

response is

{
    "@timestamp" => 2020-01-15T15:01:58.828Z,
      "@version" => "1",
       "headers" => {
           "http_version" => "HTTP/1.1",
         "request_method" => "POST",
            "http_accept" => "*/*",
        "accept_encoding" => "gzip, deflate",
          "postman_token" => "5ae8b2a0-2e94-433c-9ecc-e415731365b6",
          "cache_control" => "no-cache",
           "content_type" => "text/plain",
             "connection" => "keep-alive",
        "http_user_agent" => "PostmanRuntime/7.21.0",
              "http_host" => "localhost:8080",
         "content_length" => "13",
           "request_path" => "/"
    },
          "host" => "0:0:0:0:0:0:0:1",
        "message" => "day la text"   <===== see the extra inbuilt `\"` gone.
}

i do like you suggested but it doesn't work. Acctually, i want to handle quotation marks. Because the the output at the kibana have " ". Example for my data, at the kibana will show is. word : "day la text". I just want the text only => day la text. — Hoang Hiep, Jan 16 '20 at 00:27
Ok I thought you only needed to delete the last quotation mark. Let me modify the code to remove the first quotation as well — JBone, Jan 16 '20 at 00:49
see the change I made to the `gsub` block. I removed `$` to not just select the end `\"` but both — JBone, Jan 16 '20 at 01:25

how to handle special characters ( " ) in input file logstash

1 Answers1