3

I'm trying to setup Logstash because I want to forward Apache log info into Elasticsearch. And I want to start Logstash process automatically when the server start.

I could resister Logstash as a service and start the process automatically, but it seems configure file has not been read by the service. And when I start logstash using -f option it work correctly, so, I can say that contents of the config file has no problem.

It's manual says , if I place *.conf file under /etc/logstash/conf.d/ , the service read the configure file when it start, but it never work as I expected.

How should I configure logstash so that it read the my config file when it start?

this is my config file and it has been placed on /etc/logstash/conf.d/01-httpdlog.conf

input {
    file {
        path => "/etc/httpd/logs/access_log"
        start_position => beginning
    }
}
filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
}
output {
    elasticsearch {
       hosts => ["xxx.xxx.xxx.xxx:9200"]

    }
    file{
        path => "/tmp/result.txt"
    }
    stdout {}
}
Naga
  • 10,944
  • 2
  • 21
  • 38

2 Answers2

1

It's pretty certain that the user running logstash as a service doesn't have the permission to read your Apache log files.

You need to chmod 755 your Apache logs folder (in /etc/httpd/logs) and then it should work much better.

Val
  • 207,596
  • 13
  • 358
  • 360
  • when I start logstash as a service, who is the owner the process? – Naga Aug 11 '16 at 05:35
  • 1
    By default it's the `logstash` user in the `logstash` group. See your `/etc/init.d/logstash` config file. – Val Aug 11 '16 at 05:39
  • yes!, as you expected it seems logstash couldn't read the httpd log file because permission was not enough, I misunderstood that every processes which are started by service command, the owner is root but I realized that my understanding was wrong because of your comment :) – Naga Aug 11 '16 at 06:34
  • 1
    Yep, glad we figured it out!! – Val Aug 11 '16 at 06:35
  • I'm deeply appreciate for your help. – Naga Aug 11 '16 at 06:35
  • 1
    My pleasure to help! – Val Aug 11 '16 at 06:36
  • and I realized that it is important to see logstash logfile under /var/log/logstash/logstash.log to see what is happening when it start. – Naga Aug 11 '16 at 06:39
  • Yes, that's correct, you can also run logstash in `--debug` mode or `--verbose` mode in order to get more insights of what it's doing. – Val Aug 11 '16 at 06:41
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/120760/discussion-between-naggi-nagase-yoshi-and-val). – Naga Aug 12 '16 at 06:57
  • sorry, I did miss operation. I don't have farther question so far. – Naga Aug 13 '16 at 06:47
1

This worked for us as well. Checked the permissions of the folder by using the command ll and saw that the folder had root permissions. Then changed the permissions for our /etc/logstash and /usr/share/logstash folders. Used the command; sudo chown -R logstash:logstash logstash while in the /etc folder. For our /usr/share/logstash folder there was only one file that didnt have the right permissions so we used the cmd; sudo chown logstash:logstash to change the permissions for that file. Everything booted correctly when we used the sudo systemctl start logstash command after the permissions were updated.

Ryan Downey
  • 11
  • 1