1

I'm following the instructions from this link to write apache access logs to mongo db- http://docs.fluentd.org/articles/apache-to-mongodb.

But nothing is getting written to mongo DB. I'm checking using db["access"].findOne();.

Contents of my td-agent.conf file are as follows-

<source>
  type tail
  format apache2
  path /var/log/httpd/access_log
  pos_file /var/log/td-agent/apache2.access_log.pos
  tag mongo.apache.access
</source>

<match mongo.*.*>
  # plugin type
  type mongo

  # mongodb db + collection
  database apache
  collection access

  # mongodb host + port
  host localhost
  port 27017

  # interval
  flush_interval 10s

  # make sure to include the time key
  include_time_key true
</match>

I know that td-agent is looking at this config file because /etc/init.d/td-agent start fails if there are errors in the conf file.

Contents of the mentioned pos_file are as follows-

/var/log/httpd/access_log   0000000000000000    00000000

I'm looking for help on why this may not be working. Is there anything that I'm missing?

Update 1: If I add a match for the above source (apache access log) that writes to stdout, I'm able to see the events being logged to td-agent's log. So it appears to be a problem writing to mongodb. I'm able to connect to mongo from command line using the mongo command.

Update 2: All of a sudden this has started to work and I can't explain why. I just changed the td-agent config to match the output to stdout and back. Now I don't know what was wrong or why it started to work. :(

Thanks,

ksrini
  • 121
  • 5

1 Answers1

1

It turned out to be a permission issue. The reason it worked later was that I had changed td-agent init.d script to run the process as root. By default it runs as user td-agent and in that case it complains that it doesn't have permission to access apache access log file even thought the log file has permissions 666.

ksrini
  • 121
  • 5