0

I am trying to configure sending pgsql log data to elasticsearch. The architecture is like this: packetbeat -> logstash -> elasticsearch.

After running packetbeat, logstash log says:

:message "Beats Input: Remote connection closed", 
:peer=>"10.1.1.103:51748", 
:exception=>#Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: EOFError, End of file reached, 
:level=>:warn}

packetbeat log output looks normally:

logstash.go:106: INFO Max Retries set to: 3
outputs.go:119: INFO Activated logstash as output plugin.
publish.go:288: INFO Publisher name: pgsqldb01
async.go:78: INFO Flush Interval set to: 1s
async.go:84: INFO Max Bulk Size set to: 2048
beat.go:147: INFO Init Beat: packetbeat; Version: 1.1.0
procs.go:88: INFO Process matching enabled

packetbeat.yml contains:

interfaces:
  device: any
  buffer_size_mb: 100
protocols:
  pgsql:
    ports: [5432]
procs:
  enabled: true
  monitored:
    - process: pgsql
      cmdline_grep: postgres
output:
  logstash:
    hosts: ["10.1.1.1:5044"]
    bulk_max_size: 2048
    index: packetbeat
    tls:
      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

logstash input conf looks like:

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}

packetbeat version is: 1.1.0

logstash/elasticsearch version is: 2.2.0

What could be the problem?

Kadir
  • 126
  • 1
  • 8
  • Can you post your LS config, or at least the `input` section? – GregL Feb 11 '16 at 14:17
  • Also, other than the errors in the LS log file, what exactly is the problem you're having? – GregL Feb 11 '16 at 14:18
  • Updated the question. I also have a filebeat configuration which is running normally. What i expected that Kibana sould display some packetbeat indexes. – Kadir Feb 11 '16 at 14:59
  • Do you actually *have* any `packetbeat` indices in ES? – GregL Feb 11 '16 at 16:30
  • No I don't have any. – Kadir Feb 11 '16 at 16:33
  • Does it imply there are no transactions running on pgsql server? – Kadir Feb 11 '16 at 16:34
  • Not sure. You'd need to do more testing to find out: 1) try running a dummy LS instance which outputs to `stdout` and pointing Packetbeat at it; 2) check `tcpdump` output for relevant traffic on the servers. – GregL Feb 11 '16 at 16:41

0 Answers0