Suricata IDS/Telegraf error: Client message is too long, disconnect him

Question

I have a problem connecting Suricata with Telegraf, using unix_stream socket:

Host: Ubuntu 20.04
Docker: SURICATA_VERSION=6.0.6
Docker: INFLUXDB_VERSION=2.1.1
Docker: TELEGRAF_VERSION=1.21

Suricata confg:

 - eve-log:
 enabled: yes
 filetype: unix_stream
 filename: /var/run/suricata/suricata-command.socket
 types:
   - stats:
       totals: no       # stats for all threads merged together
       threads: yes       # per thread stats

Telegraf config:

# Suricata stats and alerts plugin
[[inputs.suricata]]
  ## Data sink for Suricata stats and alerts logs
  # This is expected to be a filename of a
  # unix socket to be created for listening.
  source = "/tmp/suricata-command.socket"

  # Delimiter for flattening field keys, e.g. subitem "alert" of "detect"
  # becomes "detect_alert" when delimiter is "_".
  delimiter = "_"

  ## Detect alert logs
  # alerts = false

Error displayed in Suricata container log:

25/7/2022 -- 09:56:27 - <Warning> - [ERRCODE: SC_ERR_SOCKET(200)] - Write error on Unix socket "/var/run/suricata/suricata-command.socket": Broken pipe; reconnecting...
25/7/2022 -- 09:56:27 - <Notice> - Reconnected socket "/var/run/suricata/suricata-command.socket"
25/7/2022 -- 09:56:27 - <Info> - Command server: client message is too long, disconnect him.

score 0 · Answer 1 · answered Jul 26 '22 at 06:11

Update: the socket issue is solved by sharing a volume between containers:

Suricata service configuration:

  suricata:
    image: jasonish/suricata:${SURICATA_VERSION}
    container_name: suricata
    #user: root
    #profiles: ["suricata"]
    restart: on-failure
    depends_on:
      - telegraf
    env_file:
      - './suricata/env.suricata'
    network_mode: "host"
    cap_add: 
      - NET_ADMIN
      - SYS_NICE
      - NET_RAW
    volumes:
      - ./suricata/suricata.yaml:/etc/suricata/suricata.yaml
      - ./suricata/log:/var/log/suricata
      - ./suricata/rules:/var/lib/suricata/rules
      - /var/run/shared:/var/run/suricata/

Telegraf service configuration:

  telegraf:
    container_name: telegraf
    image: telegraf:${TELEGRAF_VERSION}
    user: root
   #profiles: ["telegraf"]
    networks:
      - influx
    ports:
      - 8125:8125/udp
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8086/ping"]
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      - influxdb
    env_file:
      - ./telegraf/telegraf.env
    volumes:
      - ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
      - /var/run/shared:/var/run/

The socket contained in the host (shared volume):

ubuntu@ip-172-31-31-38:~/composer-suri-tele-infl-graf$ ls -la /var/run/shared/
total 0
drwxrwxrwx  2 lxd   996   60 Jul 25 21:21 .
drwxr-xr-x 29 root root 1060 Jul 25 20:53 ..
srw-rw----  1 lxd   996    0 Jul 25 21:21 suricata-command.socket
ubuntu@ip-172-31-31-38:~/composer-suri-tele-infl-graf$

Inside Suricata container is used suricatasc tool to test the socket:

[root@ip-172-31-31-38 /]# suricatasc
Command list: shutdown, command-list, help, version, uptime, running-mode, capture-mode, conf-get, dump-counters, reload-rules, ruleset-reload-rules, ruleset-reload-nonblocking, ruleset-reload-time, ruleset-stats, ruleset-failed-rules, register-tenant-handler, unregister-tenant-handler, register-tenant, reload-tenant, unregister-tenant, add-hostbit, remove-hostbit, list-hostbit, reopen-log-files, memcap-set, memcap-show, memcap-list, dataset-add, dataset-remove, iface-stat, iface-list, iface-bypassed-stat, ebpf-bypassed-stat, quit
>>> version
Success:
"6.0.6 RELEASE"
>>> capture-mode
Success:
"AF_PACKET_DEV"
>>> iface-list
Success:
{
    "count": 1,
    "ifaces": [
        "eth0"
    ]
}
>>> iface-stat eth0
Success:
{
    "bypassed": 0,
    "drop": 0,
    "invalid-checksums": 0,
    "pkts": 9198
}
>>>

However, I still don't see suricata's metrics in Influx:

Any recommendation?

score 0 · Answer 2 · answered Jul 26 '22 at 06:58

Update. I have installed suricatasc tool on telegraf container in order to test the socket:

root@dd39f97b4f3f:/suricata/python# suricatasc
Unable to connect to socket @e_localstatedir@/suricata-command.socket: [Errno 2] No such file or directory
root@dd39f97b4f3f:/suricata/python# suricatasc /var/run/suricata-command.socket
Command list: shutdown, command-list, help, version, uptime, running-mode, capture-mode, conf-get, dump-counters, reload-rules, ruleset-reload-rules, ruleset-reload-nonblocking, ruleset-reload-time, ruleset-stats, ruleset-failed-rules, register-tenant-handler, unregister-tenant-handler, register-tenant, reload-tenant, unregister-tenant, add-hostbit, remove-hostbit, list-hostbit, reopen-log-files, memcap-set, memcap-show, memcap-list, dataset-add, dataset-remove, iface-stat, iface-list, iface-bypassed-stat, ebpf-bypassed-stat, quit
>>> iface-list
Success:
{
    "count": 1,
    "ifaces": [
        "eth0"
    ]
}
>>> iface-stat eth0
Success:
{
    "bypassed": 0,
    "drop": 11284,
    "invalid-checksums": 2,
    "pkts": 183099
}

score 0 · Accepted Answer · answered Aug 03 '22 at 12:33

The problem has been solved by the orderly deployment of the containers due to the creation of the socket between Suricata and Telegraf.

The proper deployment order would be InfluxDB, Telegraf, and Suricata.

Additionally, the permissions given to the socket should be considered.

All procedures followed have been documented in the following GitHub repository. I have also included the troubleshooting carried out.

Suricata IDS/Telegraf error: Client message is too long, disconnect him

3 Answers3