-3

I am new to this command I am sending a file data on UDP 9415 port . --> I have file which has 1000 rows --> I am firing here IP is my destination IP

cat test1 | netcat -u 172.x.x.x 9514

--> Now on destination i have syslog-ng which recieves data and store it in a folder and after step 2 I am only getting first 3 records of the file .

Does this means UDP has some limit . So I have kill the command in second step and rerun the command then again I recieve 3 records and so on .

Can someone suggest what could be an issue ?

Regards VG

user3332404
  • 411
  • 1
  • 4
  • 7
  • 1
    Why are you using UDP? You know UDP is connection-less right? So depending on the protocol that may mean everything you are sending must fit in a single datagram(packet)? – Zoredache Jun 28 '17 at 17:07
  • Also, i with UDP it is not guaranteed that the packets arrive in the same order you send them. Or that they arrive at all. – Gerald Schneider Jun 28 '17 at 17:17

1 Answers1

1

Despite the haters in the comments, UDP is awesome for certain things - such as high performance logging.

I would look at the receive buffer size - if you send a datagram larger than the buffer, it will truncate the datagram.

When receiving messages using the UDP protocol, increase the size of the UDP receive buffer on the receiver host (that is, the syslog-ng OSE server or relay receiving the messages). Note that on certain platforms, for example, on Red Hat Enterprise Linux 5, even low message load (~200 messages per second) can result in message loss, unless the so_rcvbuf() option of the source is increased. In such cases, you will need to increase the net.core.rmem_max parameter of the host (for example, to 1024000), but do not modify net.core.rmem_default parameter.

As a general rule, increase the so_rcvbuf() so that the buffer size in kilobytes is higher than the rate of incoming messages per second. For example, to receive 2000 messages per second, set the so_rcvbuf() at least to 2 097 152 bytes.

(source: https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guides/en/syslog-ng-ose-v3.3-guide-admin-en/html/reference_source_tcpudp.html)

I also haven't used nc for UDP specifically, so I'm not sure if it does any modification of the data. You could use tcpdump -xvv dst port 9514 to validate what the UDP datagrams actually are being sent as. Typically, I just pipe directly to: tail -f -n0 /var/log/nodejs/log.log > /dev/udp/serverhostnamehere/9996

Community
  • 1
Brennen Smith
  • 1,742
  • 8
  • 11
  • Thanks Smith, I like the way you elaborate the answer thanks for that . So any idea how I can increase the buffer size on Ubuntu . I mean is there any direct command or any file where I can modify the settings . – user3332404 Jun 28 '17 at 19:22
  • Could do something like: `input(type="imudp" port="9514" rcvbufSize="256M")` for rsyslog – Brennen Smith Jun 28 '17 at 20:39