2

I'm troubleshooting some memory issues I'm encountering while sending messages to Flume from some Java code.

The code runs two EmbeddedAgents, each with a memory channel and some sinks pointing to a remote server.

I read in Flume documentation about memory channel configurations:

byteCapacity

Maximum total bytes of memory allowed as a sum of all events in this channel. The implementation only counts the Event body, (...). Defaults to a computed value equal to 80% of the maximum memory available to the JVM (i.e. 80% of the -Xmx value passed on the command line)

From what I understand this means that in worst case scenario each channel will use 80% of Xmx so 160% in my case (2 agents) which would obviously lead to OOM errors.

Am I right?

If so, I guess it means I have to setup a proper value for this configuration (not set in my current configuration) knowing that my code is actually running two memory channels?

Edit: my question assumes that capacity (maximum number of events in the channel) settings is quite high and the limiting factor would be the size rather than number of events.

Gaël J
  • 11,274
  • 4
  • 17
  • 32
  • I do not know anything about `Flume`, but that name of "Memory Channel" makes me think this is off heap they are talking about? – Eugene Jul 28 '21 at 16:52
  • I don't think so from my observations but I sure can be wrong. – Gaël J Jul 29 '21 at 07:04

0 Answers0