0

When trying to simulate an event-stream with JMeter and use it as source on siddhi it works for a little time but ends with RAM being overused and the execution of the program stops.

I tried executing the code with a database,without a database, using a partition to get the events one by one.

This is the stream code:

@Source(type = 'http',       
        receiver.url='http://172.23.3.22:8007/insertSweetProduction',
        basic.auth.enabled='false',
        @map(type='json', @attributes( tipoDato='$.tipoDato', fecha='$.fecha', valor='$.valor', servicio='$.servicio')))
define stream insertSweetProduction (tipoDato string, fecha string, valor double, servicio string);

This is the sink stream:

@Sink(type='file',
      @map(type='json'),
      @attributes( tipoDato='$.tipoDato', fecha='$.fecha', valor='$.valor', servicio='$.servicio'),
      file.uri='/dev/null')
define stream fileSweetProduction (tipoDato string, fecha string, valor double, servicio string);

And this is the query executed to copy from one stream to another:

@info(name='query2')
from insertSweetProduction
select tipoDato,fecha,valor,servicio
insert into fileSweetProduction;

The expected results are that the wso2worker would show that all the events were processed and inserted on the sink stream. On JMeter I am simulating 1 user introducing 6000 events during 1 hour and it looks like the memory ends up overused and the simulation stops. Tried with partition and the memory usage improved a lot but still ended up in failure. All I can think is that is a coding problem but cant seem to find anything that could cause this.

//Sorry for the poor english, not my first language//

SereniFXX
  • 1
  • 1

1 Answers1

0

The recommended heap size is 2 Gb[1]. How much have you allocated to the worker profile?

[1] https://docs.wso2.com/display/SP430/Installation+Prerequisites

Niveathika
  • 1,319
  • 2
  • 8
  • 17
  • 8 GB Memory , 4 CPU – SereniFXX Apr 24 '19 at 13:46
  • Could u check the <>SP_HOME>/wso2/worker/bin/carbon.sh file? How much Xmx and Xms bein allocated to the process? We do expect around 2 GB memory usage but anything more than that is an issue. Also, does the memory usage goes down after some time or does it keep increasing? – Niveathika Apr 25 '19 at 03:45
  • -Xms256m -Xmx2G . The memory usage keeps increasing until the execution stops with an error. – SereniFXX Apr 25 '19 at 11:22
  • Is the file getting correctly added? I see that file.uri in @Sink is null? – Niveathika Apr 25 '19 at 16:08
  • By default, a heap dump should be created for OOM scenarios, in wso2/worker/logs directory. Have you examined it for any clues – Niveathika Apr 25 '19 at 16:17
  • Okey,got something. Inserting into /dev/null should not be a problem because the info just gets destroyed in there. Anyway changed the code and now Im introducing the events in a file. Now the RAM doesnt even seem to notice it and the events processed are way more than before. Thank you for your help tho! – SereniFXX Apr 26 '19 at 10:26