1

I am using a MapR (YARN) cluster with 3 nodes. I am trying to deploy 6 Samza jobs on the cluster for some processing on data streams. All jobs are correct. I tried deploying 2-3 in parallel and they work. However when I deploy all the 6 Samza jobs in parallel I see following logs. The tasks continue to run and dont produce expected output data stream.

enter image description here

The status of the nodes on my ResourceManager web dashboard is as follows- enter image description here

Can anyone suggest how can this be resolved. I think that maybe the application does not have sufficient resources to run all of them in parallel. What change can I try?

Zeeshan
  • 1,248
  • 1
  • 12
  • 19
  • Can you show your configuration of Hadoop cluster. How many vCore, memory... Because each container need 1 core and each Samza worker need 1024mb (default) memory. – MaximeF Jun 19 '16 at 13:19

1 Answers1

0
no more pending requests in queue.

This message means that still more messages in your Kafka Topic.

MaximeF
  • 4,913
  • 4
  • 37
  • 51