6

I am running CP3.2 in a distributed mode and some of the connector which are defined even with "tasks.max": "1" have task "UNASSIGNED" state. I have increased the memory allocated to worker and restart the worker has solved me the problem or adding one more worker has solved this.

Its ok for me if "tasks.max" > 1 have some task in "UNASSIGNED" state but if I define only one task its should be in "RUNNING" state.

But I need to understand in what all condition does the task goes to "UNASSIGNED" state and how to solve this (make it running).

Regards,

Aradhya

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Renukaradhya
  • 812
  • 2
  • 19
  • 31

2 Answers2

3

A task goes into UNASSIGNED state if a successful shutdown has occurred on the worker task that has been assigned to run this connector task. This is regardless of the total number of tasks this connector is supposed to spawn (tasks.max property). You may track this in the code by following the calls to onShutdown method in AbstractHerder. Transitioning to UNASSIGNED state requires that no failure has happened or exception has been thrown by the running worker task and that normal shutdown has been triggered.

Is there a reason that your connector task might be stopping at the very start of its regular iteration loop? Can you give a bit more information? Is it a Source or a Sink?

Konstantine Karantasis
  • 1,855
  • 14
  • 15
  • Some times my Sink Connector is going to UNASSIGNED state even though worker is up and running. I felt like running more connector in less worker machine can also result in this. Currently I have 3 worker machines and running 6-7 total tasks in them. If I increase the total tasks to more than 10 I was able to see UNASSIGNED state. – Renukaradhya Apr 17 '17 at 06:00
  • Again, it will be good to know about what type of connectors are we talking about. Also, in this topic, it will help to be concrete about what a worker is. Based on your description I understand that you refer to the java process started in each node as the *worker*. That is correct. But also that differs from a worker task, which is a runnable task that is submitted to the task executor of a particular *worker*. Currently this executor is caching threads but is unbounded. Which means that a thread will executor your connector's task, regardless of the current load of the worker. – Konstantine Karantasis Apr 17 '17 at 22:36
  • Additionally, as I mentioned above, a task goes to *UNASSIGNED* state before it goes to *RUNNING* state if a shutdown was signaled right when it starts. In any case, in order to figure out whether there's an unintended state transition because of the framework there, we'll need more information about what is attempted to run on a worker. – Konstantine Karantasis Apr 17 '17 at 22:41
1

In my case my connector went to UNASSIGNED just because i ran 2 or 4 connectors parallel and while debezium start working on connectors it gets confuse and stop working on that connector i.e connector went to UNASSIGNED state in my case.

  • May be this happens just because my all connectors are collecting heavy data from different -2 databases of different location and when i check debezium log it shows that stopping connector and then finally if tolerance limit of debizium cross it stop MYSQL connection

For solving this issue i just restart docker compose or connector both help me to put my connector to running state:

docker-compose restart

or

curl -X POST localhost:8083/connectors/<connector-name>/restart

ankit
  • 2,591
  • 2
  • 29
  • 54