I had set vm_memory_high_watermark
to 0.1, and it seems also block the publishers, but the system's memory spend until it trigger OOM-killer to kill rabbitmq
.
Question:
- 1, why the
rabbitmq
occur the memory leak? - 2, how could I config
rabbitmq
auto restart while memory used reach limit? Not the periodcrontab
task or other customize method.
rabbitmq/rabbit@XXX-36.log
2022-01-08 06:54:23.034 [error] <0.20645.6235> closing AMQP connection <0.20645.6235> (xx.xx.xx.36:40530 -> xx.xx.xx.36:5672 - nova-compute:6:f8XXXXXXXXXX58):
2022-01-08 06:54:24.046 [info] <0.24196.7907> accepting AMQP connection <0.24196.7907> (xx.xx.xx.36:33010 -> xx.xx.xx.36:5672)
2022-01-08 06:54:24.047 [info] <0.24196.7907> Connection <0.24196.7907> (xx.xx.xx.36:33010 -> xx.xx.xx.36:5672) has a client-provided name: nova-compute:6:f8XXXXXXXXXX58
2022-01-08 06:54:24.048 [info] <0.24196.7907> connection <0.24196.7907> (xx.xx.xx.36:33010 -> xx.xx.xx.36:5672 - nova-compute:6:f8XXXXXXXXXX58): user 'openstack' authenticated and granted access to vhost '/'
2022-01-08 06:54:24.694 [info] <0.329.0> vm_memory_high_watermark set. Memory used:81022431232 allowed:81003041177
2022-01-08 06:54:24.694 [warning] <0.327.0> memory resource limit alarm set on node 'rabbit@node-36'.
**********************************************************
*** Publishers will be blocked until this alarm clears ***
**********************************************************
2022-01-08 06:54:37.156 [error] <0.19027.6387> closing AMQP connection <0.19027.6387> (xx.xx.xx.35:57244 -> xx.xx.xx.36:5672 - nova-compute:6:c5XXXXXXXXXX63):
2022-01-08 06:54:38.165 [info] <0.31670.8030> accepting AMQP connection <0.31670.8030> (xx.xx.xx.35:49700 -> xx.xx.xx.36:5672)
2022-01-08 06:54:38.165 [info] <0.31670.8030> Connection <0.31670.8030> (xx.xx.xx.35:49700 -> xx.xx.xx.36:5672) has a client-provided name: nova-compute:6:c5XXXXXXXXXX63
2022-01-08 06:54:39.001 [warning] lager_file_backend dropped 1 messages in the last second that exceeded the limit of 50 messages/sec
2022-01-08 06:56:58.966 [error] <0.7589.7679> closing AMQP connection <0.7589.7679> (xx.xx.xx.35:45454 -> xx.xx.xx.36:5672 - nova-compute:6:a7XXXXXXXXXXee):
2022-01-08 06:56:59.974 [info] <0.23603.1118> accepting AMQP connection <0.23603.1118> (xx.xx.xx.35:37474 -> xx.xx.xx.36:5672)