I have been trying to get graphite to scale for the last few hours and for some reason it seems that carbon-relay is not passing data to my carbon cache endpoints. If i run carbon-cache on each host and send data via those ports everything works as expected, any ideas?
To discount the obvious:
- I can ssh between the hosts on the relevant ports
carbon.conf
[cache]
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2013
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
PICKLE_RECEIVER_PORT = 2014
[relay]
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2003
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
PICKLE_RECEIVER_PORT = 2004
DESTINATIONS = 1.1.1.1:2014, 2.2.2.2:2014
relay-rules.conf
default = true
destinations = 1.1.1.1:2014, 2.2.2.2:2014
On each host I do:
/opt/graphite/bin/carbon-cache.py start /opt/graphite/bin/carbon-relay.py start
Sending data to port 2013 on each host individually works just fine, meaning, that everything gets created under the whisper directory.
If I send the same data to port 2003, I see the following in the listening.log for the carbon-relay service, however I see nothing in the carbon-cache log for that server:
31/07/2014 19:18:56 :: MetricLineReceiver connection with 70.114.169.16:53421 established
31/07/2014 19:18:56 :: MetricLineReceiver connection with 70.114.169.16:53421 closed cleanly
Any help would be much appreciated.
