1

Graphite-0.9.15

Python client written in python-3.7 connecting directly to Carbon.

I am following the docs here:

https://graphite.readthedocs.io/en/latest/feeding-carbon.html

Once you’ve formed a list of sufficient size (don’t go too big!), and pickled it (if your client is running a more recent version of python than your server, you may need to specify the protocol) send the data over a socket to Carbon’s pickle receiver (by default, port 2004).

payload = pickle.dumps(listOfMetricTuples, protocol=2)
header = struct.pack("!L", len(payload))
message = header + payload

How big is "too big"?

When using the pickle protocol, what is the maximum payload and/or message size in terms of:

  • number of tuples
  • actual byte size uncompressed
  • actual byte size compressed with protocol=2

What are the limits here on the server side?

Florin Andrei
  • 1,067
  • 3
  • 11
  • 33

1 Answers1

2

Ran into this a couple of years after you. Best as I can tell by trial and error, the maximum payload size from your example is 1048703 which happens to be 1MiB plus 127 bytes. I did not encounter a limit to the number of tuples. It would be nice if the actual limits were documented, but it is easy enough to send batches of tuples once you know that you need to.

Awful
  • 21
  • 3