(cassandra noob) I have been doing some basic config/perf tests to and try an get a better understanding of how cassandra works, and was a bit suprised at the results.
My tests essentially do: (multi-threaded (each thread simply doing an insert), 50k messages total).
Test 1: Single-Node Only (no cluster) Insert with Consistency=1. (note: test is run on this host, pointing to the local cassandra)
Result: tps=5000tps.
For Test 2,3 we using 4 Datacenters, (1 node in each), where the table is configured USING REPLICATION = 4
** Tests 2/3 (the client is again running directly on one of the nodes with Cassandra installed and is pointing directly at that localhost cassandra).
Test 2: Insert with Consistency=1
Result: tps = 423
Test 3: Insert with Consistency=4
Result: tps = 149
Originally I was expecting Test 2 to perform in a similar fashion to Test 1, operating under the assumption that: the insert would block until the insert was written to the commit log of the local cassandra node receiving the request.
Seeing that the performance is vastly slower, I can only conclude that when the insert is received by the local node, it computes a hash and this hash/partition may in fact be hosted by another Cassandra Node as primary? Even though it will eventually be 'replicated' 'back' to the local node since ConsistencyFactor==Number of nodes in cluster), which is causing the insert to block until a remote node has committed the insert to the commit log.
Is this correct? If yes..is there a way to 'change' this behavior? i.e. if all nodes in the Cluster have a copy, then when an insert is received with C=1, ensure the receiving node commits it locally, rather then delegating it out?
Note: for my tests I am using the java DataStax 2 driver (could it be this driver, even though when I connecting explicitly to the local node is in fact creating connections to the entire cluster, and dispatching requests to all nodes?)
Thanks Dano
