0

During sending a lot of transactions using differenr rest servers (we use 2 or 4 nodes/servers) mempool always overflowing and servers starts to overload, the blocks closed for a very long time, it's depending on the amount of transaction, the more transactions the more time for closing blocks need, it always between from 20sec to 20min. For example when i send 20k txs, mempool is filled up to 20k transaction, then after 20 sec block closed with all transactions. If we send 300k transactions, mempool filled up to +-280k, and during 20 minutes closed 3 blocks with ~123, ~123, ~25 txs, but in this 20 minutes tendermint always errors like "err="pong timeout"", "err=EOF", "used of closed network connection" and so on. Also we use cosmos-sdk we can't turn off autogeneration block. https://github.com/tendermint/tendermint/issues/3905 - it is github issue, check it for more information

I tried change nodes configuration, but not much changed

I don't know what I need to do, but we have troubles with mempool and we haven't got resolve. Guys do u have some idea what can I do with this?

godvlpr
  • 141
  • 1
  • 3
  • 11

1 Answers1

0

Tendermint employs two routines p2p/conn/connection.go#recvRoutine and p2p/conn/connection.go#sendRoutine to handle incoming messages and send responses, and which are both sequential. Therefore, if one node is processing new txs and calling reactors iteratively, it is blocked before all reactors (pl reference to node/node.go#createSwitch for basic reactors).

Tendermint verifies signatures before adding txes to mempool, which takes about .5ms according to config/config.go#DefaultMempoolConfig, so verifying 20k txes literally takes about 10s not including other operation costs.

Derek Xiao
  • 1
  • 2
  • Ok, but we send a lot of transactions and blocks doesn't close before all of them will not get to the mempool (it's work uncorrectly, because blocks must be closed every 5+- second with some transactions amount). And what about situation when first node mempool fuller then mempool of the second node? I added two diagrams to the GitHub issue (in comments) yesturday, on first of them we can see that the mempool work more o less syncronyzed but mempool of first node has more transaction (stuck aboute 3k transactions) that mempool of second node. – godvlpr Aug 21 '19 at 09:22
  • Perhaps you can limit block size thus reduce the time for closing blocks. types/params.go#BlockParams – Derek Xiao Aug 21 '19 at 09:53
  • I tried this, but it didn't work, I set `MaxBlockSizeBytes = 1048576` (it's aboute 1MB), but closed blocks was bigger (aboute 1.5mb). Did I must change `MaxBlockSizeBytes` in other part of code? – godvlpr Aug 21 '19 at 12:04