0

I executed Hyperledger Fabric test network and it had no errors. When I benchmarked its performance with Hyperledger Caliper tool, it was successful. But when the transactions count is increased to 800 or 1000 or above it throws the below error,

2023.02.13-11:49:56.059 error [caliper] [connectors/peer-gateway/PeerGateway]   Failed to perform submit transaction [Admin_createPatient] using arguments [Client3_CAR557,Toyota,S,brown,Shotaro,29,123456890],  with error: EndorseError: 2 UNKNOWN: too many requests for /gateway.Gateway, exceeding concurrency limit (500)

To rectify the error I changed the value of gatewayService to 2500 from its default value 500, but still the error not rectified.

limits:

    concurrency:
        # endorserService limits concurrent requests to endorser service that handles chaincode deployment, query and invocation,
        # including both user chaincodes and system chaincodes.
        endorserService: 2500
        # deliverService limits concurrent event listeners registered to deliver service for blocks and transaction events.
        deliverService: 2500
        # gatewayService limits concurrent requests to gateway service that handles the submission and evaluation of transactions.
        gatewayService: 2500
Gowthami
  • 1
  • 2

1 Answers1

0

In the error message I assume that it was updated to say 2500 ? If not then the change wasn't picked up. If it was picked up then you will need to increase the value further. The number corresponds to in flight transactions, so you can only have that many transactions submitted and not committed being processed. This should track with the unfinished transaction count that caliper outputs during a benchmark run.

david_k
  • 5,843
  • 2
  • 9
  • 16
  • Even if the value of gateway service in config.yaml is updated to 25,000, the change is not reflected and the same error is thrown. Is there any other way to update the gateway concurrency limit value of hyperledger fabric? – Gowthami Feb 13 '23 at 11:38
  • If the peer is still reporting 500 then it doesn't see the change made to core.yaml. So you need to make sure that the peer is really reading the core.yaml file you have changed. Also the peer will only read the change at startup so you need to restart the peer – david_k Feb 13 '23 at 13:00
  • I have configured the FABRIC_CFG_PATH to point to the core.yaml but the change is not reflected. Even I have used the environment variable CORE_PEER_LIMITS_CONCURRENCY_GATEWAYSERVICE to override the value but nothing gets reflected while using Caliper. – Gowthami Feb 16 '23 at 09:13
  • Sorry it's nothing to do with Caliper, if it still reports the limit as 500 then it's definitely an issue with how you are running the peer. Are you running the peer as a docker container via docker-compose or swarm ? are you running the peer natively ? are you running the peer as a container in a K8s environment ? Are you running on linux/windows/Mac (m1 or x86) ? – david_k Feb 16 '23 at 09:29
  • Yah It's a problem associated with Fabric not with Caliper. I am using Docker via docker-compose in Ubuntu 18.04.6. I am troubling to find out the exact location from where the fabric is picking up the core.yaml – Gowthami Feb 16 '23 at 10:21
  • If you are using docker/docker-compose then you should be able to add CORE_PEER_LIMITS_CONCURRENCY_GATEWAYSERVICE to the environment section of your peer service to override the value stored in core.yaml. See the testnetwork as an example https://github.com/hyperledger/fabric-samples/blob/c723d812e9bf0ef12ddc0a099805aae94e10d6af/test-network/compose/compose-test-net.yaml#L61 – david_k Feb 16 '23 at 12:06
  • Yahhh I got the value overrided. Still the time got exceeded when I try to submit larger transactions. error [caliper] [connectors/peer-gateway/PeerGateway] Failed to perform submit transaction [Admin_deletePatient] using arguments [Client4_CAR997], with error: CommitStatusError: 4 DEADLINE_EXCEEDED: Deadline exceeded.... Which value have to be modified to extend this time? – Gowthami Feb 17 '23 at 05:04
  • I suggest you create another stackoverflow question or ask on discord. One thing I will point out for benchmarking. Don't try to run caliper on the same machine as your fabric network as they will contend for the limited resources on that machine. 2 Cal;iper requires more CPU/Memory usually than the network it's trying to benchmark. – david_k Feb 17 '23 at 14:36