Hyperledger Fabric v2.1: Fabcar with asLocalhost set to false in Gateway connection options

Question

I am trying to get a multimachine network of Hyperledger Fabric running. I encountered some errors. I was able to reproduce the same errors on a single machine in the Fabcar example of Fabric v2.1 by changing one line in fabric-samples/fabcar/javascript/query.js. I changed the line

await gateway.connect(ccp, { wallet, identity: 'appUser', discovery: { enabled: true, asLocalhost: true } });

to the line

await gateway.connect(ccp, { wallet, identity: 'appUser', discovery: { enabled: true, asLocalhost: false } });

So I am setting discovery.asLocalhost to false instead of true. When I run node query.js in the fabric-samples/fabcar/javascript/ directory. I get the following errors.

Wallet path: /home/userName/my/code/fabric-samples/fabcar/javascript/wallet
2020-10-23T06:09:56.505Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: orderer.example.com:7050, url:grpcs://orderer.example.com:7050
2020-10-23T06:09:56.507Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server orderer.example.com:7050 url:grpcs://orderer.example.com:7050 timeout:3000
2020-10-23T06:09:56.508Z - error: [DiscoveryService]: _buildOrderer[mychannel] - Unable to connect to the discovered orderer orderer.example.com:7050 due to Error: Failed to connect before the deadline on Committer- name: orderer.example.com:7050, url:grpcs://orderer.example.com:7050
2020-10-23T06:09:59.522Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com:7051, url:grpcs://peer0.org1.example.com:7051
2020-10-23T06:09:59.523Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com:7051 url:grpcs://peer0.org1.example.com:7051 timeout:3000
2020-10-23T06:09:59.523Z - error: [DiscoveryService]: _buildPeer[mychannel] - Unable to connect to the discovered peer peer0.org1.example.com:7051 due to Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com:7051, url:grpcs://peer0.org1.example.com:7051
2020-10-23T06:10:02.528Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org2.example.com:9051, url:grpcs://peer0.org2.example.com:9051
2020-10-23T06:10:02.528Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org2.example.com:9051 url:grpcs://peer0.org2.example.com:9051 timeout:3000
2020-10-23T06:10:02.529Z - error: [DiscoveryService]: _buildPeer[mychannel] - Unable to connect to the discovered peer peer0.org2.example.com:9051 due to Error: Failed to connect before the deadline on Endorser- name: peer0.org2.example.com:9051, url:grpcs://peer0.org2.example.com:9051
2020-10-23T06:10:02.564Z - error: [SingleQueryHandler]: evaluate: message=Query failed. Errors: [], stack=FabricError: Query failed. Errors: []
    at SingleQueryHandler.evaluate (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/node_modules/fabric-network/lib/impl/query/singlequeryhandler.js:45:23)
    at Transaction.evaluate (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/node_modules/fabric-network/lib/transaction.js:287:49)
    at Contract.evaluateTransaction (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/node_modules/fabric-network/lib/contract.js:115:45)
    at main (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/query.js:46:39)
    at processTicksAndRejections (internal/process/task_queues.js:85:5), name=FabricError
Failed to evaluate transaction: FabricError: Query failed. Errors: []

I followed the instructions in the Fabcar tutorial. Here are the steps to reproduce the error.

cd fabric-samples/fabcar
./startFabric.sh javascript
cd javascript
npm install
node enrollAdmin.js
node registerUser.js
[Change the line in query.js]
node query.js

If I can figure out how/where to specify the grpcs URLs, I think I can get my multimachine network to work. Any help is appreciated.

More detailed error logs can be obtained by setting export GRPC_TRACE=all and export GRPC_VERBOSITY=DEBUG as suggested by this answer.

score 3 · Accepted Answer · answered Oct 23 '20 at 10:08

In the single machine case your client application is running on your local machine but the nodes (peers and orderers) are running in Docker containers on your local machine, and have hostnames like peer0.org1.example.com.

Within the Docker network the nodes can talk to each other using their hostnames, e.g. peer0.org1.example.com. Your client (outside the Docker network) can't talk to peer0.org1.example.com because that DNS name does not exist. Instead it needs to connect to localhost on ports mapped (by Docker) to/from specific hosts/ports within the Docker network.

Your client application gets node endpoints in two ways:

Entries in its local connection profile.
Network topology returned by service discovery.

You can set appropriate (localhost) endpoint URLs in your connection profile. However, the endpoint URLs returned by discovery will be the ones exposed within the Docker network, e.g. peer0.org1.example.com. To facilitate this scenario, the SDK provides the discovery.asLocalhost setting which, when enabled, maps all endpoint addresses returned by discovery to the same port at localhost. An alternative approach is to add entries to your local hosts file that map the node names to localhost.

You should not have the discovery.asLocalhost setting enabled if your nodes are accessible on the real network using their configured hostnames. These hostnames must be resolvable in DNS (so for a real deployment you can't use non-resolvable addresses like example.com), and must match your Fabric network configuration (and server certificate details if using TLS).

After I posted the question, I noticed a discussion in the #fabric-quesions channel of Hyperledger Chat between users iambatman and aim8526. The latter suggested setting the CORE_PEER_GOSSIP_EXTERNALENDPOINT variable in the docker compose files. I tried that and it reduced the errors. See commit and comment for the reduced errors at https://github.com/avras/fabric-samples-temp/commit/71c3ceaa135f64ff8be439639a336c56b9516540 I tried setting grpcs url in the local connection profile https://gist.github.com/avras/aef24cbf7a3029a4176bd8ba4bfc99f2 But that didn't work. Am I missing fields? — sarva, Oct 23 '20 at 11:03
I missed your suggestion about adding a mapping to the `/etc/hosts` file. The following entry in the /etc/hosts file fixed the error I was getting after the change to the CORE_PEER_GOSSIP_EXTERNALENDPOINT variable. `127.0.0.1 orderer.example.com` Thanks! — sarva, Oct 23 '20 at 11:15
Adding entries `127.0.0.1 peer0.org1.example.com` and `127.0.0.1 peer0.org2.example.com` to `/etc/hosts/` makes its unnecessary to change the CORE_PEER_GOSSIP_EXTERNALENDPOINT variable in the docker compose files. Mentioning it here just in case someone else faces a similar issue. — sarva, Oct 23 '20 at 11:40

score 0 · Answer 2 · answered Oct 23 '20 at 14:24

For setting up the multiMachine network. The changes you have to make is on the config.json and connection.yaml ( basically connection profiles).

await gateway.connect(ccp, { wallet, identity: 'appUser', discovery: { enabled: true, asLocalhost: false } });

The localhost Property should be set as "false" as you are going to setup a multi-host Blockchain Network. You can create an overlay network to connect to it.

    ---
  name: mychannel.firstnetwork.connectionprofile
  x-type: "hlfv1"
  description: "connection profile for 2.1  network"
  version: "1.0"
  
  channels:
    mychannel:
      orderers:
        - orderer.example.com
      peers:
        peer0.org1.example.com:
          endorsingPeer: true
          chaincodeQuery: true
          ledgerQuery: true
          eventSource: true
        peer0.org2.example.com:
          endorsingPeer: true
          chaincodeQuery: true
          ledgerQuery: true
          eventSource: true
  
  organizations:
    Org1:
      mspid: Org1MSP
      peers:
        - peer0.org1.example.com
      certificateAuthorities:
        - ca.org1.example.com
      adminPrivateKey:
        path:  ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore/8717c2e82c3c4caff76fca964bb70_sk
      signedCert:
        path:  ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/signcerts/cert.pem
    
    Org2:
      mspid: Org2MSP
      peers:
        - peer0.org2.example.com
      certificateAuthorities:
        - ca.org2.example.com
      adminPrivateKey:
        path:  ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/keystore/9592495c719c2e87fc4a8_sk
      signedCert:
        path:  ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp/signcerts/cert.pem
  orderers:
    orderer.example.com:
      url: grpcs://orderer.example.com:7050
      grpcOptions:
       ssl-target-name-override: orderer.example.com
      tlsCACerts:
        path:  ./caro-blockchain-commonfiles/config/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
  
  peers:
    peer0.org1.example.com:
      url: grpcs://peer0.org1.example.com:7051
      grpcOptions:
        ssl-target-name-override: peer0.org1.example.com
      tlsCACerts:
        path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp/cacerts/localhost-7054-ca-org1.pem
  
   
    peer0.org2.example.com:
      url: grpcs://peer0.org2.example.com:9051
      grpcOptions:
        ssl-target-name-override: peer0.org2.example.com
      tlsCACerts:
        path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/msp/cacerts/localhost-8054-ca-org2.pem
  
   
  
  certificateAuthorities:
    ca.org1.example.com:
      url: https://ca.org1.example.com:7054
      httpOptions:
        verify: false
      tlsCACerts:
        path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/ca/ca.org1.example.com-cert.pem
      registrar:
        - enrollId: admin
          enrollSecret: adminpw
      caName: ca-org1
    ca.org2.example.com:
      url: https://ca.org2.example.com:8054
      httpOptions:
        verify: false
      tlsCACerts:
        path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/ca/ca.org2.example.com-cert.pem
      registrar:
        - enrollId: admin
          enrollSecret: adminpw
      caName: ca-org2

The URLs property here like grpcs://peer0.org1.example.com:7051 should be changed from hostname to the IP of the other machine to VM ( grpcs://XX.XX.XX.XX:7051)where peer container is running.

Thanks Shubham for posting this connection profile. I will try it out. One question about the secrets from both organization being referenced. Is this normal? For example, the `adminPrivateKey` from both organizations are available . So if the node.js application is under the control of org1, will they need access to the secrets of org2? I guess for a demo it wouldn't matter. — sarva, Oct 23 '20 at 17:50
Normally in the case of multi organisation setup we will have node.js application running on each organisation and we configure the network-profile.json file to include only the attributes of that organisation which is running the application. So we need not keep the adminPrivateKey of another organisation in our network-profile. — GPC, Oct 28 '20 at 13:30

Hyperledger Fabric v2.1: Fabcar with asLocalhost set to false in Gateway connection options

2 Answers2