0

I see there are a lot of questions about this error, I have seen this solution Raft bad format but I doubled checked and the folders are right and the certs are in there, I also looked at Sans problem but for what I understand I don't need Sans when using Raft (I may be wrong). I think my problem its because I'm not handling the intermediate certificates correctly and I'm getting the error both creating a channel and in the Raft consensus.

So here is what I've done so far:

I created my genesis block using a configtx.yaml and this msp folder structure:

configtx.yaml

    Organizations:
    - &ordererOrg
        Name: orderer
        ID: orderer
        MSPDir: /crypto/msp
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('orderer.member')"
            Writers:
                Type: Signature
                Rule: "OR('orderer.member')"
            Admins:
                Type: Signature
                Rule: "OR('orderer.admin')" 
    Capabilities:
        Channel: &ChannelCapabilities
            V1_4_3: true
        Orderer: &OrdererCapabilities
            V1_4_2: true
        Application: &ApplicationCapabilities
            V1_4_2: true
    Application: &ApplicationDefaults
        Organizations:
        Policies:
            Readers:
                Type: ImplicitMeta
                Rule: "ANY Readers"
            Writers:
                Type: ImplicitMeta
                Rule: "ANY Writers"
            Admins:
                Type: ImplicitMeta
                Rule: "MAJORITY Admins"
        Capabilities:
            <<: *ApplicationCapabilities
    Orderer: &OrdererDefaults
        OrdererType: solo
        BatchTimeout: 2s
        BatchSize:
            MaxMessageCount: 10
            AbsoluteMaxBytes: 99 MB
            PreferredMaxBytes: 512 KB
        Kafka:
            Brokers:
                - 127.0.0.1:9092
        Organizations:
        Policies:
            Readers:
                Type: ImplicitMeta
                Rule: "ANY Readers"
            Writers:
                Type: ImplicitMeta
                Rule: "ANY Writers"
            Admins:
                Type: ImplicitMeta
                Rule: "MAJORITY Admins"
            BlockValidation:
                Type: ImplicitMeta
                Rule: "ANY Writers"
    Channel: &ChannelDefaults
        Policies:
            Readers:
                Type: ImplicitMeta
                Rule: "ANY Readers"
            Writers:
                Type: ImplicitMeta
                Rule: "ANY Writers"
            Admins:
                Type: ImplicitMeta
                Rule: "MAJORITY Admins"
        Capabilities:
            <<: *ChannelCapabilities
    Profiles:
    SampleEtcdRaftProfile:
        <<: *ChannelDefaults
        Capabilities:
            <<: *ChannelCapabilities
        Orderer:
            <<: *OrdererDefaults
            OrdererType: etcdraft
            Addresses:
                - orderer1.xxxx.eastus.aksapp.io:443
                - orderer2.xxxx.eastus.aksapp.io:443
            Organizations:
            - *ordererOrg
            EtcdRaft:
                Consenters:
                    - Host: orderer1
                    Port: 7050
                    ClientTLSCert: /crypto/orderers/orderer1/tls/server.crt
                    ServerTLSCert: /crypto/orderers/orderer1/tls/server.crt
                    - Host: orderer2
                    Port: 7050
                    ClientTLSCert: /crypto/orderers/orderer2/tls/server.crt
                    ServerTLSCert: /crypto/orderers/orderer2/tls/server.crt
            Capabilities:
                <<: *OrdererCapabilities
        Application:
            <<: *ApplicationDefaults
            Organizations:
                - <<: *ordererOrg
        Consortiums:
        SampleConsortium:
            Organizations:
                - *ordererOrg

MSP folder structure:

+ /crypto
  configtx.yaml
  + msp
    + cacerts > ca.crt
    + tlscacerts > ca.crt
    + intermediatecerts > intermediate.crt
    + tlsintermediatecerts > intermediate.crt
    + admincerts > admin.crt
  + orderers
    + orderer1/tls > server.crt
    + orderer2/tls > server.crt

I created my genesis block using this:

configtxgen -profile SampleEtcdRaftProfile -outputBlock genesis.block -channelID mychannel

Now here I have a doubt inside my orderer the msp structure is like this:

+ /var/hyperledger/orderer
  genesis.block
  + msp
    + cacerts > ca.crt
    + intermediatecerts > intermediate.crt
    + admincerts > admin.crt
    + signcerts > cert.pem
    + keystore > key.pem
  + tls
    server.crt
    server.key
    ca.crt
    intermediate.crt

And these are my env variables:

ORDERER_GENERAL_TLS_ENABLED=true
ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
ORDERER_GENERAL_TLS_CLIENTROOTCAS=/var/hyperledger/orderer/tls/ca.crt
ORDERER_GENERAL_TLS_CLIENTAUTHREQUIRED=false
ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key
ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt
ORDERER_GENERAL_CLUSTER_ROOTCAS=/var/hyperledger/orderer/tls/ca.crt

I'm not sure why the structure is different and the tls files are somewhere else but I am copying the configuration from the azure hyperledger template That I have already used successfuly.

Now my orderers are running but orderer1 keeps starting a new election and orderer 2 becomes precandidate and finally fails with a TLS handshake error.

These are the error logs in orderer2:

2021-03-23 22:15:21.969 UTC [orderer.consensus.etcdraft] Step -> INFO f96 2 is starting a new election at term 1 channel=canalenergia node=2
2021-03-23 22:15:21.969 UTC [orderer.consensus.etcdraft] becomePreCandidate -> INFO f97 2 became pre-candidate at term 1 channel=canalenergia node=2
2021-03-23 22:15:21.969 UTC [orderer.consensus.etcdraft] poll -> INFO f98 2 received MsgPreVoteResp from 2 at term 1 channel=canalenergia node=2
2021-03-23 22:15:21.969 UTC [orderer.consensus.etcdraft] campaign -> INFO f99 2 [logterm: 1, index: 2] sent MsgPreVote request to 1 at term 1 channel=canalenergia node=2
2021-03-23 22:15:26.673 UTC [core.comm] ServerHandshake -> ERRO f9a TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=x.x.x.x:45472

I tried deleting intermediate.crt and mixing ca.crt and intermediate.crt into one file in ca.crt in the tls folder of the orderer like this:

-----BEGIN CERTIFICATE-----
ROOTCERTxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
INTERMEDIATExxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END CERTIFICATE-----

But it didn't work either.

I tried openssl verify -CAfile chain.crt orderer1-tls.crt and returns OK.

This is what happens when I try to create a new channel:

peer channel create -o orderer1.xxxx.eastus.aksapp.io -c testchannel -f ./channel.tx --tls --cafile /var/hyperledger/peer/msp/tlscacerts/ca.crt --clientauth --certfile /var/hyperledger/peer/tls/cert.pem --keyfile /var/hyperledger/peer/tls/key.pem
2021-03-24 00:04:40.331 UTC [comm.tls] ClientHandshake -> ERRO 001 Client TLS handshake failed after 939.077µs with error: EOF remoteaddress=x.x.x.x:443

I tested my urls with telnet and they are ok.

I created my certificates using openSSL but I don't see anything wrong in them, the only difference is that they aren't signed by a fabric-ca but by an intermediate CA from a big company.

I have double checked all the values but I guess orderer wouldn't even be running if they weren't right and followed this script from azure for the creation of the genesis block only adding the intermediate info.

Any advice would be great.

Thanks

UPDATE:

I activated the debug logs with this variable:

FABRIC_LOGGING_SPEC="grpc=debug:info"

And found the problem is this:

transport: authentication handshake failed: x509: certificate is not valid for any names, but wanted to match orderer1

My certificate has this subject:

CN=orderer1-tls@blockchain.company.com,O=Company,L=CITY,ST=STATE,C=US

Now, I don't understand why its telling me it doesn't have a name, I though the CN orderer1-tls@blockchain.company.com was the name, and, also, where did I tell the orderer that the name to search is "orderer1"?

UPDATE 2:

I changed my TLS certificates to CN=orderer.company.com and then the error was this:

x509: certificate is valid for orderer1.company.com, not orderer1

So as 李可以 says, the orderer is expecting the hostname in the certificate CN and my hostname is orderer1 so I changed it to that.

Now I'm getting a new error:

UTC [comm.grpc.server] 1 -> INFO 118 streaming call completed grpc.service=orderer.Cluster grpc.method=Step grpc.peer_address=x.x.x.x:39424 error="no TLS certificate sent" grpc.code=Unknown grpc.call_duration=161.713µs

I guess This is a new error so I'm going to open a new question. thanks!

Ana Franco
  • 1,611
  • 3
  • 24
  • 43

2 Answers2

1

Ana

I have same problem when i study fabric.and i have solve them,hope this can help you.

for example,when you execute in linux terminal

export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/home/www/byfn-on-k8s/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/home/www/byfn-on-k8s/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=peer0.org1.example.com:30011

peer channe list

you will get correct result

and change CORE_PEER_ADDRESS to exmaple.com(example.com link same ip to peer0.org1.example.com,you can setup by edit /etc/hosts),

export CORE_PEER_ADDRESS=example.com:30011

peer channe list

and you will get error "TLS handshake failed with error remote error: tls: bad certificate server=PeerServer"in peer log

but this is not the only scene when you meet error "tls: bad certificate",

and i think this error is caused by the "hostname vertify"

for instance,you want to access peer peer0.org1.example.com,and this peer enable server tls,you can find the server.crt and server.key in peer env.

if you parse the server.crt,you will find the CN of this crt is "peer0.org1.example.com"

when you contact to peer "peer0.org1.example.com", the peer will send you its cert,and you find the CN of th cert is "peer0.org1.example.com",so you trust this server,

but when you contact to "example.com" (point to same IP with peer0.org1.example.com),and the peer send you its cert ,you find the CN of the cert is "peer0.org1.example.com" ,id not equal "example.com",so you dont trust this server and get error.

Li Xian
  • 421
  • 4
  • 4
  • Thank you 李可以 I don't see how to disable the hostname verify but I guess its a good thing and as for now I can't change my hostname I changed the certificates and it worked. – Ana Franco Mar 24 '21 at 21:23
  • when you use enable tls on server side,you can't disable hostname vertify,but you can slove "tls:bad certificate" by these :1.change the CN of your server.crt.2.change the server name which you are contcat to match the CN of your server.crt.3.disable tls on your server side.about hostname vertify you can see https://medium.com/@technospace/host-name-verification-failed-for-host-5233ab2c9e6d – Li Xian Mar 25 '21 at 01:05
1

I think the new error "no TLS certificate sent" was caused by you have set CORE_PEER_TLS_CLIENTAUTHREQUIRED=true on your orderer env.

so I try to test when CORE_PEER_TLS_CLIENTAUTHREQUIRED=true,I meet another error "tls:bad certificate" when raft elect,so I change the orderer env like these:

 - ORDERER_GENERAL_TLS_ENABLED=true
 - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
 - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
 - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
 - ORDERER_KAFKA_TOPIC_REPLICATIONFACTOR=1
 - ORDERER_KAFKA_VERBOSE=true
 - ORDERER_GENERAL_TLS_CLIENTAUTHREQUIRED=true
 - ORDERER_GENERAL_TLS_CLIENTROOTCAS=/var/hyperledger/orderer/tls/ca.crt
 - ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt
 - ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key
 - ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]

and there are no error during elect,but when I try to create channel,I did't set authclient

peer channel create -o orderer.example.com:7050 -c mychannel -f ./channel-artifacts/channel.tx --tls true --cafile /root/go/src/github.com/hyperledger/fabric-samples/first-network/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem

I did't set authclient,I meet another error

TLS handshake failed with error tls: client didn't provide a certificate server=Orderer remoteaddress=192.168.192.11:57372

so I change my command

peer channel create -o orderer.example.com:7050 -c mychannel -f ./channel-artifacts/channel.tx --tls true --cafile /root/go/src/github.com/hyperledger/fabric-samples/first-network/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --clientauth --certfile /root/go/src/github.com/hyperledger/fabric-samples/first-network/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt --keyfile /root/go/src/github.com/hyperledger/fabric-samples/first-network/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key

and it exec success,you can see the --certfile value is peer's server.crt and --keyfile value is peer's server key.

so I think the problem you had meet is caused by client side tls,you can check the client side crt and key is correct or not.

Hope these are useful to you.

Li Xian
  • 421
  • 4
  • 4