0

I'm unable to add a new block to the chain because the sawtooth-validator-default is reporting: [2019-01-25 18:18:54.029 WARNING block_validator] Block 7f3...370e (block_num:1, state:c3a...954, previous_block_id:0d8...09d) failed validation: Block 7f3...370e (block_num:1, state:c3a...954, previous_block_id:0d8...09d) rejected due to invalid predecessor 0d8...09d (block_num:0, state:66e...ee1, previous_block_id:0000000000000000)

I'm guessing this is a timing issue because it just started happening and still works ok on a colleague's environment.

I tried running through the following sequence of operation 7 times this morning:

  1. docker-compose down
  2. gradle clean build fatJar docker
  3. docker-compose up --build
  4. GET localhost:8008/blocks --> 1 block as expected
  5. POST locApplication to chain
  6. GET localhost:8008/blocks --> shows block added ... or not
  7. GET localhost:20005/api/v1/lettersofcredit/applications/?id=d26...f2d

Works? Y|N|N|Y|Y|Y|N

I'm not seeing any output from settings_tp, only from the default validator. Every time the new block submission returns 202 ACCEPTED.

I'm running in developer mode, using docker-compose to run all containers on a single Ubuntu 18.0.4 VM. Here's the docker-compose.yaml:

version: "2.1"

services:

  settings-tp:
    image: hyperledger/sawtooth-settings-tp:1.1
    container_name: sawtooth-settings-tp-default
    depends_on:
      - validator
    entrypoint: settings-tp -vv -C tcp://validator:4004

  intkey-tp-python:
    image: hyperledger/sawtooth-intkey-tp-python:1.1
    container_name: sawtooth-intkey-tp-python-default
    depends_on:
      - validator
    entrypoint: intkey-tp-python -vv -C tcp://validator:4004

  xo-tp-python:
    image: hyperledger/sawtooth-xo-tp-python:1.1
    container_name: sawtooth-xo-tp-python-default
    depends_on:
      - validator
    entrypoint: xo-tp-python -vv -C tcp://validator:4004

  validator:
    image: hyperledger/sawtooth-validator:1.1
    container_name: sawtooth-validator-default
    expose:
      - 4004
    ports:
      - "4004:4004"
    # start the validator with an empty genesis batch
    entrypoint: "bash -c \"\
        sawadm keygen && \
        sawtooth keygen my_key && \
        sawset genesis -k /root/.sawtooth/keys/my_key.priv && \
        sawadm genesis config-genesis.batch && \
        sawtooth-validator -vv \
          --endpoint tcp://validator:8800 \
          --bind component:tcp://eth0:4004 \
          --bind network:tcp://eth0:8800 \
          --bind consensus:tcp://eth0:5050 \
        \""

  devmode-engine:
    image: hyperledger/sawtooth-devmode-engine-rust:1.1
    container_name: sawtooth-devmode-engine-rust-default
    depends_on:
      - validator
    entrypoint: devmode-engine-rust -C tcp://validator:5050

  rest-api:
    image: hyperledger/sawtooth-rest-api:1.1
    container_name: sawtooth-rest-api-default
    expose:
      - 8008
    ports:
      - "8008:8008"
    depends_on:
      - validator
    entrypoint: sawtooth-rest-api -C tcp://validator:4004 --bind rest-api:8008

  shell:
    image: hyperledger/sawtooth-all:1.1
    container_name: sawtooth-shell-default
    depends_on:
      - rest-api
    entrypoint: "bash -c \"\
        sawtooth keygen && \
        tail -f /dev/null \
        \""

  importer-webserver:
    image: sawtooth-trade-finance/importer-web:latest
    container_name: importer-webserver
    depends_on:
      - rest-api
    ports:
      - "20005:8080"

  importer-processor:
    container_name: importer-processor
    image: sawtooth-trade-finance/importer-processor:latest
    build:
      context: .
      dockerfile: ./tf_processor/Dockerfile
    depends_on:
      - validator

Any suggestions for how I might go about debugging this issue would be gratefully received. Thanks.

Nick Southwell
  • 11
  • 1
  • Review: https://stackoverflow.com/help/how-to-ask and perhaps provide minimal and repeatable focus to your issue. – Frank C. Jan 28 '19 at 09:30

3 Answers3

1

To work with in Developer Mod you should install and run DevMod consensus.
to install 

sudo apt-get install sawtooth-devmode-engine-rust

After starting validator, start consensus in another terminal 

sudo -u sawtooth devmode-engine-rust -vv --connect tcp://localhost:5050
Harun-Ur-Rashid
  • 3,338
  • 1
  • 15
  • 22
1

Did you try to remove all containers, images, and volumes in docker? It works for me. Docker remove all containers, images, and volumes

avinash
  • 11
  • 1
  • Yes. I've tried that. No joy. We had the same issue when we deployed to AWS using Kubernetes instead of docker-compose, so I guess that the dependencies in the docker-compose file is not the problem. – Nick Southwell Feb 22 '19 at 16:34
1

In ideal world failed block validation is equivalent to Byzantine behaviour in the network. The Byzantine behaviour here could be in your node or the node that published the block.

Given this, if you think in your network there's no Byzantine node and all the nodes are trusted. You need to check either your deployment environment or the smart contract.

Deployment: It could be possible that when you setup your Hyperledger Sawtooth network node, it picked up an existing blockstore file and initialized with the unintended block. All the later block validations would fail because your final global state will not match with what is there in the new block. Please clear your environment/stale files before you run the new network setup.

Smart Contract/Transaction Processor (TP): If your TP is not written such that you get deterministic result, then you need to dig into your application and identify that. This means that your TP is producing unique results on different machines or different time instances, hence one of your node is producing a global state hash which doesn't match with that in the new block. Make sure your TP is deterministic.

Arun
  • 592
  • 3
  • 13