Why does this Dafny function return an empty sequence?

Question

I'm trying to prove that encoding/decoding a LEB128 (well actually LEB64) varint is lossless. Here's my code:

function decode_varint(input: seq<bv8>) : bv64
    requires |input| > 0
{
    var byte := input[0];
    var val := (byte & 0x7F) as bv64;
    var more := byte & 0x80 == 0 && |input| > 1;

    if more then val | (decode_varint(input[1..]) << 7) else val
}

function encode_varint(input: bv64) : seq<bv8>
{
    var byte := (input & 0x7F) as bv8;
    var shifted := input >> 7;
    if shifted == 0 then [byte | 0x80] else [byte] + encode_varint(shifted)
}

lemma Lossless(input: bv64) {
    var test := encode_varint(128);
    var encoded := encode_varint(input);
    var decoded := decode_varint(encoded);
    assert decoded == input;
}

Unfortunately the assertion doesn't hold. I used the VSCode plugin's counter-example feature (F7) to inspect the values in Lossless, and it picks input = 0x8000000000000000. Fine, I think that should work, but the counter-example also shows that test is test:seq<bv8> = ().

I don't understand that. Firstly don't sequences use square brackets? Second it doesn't look like it is possible for encode_varint() to return an empty sequence in any case. In fact Dafny proves this successfully!

lemma NeverEmpty(input: bv64) {
    var encoded := encode_varint(input);
    assert |encoded| > 0;
}

What's going on here?

Edit: Also if I add these examples...

lemma Examples()
{
    assert encode_varint(1 << 7) == [0x00, 0x81];
    assert encode_varint(1 << 14) == [0x00, 0x00, 0x81];
    assert encode_varint(1 << 28) == [0x00, 0x00, 0x00, 0x00, 0x81];
    assert encode_varint(1 << 42) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x81];
    assert encode_varint(1 << 56) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x81];
    assert encode_varint(1 << 63) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x81];
    assert encode_varint(0x8000000000000000) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x81];
}

Then they are all proven, but if I only have the last example (the counter-example Dafny generates) then it isn't!

score 2 · Accepted Answer · answered Dec 17 '22 at 20:03

2

The fact that decode(encode(x)) == x requires inductive proof (since the functions are recursive). Luckily, Dafny's automatic induction is smart enough to prove it if you state it as a lemma with a postcondition like this:

lemma Lossless(input: bv64)
  ensures decode_varint(encode_varint(input)) == input
{}

You can then call that lemma to prove the assert.

lemma Main(input: bv64) {
  var test := encode_varint(128);
  var encoded := encode_varint(input);
  var decoded := decode_varint(encoded);
  Lossless(input);
  assert decoded == input;
}

(If you're wondering why Dafny proves my lemma automatically but not the assert, one reason is because (roughly speaking) Dafny only attempts automatic induction on lemmas, not on asserts.)

And let me never pass up an occasion to mention the slogan "Just because Dafny reports an error doesn't mean the program is incorrect"

answered Dec 17 '22 at 20:03

James Wilcox

5,307
16
25

Ah thanks that makes perfect sense. I also found [this bit of the user guide](https://dafny.org/latest/OnlineTutorial/Lemmas) quite illuminating around proving inductive functions. If I'm understanding things correctly Dafny reports `assertion might not hold` both for things that are obviously untrue (`assert 1 == 2;`) and for things it just hasn't been able to prove? That seems pretty mad. Shouldn't it say something like `assertion is false` and `could not prove assertion`? – Timmmm Dec 17 '22 at 20:18
That's right, it doesn't distinguish between "assertion is false" and "could not prove assertion". It would be nice if it did, but it's actually tricky in for Dafny to know when something is false. (Maybe it could try to prove the negation?) – James Wilcox Dec 18 '22 at 00:08
Ah right, but confusing to display a counter-example that may be fine too then! This is my first exposure to software formal verification. I've done a bit of hardware formal (SVA) and that seems to properly distinguished proven/falsified/inconclusive (and vacuously proven) and it only provids valid counterexamples. I might open a GitHub issue for this. Thanks for the help! – Timmmm Dec 18 '22 at 07:48
One challenge is that Dafny is trying to solve an undecidable problem (checking whether code satisfies its specification), so we know it's not possible for it to always give the right answer (and terminate). Many hardware verification problems are hard but not undecidably hard. – James Wilcox Dec 19 '22 at 20:57

Why does this Dafny function return an empty sequence?

1 Answers1