0

I'm seeing this:

Missing fields! Expected 8 fields but only got 8! Last field end 2876176 and serialize buffer end 2876175.

in my Hive log. It clearly does not make sense "expect 8 got 8", so I checked the source code:

    // Missing fields?
    if (!missingFieldWarned && lastFieldByteEnd > structByteEnd) {
      missingFieldWarned = true;
      LOG.warn("Missing fields! Expected " + fields.length + " fields but " +
              "only got " + fieldId + "! " +
          "Last field end " + lastFieldByteEnd + " and serialize buffer end " + structByteEnd + ". " +
          "Ignoring similar problems.");
    }

It seems the expected bytes range went over actual byte so the Serde thinks there might be missing fields, but field id and field length are the same.

Is there a bug in this code?

      if (7 == (i%8)) {
        if (lastFieldByteEnd < structByteEnd) {
          nullByte = bytes[lastFieldByteEnd];
          lastFieldByteEnd++;
        } else {
          // otherwise all null afterwards
          nullByte = 0;
          lastFieldByteEnd++;
        }        
      }   

The last lastFieldByteEnd++; seems to be unnecessary.

dtolnay
  • 9,621
  • 5
  • 41
  • 62
dz902
  • 4,782
  • 38
  • 41

0 Answers0