Filter data from the end of a byte array (smart card response)

Question

I have a byte[] for example [83, 97, 103, 117, 110, -1, -1, -1, .....]. My data bytes is represented by bytes from 83 to 110 and the remaining are default bytes. How do I extract by data filtering out the -1's at the end of the byte array?

Actually, I was trying to read data out from a smart card. I need to read data from the specific sector of the card that has fixed size. To read the data, I have to specify the length of the data that I want to read. Now the problem is, while reading, my application cannot say what the length of data actually is. So, I have to read the entire sector and then parse/filter out the data from the read byte array. As I said, after the end of my data bytes in the array, the remaining are all -1.

Any better way of achieving this?

Have a look [here](http://stackoverflow.com/a/17004488/5128464) and [here](http://stackoverflow.com/a/7668864/5128464) (and replace `0`/`32` with `-1`)...Good luck! — vlp, Apr 02 '16 at 17:58
Note that the value -1 is the same as 0xFF when viewed as unsigned bytes, i.e. it's a byte with all bits set, represented in two-complement form. The fact that all bits are set is probably the *meaning* of the value, so I would rather compare to `(byte) 0xFF` as a constant in my code. — Maarten Bodewes, Apr 03 '16 at 11:22

Maarten Bodewes · Accepted Answer · 2016-04-03T12:07:14.273

You can look for the first value that isn't valued -1, starting from the end. You then use one of the Arrays methods to copy the remaining values into a new array. As the array is definitely less than 65KiB and probably less than 256 bytes that would not matter with regards to memory or speed. You could also just store the length or wrap the result in a appropriately sized ByteBuffer if you really don't want to copy the bytes.

So here's some Java code to do above. Note that the methods are made private because I don't test the start & end values (they could be switched around or negative). You might also want to include some tests with different start values and other general bounds checking.

This code doesn't check if the values are in the range defined in the question.

private static final byte PADDING_BYTE = (byte) 0xFF;

private static int determineUnpaddedEnd(byte[] array, int start, int end) {
    // end is exclusive
    int paddingOffset = end - 1;
    while (paddingOffset >= start
            && array[paddingOffset] == PADDING_BYTE) {
        paddingOffset--;
    }
    // returns either start or the location of the last padding byte
    return paddingOffset + 1;
}

private static byte[] removePadding(byte[] array) {
    int unpaddedEnd = determineUnpaddedEnd(array, 0, array.length);
    return Arrays.copyOf(array, unpaddedEnd);
}

private static ByteBuffer removePadding(ByteBuffer buf) {
    int start = buf.position();
    int paddingOffset = buf.limit() -1;
    while (paddingOffset >= start
            && buf.get(paddingOffset) == PADDING_BYTE) {
        paddingOffset--;
    }
    buf.limit(paddingOffset + 1);
    return buf;
}

public static void main(String[] args) {
    Map<byte[], Integer> testVectors = new HashMap<>();
    testVectors.put(new byte[0], 0);
    testVectors.put(new byte[] {-1}, 0);
    testVectors.put(new byte[] {1, -1}, 1);
    testVectors.put(new byte[] {83, 97, 103, 117, 110, -1, -1, -1}, 5);

    int testcase = 1;
    for (Entry<byte[], Integer> test : testVectors.entrySet()) {
        System.out.println(testcase++);
        byte[] array = test.getKey();
        int value = determineUnpaddedEnd(array, 0, array.length);
        if (value != test.getValue()) {
            throw new IllegalStateException("Maarten cannot program anymore");
        }
        System.out.println(Arrays.toString(removePadding(array)));
        System.out.println(removePadding(ByteBuffer.wrap(array)));
    }
}

The better way is probably to read out the FCP information using the correct ISO 7816-4 SELECT command, which contains the length of the file being read. This presumes your smart card implementation complies to that format, and the data is in an elementary file. If the file size is larger than the information contained in the file then you can only hope that the protocol defines the size somehow.

The first method is exactly what came across my mind. The protocol doesn't defines the data length, so in cases when the length of the file is larger than the length of data contained, my only bet is to read the entire file. I have come up with a solution though. When writing data in the card, I append a null character at the end. So every time I read the entire file, then strip the data referencing the null character. — yuva, Apr 03 '16 at 06:54
That would work, although prepending a size may be smarter if your data could include a null character. Or, if you want to do it the smart card way, wrap the data in a DER TLV structure. — Maarten Bodewes, Apr 03 '16 at 12:17
Thank you so much, I think prepending the data length will be a smarter way to achieve it. I am trying that out. — yuva, Apr 03 '16 at 12:33

Filter data from the end of a byte array (smart card response)

1 Answers1