You can't simply declare a byte as delimiter if you're working with random unstructured data (which compressed/encrypted data resembles quite closely), because the delimiter can always appear as a regular data byte in such data.
If the size of the data is already known when you start writing, just generally write the size first and then the data. When reading back you then know you need th read the size first (e.g. 4 bytes for an int), and then as many bytes the size indicates.
This will obviously not work if you can't tell the size while writing. In that case, you can use an escaping mechanism, e.g. select a rarely appearing byte as the escapce character, escape all occurances of that byte in the data and use a different byte as end indicator.
e.g.
final static byte ESCAPE = (byte) 0xBC;
final static byte EOF = (byte) 0x00;
OutputStream out = ...
for (byte b : source) {
if (b == ESCAPE) {
// escape data bytes that have the value of ESCAPE
out.write(ESCAPE);
out.write(ESCAPE);
} else {
out.write(b);
}
}
// write EOF marker ESCAPE, EOF
out.write(ESCAPE);
out.write(EOF);
Now when reading and you read the ESCAPE byte, you read thex next byte and check for EOF. If its not EOF its an escaped ESCAPE that represents a data byte.
InputStream in = ...
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
while ((int b = in.read()) != -1) {
if (b == ESCAPE) {
b = in.read();
if (b == EOF)
break;
buffer.write(b);
} else {
buffer.write(b);
}
}
If the bytes to be written are perfectly randomly distributed this will increase the stream length by 1/256, for data domains that are not completely random, you can select the byte that is least frequently appearing (by static data analysis or just educated guess).
Edit: you can reduce the escaping overhead by using more elaborate logic, e.g. the example can only create ESCAPE + ESCAPE or ESCAPE + EOF. The other 254 bytes can never follow an ESCAPE in the example, so that could be exploited to store legal data combinations.
