Multi-dimensional byte arrays with LinkedHashMap... Is there a better way?

Question

I'm VERY new to Java Programming, so please forgive my newbish questions :).

I am using a LinkedHashMap as a file cache for an application I am modifying for the sake of assisting development. I am doing this to reduce I/O overhead, and hence improve performance. The problem with this is that overhead is introduced in many other ways.

The relevant source looks like this.

// Retrieve Data From LinkedHashMap  
byte grid[][][] = null;  
if(file.exists())
{  
    if (!cache.containsKey(file))  
    {
        FileInputStream fis = new FileInputStream(file);
        BufferedInputStream bis = new BufferedInputStream(fis, 16384);
        ObjectInputStream ois = new ObjectInputStream(bis);
        cache.put(file, ois.readObject());
        ois.close();
    }
    grid = (byte[][][]) cache.get(file);
} else {
    grid = new byte[8][8][];
}

The following is what I use to save data. The method to load the data is the exact opposite.

ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gos = new GZIPOutputStream(baos){{    def.setLevel(2);}};
BufferedOutputStream bos = new BufferedOutputStream(gos, 16384);
DataOutputStream dos = new DataOutputStream(bos);
// Some code writes to dos
dos.close();
byte[cx][cz] = baos.toByteArray();
baos.close();
cache.put(file, grid);

And here is the declaration for the cache.

private static LinkedHashMap<File, Object> cache = new LinkedHashMap<File, Object>(64, 1.1f, true)
{protected boolean removeEldestEntry(Map.Entry<File, Object> eldest) 
    {
        return size() > 64;
    }
}

Since I'm very unfamiliar with Java stream etiquette, it's very likely that the above code looks sloppy. I am also sure that there are more efficient ways to do the above, such as where to put the buffer.

Anyway, my primary problem is this: Whenever I need to do anything to a single chunk, I have to convert all the grid data to an object, send it to the cache, and write the file. This is a very inefficient way of doing things. I would like to know if there is a better way of doing this, so that I don't have to get(); the entire byte[8][8][] array when I only need access to that one chunk. I'd love to do something like chunk = cache.get[cx]cz, but I'm sure it isn't that simple.

Anyway, as I said earlier, please excuse the question if the answer is obvious, I am but a lowly newb :D. I greatly appreciate any input :).

Thanks.

score 1 · Answer 1 · answered Dec 21 '10 at 21:17

If your aim is to reduce I/O overhead, how about placing the byte[][][] object in a wrapper object which adds the concept of a dirty flag?

That way you can reduce the number of times a file is written upon modification, only writing dirty objects to disk when you either are done using the cache or your are about to remove an eldest object while inserting into a full cache.

score 0 · Answer 2 · answered Dec 21 '10 at 21:18

I would start by creating a new class -- call it ByteMatrix3D -- to hold the data. And rather than using byte[][][], I'd use a single-dimension array with calculated offsets (eg, in an 8x8x8 array, the offset of [1][2][3] could be calculated as 1 * 64 + 2 * 8 + 3. This change will eliminate quite a bit of object-management overhead, and also let you make additional changes without affecting the higher-level code.

And the first change that I'd make would be to use a MappedByteBuffer to access the files. This would let the operating system manage the actual data, and make reads and writes transparent to the program.

Multi-dimensional byte arrays with LinkedHashMap... Is there a better way?

2 Answers2