2

I have an applet that retrieves a byte array from a backend server. This byte array contains a dynamic library (DLL or SO, depending on which OS the applet is running on), that must be written to disk and subsequently loaded from the disk by a call to System.load().

I need to ensure that the file is not tampered with after it's been written to disk and before it's loaded by the OS through the call to System.load(). I obtain an exclusive lock on the file while it's written to disk, but my testing shows that I must release this lock before the call to System.load(), or it'll fail to load the library.

Is there some way I can keep the lock on the file while I load it?

Sample code:

File f = File.createTempFile("tmp", "");
RandomAccessFile raf = new RandomAccessFile(f, "rwd");
FileChannel channel = raf.getChannel();
FileLock lock = channel.lock(0, Long.MAX_VALUE, false);

// This would be where I write the DLL/SO from a byte array...             
raf.write((int)65); // 'A'
raf.write((int)66); // 'B'
raf.write((int)67); // 'C'

System.out.println("Wrote dynamic library to file...");

// Close and release lock
raf.close();

System.out.println("File closed. Lock released.");

// This call fails if the file is still locked.             
System.load(f.getAbsolutePath());

Any help is greatly appreciated. The solution (if there is any) must not be native to any OS, but work on all platforms supported by Java. It is also a requirement that the solution be compatible with Java 1.4.

Peter Vils Hansen
  • 21
  • 2
  • 1
    I am not sure, but calculating a md5/sha for the file would help? You could check the signature before doing the system.load.. – Manish Nov 01 '11 at 10:37
  • @ManishSharma Agreed with the hash digest idea. Nothing stops a user from "faking" an OS lock and still altering the file. While it'd be quite impressive for someone to alter file system code for that purpose, you can't count on no-one being crazy enough to try it. – G_H Nov 01 '11 at 10:41
  • A digest will not suffice for the following reasons: 1) It will only show the file has been tampered with *before* or *after* the file has been loaded by the OS, and 2) If an attacker can replace the contents of the before the call to System.load(), he can also do it after that call and before the digest computation, hence making the attack invisible. Thanks for the suggestion though! – Peter Vils Hansen Nov 01 '11 at 11:00
  • @PeterVils was the last line of your code supposed to be 'System.load(f.getAbsolutePath());'. your code didn't work for me. – Naveen Babu Nov 01 '11 at 14:24
  • @PeterVils so is the question still open? or you face some other issue? – Naveen Babu Nov 02 '11 at 05:26

1 Answers1

0
  1. In Java 7 you can implement in-memory file system (see java.nio.file.spi.FileSystemProvider), so library content will be completely in memory thus making attacker's life much harder.
  2. Another possible approach is to sign the library and let OS do security checks after reading file from disk; might be not very portable though.
  3. Most important - is it really the biggest security issue you're facing? The time between 2 calls will be some micro- (ok, maybe milli-) seconds - one must hack really deep into filesystem to do bad things. Wouldn't it be easier to alter file while it is transferred over network? Or don't you think an attacker that advanced may... let's say hack JVM and substitute content while library is written to disk? Nothing is bulletproof, maybe this is the risk you can accept?

Just out of interest - what exactly is the error you're getting?

Andrey Nudko
  • 697
  • 5
  • 5
  • (1) is definitely an interesting feature, but alas, this particular piece of code must be compatible with Java 1.4! – Peter Vils Hansen Nov 01 '11 at 19:07
  • (2) I've played with the idea of having the DLL/SO constructor do a self-check on the file when executed, but in this particular setting, that would be too much work for something I cannot convince myself would work. – Peter Vils Hansen Nov 01 '11 at 19:09
  • (3) I agree completely that this is more a theoretical hack than anything else, but the powers that be dictate that I look into this and provide irrefutable evidence that a) it cannot be done, or b) the amount of work required to solve this outweighs the actual danger and/or risk of the attack. – Peter Vils Hansen Nov 01 '11 at 19:11