ChronicleQueue - Unable to cleanup older CQ4 files

Question

I am using CronicleQueue and I have only one writer/reader. Wanted to cleanup as soon as the reader is done with a CQ4 file. The following code wasn't able to remove the file, is the file reference still held by CQ during onReleased() event?

public class ChronicleFactory {
public SingleChronicleQueue createChronicle(String instance, String persistenceDir, RollCycles rollCycles) {
    SingleChronicleQueue chronicle = null;

    String thisInstance = instance;
    try {
    chronicle = SingleChronicleQueueBuilder.binary(persistenceDir).rollCycle(rollCycles).storeFileListener(new StoreFileListener() {

        @Override
        public void onReleased(int i, File file) {
        String currentTime = LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy??-MM-dd HH:mm:ss.SSS"));
        System.out.println(instance + "> " + currentTime + ": " + Thread.currentThread().getName() + " onReleased called for file: " + file.getAbsolutePath() + " for cycle: " + i);
        if(instance.equals("Reader")) {
            System.out.println("Removing previous CQ file: " + file.getName() + ", deleted? " + file.delete());  //==> Not able to delete the file !!!
        }
        }
    .....

usually the delete should work, is there any error? – Devas Feb 25 '18 at 06:01 — Devas, Feb 25 '18 at 06:01

score 0 · Answer 1 · answered Feb 19 '18 at 05:23

Don't know whether this is an optimal solution, please let me know if there is a cleaner way, but the following works [except it will have to loop few times before deleting, as the file is actually released after few milliseconds it seems, i see "file is closed" printed few time before the actual delete happens)

                @Override
            public void onAcquired(int cycle, File file) {
                String currentTime = LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy‌-MM-dd HH:mm:ss.SSS"));
                System.out.println(instance + "> " + currentTime + ": " + Thread.currentThread().getName() + " onAcquired called for file: " + file.getAbsolutePath() + " for cycle: " + cycle);
                if(prevCycleFile != null && instance.equals("Reader")) {
                    System.out.println(">>> Trying to remove: " + prevCycleFile);
                    boolean bDelete;
                    while((bDelete=prevCycleFile.delete())==false) {

                        System.out.println(">>> file is closed");    
                        try {
                            Thread.sleep(100);
                        } catch (InterruptedException e) {
                            // TODO Auto-generated catch block
                            e.printStackTrace();
                        }
                    }
                    System.out.println(">>> Removing previous CQ file: " + prevCycleFile.getAbsolutePath() + ", deleted? " + bDelete);
                }
            }

score 0 · Answer 2 · answered Feb 19 '18 at 08:22

It's reasonable to expect a small delay between the Queue reporting that a file is released, and the OS actually detecting/acting upon it.

The StoreFileListener is invoked on the same thread that is reading or writing to the queue, so you will incur latency while the while loop is executing. You'll get better results by handing off the work of closing files to another thread (e.g. via a j.u.c.ConcurrentLinkedQueue).

ChronicleQueue - Unable to cleanup older CQ4 files

2 Answers2

Linked