2

I am trying to implement Chronicle Queue into our system and had a question regarding rolling of files daily but at a specific time as per the local time zone of the process. I read few write-ups regarding how to specify roll cycle but as per documentation the epoch time works as per midnight UTC. What would I need to do to configure a roll cycle let's say every day at 5PM local time zone of the process running? Any suggestions?

public class TestRollCycle {

    public class TestClass implements TestEvent {
        private int counter = 1;

        @Override
        public void setOrGetEvent(String event) {
            System.out.println("Counter Read Value: " + counter);
            counter++;
        }
    }

    public interface TestEvent {
        void setOrGetEvent(String event);
    }

    @Test
    public void testRollProducer() {
        int insertCount = 1;
        String pathOfFile = "rollPath";
        // Epoch is 5:15PM EDT
        SingleChronicleQueue producerQueue = SingleChronicleQueueBuilder.binary(pathOfFile).epoch(32940000).build();
        ExcerptAppender myAppender = producerQueue.acquireAppender();
        TestEvent eventWriter = myAppender.methodWriter(TestEvent.class);

        while (true) {
            String testString = "Insert String";
            eventWriter.setOrGetEvent(testString);

            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println("Counter Write Value: " + insertCount);
            insertCount++;
        }
    }

    @Test
    public void testRollConsumer() throws InterruptedException {
        String pathOfFile = "rollPath";
        // Epoch is 5:15PM EDT
        SingleChronicleQueue producerQueue = SingleChronicleQueueBuilder.binary(pathOfFile).epoch(32940000).build();
        TestClass myClass = new TestClass();
        ExcerptTailer trailer = producerQueue.createTailer();
        MethodReader methodReader = trailer.methodReader(myClass);

        while (true) {
            if (!methodReader.readOne()) {
                Thread.sleep(1000);
            } else {
                //System.out.println(trailer.index());
            }
        }
    }
}
ATO
  • 574
  • 4
  • 14

2 Answers2

1

This is a feature we added to Chronicle Queue Enterprise. I suggest you contact sales@chronicle.software if you are will to pay for it.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • ahh.. thanks for the response. Will have to confirm and check. Though another related question lets say I use the default roll functionality which is midnight UTC so what are the state of affairs during the moment of switch? I mean if a constant stream of messages is waiting to get persisted during that time is there any chance chance of loosing messages or is everything is seamless? – ATO Aug 29 '17 at 14:24
  • You can specify `epoch` in the builder as a fixed time of day, just not a timezone with daylight savings changes. We are not aware of any situation where you might lose a written, or miss reading a message. – Peter Lawrey Aug 29 '17 at 16:05
  • Peter - I did the following. Because the benchmark for roll is UTC Midnight and I want the roll to happen at 5PM EDT which mean 9PM UTC. So in the epoch I specified 32400000 Milliseconds(==9 hours). I started my sample at 3.30PM EDT. I was expecting at 5PM EDT a role should happen to the new day but it never happened. Is this expected? Documentation mentioned otherwise. – ATO Aug 31 '17 at 19:33
  • @ATO can you include a self contained test which shows this? – Peter Lawrey Sep 01 '17 at 07:23
  • Peter - I edited my question and added the sample code. Just to give you little more color. The file roll does work but in some conditions. I tested few times in past couple of days close to midnight EDT and test worked fine. If I put roll time 1.00am EDT(5am UTC) switch happens. But if I add 5PM EDT(9PM UTC) the switch doesn't work. Same code above two different results. – ATO Sep 05 '17 at 21:38
1

I think there's a problem in your test - the epoch of 32940000 supplied to the queue builder is 9hr 15m from midnight, so 9:15AM UTC or 5:15AM EDT. It should be another 12 hours later for the roll-time to be 5:15PM.

I've added a test that documents the current behaviour for your use-case, and it passes as expected. Can you double-check that you're supplying the correct epoch offset, and perhaps implement a StoreFileListener in order to capture/log any roll events.

The roll will not actually occur until an event is written to the queue that is after the roll-time boundary. So an idle queue that is not being written-to will not roll without input events.

The test is on github:

https://github.com/OpenHFT/Chronicle-Queue/blob/master/src/test/java/net/openhft/chronicle/queue/impl/single/QueueEpochTest.java

Mark Price
  • 296
  • 1
  • 3
  • Ahh.. I understood the problem... 9 vs 21...Thanks.. Regarding the test code you added, it seems method rollTime which replaces epoch is not part of the latest release v4.5.27. Is this a BETA api or paid version? Also Class DirectoryUtils. – ATO Sep 06 '17 at 17:39
  • The latest release published to maven is quite out of date; artifacts are no longer published to maven central (see https://github.com/OpenHFT/Chronicle-Queue#download). I suggest you build from the source and host the artifacts in your own repository. – Mark Price Sep 07 '17 at 09:13
  • Thanks Mark. Is master the right branch to retrieve? Or Tag - chronicle-queue-4.6.15? – ATO Sep 07 '17 at 14:14
  • You should use the release tags for stable versions. The master branch is being actively developed. – Mark Price Sep 11 '17 at 06:57
  • Thanks Mark. One last question - In the example producer is doing setCurrentTime to force the queue to use the time which we are setting. If there is a simultaneous reader also on the other end will the reader also have to do setCurrentTime so both reader and writer are working on the same cq4? I have this situation where the producer and the consumer using Chronicle doesn't need to shutdown but roll daily at a set time. I don't I might need to call setCurrentTime after each roll. – ATO Sep 12 '17 at 02:36
  • The example sets the current time just to get the roll behaviour in a unit-test. You should not override the TimeProvider unless there's a very good reason for it. A reader shouldn't really care about 'time', it just reads events from the end of a queue. – Mark Price Sep 12 '17 at 14:36