Using Chronicle-Queue as a file based FIFO queue

Question

I got to know Chronicle-Queue from post: Implementing a file based queue

Here's my use case:

• I have a web server (say tomcat) which serves http requests
• Each request processing might generate some tracing info.
• I'll write these tracing info into a Chronicle-Queue (in bytes[], I'll do the marshalling/unmarshalling by my own, like using protobuf)
• I'll have a dedicate thread to use a tailer to read from the Chronicle-Queue. Each message will be processed ONLY once, if failed, I'll have my own retry policy to put it back to the queue to allow next try.

Based on above use case, I have below questions:

1. How many appenders should be used? Multiple threads share 1 appender or each thread has its own appender?

2. is queue.acquireAppender() a heavy operation? Shall I cache the appender to avoid calls to acquireAppender()?

3. If for some reason server is down, can tailer remember the last success read entry and continue with next entry ? (like a millstone feature)

4. How can I purge/delete old files? Any API to do the purge?

And another irrelevant question:

Is it possible to use Chronicle-Queue to implement a file based BlockingQueue?

Thanks

Leon

Answer 1

How many appenders should be used? Multiple threads share 1 appender or each thread has its own appender?

I suggest you use queue.acquireAppender() and it will create Appenders as needed.

is queue.acquireAppender() a heavy operation? Shall I cache the appender to avoid calls to acquireAppender()?

It's not free but costs a ~100 of nanoseconds.

If for some reason server is down, can tailer remember the last success read entry and continue with next entry ? (like a millstone feature)

We suggest recording to another queue the outcomes of processing the first queue. In this you can record the index it is up to. This is a feature we are considering without the need to add a queue.

How can I purge/delete old files? Any API to do the purge?

If you set a StoreFileListener on the builder you can be notified when a file isn't needed any more.