1

The KDB Ticker Plant process has a log file which keeps data for a second. Why does it need a log file in a first place? What is the purpose of it?

Donald_W
  • 1,773
  • 21
  • 35
user3914448
  • 469
  • 1
  • 11
  • 22
  • btw the tickerplant keeps data for 1s by default. You can set the timer to 1, i.e. 1ms, or whatever. From your question it sounds like you're saying the log file keeps data for 1 second - it's the tickerplant that keeps it for a certain amount of time. The log file has a whole day's worth of data in it. – Manish Patel Jan 16 '15 at 22:32
  • Ok, but why TP keeps then data also ib runtime for one sec? It may just piblish after received and purge it. – user3914448 Jan 19 '15 at 15:17
  • because for very data intensive data surges that'll cause consistent high CPU usage, which will cause other issues. – Manish Patel Jan 19 '15 at 15:31

1 Answers1

2

It is just to restore the data mainly in case of RDB process crash which ensures system stability. You can assume it as a backing store or as a permanent storage.

First let's understand the tickerplant architecture:

Tickerplant takes data from a source (feed handler) , logs it in file and forwards the data to subscriber (Realtime service).

Case : Realtime process (RDB) goes down:

Realtime service(RDB) is a service which subscribes to tickerplant and stores data in memory. Lets say your ticker plant is taking real-time stock data from some exchange and publishing it to realtime service. And it is being stored in memory as a table(or any other format) by realtime service.

Now after 1 hour RDB process goes down for some reason, in that case it will lost all in-memory data. Then it will require some way to restore that data. It will ask tickerplant to provide that data again.

Now, tickerplant will read the required data from from log file and pass that to RDB. This ensures system stability.

Rahul
  • 3,914
  • 1
  • 14
  • 25
  • 3
    Just to clarify: the TP does *not* read all of the data from the log file and send it to RDB. The TP sends only the log location and the current log count to the RDB, then the RDB re-runs the data from the log to get up to speed. – terrylynch Jan 16 '15 at 16:38
  • @Rahul 's answer covers it. Just an addendum, with a tiny bit of tweaking you can in fact bypass the logging (if it's an issue...) – Manish Patel Jan 16 '15 at 22:31