0

Basically subj. In a case of ChronicleMap: if process was killed or failed with OOM error or something else, is there a probability that actual stored data was corrupted and map became not usable any more?

leventov
  • 14,760
  • 11
  • 69
  • 98
Konstantin Kulagin
  • 684
  • 6
  • 16

1 Answers1

1

Entries altered by queries which are in-flight when the process is interrupted, might be corrupted, however, Chronicle Map 3 release will have an option to exclude this possibility, i. e. the entries will either have the new value or the old value, but not some corrupted bytes as the value.

The entries not updated right when crash occurs, are safe.

leventov
  • 14,760
  • 11
  • 69
  • 98
  • Thanks Roman. I was not able to correctly resolve maven dependencies for the 3rd version it is alpha, so I do not spent much time on that. For the second version: 1) I assume that the data for entry transferred to another node only after successful save, i.e. I cannot get corrupted data on the second node if the first one failed. Correct? 2) How about question similar to original one: if something happened to network connection: it is possible that I will get invalid entry on target node? – Konstantin Kulagin Oct 27 '15 at 00:16
  • @KostyaKulagin yes, unfortunately Chronicle Map 3 current alpha version is not usable due to a mistake in Maven pom file. However, the option I'm talking about, in not present in alpha releases yet, and won't appear in Chronicle Map 3 beta as well, only the production release, which takes some time to prepare. – leventov Oct 27 '15 at 00:19
  • I've updated my comment above (always getting confused with press-enter-send-message behavior :) ), sorry for a little mess. And probably final question: from my understanding having some kind of MMFile 'snapshots' should be pretty non trivial thing specially in a life environment when file itself is frequently updated. So the only option in this case to have some resiliency is to have several copies/nodes per map (i.e. map distributed over several nodes). Correct? – Konstantin Kulagin Oct 27 '15 at 00:29
  • 1) If the process/machine crashes, the changes won't be replicated until you take it up again, before that you are supposed to run a special "recovery" procedure, which clears locks, corrupted enties etc. (This procedure is also the target for Chronicle Map 3 release). But yes, replication worker is notified about new entries to replicate only after the update event fihishes. 2) Chronicle Map replicates via TCP and/or UDP, both protocols guarantee integrity AFAIK. – leventov Oct 27 '15 at 00:38
  • Making a snapshot should be possible, you should just acquire exclusive locks of all Chronicle Map's segments. You will only have bad latencies of operations which are blocked by taking a snapshot. – leventov Oct 27 '15 at 00:40
  • Just calling `msync` seems simpler. But generally, I don't like the idea of trying to make a durable database from Chronicle Map – leventov Oct 27 '15 at 00:41
  • 2) TCP/UDP - if target node got just a 'half' of an object and after that connection died will it cause problems? Or map will function normally just without this 'half-read' entry? As for database: you mean CM basically designed to be more like a cache i.e. if something fails - you should be able to re-create it and before that just call real 'durable database'. Correct? – Konstantin Kulagin Oct 27 '15 at 01:01
  • No, of cause replication won't apply half-recieved update. Yes, caches, IPC, moving off-heap the state otherwise you keep on-heap in ConcurrentHashMap. "before that just call real 'durable database" -- I don't understand this, we gave durability promises somewhere? – leventov Oct 27 '15 at 01:08
  • ) in the last part I meant that app should fall back to some durable source. It is not about CM. Sorry for confusion. And thanks for the answers, really appreciate. – Konstantin Kulagin Oct 27 '15 at 01:12