3

I have a rocksdb instance with multithreaded read/write access. At some point an arbitrary thread needs to process a request to clear the whole database, basically delete all keys. How can I do it with the smallest disturbance to the other threads? Obviously, as everything is parallel, there is no need for a definite moment at which the database gets cleared and the new writes go to an empty one, and it is okay if some parallel reads are still getting the old data for some time.

  • I see DeleteRange, but my keys are irregular, there is no such thing as an upper bound
  • I see DeleteFile, but the comment says it will be gone in rocksdb 7.0. Also, this looks like a bad idea in a multithreaded environmnet

Interestingly, I could not find a recipe for such seemingly common use case

Aviaconstructor
  • 31
  • 4
  • 1
    how is that a common use case...? usually one would need to clear the whole database sometimes when developing or testing, but not as part of normal operations – eis Feb 21 '21 at 12:43
  • Well, SQL databases have `drop` and redis has 'flush*', but I can accept this is not a common use case. In my case this is a must-have, though – Aviaconstructor Feb 21 '21 at 13:01

1 Answers1

0

I see DeleteRange, but my keys are irregular, there is no such thing as an upper bound

Do they have a common prefix? Generally you would prefix the keys with the 'table name' like 'users' or 'messages' and then you can drop the entire messages range which would be like dropping the entire table

If you don't - then I would suggest rereading the docs to make sure you are using rocksdb correctly but the only other option is to loop over and delete each entry

An alternative, is to grab a lock and swap out to a new clean DB and delete the old data folder entirely

Asad Awadia
  • 1,417
  • 2
  • 9
  • 15
  • | An alternative, is to grab a lock and swap out to a new clean DB and delete the old data folder entirely Yeah, since the operation is rare I did – Aviaconstructor Apr 03 '22 at 06:09
  • | An alternative, is to grab a lock and swap out to a new clean DB and delete the old data folder entirely Yeah, since the operation is very rare, I take the app offline and delete the database with a script. Another solution would be to try deleting from '\0' to some "\xFF\xFF\xFF\xFF", with DeleteRange. I will need to find time for investigation. Thanks! – Aviaconstructor Apr 03 '22 at 06:24