How do I use multiple writers with tdb?

Question

I'm using tdb to try to get acquainted with database management in C on Linux. Per tdb's description

tdb is a Trivial database.

In concept, it is very much like GDBM, and BSD's DB except that it allows multiple simultaneous writers and uses locking internally to keep writers from trampling on each other. tdb is also extremely small. Interface

The interface is very similar to gdbm except for the following:

• different open interface. The tdb_open call is more similar to a traditional open()
• no tdbm_reorganise() function
• no tdbm_sync() function. No operations are cached in the library anyway
• added transactions support

A general rule for using tdb is that the caller frees any returned TDB_DATA structures. Just call free(p.dptr) to free a TDB_DATA return value called p. This is the same as gdbm.

Now I wanted to make a small test program to test multiple write-connections to my database. But it fails.

#include <tdb.h>

int main(int argc, char** argv) {
   struct tdb_context * tdb = tdb_open("test.tdb", 0, 0,
           O_RDWR | O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP);
   if(!tdb){
       printf("%s\n", tdb_errorstr(tdb));
   } else {
       printf("Database successfully opened!\n");
   }
   
   struct tdb_context * anothertdb = tdb_open("test.tdb", 0, 0,
           O_RDWR| O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP);  //why does this fail?
   if(!anothertdb){
       printf("%s\n", tdb_errorstr(anothertdb));
   } else {
       printf("Another reader successfully opened!\n");
   }
   
   if(tdb_close(anothertdb)){
       printf("Error while closing database!\n");
   } else {
       printf("closing anothertdb-connection\n");
   }
   
   if(tdb_close(tdb)){
       printf("Error while closing database!\n");
   } else {
       printf("closing tdb-connection\n");
   }
return 0;
}

The output is:

Database successfully opened!

RUN FINISHED; Segmentation fault; core dumped; real time: 240ms; user: 0ms; system: 20ms

If I open just a single connection to the database there are no problems. How do I open multiple readers on a database?

Answer 1

The first problem you have is the call to tdb_errorstring after tdb_open fails:

if(!anothertdb){
       printf("%s\n", tdb_errorstr(anothertdb));
}

It's perhaps not obvious that this can't work, but there's good reason to suspect that it can't work, because anothertdb is most certainly NULL when tdb_errorstr is called, and it's reasonable to assume that tdb_errorstr's argument must be a valid pointer to a struct tdb_context. In fact, that's true; practically the first thing that tdb_errorstring does is to try to dereference its argument in order to extract the last error code.

In short, there's no way to extract a custom error string from the result of a tdb_open failure; all you can do is what you would do for a NULL return from, for example, fopen(): call perror() in the hopes that errno will be set to something meaningful. If you do that, you will see an error message instead of a crash:

$ ./tdb_test 
Database successfully opened!
open 2: Device or resource busy

The library seems to have a way to send tdb_open errors to a configured logging system, but the default configured logging system just throws the error calls away without doing anything. I didn't read the docs to see if there is a description of how to configure a logger; you might want to look into that.

I'm not defending this API design, by the way. Just reporting it.

OK, so now on to your actual question: how do you open multiple connections to a TDB database? And the answer is, you have to do that from multiple processes. An individual process can open as many TDBs as it wants to, but it can only open each of them once (at a time). According to a comment in the source code, that's a result of the way that fcntl() locks work, which seems like a reasonable explanation to me.

So if you want to try a multiple-reader, multiple-writer scenario, you're going to have to either run multiple processes, or fork multiple children from a parent process. Threading won't work, because two threads in the same process can't open the same TDB either. It's one open per process.

Also, if you choose to fork multiple children, remember that you will need to call tdb_open only after you fork, because if you open a TDB in the parent process, it will still be open in the child.

As a final note, please get in the habit of writing warning and error messages to stderr, which is normally not buffered so that you have a much better chance that the error message will become visible when you output it, as opposed to some later moment when the stdio library chooses to flush the stdout output buffer. That's not actually a problem here, but it could have been and you're much better off using stderr for that purpose (which is its intended purpose, hence the name).