GWAN Key-Value persistent multiple store

Question

I want to record a key-value in persistent mode but when I want to use 2 or more different stores it doesn't work.

Here's my script:

    ...
    typedef struct{
       kv_t  *kv;
       char  *name;
    } kv_data;

    int main(int argc, char *argv[])
    {
       kv_data **data = (kv_data**)get_env(argv, US_SERVER_DATA);
       if(!data[0]){
          data[0] = (kv_data*)calloc(1, sizeof(kv_data));
          if(!data[0]){
             return 500;
          }
          kv_t users;
          kv_init(&users, "users", 10, 0, 0, 0);
          data[0]->kv = &users;

          kv_item item;
          item.key = "pierre";
          item.klen = sizeof("pierre") - 1;
          item.val = "pierre@example.com";
          item.flags = 0;
          kv_add(data[0]->kv, &item);

          data[0]->name = strdup("users");
       }
    ...

This error is in line 15 with the kv_init() function.

My wish would be for example to use data[0]->kv to read and record value in "users" store and to use data[1]->kv to read and record value in other store...

Can you clarify which error you get? – Nagi Apr 27 '14 at 06:24 — Nagi, Apr 27 '14 at 06:24

score 1 · Answer 1 · answered Apr 26 '14 at 11:54

John,

Your problem comes from the difficulty to play with pointers on pointers. We could fix this code but it will probably break later in your scripts because of its complexity.

For the long term, a simpler solution should be used to avoid bugs.

The structure attached to US_SERVER_DATA can be made simpler:

typedef struct
{
   char *name;
   kv_t *kv_1;
   kv_t *kv_2;
   void *callback;
   u32   current_time;
}

This is the way to go: allocate memory with calloc(sizeof(my_struct));, and attach this structure to your US_SERVER_DATA pointer.

Then, providing you do all this in a G-WAN handler init() call (or in the gwan/main.c maintenance script, or even in the recently added gwan/init.c script), you will have exclusive access (one single thread is doing the initialization) and you will be able to attack G-WAN KV stores (and other objects) as needed.

If, later, G-WAN servlets or handlers are modifying these structure pointers then you will have to use a lock, spinlock, etc. or atomic variables for the pointers themselves.

Hope this helps.

score 1 · Accepted Answer · answered Apr 27 '14 at 11:01

1

I just attach a kv store that contain all the kv stores I need to the persistence pointer. G-WAN kv is fast, plus the number of records is small, it should not hurt performance.

  xbuf_t *reply = get_reply(argv);
  kv_t **kv_db = (kv_t **)get_env(argv, US_VHOST_DATA);

  if (!kv_db[0])
  {
     kv_db[0] = (kv_t*) malloc(sizeof(kv_t));

     if (!kv_db[0])
     {
        puts("Could not allocate memory for the v-host kv store");
        return 500;
     }

     kv_init(kv_db[0], "kv_db", 10, 0, 0, 0);
  }

  kv_t *blog_db = (kv_t *) kv_get(kv_db[0], "blog_db", 7);

  if (blog_db)
  {
     xbuf_cat(reply, "Blog already installed. GTFO.");
     return 200;
  }

  blog_db = (kv_t *) malloc(sizeof(kv_t));
  kv_init(blog_db, "blog_db", 0, 0, 0, 0);

  if (!blog_db)
  {
     puts("Could not allocate memory for the blog kv store");
     return 500;
  }

  _KV_ADD(kv_db[0], item, blog_db->name, blog_db, 0);

(_KV_ADD is a custom macro)

answered Apr 27 '14 at 11:01

Nagi

256
1
2
4

thanks for your help, your solution works. But do you imply that it's less optimized (from a performance point of view) than the solution proposed by Gil below ? (which by the way I didn't manage to make work) Because I might have a very large number of data to process simultaneously. Regards. – John S Apr 27 '14 at 16:25
In my solution I have to perform a look up (`*blog_db = (kv_t *) kv_get(kv_db[0], "blog_db", 7);`) whenever I want to access my blog_db kv; in Gil's solution he has direct access to it since it's just a field in a struct. So of course my solution is less optimized. – Nagi Apr 28 '14 at 03:36
Btw, the size of your dataset (the size of each kv store) is irrelevant in this case, since what matters in my solution is the number of kv stores. Anyway, if you have too many kv stores, you might want to reconsider your design. Think about merging kv stores by using key prefixes instead (ex: "user_db:" prefix for user kv, "blog_db:" prefix for posts etc.) – Nagi Apr 28 '14 at 03:39

GWAN Key-Value persistent multiple store

2 Answers2

Linked