Postgres custom type mapping to C struct?

Question

I am writing a custom Postgres C AGGREGATE extension. I would like to know how to properly map a type defined in PSQL to a struct defined in C. I have tried doing it as best as I know how but I do not get results, I get an empty row when I test.

The C struct has the following form:

typedef struct {
    int64_t row_idx;     //Row number in the processing order (NOT thread safe)
    int64_t data1_size;   
    int64_t data2_size;     
    char data2[ 1<<10 ];    
    char data1[ 1<<10 ];    
} search_state;

I am not sure if it is better to get char pointers for the data, or if to allocate statically upfront as above. I have tried both ways to the same effect; I know to allocate with pmalloc to retain data between calls to the rows. I have also tried using the postgres type "text", and I still get no data at the end of the aggregation, just no rows as the result.

On the Psql side I have:

CREATE TYPE search_state AS (
    row_idx    int8,
    data1_size int8,
    data2_size int8,
    data1      char[1<<10],
    data2      char[1<<10]
);

Again, I have tried text type for data1 or data2, but I do not know how to make it work.

The question is, what is the proper way to get a type in psql that will precisely map to a struct in C, as above? It seems my problem arises from postgres being unable to find the data when i modify my struct in C.

The function that uses this struct as intermediary state during the aggregation is something like:

 Datum fancy_select(PG_FUNCTION_ARGS){
        //Get the Postgres C-Extension Parameters
        search_state *state   = (search_state * ) PG_GETARG_POINTER(0);
        char *inputText       = PG_GETARG_CSTRING(1);
        uint64_t numeric_id   = PG_GETARG_INT64(2);

        //Make object to store state from now on.
        search_state *new_state   = (search_state *) palloc( 2*(1<<10) + 8*3 );

        //Small Test updating the state
        new_state->row_idx    = 1;
        new_state->data1[0]   = 'T'
        new_state->data1[1]   = '\0'
        new_state->data2[0]   = 'H'
        new_state->data2[1]   = '\0'
        new_state->data1_size = strlen( new_state->data1 ) ;
        new_state->data2_size = strlen( new_state->data2 ) ;

        //Return state
        PG_RETURN_POINTER( new_state );
   };
}

There is a final function for this aggregation which essentially takes the struct, gets data1, and returns the text that is stored there by the end. I should also mention that this extension is in C++, not plain C, but I am using as much C and as little C++ as possible.

Note 1: Testing with different structures the behaviour ranges from return initcond, no rows at all and the following message:

ERROR:  type with OID 0 does not exist
CONTEXT:  PL/pgSQL function final_func(search_state) while storing call arguments into local variables

score 0 · Answer 1 · answered Sep 11 '20 at 08:43

0

I can give you a few pointers.

First, I assume you know about the caveats when writing server code in C++.

The internal representation of your data type could be anything, but I wouldn't allocate more memory than needed.

The mapping between the internal and the external representation is entirely up to you and is determined by the type input and output function you write.

But I don't see why you need a special type written in C if all you want to write is an aggregate function.

I'd simply

CREATE TYPE search_state AS (
   row_idx bigint,
   data2 text,
   data1 text
);

and use this composite type with your functions. It is not hard to get the components of that type in your C code, and it would save you the trouble of having to write a lot of boiler plate code.

answered Sep 11 '20 at 08:43

Laurenz Albe

209,280
17
206
263

The example I gave is simple and to the point. The real code is complex and there is a lot of fancy computation happening on the string data, the extension in C++ is required due to dependencies and fancy libraries. – ReverseFlowControl Sep 11 '20 at 08:47
I figured the latter. But I don't understand why that requires a type written in C. – Laurenz Albe Sep 11 '20 at 08:50
Could you write a simple example about how to write/read to data1, data2 in that type from C/C++ in an aggregate function? – ReverseFlowControl Sep 11 '20 at 10:15
https://www.postgresql.org/docs/current/xfunc-c.html#id-1.8.3.13.10 – Laurenz Albe Sep 11 '20 at 10:44
So, this approach's complexity is too high for me...lots of postgres framework stuff to worry about... and not one clear example/template to go from. The official documentation is not helpful in this particular regard. i decided to implement this differently. – ReverseFlowControl Sep 16 '20 at 21:59
Yes; even though the code is well documented and well written, it takes a while to get comfortable. – Laurenz Albe Sep 17 '20 at 02:24

Postgres custom type mapping to C struct?

1 Answers1