Deleting an element from a Riak set via protocol buffers using the Erlang client without fetching the whole set

Question

Via the HTTP API, we can delete an arbitrary element from a set without fetching the whole content:

curl -X POST http://127.0.0.1:8098/types/sets/buckets/travel/datatypes/cities -H "content-type: application/json" -d '{ "remove" : "Toronto" }'

(to verify: tcpdump -i any -s 0 -n 'src port 8087 or src port 8098 and host 127.0.0.1')

However via protocol buffers client, we need to perform the following steps in order to delete an element from a set:

{ok, MySet} = case riakc_pb_socket:fetch_type(Pid, {<<"sets">>, <<"travel">>}, <<"cities">>) of {error,{notfound,set}}-> {ok, riakc_set:new()}; {ok, Set} -> {ok, Set}  end.
ModSet=riakc_set:del_element(lists:last(ordsets:to_list(riakc_set:value(MySet))), MySet).
riakc_pb_socket:update_type(Pid, {<<"sets">>, <<"travel">>}, <<"cities">>, riakc_set:to_op(ModSet)).

As its name suggests, riakc_pb_socket:fetch_type retrieves the whole set. I could not find any methods in the Erlang client using protobuf to just send the delete request without retrieving the whole set first.

Is there a way to avoid fetching the whole set object via the protobuf client when deleting an element?

Update: protocol buffers API to update datatypes seems useful:

• http://docs.basho.com/riak/latest/dev/references/protocol-buffers/dt-set-store/
• riak_pb/src/riak_pb_dt_codec.erl

Answer 1

The last argument to riakc-pb-socket:modify_type (source code) is a set of changes. If you already know which element you want removed it looks like you could, in theory, create a new empty set and build a remove operation

Empty = riakc_set:new(Context),
Removal = riakc_set:del_element(<<"Toronto">>,Empty),
Op = riakc_set:to_op(Removal),
riakc_pb_socket:update_type(Pid, {<<"sets">>, <<"travel">>}, <<"cities">>, Op).

The key here is the Context which is an opaque value generated by the server. You may be able to send the request without one, or with an empty one (<<>>), but that is probably not a Good Thing(tm). The context is how Riak determines causality. It is updated by each actor each time an action is taken and is used to determine the final consistent value. So if you send a set operation with no context it may fail or be processed out of order, especially if any other updates are happening around the same time.

In the case of the HTTP API the entire object is fetched by a coordinator to get the context, then the operation is submitted with that context.

When performing a regular get operation, you can specify head in the options to get back just the metadata, which include the context, but not the data. I haven't tested with fetch_type yet, but there may be similar functionality for convergent types. If there is, you would just need to fetch the head to get the context, and submit your operation with that context.

-EDIT-
According to the docs:

%% You cannot fetch a Data Type's context directly using the Erlang
%% client. This is actually quite all right, as the client automatically
%% manages contexts when making updates.

It would appear that you can pass a fun to riakc_pb_socket:modify_type so that you don't have to explicitly fetch the old value, but that will just fetch it behind the scenes, so you only really save a tiny bit of code.

riakc_pb_socket:modify_type(Pid,
            fun(MySet) -> riakc_set:del_element(lists:last(ordsets:to_list(riakc_set:value(MySet))), MySet)
            end, {<<"sets">>, <<"travel">>}, <<"cities">>,[create]).