How do I load data from a Java object to Kudu table?

Question

I have a Java code where I am converting my JSON string to a Java object. I am storing the values of that string to the object.

What I need to do next is store these values in a Kudu table. I just want to know how can this be done using Docker containers if possible. Or is there any other way this can be done.

Below is the code I have written:

import com.google.gson.Gson;
import java.util.Map;

class UserDetails {public String id; public int type; public Map<String, String> source; public String source_uri; public long timestamp; public String options; public Map<String, String> payload;}

public class JSONToJSONObject {
    public static void main(String[] args) {
        String json = "{'id':'46B56A42-6500-6500-59DF-68E26AD11BCE','type':1,'source':{'org':'JJV','site':'US11','area':'MFG','gen':'3GT','line':'TAM04','cell':'LF','zone':'Z2B','process':'Takeover'},'source_uri':'JJV/US11/MFG/3GT/TAM04/LF/Z2B','timestamp':1557254610759,'options':null,'payload':{'dcp_id':'DCP003','trigger_count':1593,'lot_number':'B00SDZK','pallet_id':0,'status_code':1,'station_code':1,'pallet_ts':1557254568000,'mold_open_1_ts':1557254570000,'mold_open_2_ts':1557254568000,'mold_open_3_ts':1557254573000,'mold_open_4_ts':1557254570000,'butterfly_pick_a_ts':1557254574000,'butterfly_pick_b_ts':1557254572000,'robot_pick_1_ts':1557254572000,'robot_pick_2_ts':1557254574000,'pickpallet_xfer_ts':1557254610764,'x_200ps_vac_peak':0,'x_201ps_vac_peak':0,'x_202ps_vac_peak':0,'takeover_xfer_1_PS206':0,'takeover_xfer_2_PS206':0,'takeover_xfer_3_PS206':0,'nest_a_vac':0,'nest_b_vac':0,'mold_open_1_hs':9,'mold_open_2_hs':7,'mold_open_3_hs':2,'mold_open_4_hs':9,'butterfly_pick_a_hs':4,'butterfly_pick_b_hs':5,'robot_pick_1_hs':5,'robot_pick_2_hs':4,'pallet_xfer_hs':0}}";
        Gson gson = new Gson();
        UserDetails user = gson.fromJson(json, UserDetails.class);
        System.out.println("ID is: " + user.id);
        System.out.println("Type is: " + user.type);
        System.out.println("Source is: " + user.source);
        System.out.println("Source URI is: " + user.source_uri);
        System.out.println("Timestamp is: " + user.timestamp);
        System.out.println("Options is: " + user.options);
        System.out.println("Payload is: " + user.payload);
    }
}

Also, after installing Docker on my Ubuntu VM, I am trying to follow the steps given in the https://kudu.apache.org/docs/quickstart.html page. But cloning the git repo is throwing an error as below:

fatal: pack has bad object at offset ____: returned -5

score 0 · Answer 1 · answered Nov 10 '20 at 22:15

I created an example and documented it in my github repo. In order to launch kudu locally I use andreysabitov/impala-kudu docker image and start the container using following docker command:

docker run -d --name kudu-impala \
    -p 8050:8050 -p 8051:8051 -p 7050:7050 -p 7051:7051 \
    -p 25000:25000 -p 25010:25010 -p 25020:25020 \
    -p 50070:50070 -p 50075:50075 \
    -p 21050:21050 -p 28000:28000 \
    -e HOSTNAME=localhost \
    --hostname localhost \
    andreysabitov/impala-kudu:latest

Storing the java object in Kudu is done in this class. The part responsible for translating object into Kudu column looks like this:

KuduTable table = clientProvider.get().openTable(tableName());
Upsert upsert = table.newUpsert();
upsert.getRow().addString("id", userInfo.getId());
upsert.getRow().addString("name", userInfo.getName());
upsert.getRow().addLong("counter", userInfo.getCounter());
session.apply(upsert);

How do I load data from a Java object to Kudu table?

1 Answers1