1

I am new JSON and Cassandra part i am doing a POC in which i receive a stream of JSON object. I have a configuration JSON which is like the subset of the incoming JSON Object. Right now what i am doing is i have method which will extract the required fields from JSON object and map to a POJO object like below

@Table(keyspace = KEYSPACE_NAME, name = DATA_HISTORY_TABLE_NAME, readConsistency = QUORUM,
        writeConsistency = QUORUM,
        caseSensitiveKeyspace = false,
        caseSensitiveTable = false)
public class Table1 implements Serializable {
    @PartitionKey
    @Column(name = "FIRSTNAME")
    private String firstName;

    @Column(name = "USER_ID")
    private String userId;

    @Column(name = "TRASACTION")
    private double transactionAmount;

    @Column(name = "CREATED_TIMESTAMP")
    private Date createdDateTime;

    @Column(name = "CREATED_DATE")
    private LocalDate createdDate;
}

Everything is going good now. I am trying to make this more dynamic way, i have configuration file stored in database which will be pulled through API call during run time. The sample of configuration file structure looks like below and i can have multiple conditions with JSON paths

{
    "Condition1": {
        "path": ["JSON PATH OF VALUE I WANT TO FETCH"]
    }
}

Sample Condition:

{
    "Condition1": {
        "path": ["firstName"]
    },
    "Condition2": {
        "path": ["userID"]
    }
}

Incoming JSON can have any values but i want to extract above two values from the incoming JSON object stream and store to Cassandra database. I thought of a solution in which converting the incoming JSON object to a java POJO class and extract the information but since this is a streaming platform this might kill JVM. Please suggest me any efficient way to dynamically map the JSON attributes. Thanks in advance.

user13906258
  • 161
  • 1
  • 13

1 Answers1

0

Since you have a streaming use case, have a look at the DataStax Apache Kafka Connector.

With the Kafka connector, you can map the JSON fields to CQL columns. Data from the stream will then get written to Cassandra. Cheers!

Erick Ramirez
  • 13,964
  • 1
  • 18
  • 23