2

I am new on Apache Nifi

We create Nifi flow, which is consuming json data from kafka and the results are being sent to another kafka topic after enrichment. However HBase lookup does not return value of key. Instead it returns key, value pair like MapRecord[{SERIAL_NUM=123456789}]. But I require only value as '123456789'.

I could not resolve this issue. can anybody help me about this topic?

I am sharing the output of my flow. We want to see value of "hbase_integid" as "123456789", but lookup service returns

MapRecord[{SERIAL_NUM=123456789}]"

How can we extract only value of SERIAL_NUM value ("123456789")??

[
  {
    "table" : SIEBEL.S_ASSET,
    "op" : Update,
    "hbase_integid" : **MapRecord[{SERIAL_NUM=123456789}]**,
    "op_type" : U,
    "op_ts" : 2018-04-02 05:48:35.055299,
    "current_ts" : 2018-04-02T08:48:40.071000,
    "pos" : 00000000020530642196,
    "before" : {
        "ROW_ID" : 1-G7B7EGF,
        "BILLACCT_ID" : ,
        "BILL_ACCNT_ID" : 1-G79GNWP,
        "BILL_PROFILE_ID" : 1-1FJHFB0,
        "INTEGRATION_ID" : 1-G79GU5K,
        "NAME" : ,
        "OWNER_ACCNT_ID" : 1-G79GEVV,
        "OWNER_CON_ID" : 1-G79GEW3,
        "PROD_ID" : 1-Q5B470,
        "PROMOTION_ID" : ,
        "PROM_INTEG_ID" : ,
        "PROM_ITEM_ID" : ,
        "PR_ACCNT_ID" : ,
        "PR_CON_ID" : No Match Row Id,
        "ROOT_ASSET_ID" : 1-G7B7EFI,
        "SERIAL_NUM" : ,
        "X_VF_MSISDN" : ,
        "X_VF_PERMISSONS" : 
    },
    "after" : {
        "ROW_ID" : 1-G7B7EGF,
        "BILLACCT_ID" : ,
        "BILL_ACCNT_ID" : ,
        "BILL_PROFILE_ID" : ,
        "INTEGRATION_ID" : ,
        "NAME" : ,
        "OWNER_ACCNT_ID" : ,
        "OWNER_CON_ID" : ,
        "PROD_ID" : ,
        "PROMOTION_ID" : ,
        "PROM_INTEG_ID" : ,
        "PROM_ITEM_ID" : ,
        "PR_ACCNT_ID" : ,
        "PR_CON_ID" : ,
        "ROOT_ASSET_ID" : ,
        "SERIAL_NUM" : ,
        "X_VF_MSISDN" : ,
        "X_VF_PERMISSONS" : 
    }
  }
]
Sivaprasanna Sethuraman
  • 4,014
  • 5
  • 31
  • 60
  • `How can we extract only value of SERIAL_NUM value ("123456789")??` -- it's not clear is your question to `hbase` or to `nifi` ? – daggett Apr 06 '18 at 15:04
  • My queston related with Nifi, I am trying to enrich data using LookupRecord. Habse Lookup Services has been enabled to get value of the Key field. – erkan.oktay Apr 09 '18 at 05:52

2 Answers2

0

In case your are using LookupRecord processor, the documentation is somewhat misleading. Set up your processor as shown in the screenshot. Ensure the following two properties are set propery:

  1. Record result contents must be set to 'Insert record fields'
  2. Record update strategy should be set to 'Use property'
  3. Lastly Result Recordpath must be set to / and not hbase_integid (This is a bug in documentation it seems)

Hope that helps!

enter image description here

MojoJojo
  • 3,897
  • 4
  • 28
  • 54
0

The Lookup Record is giving output as a MapRecord; you may need to use other processors subsequently to extract the value of SERIAL_NUMBER.

A working example is given below.

enter image description here

HBase Lookup Configuration

enter image description here

Once you have result from lookup, you can use 2 UpdateRecord processors to extract and update the value of SERIAL_NUM in the flow file.

Update Record 1 configuration

enter image description here

Update Record 2 configuration

enter image description here

Akhil
  • 498
  • 2
  • 6
  • 22