Working with TEDA 4.2 architecture.How to call LookUpCache operator in a loop?

Question

I have input as a1;b1;c1|a2;b2;c2|a3;b3;c3

Now I want to use a1 , a2 and a3 as look up key and generate a tuple for each (as output). I have extracted a1,a2,a3 using tokenizer but how to feed it to LookupCache operator so that it uses all keys one by one and generate 3 tuples.

score 0 · Answer 1 · answered Oct 12 '17 at 11:14

Using the TEDA application framework, you need to prepare the TypesCommon.ReaderOutStreamType schema, which is the output of the parsing and the input to the data enrichment. It is extended with the attributes that are specified in the xxx.streams.custom::TypesCustom.LookupType type.

In xxx.streams.custom.TypesCustom.spl

    static LookupType = tuple<
        // ------------------------------------------------
        // custom code begin
        // ------------------------------------------------
        // add your custom attributes here
        // key attribute for LookupCache, e.g. a1, a2, a3 from input
        rstring a
        // ------------------------------------------------
        // custom code end
        // ------------------------------------------------
    >;

In xxx.chainprocessor.transformer.custom.DataProcessor.spl you can prepare extract the key attribute for your lookup, fill the a attribute and submit n tuples (LookupStream) to the LookupCache operator from a single input tuple (InRec)

    (stream<I> LookupStream as O) as PrepareLookup = Custom(InRec as I) {
        logic
            onTuple I: {
                mutable list<rstring> list1 = tokenize(attrInput, "|", true);
                for (rstring val in list1) {
                    mutable list<rstring> list2 = tokenize(val, ";", true);
                    I.a = list2[0];
                    submit(I, O); // submit for a1, a2 ...
                }   
            }
            onPunct I: {
                if (currentPunct() == Sys.WindowMarker) {
                    submit(Sys.WindowMarker, O);
                }
            }
    }


    stream<I> EnrichedRecords = LookupCache(LookupStream as I) {
        param
            segmentName:    "yourSeg"; // The name of the physical memory segment
            containerName:  "yourContainer"; // The name of the store
            useMutex:   false; // Don't protect data with mutex
            mutexName:  "";
            key:        I.a;
            keyType:        rstring;           // SPL type of key
            found:      I.lookupFound;     // attribute that takes lookup success
            valueType:  TypesCustom.YourType_t; //type of lookup table value as also defined in LookupManager
            disableLookup:  $disableLookup;   // always pass through this parameter
            applControlDir: $applControlDir;   // always pass through this parameter
    }

I am having an issue here.Suppose I have 10 buckets in input field and I need to generate output for those which have successful lookup(lets say 1 successful lookup in this case) , then with this approach all the streams will go to cpp once .It means 9 unwanted iterations will be done. Can we control that in some way? — shruti, Dec 05 '17 at 06:01
For example, input is like "1;0;20170528;|2;0;20170528;|3;0;20141231;|5;0;20170528;|6;5401;20141224;|20;120;20151203;|30;0;20170527;|39;2;20151206;|42;163101970;20150105;|43;0;20150105;|47;0;20141224;|48;0;20141224;|49;113100000;20141224;|50;0;20141224;|54;48114;20141224;|76;163100512;20141224;|88;162100000;20141224;|115;3;20170528;|" Out of these the bucket with 88 in first field has successful lookup.Now if we go with above given solution, code will iterate for all buckets unnecessary. We need output for 88 value only. @Mark.Heger Please see if you could help. — shruti, Dec 05 '17 at 06:05
If you need to quit the loop after a successful lookup, then you need to have one custom operator doing the loop and the lookup and exit the loop on a specific condition. For this you would need to copy the Operator code from the LookupCache into the Custom Operator doing the for-loop. — Mark.Heger, Dec 08 '17 at 12:01

Working with TEDA 4.2 architecture.How to call LookUpCache operator in a loop?

1 Answers1