0

I have a lot of telemetry data being sent by various sensors and it often happens that multiple parameters for a single device is changed within the same second (with millisecond differences). I want a method to merge these records for the same second in Azure stream analytics

I tried to add a trigger to the main SQL table,

ALTER TRIGGER [dbo].[trgOEESignals_updated_upsert] ON [dbo].[TBLOEESIGNALS] INSTEAD OF INSERT
AS
BEGIN
    set nocount on;
    MERGE TBLOEESIGNALS AS old
    USING inserted AS new
        ON  LTRIM(RTRIM(new.SITE_CODE)) = LTRIM(RTRIM(old.SITE_CODE)) and
                LTRIM(RTRIM(new.MACHINENAME)) = LTRIM(RTRIM(old.MACHINENAME))
        and DATEDIFF(SECOND, old.SIGNALRECEIVEDTIME,new.SIGNALRECEIVEDTIME) = 0
    WHEN MATCHED THEN 
        UPDATE SET
            old.REC_READ = 0, 
            old.CURRENTMODE = new.CURRENTMODE, 
            old.CURRENTSPEED= new.CURRENTSPEED, -- same logic for here
            old.TOTALPARTS = new.TOTALPARTS,
            old.PRODUCEDPARTSPERCYCLE= new.PRODUCEDPARTSPERCYCLE, 
            --- add a condition for cycle counter if 1 is present in existing or in the new one
            old.CYCLECOUNTER= new.CYCLECOUNTER

    WHEN NOT MATCHED THEN
        INSERT(signalreceivedtime,currentmode,currentspeed,totalparts,machinename,
                currentjob,producedpartspercycle,rec_read,site_code,scrap,
                cyclecounter,scrap_code,target_cycle_time)
        VALUES (new.SIGNALRECEIVEDTIME,new.CURRENTMODE,new.CURRENTSPEED,new.TotalParts,new.MachineName, COALESCE(new.CurrentJob, ''), new.PRODUCEDPARTSPERCYCLE,
        new.rec_read, new.site_code, new.scrap, new.cyclecounter, new.scrap_code, new.target_cycle_time);
    
END;

but still the same issue persists. I have noticed, there is a ASA function output also doing the similar approach and first wanted to try it directly on SQL to see the results. The sensor readings are continuous stream by nature and I dont know a good approach for now

0 Answers0