0

We want to use a stream to insert data from a source table to multiple target tables. For that purpose, we have created steam on the source table.

We are using a Java script procedure for this. We are inserting stream data into a temporary table first and then we have written a few queries to insert data in multiple target tables using the temporary table.

If the procedure fails at any query, We are losing stream data as well. How can we protect stream data if the procedure fails?

Below is the sample code of my procedure

CREATE OR REPLACE PROCEDURE test_proc()
RETURNS string
LANGUAGE javascript
strict
EXECUTE AS owner
AS
$$
try {
    snowflake.execute({
        sqlText: "BEGIN NAME test_update;"
    });
        snowflake.execute({
            sqlText: "CREATE OR REPLACE TEMPORARY TABLE temp_1 as SELECT * FROM STREAM_TEST_1;"
        });
        
        snowflake.execute({
            sqlText: "DELETE FROM TAB_2 WHERE ID IN (Select ID from temp_1 where name = 'TEST');"
        });
        
        snowflake.execute({
            sqlText: "DELETE FROM TAB_3 WHERE ID IN (Select ID from temp_1 where name = 'DUMMY');"
        });
        
        snowflake.execute({
            sqlText: "INSERT INTO TAB_4 SELECT ID, NAME FROM temp_1 where name = 'ABC';"
        });

        snowflake.execute({
            sqlText: "INSERT INTO TAB_5 SELECT ID, NAME FROM temp_1 where name = 'XYZ';"
        });     

        
    return "success";
} catch (err) {
        
        snowflake.execute({
            sqlText: "ROLLBACK;"
        });         
}
$$;
YogeshR
  • 1,606
  • 2
  • 22
  • 43
  • Have you considered creating a separate stream for each DML and then removing the temp table create? Or, you could use a transient table instead of a temp table, which would then allow you to manually complete the remaining steps of the job before dropping the transient table. – Mike Walton May 28 '23 at 20:07
  • If any query fails, does the ROLLBACK statement get executed? If/when it does, are you still losing data from the stream? – NickW May 30 '23 at 14:04
  • I'm interested to know if you've find the answer, as I have the exact same issue. – zemir Jun 21 '23 at 22:37
  • @zemir I used a transient table instead of a temp table. – YogeshR Jun 22 '23 at 03:55

0 Answers0