SOLVED All it takes is quite a bit of patience (1 hour or so)
I'm working through the on the fly decryption example from strmprivacy.io which works fine, but the lineage tab in BigQuery shows a stand-alone table, instead of something derived. This is my query, and the resulting table is fine. But no lineage at all.
CREATE TABLE on_the_fly_decryption.decrypted_retail_consent_ge_1 AS
SELECT * FROM (
SELECT
e.strmMeta_keyLink,
DETERMINISTIC_DECRYPT_STRING(k.key, FROM_BASE64(e.Customer_ID), '') AS decryptedCustomerID,
e.Customer_ID,
( SELECT MAX(CAST(num AS Int64)) FROM
UNNEST(JSON_VALUE_ARRAY(e.strmMeta_consentLevels)) AS num ) max_consent
FROM
on_the_fly_decryption.encrypted e
INNER JOIN (
SELECT
k.keyLink,
KEYS.KEYSET_FROM_JSON( k.encryptionKey ) key
FROM
on_the_fly_decryption.keys k) k
ON e.strmMeta_keyLink = k.keyLink
)
WHERE max_consent >= 1;
This should work according to the docs.
Data lineage is enabled for my project. Any hints of what (is going|am I doing) wrong?
P.S the query is a slightly modified one, for the UCI online retail II set instead of the earlier version, but it's pretty much equal.
