I have an avro input_stream something like this:
VERSION STRING,
ID STRUCT<
NAME STRING,
STATUS STRING
>,
ROLES ARRAY<
STRUCT<
DESC STRING,
TYPE STRING
>
>
and I want to falatten it and do some filtering as well.
This is working:
select version, explode(roles)->type from input_stream where version='1.0' emit changes;
but this fails:
select version, explode(roles)->type from input_stream where explode(roles)->type='master' emit changes;
Invalid Predicate: Can't find any functions with the name 'EXPLODE'. expression:(explode(roles)->type='master')
How can I filter directly the array elements based on type without creating a new stream that will query and filter from this exploded one?
Thanks!
Example how to reproduce:
>create stream test_for_robin_moffatt (VERSION STRING,ID STRUCT<NAME STRING,STATUS STRING>,ROLES ARRAY<STRUCT<DESC STRING,TYPE STRING>>) with (kafka_topic='test_for_robin_moffatt', value_format='json', partitions=1);
>insert into TEST_FOR_ROBIN_MOFFATT (VERSION,ID, ROLES) values ('1.1', STRUCT(name:='zoltan', status:='active'), ARRAY[STRUCT(desc:='blabla', type:='master')]);
>SET 'auto.offset.reset' = 'earliest';
>select version, id->name, id->status, explode(roles)->desc, explode(roles)->type from TEST_FOR_ROBIN_MOFFATT emit changes;
>select version, id->name, id->status, explode(roles)->desc, explode(roles)->type from TEST_FOR_ROBIN_MOFFATT where version='1.1' emit changes;
>select version, id->name, id->status, explode(roles)->desc, explode(roles)->type from TEST_FOR_ROBIN_MOFFATT where id->status='active' emit changes;
The statements above work fine:
+---------------------------------------------+---------------------------------------------+---------------------------------------------+---------------------------------------------+---------------------------------------------+
|VERSION |NAME |STATUS |DESC |TYPE |
+---------------------------------------------+---------------------------------------------+---------------------------------------------+---------------------------------------------+---------------------------------------------+
|1.1 |zoltan |active |blabla |master |
But, this fails:
>select version, id->name, id->status, explode(roles)->desc, explode(roles)->type from TEST_FOR_ROBIN_MOFFATT where explode(roles)->type='master' emit changes;
Invalid Predicate: Can't find any functions with the name 'EXPLODE'. expression:(EXPLODE(ROLES)->TYPE = 'master'), schema:`VERSION` STRING, `ID` STRUCT<`NAME` STRING, `STATUS` STRING>, `ROLES` ARRAY<STRUCT<`DESC` STRING, `TYPE` STRING>>, `ROWTIME` BIGINT