Questions tagged [glue-streaming]

2 questions

vote

1 answer

Streaming ETL jobs in AWS Glue, How does it cost?

I want to use AWS Glue job that streams data from kafka source (it runs continuously). How much does it cost ? (let's suppose that we're using the minimum possible DPUs) Any idea please

apache-spark aws-glue glue-streaming

asked Dec 12 '22 at 11:15

Smaillns

2,540
1
28
40

votes

1 answer

pyspark.sql.utils.AnalysisException: 'writeStream' can be called only on streaming Dataset/DataFram

I am having glue streaming job, and I need to write the data as stream but after applying some processing, so I did the following: data_frame_DataSource0 = glueContext.create_data_frame.from_catalog( database=database_name, …

pyspark amazon-kinesis apache-hudi glue-streaming

asked Feb 12 '23 at 14:15

Mee

1,413
5
24
40