Broadcast' object has no attribute 'destroy'?

Question

In my pyspark scirpt code, I declare a Broadcast variable. At last, I want to destory this variable, but get

Blockquote AttributeError: 'Broadcast' object has no attribute 'destroy'

My code like this：

br = sc.broadcast(my_value)
rdd.map(__my_map_function) # __my_map_function() used br variable
br.destroy()

My spark version is 1.6.1, and I know the class BroadCast has the destroy() method, but Why the code throw this exception? thanks.

score 3 · Accepted Answer · edited Jul 19 '17 at 07:15

3

The destroy method wasn't available in pyspark before spark-2.x.

You could check out the sources when in doubt.

edited Jul 19 '17 at 07:15

Jacek Laskowski

answered Jul 19 '17 at 06:47

eliasah

1 Answers1