In our Data Science project we are playing around pandas dataframe, numpy and scipy libraries and we want to change the code into Pyspark, We are facing issues like:
wst = cur_buck[:, [0]]
cur_buck[:, :-1] = cur_buck[:, 1:] - wst
cur_buck[:, -1] = cur_buck[:, -2]
number_of_particles=100
particles_matrix[10] = cur_buck[:, -1].reshape(number_of_particles).copy()
Same thing how can we achieve in Pyspark?
Any leads would be highly appreciable like these 4 lines I want to convert in pyspark.
Thanks in advance