I'm currently using graphframes
to generate a graph and then I need to find the paths between all vertices. (That is all pairs of vertices are tested to find the minimum path between them.)
Both bfs
and find
in graphframes
generate dataframes meaning they cannot be used in any kind of mapping function. Is there a way I can work around this? (For instance, doing a for loop on all RDD entries in a partition [mapPartitions
doesn't work, I tried it.])