DataFrame based graph library for Apache Spark
GraphFrames is DataFrame base alternative to core GraphX with cross language support:
External resources:
Related tags:
Questions tagged [graphframes]
186 questions
2
votes
1 answer
Using GraphFrames (Scala) to compute hierarchy
I have a dataframe below:
employee_id|employee_name|manager_employee_id|
----------------------------------------------
1 eric (ceo) 1
2 edward 1
3 john 1
4 james …
Edd
- 65
- 5
2
votes
1 answer
Computing PageRank on a digraph with edge weights using GraphFrames
Assume I use GraphFrames to construct a digraph g with edge weights from the positive real numbers. I would then like to compute the PageRank with taking the edge weights into account. I don't see how this can be achieved by looking at the…
NahsiN
- 119
- 3
2
votes
1 answer
Spark graphFrames - Label Propagation vs. Strongly Connected Components
In the https://docs.databricks.com/spark/latest/graph-analysis/graphframes/user-guide-scala.html standard example:
The Strongly Connected Components seem reasonable computationally when looking also at them visually on a drawing.
Therefore I am…
thebluephantom
- 16,458
- 8
- 40
- 83
2
votes
1 answer
How to find the hierarchy levels of a person(employee,manager etc.) using graphframes in pyspark?
I have a graph frame with vertices and edges as below. I am running this on pyspark in jupyter notebook.
vertices = sqlContext.createDataFrame([
("12345", "Alice", "Employee"),
("15789", "Bob", "Employee"),
("13467", "Charlie",…
Joe14_1990
- 89
- 1
- 9
2
votes
2 answers
PySpark exception with GraphFrames
I am building a simple Network Graph with PySpark and GraphFrames (running on Google Dataproc)
vertices = spark.createDataFrame([
("a", "Alice", 34),
("b", "Bob", 36),
("c", "Charlie", 30),
("d", "David", 29),
("e", "Esther", 32),
("f",…
Alex
- 1,447
- 7
- 23
- 48
2
votes
3 answers
How to make GraphFrame from Edge DataFrame only
From this, "A GraphFrame can also be constructed from a single DataFrame containing edge information. The vertices will be inferred from the sources and destinations of the edges."
However when I look into its API doc, it seems there is no way to…
Muhammad Mohib Khan
- 89
- 2
- 13
2
votes
1 answer
sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.11;2.1.2: not found
I want to use the connected components algo in spark 2.0. However, the packaging with sbt 0.13.15 got dependency errors. What's the cause behind this? The build.sbt file is like this:
lazy val root = (project in file(".")).
settings(
name :=…
Shuaili WANG
- 21
- 2
2
votes
0 answers
Efficient GraphFrame for different types of vertices
I need to create a graphframe in spark databricks which will have 2 types of vertices lets say N1 and N2 with different properties.
Currently I am creating a new dataframe with all columns from both N1 and N2 with relevant values as given and rest…
sopho-saksham
- 171
- 1
- 11
2
votes
1 answer
How to get MapType from column
I have working with graphframes and now I am using aggregate Message. The vertex schema is:
|-- id: long (nullable = false)
|-- company: string (nullable = true)
|-- money: integer (nullable = false)
|-- memoryLearned: map (nullable = true)
| …
David González
- 39
- 5
2
votes
0 answers
GraphFrame Spark : Get Subgraph from specific node
I'm building an simple Graph with GraphFrames on Scala 2.11 Spark 2.2.
I can create my graph without problems, but i have no idea how create a subgraph from input user.
I want to extract graph from the big one, like :
Get subgraph from node#123…
Gohmz
- 1,256
- 16
- 31
2
votes
0 answers
GraphFrames shortestPaths fine in local mode, but an error when done on a cluster
I am learning Apache Spark and graphframes trying to get shortestPaths to work on my 6 node cluster using the code provided here https://graphframes.github.io/user-guide.html
import org.graphframes.examples
val g: GraphFrame =…
ECSimm
- 21
- 3
2
votes
1 answer
How to create Directional graph with Spark Graphx or Graphframe
I'm trying to run the connected component algorithm on my dataset but on a directional graph. I don't want the connected component to transverse in both direction of the edges.
This is my sample code
import org.apache.log4j.{Level,…
Philip K. Adetiloye
- 3,102
- 4
- 37
- 63
2
votes
1 answer
spark graphframes stateful motif
graph frames has a nice example for stateful motifs.
How can I explicitly return the counts? As you see the output only contains vertices and friends but not the counts.
How can I modify it to not (only) have access to the edges but access to the…
Georg Heiler
- 16,916
- 36
- 162
- 292
2
votes
1 answer
Apache-Spark Graph-frame is very slow on BFS
I am using the Apache Spark-GraphFrames using Scala in the following Code, I am applying the BFS on above code and try to find the distance between Vertice 0 to 100.
import org.apache.spark._
import org.graphframes._
import…
Yasir Arfat
- 645
- 1
- 8
- 21
2
votes
1 answer
Graphframes BFS issue
I was testing graphframes BFS toy example:
val g: GraphFrame = examples.Graphs.friends
val paths: DataFrame = g.bfs.fromExpr("name = 'Esther'").toExpr("name <> 'Esther'").run()
The result I get is:
+-------------+------------+------------+
| …
Daniel
- 127
- 1
- 9