GeoSpark is a cluster computing system for processing large-scale spatial data. GeoSpark extends Apache Spark / SparkSQL with a set of out-of-the-box Spatial Resilient Distributed Datasets (SRDDs)/ SpatialSQL that efficiently load, process, and analyze large-scale spatial data across machines.
Questions tagged [geospark]
45 questions
1
vote
1 answer
Apache Sedona (Geospark) SQL with Java: ClassNotFoundException during SQL statement
I use the newest snapshot of Apache Sedona (1.3.2-SNAPSHOT) to do some geospatial work with my Apache Spark 3.0.1 on a docker cluster.
When trying out the first example in the tutorials section (http://sedona.apache.org/tutorial/sql/), I am…
mst94
- 21
- 6
1
vote
2 answers
How to join a point to nearest polygon boundary
I am trying to join two spatial datasets. The first contains points and the second polygons.
However, some of the points are outside of the polygons.
Is there a simple way of joining/snapping these points to the nearest polygon boundary , not the…
Stefan Gerber
- 11
- 2
1
vote
1 answer
GeoSpark using Maven UDF running Databricks on Azure?
I'm trying out Databricks@Azure with spark cluster:
Spark 3.0.0, Scala 2.12
On the cluster(!) I installed:
geospark:1.3.1
geospark-sql_2.3:1.3.1
Inspired by https://databricks.com/notebooks/geospark-notebook.html
I love SQL and would like to run…
Bjørn D. Jensen
- 21
- 2
1
vote
1 answer
GEOMETRY ST_GeomFromWKB(binary wkb) on spark using Java Language
I have a dataframe with Binary column type…
melissa maya
- 119
- 10
1
vote
1 answer
ST_WITHIN using Spark / Java
I have the following dataframe :
+-------------+-----------------+------------------+
|longitude |latitude |geom |
+-------------+-----------------+------------------+
|-7.07378166 |33.826661 [00 00 00 00 01…
HBoulmi
- 333
- 5
- 16
1
vote
0 answers
GeoSpark librairy using Spark Java
I have the following dataframe o spark :
__________________________
│Longitude │latitude |
├────────────┼─────────────┼
| -7.07378166|33.826661 |
└────────────┴─────────────┴
I want to apply the ST_Geomfromtext from GeoSpark ,…
HBoulmi
- 333
- 5
- 16
1
vote
1 answer
Pyspark: how to improve spatial intersection?
I am working with pyspark on Databriks where I have a table of data points that looks like the following
pingsGeo.show(5)
+--------------------+--------------------+----------+--------------------+
| ID| point| …
emax
- 6,965
- 19
- 74
- 141
0
votes
2 answers
How to find the closest geospatial line to a geospatial point
Context
I have 500,000+ road objects for all the roads in the state of Illinois that have a Geoshape property for a line. I additionally have a set of objects for points across the state.
Need
I would like to add to the backing dataset of the points…
lectrician1
- 77
- 1
- 13
0
votes
1 answer
use geospark sql to calculate the distance between two points
I use geospark sql to calculate the distance between two points
like this:
sql("""
SELECT ST_Distance(ST_Transform(ST_GeomFromText('Point (102.756444 25.038064 )'),"epsg:4236","epsg:3857"),ST_Transform(ST_GeomFromText('Point (102.745025 25.040838…
mentongwu
- 463
- 2
- 7
- 21
0
votes
1 answer
Geospark IllegalArgumentException: Number of partitions must be >= 0
I'm trying to run a simple intersect on a couple of tables with geometries and get this error.
IllegalArgumentException: Number of partitions must be >= 0
My script.
tableA.
join(tableB, expr("ST_Intersects(geom, point)")).
show
This is table A.…
gjin
- 860
- 1
- 14
- 28
0
votes
1 answer
Scala multipolygon to h3
I'm following this tutorial https://www.databricks.com/notebooks/geomesa-h3-notebook.html and have encountered a problem. Namely the multiPolygonToH3 function fills only the first polygon.
This is my multiPolygon
MULTIPOLYGON(((37.60574672029919…
gjin
- 860
- 1
- 14
- 28
0
votes
1 answer
Wrong computed azimut for geo coordinates using Apache Sedona in Spark
I am running a Spark application, using Apache Sedona.
I am trying to compute the bearing/azimuth between two coordinates, using following script:
import…
Christian Schou Jødal
- 53
- 1
- 6
0
votes
1 answer
scala 2.12 + Spark 3 + sedona-sql-3.0_2.12
I'm trying to use sedona with scala and spark. Here is the build.sbt file:
ThisBuild / scalaVersion := "2.12.12"
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.12" % "3.0.1",
"org.apache.spark" % "spark-sql_2.12" % "3.0.1",
…
IcePack
- 3
- 2
0
votes
1 answer
Read and Write sdo_geometry field in spark/GeoSpark(Sedona) from Oracle Table
i'm using geospark(sedona) with pyspark:
is possible read from Oracle a sdo_geometry type and write in a table in Oracle with sdo_Geometry field?
in my app:
i'm able to read :
db_table = "(SELECT sdo_util.to_wktgeometry(geom_32632) geom FROM …
Catanzaro
- 1,312
- 2
- 13
- 24
0
votes
1 answer
Firebase messaging: subscribed topics with Location Triggered Notifications
I'm using firebase cloud messaging and its notification mechanism
I've the following scenario, can you please help?
My android application allows its users to subscribe to interests (like running,movies,RESTAURANT): by using Firebase messaging…
aymenhs
- 31
- 5