Converting CSV to KML for just lat and lon

Question

Below is the code I am using to covert my CSV file to KML into Google Earth Pro; however Google Earth Pro continues to crash when I upload to files. The amount of Lat Lon Coordinates are too great for Google Earth Pro to handle? Am I missing something in this code? I have a few CSV files with over 100,000 coordinates. Trying to avoid this error....

Error Message

import simplekml
import pandas
df=pandas.read_csv("FILE NAME.CSV")

kml=simplekml.Kml()

for lon,lat in zip(df["Longitude"],df["Latitude"]):
    kml.newpoint(coords=[(lon,lat)])

kml.save("OUTPUT.KML")

You can find out if your code is correct but the amount is just too big by adding a `break` in the `for` loop. It should then only export one point. If that is read correctly, you can try exporting smaller batches of files and check if these work. — Taxel, Aug 02 '21 at 12:22
It’s not a error it’s an informational message and it seems clear — Mark Tolonen, Aug 02 '21 at 14:19
@MarkTolonen fair point.... is there a way around it? I ask cause if I click import all Google Earth Pro just crashes. The amount of data is too much for it to handle — JRoth9125, Aug 02 '21 at 14:26
Probably not. If 2500 starts to cause performance issues, what do you expect exceeding that limit by 400x? — Mark Tolonen, Aug 02 '21 at 15:05
@Mark Tolonen that makes sense. I just tested it with 500 points and it works fine so it's certainly a performance issue. So if I write a code to export a KML file at every 2000 coordinates theoretically it should work. Kind of like exporting line vectors. Kind of... — JRoth9125, Aug 02 '21 at 15:18
The data import dialog is prompted to import CSV file not load the 100K point KML file. Does GE Pro crash when you load the KML created from python or just when you try to import the CSV file with it? — CodeMonkey, Aug 02 '21 at 19:48

CodeMonkey · Accepted Answer · 2021-08-03T02:03:10.983

Creating KML with over 100,000 placemarks could have performance issues in GE Pro. In your situation, your KML file is crashing GE Pro.

Importing CSV files directly into GE Pro is limited but customizing the KML as generated in Python gives you many more options which help you tailor the output to your needs.

KML provides mechanisms to control which features or sub-KML files are displayed using time, region, or altitude level filtering.

You need to consider what makes sense based on the characteristics of your data. If your data is geographically spread out and you can bin the data into different geographic sub-areas then each sub-area can be written to a separate KML file, and the main KML file can have a networklink to each of the sub-kml files but not visible when first loaded.

Large KML files can scale using the following techniques:

NetworkLinks
Regions

SimpleKML supports creating NetworkLinks and Regions. You can use shapely module to determine if a point is inside a rectangle or polygon to bin the points to the appropriate sub-area.

NetworkLinks

A NetworkLink allows a KML file to include another KML file. You can have any level of NetworkLinks from within your root KML file from flat (single KML file with Networklinks to all other KML files within the KMZ) to deep (with each KML file with a NetworkLink to other KML files each with its own NetworkLink). Depends on how you need to structure your KML and how large the data is.

The key is that Google Earth chooses the first KML as the root KML file so you must ensure that the first file (typically named doc.kml) is the root KML file that loads the other KML files via network links. A common structure is to include additional KML files in a "kml" sub-folder to differentiate it from the root KML file if packaging them in a KMZ file.

A tutorial on Using Network Links Effectively
https://www.google.com/earth/outreach/learn/using-network-links-effectively/

Regions

A Region affects visibility of a Placemark's geometry or an Overlay's image. Regions combined with NetworkLinks enable access to massive amounts of data in KML files. A region can optionally have a min and max altitude for altitude level filtering.

See the documentation on Regions, section on "Smart" Loading of Region-Based Network Links and a tutorial on Regions in KML.

Regions are a more advanced feature of KML, so would recommend first looking into NetworkLinks and creating multiple-kml files if there is a logical way to split your data into groups.

Converting CSV to KML for just lat and lon

1 Answers1