0

I need help with writing code for a work project. I have written a script that uses pandas to read an excel file. I have a while-loop written to iterate through each row and append latitude/longitude data from the excel file onto a map (Folium, Open Street Map)

The issue I've run into has to do with the GPS data. I download a CVS file with vehicle coordinates. On some of the vehicles I'm tracking, the GPS loses signal for whatever reason and doesn't come back online for hundreds of miles. This causes issues when I'm using line plots to track the vehicle movement on the map. I end up getting long straight lines running across cities since Folium is trying to connect the last GPS coordinate before the vehicle went offline, with the next GPS coordinate available once the vehicle is back online, which could be hundreds of miles away as shown here. I think if every time the script finds a gap in GPS coords, I can have a new loop generated that will basically start a completely new line plot and append it to the existing map. This way I should still see the entire vehicle route on the map but without the long lines trying to connect broken points together.

My idea is to have my script calculate the absolute value difference between each iteration of longitude data. If the difference between each point is greater than 0.01, I want my program to end the loop and to start a new loop. This new loop would then need to have new variables init. I will not know how many new loops would need to be created since there's no way to predict how many times the GPS will go offline/online in each vehicle.

https://gist.github.com/tapanojum/81460dd89cb079296fee0c48a3d625a7

import folium
import pandas as pd

#  Pulls CSV file from this location and adds headers to the columns
df = pd.read_csv('Example.CSV',names=['Longitude', 'Latitude',])

lat = (df.Latitude / 10 ** 7)  # Converting Lat/Lon into decimal degrees
lon = (df.Longitude / 10 ** 7)

zoom_start = 17  # Zoom level and starting location when map is opened
mapa = folium.Map(location=[lat[1], lon[1]], zoom_start=zoom_start)

i = 0
j = (lat[i] - lat[i - 1])
location = []
while i < len(lat):
if abs(j) < 0.01:
    location.append((lat[i], lon[i]))
    i += 1
else:
    break

# This section is where additional loops would ideally be generated

# Line plot settings
c1 = folium.MultiPolyLine(locations=[location], color='blue', weight=1.5,         opacity=0.5)
c1.add_to(mapa)

mapa.save(outfile="Example.html")

Here's pseudocode for how I want to accomplish this.

1) Python reads csv

2) Converts Long/Lat into decimal degrees

3) Init location1

4) Runs while loop to append coords

5) If abs(j) >= 0.01, break loop

6) Init location(2,3,...)

7) Generates new while i < len(lat): loop using location(2,3,...)

9) Repeats step 5-7 while i < len(lat) (Repeat as many times as there are instances of abs(j) >= 0.01))

10) Creats (c1, c2, c3,...) = folium.MultiPolyLine(locations=[location], color='blue', weight=1.5, opacity=0.5) for each variable of location

11) Creates c1.add_to(mapa) for each c1,c2,c3... listed above

12) mapa.save

Any help would be tremendously appreciated!

UPDATE: Working Solution

import folium
import pandas as pd

#  Pulls CSV file from this location and adds headers to the columns
df = pd.read_csv(EXAMPLE.CSV',names=['Longitude', 'Latitude'])

lat = (df.Latitude / 10 ** 7)  # Converting Lat/Lon into decimal degrees
lon = (df.Longitude / 10 ** 7)

zoom_start = 17  # Zoom level and starting location when map is opened
mapa = folium.Map(location=[lat[1], lon[1]], zoom_start=zoom_start)

i = 1
location = []
while i < (len(lat)-1):
location.append((lat[i], lon[i]))
i += 1
j = (lat[i] - lat[i - 1])
 if abs(j) > 0.01:
    c1 = folium.MultiPolyLine(locations=[location], color='blue',    weight=1.5, opacity=0.5)
    c1.add_to(mapa)
    location = []

mapa.save(outfile="Example.html")
Tappy
  • 87
  • 1
  • 8
  • You should move your code to question. And also code snippets are preferably stored in [Gist Github](https://gist.github.com/) – vishes_shell Sep 22 '16 at 18:09
  • Please post the code in the question. Otherwise, when you delete the code from Github, this question becomes useless to everyone else. – CAB Sep 22 '16 at 18:12
  • This seems impossible as stated. If the GPS goes offline then you have no interim points with which to join up a route? Your second loop would be blind to any real data. If you're using OSM, use graphhopper or something similar to fill in the gaps? You'd be left with an assumption that the driver went through all the nodes of the graphhopper route – roganjosh Sep 22 '16 at 18:20
  • Thanks @vishes_shell , I just set up a Github page an hour ago and didn't know how to properly use/share it, or about the code formatting provided by stackoverflow. – Tappy Sep 22 '16 at 18:45
  • @roganjosh The data is static in an excel report. My thinking was that if I hit a point in the data where the GPS goes offline, the script will autogenerate a new loop and create a 2nd plot starting from the last cords of where the GPS came back online. OSM would then have multiple line plots instead of trying to connect broken data points in a single plot. – Tappy Sep 22 '16 at 18:48
  • @CAB thanks for the advice. I've edited my post as you suggested. – Tappy Sep 22 '16 at 18:49
  • @Arthur but I don't understand. At the end of the day, you have a start and an end GPS location. You can detect when GPS connection is broken, but you cannot meaningfully fill in that gap other than to assume that a driver followed a recommended route between those two points... and you can get that from something like graphhopper that operates on OSM data. – roganjosh Sep 22 '16 at 18:53
  • @roganjosh Sorry Josh, looks like I left out a bit of information. I'm not trying to fill in the gaps between the broken GPS locations. At this point, we don't know the exact reason for the missing gap per vehicle. I generically called it "GPS going offline" but if the vehicle is being shipped or towed, then the gps would be deactivated as well. This current assignment wants me to exclude filling in the gaps. I will check out graphhopper because I may need to fill in the gaps on future projects, thank you! – Tappy Sep 22 '16 at 19:02
  • @Arthur I honestly still don't know what you're trying to do, but I'll take your word for it for now that I have somehow helped :P If you want anything other than a straight line between the two GPS points, you'll surely need some graphing software to estimate nodes in between that lie on a road – roganjosh Sep 22 '16 at 19:13

1 Answers1

0

Your while loop looks wonky. You only set j once, outside the loop. Also, I think you want a list of line segments. Did you want something like this;

i = 0
segment = 0
locations = []
while i < len(lat):
    locations[segment] = []  # start a new segment

    # add points to the current segment until all are 
    # consumed or a disconnect is detected
    while i < len(lat):
        locations[segment].append((lat[i], lon[i]))
        i += 1
        j = (lat[i] - lat[i - 1])
        if abs(j) > 0.01:
            break
    segment += 1

When this is done locations will be a list of segments, e.g.;

 [ segment0, segment1, ..... ]

each segment will be a list of points, e.g.;

 [ (lat,lon), (lan,lon), ..... ]
CAB
  • 1,106
  • 9
  • 23
  • Ah, that makes so much sense! The only thing missing is the 2nd part. For each segment I need a line output stating: graph[segment]= folium.MultiPolyLine(locations=[location[segment]], color='blue', weight= 3, opacity=0.5) I'm going to try to work out this 2nd part now but still willing to accept help! Thanks again for your help. I've found myself in a new job working with code even though I have 0 CS background. Trying to wrap my head around all of this. – Tappy Sep 22 '16 at 20:00
  • I'm not familiar with folium, but I notice they have both `polyline` and `multipolyline` methods. I would think `multipolyline` might do exactly what you need, – CAB Sep 26 '16 at 13:32
  • Yeah you're right about the multipolyline. I've updated my post to show the working solution. – Tappy Sep 26 '16 at 17:53