mapnik vector tile vs raw geojson performance

Question

I am experiencing > 2s processing time for converting a 120 MB geojson file to protobuf through Mapnik Vector Tile node bindings.

On the other end serving the raw geosjon file takes under 200 ms.

Is it normal ? If yes, what is the point of serving vector tiles over geojson (I am viewing it with mapbox-gl-js)?

Here is an extract of my code :

// Load GeoJson into memory

var fs = require("fs");
var content = JSON.parse(fs.readFileSync("us_counties.json"));


// Initialise Mapnik and mercator object

mapnik.register_default_fonts();
mapnik.register_default_input_plugins();

var mercator = new SphericalMercator({
  size: 256
 });

// Vector Tile Router

router.get('/:z/:x/:y.pbf', function(req, res) {

  var bbox = mercator.bbox(
    +req.params.x,
    +req.params.y,
    +req.params.z,
    false,
    '4326'
  );

  // Convert GEOJSON to protobuf VectorTile and Serve

  var vtile = new mapnik.VectorTile(+req.params.z, +req.params.x, +req.params.y)
  vtile.addGeoJSON(JSON.stringify(content), 'fixture_layer')

  res.setHeader('Content-Encoding', 'deflate')
  res.setHeader('Content-Type', 'application/x-protobuf')


  zlib.deflate(vtile.getData(), function(err, pbf) {
    res.send(pbf);
  })
});

score 5 · Accepted Answer · answered Aug 08 '16 at 18:22

5

GeoJSON and vector tiles each have strengths and weaknesses as data formats for Mapbox GL.

vector tiles require more preprocessing than GeoJSON
GeoJSON must be downloaded in its entirety whereas vector tiles can be downloaded in tiles as needed to fill the viewport (it would be near impossible to download all of Open Street Map as a single X00 GB GeoJSON!)
GeoJSON may be modified clientside, whereas vector tiles cannot
GeoJSON is human-readable whereas vector tiles are not

If GeoJSON is working for you, I encourage you to use it! Just be aware of cases where your users may be unable to quickly download 120 MB.

answered Aug 08 '16 at 18:22

Lucas Wojciechowski

3,695
16
19

Thanks, however in this particular case, am I doing anything wrong, ie is it normal that mapnik takes so much time to process the file ? Thanks for your answer. – Clouddie Aug 08 '16 at 18:50
It's a very CPU bound operation on a lot of serial data the way you are going at it so, yes, it is not surprising to hear the operation takes so long. As far as what you are doing wrong, that amount pre-processing, you probably want to save off the result or cache it. The binary encoding used by MVT is very efficient for transfer but requires pre-processing you are discarding by answering the query without storing off the result for reuse. To increase cache-ability, that binary, difference encoded format is broken into tiles (and there you have it, MVT). – marr75 Jun 09 '17 at 15:59

score 1 · Answer 2 · answered Nov 16 '16 at 18:11

Vector tile is NOT lossless, vector tile simplify the geometry also round the corner to tile grid intersection. So here comes problem, some of my polygon in vector tile are not correctly drawing. You must be aware of this problem, because it makes my parcels not usable since always I can easily found some parcels lines are not correctly drawing.

second vector tile only support by mapbox GL js, geojson supported by google map, openlayer, leaflet, bing map, mapquest, etc....mapbox js.

If you worry about performance of geojson, here is the solution:

on server side, use geobuf, to serialize(encode) geojson to protocol buffer binary format, it compress to 20% of original geojson size. 120MB-geojson- 20MB geobuf.

transfer 20MB geobuf from server via internet to client browser, use geobuf(javascript lib) again to decode to geojson.

In brower, render geojson directly with google map, bing map, openlayer, leaflet, etc....

This give you a lossless and quick response solution.

This is utter madness. If you're going to the trouble of encoding/decoding geoJSON from protofbufs then you should just use vector tiles and be done with it. Your approach leaves you with none of the benefits of geoJSON and none of the benefits of vector tiles. If your data is so big that you're worrying about transfer time, that's when you should be using tiles. — JeneralJames, Feb 25 '20 at 10:40

mapnik vector tile vs raw geojson performance

2 Answers2