Javascript to download and process GTFS zip file

Question

I try to download, unzip and process a GTFS file in zip format. Downloading and unzipping are working, but I get error message when try to use txt files with gtfs-utils module in gtfsFunc(). Output is undefined. Delays are hardcoded just for testing purpose.

const dl = new DownloaderHelper('http://www.bkk.hu/gtfs/budapest_gtfs.zip', __dirname);
dl.on('end', () => console.log('Download Completed'))
dl.start();

myVar = setTimeout(zipFunc, 30000);
function zipFunc() {
    console.log('Unzipping started...');
    var zip = new AdmZip("./budapest_gtfs.zip");
    var zipEntries = zip.getEntries();
    zip.extractAllTo("./gtfsdata/", true);
  }

myVar = setTimeout(gtfsFunc, 40000);
function gtfsFunc() {
    console.log('Processing started...');
    const readFile = name => readCsv('./gtfsdata/' + name + '.txt')
    const filter = t => t.route_id === 'M4'

    readStops(readFile, filter)
    .then((stops) => {
        const someStopId = Object.keys(stops)[0]
        const someStop = stops[someStopId]
        console.log(someStop)
    })
}

Can you give more details on readCsv process? Just to make sure that the readCsv process is working well first. Like this: ```readCsv('./gtfsdata/' + name + '.txt') .on('error', console.error) .on('data', console.log)``` — ChickenSoups, Aug 10 '20 at 08:22
@ChickenSoups : No, I get same error, undefined. I checked path again, given folder is located in project root. — plaidshirt, Aug 10 '20 at 10:31
It's possible to you show us the readCsv and readStops functions also? So this way we could reproduce your code 100%. — Luis Paulo Pinto, Aug 11 '20 at 13:00
@LuisPauloPinto : These are part of module: https://github.com/public-transport/gtfs-utils#readcsvpath — plaidshirt, Aug 12 '20 at 07:37

ChickenSoups · Answer 1 · 2020-08-10T14:24:35.083

1

Stop data don't have route_id field.

You should try other data, such as Trip or Route

You can look at the first row in your data file to see which field do they have.

GTFS data structure here

edited Aug 10 '20 at 14:24

answered Aug 10 '20 at 14:04

ChickenSoups

937
8
18

score 1 · Accepted Answer · answered Aug 12 '20 at 10:38

As @ChickenSoups said, you are trying to filter stops files with route_id field and this txt doesnt have this field.

The fields that stops has are:

stop_id, stop_name, stop_lat, stop_lon, stop_code, location_type, parent_station, wheelchair_boarding, stop_direction

Perhaps what you need is read the Trips.txt file instead Stops.txt, as this file has route_id field. And you can accomplish this using readTrips function:

const readTrips = require("gtfs-utils/read-trips");

And your gtfsFunc would be:

function gtfsFunc() {

  console.log("Processing started...");
  const readFile = (name) => {
    return readCsv("./gtfsdata/" + name + ".txt").on("error", console.error);
  };

  //I used 5200 because your Trips.txt contains routes id with this value
  const filterTrips = (t) => t.route_id === "5200";

  readTrips(readFile, filterTrips).then((stops) => {
    console.log("filtered stops", stops);
    const someStopId = Object.keys(stops)[0];
    const someStop = stops[someStopId];
    console.log("someStop", someStop);
  });
}

Or if what you really want is to read Stops.txt, you just need to change your filter

const filter = t => t.route_id === 'M4'

to use some valid field, for example:

const filter = t => t.stop_name=== 'M4'

Javascript to download and process GTFS zip file

2 Answers2