GTFS-realtime feed is incorrectly reported to be missing a header

Question

I am trying to parse a gtfs-realtime transit feed in Android.

I have made a barebones Android app, that uses the Java GtfsRealtime binding library based on the sample code shown here: https://github.com/MobilityData/gtfs-realtime-bindings/tree/final-google-version/java. The feed I am trying to parse is the GRT tripupdates feed: http://webapps.regionofwaterloo.ca/api/grt-routes/api/tripupdates. However, upon doing so I get an error:

W/System.err: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: header

I have experimented parsing the feed with the nodejs code shown at the bottom of this question and it appears that the GRT tripupdate feed infact contains a header!

If I replace the GRT feed in the Android app, with the Thunderbay feed (http://api.nextlift.ca/gtfs-realtime/tripupdates.pb), the gtfs library in Android is able to parse the feed just fine, and correctly prints info contained in the header. So, I do know that a tripupdate protobuf file is parseable using the library.

Additionally, taking a look at the data contained within the 2 different sources, by using the external viewer at http://transitfeeds.com/p/thunder-bay-transit/431/source and http://transitfeeds.com/p/grand-river-transit/624/source shows that both feeds have a similar structured header, with the same gtfs_realtime_version.

I’d appreciate any help on parsing the GRT feed in Android. I am not sure why protobuf thinks there isn’t a header in the GRT feed.

Android code:

package com.myapp.grttracker;

import android.os.AsyncTask;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.Log;

import java.net.URL;

import com.google.transit.realtime.GtfsRealtime.FeedEntity;
import com.google.transit.realtime.GtfsRealtime.FeedMessage;

public class MainActivity extends AppCompatActivity {
    private static final String TAG = MainActivity.class.getName();

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        new RetrieveFeedTask().execute("");
    }
}

class RetrieveFeedTask extends AsyncTask<String, Void, FeedMessage> {
    private static final String TAG_TASK = "---ASYNC_TASK---";
    private Exception exception;
    private FeedMessage feed;

    protected FeedMessage doInBackground(String... urls) {
        try {
            URL url = new URL("http://webapps.regionofwaterloo.ca/api/grt-routes/api/tripupdates");
            //http://api.nextlift.ca/gtfs-realtime/tripupdates.pb
            feed = FeedMessage.parseFrom(url.openStream());
            System.out.println(feed.getHeader().toString());
            return feed;
        } catch (Exception e) {
            this.exception = e;
            return null;
        }
    }

    protected void onPostExecute(FeedMessage feed) {
        if (this.exception != null) {
            Log.e(TAG_TASK, "Error getting feed");
            this.exception.printStackTrace();
        } else {
            for (FeedEntity entity : feed.getEntityList()) {
                // do stuff
            }
        }
    }
}

app's build.gradle

apply plugin: 'com.android.application'

android {
    compileSdkVersion 28
    defaultConfig {
        applicationId "com.myapp.grttracker"
        minSdkVersion 26 // 26 to support gtfs
        targetSdkVersion 28
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation 'com.android.support:appcompat-v7:28.0.0'
    implementation 'com.android.support.constraint:constraint-layout:1.1.3'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'com.android.support.test:runner:1.0.2'
    androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
    //https://search.maven.org/search?q=a:gtfs-realtime-bindings
    implementation group: 'io.mobilitydata.transit', name: 'gtfs-realtime-bindings', version: '0.0.5'
}

index.js

var GtfsRealtimeBindings = require('gtfs-realtime-bindings');
var request = require('request');

var requestSettings = {
  method: 'GET',
 // url: 'http://api.nextlift.ca/gtfs-realtime/tripupdates.pb',  /// Thunderbay
  url: 'http://webapps.regionofwaterloo.ca/api/grt-routes/api/tripupdates', /// GRT
  
  encoding: null
};
request(requestSettings, function (error, response, body) {
  if (!error && response.statusCode == 200) {
    var feed = GtfsRealtimeBindings.transit_realtime.FeedMessage.decode(body);
    console.log(feed)
  }
});

Output of Thunderbay feed, using nodejs

FeedMessage {
  entity:
   [ FeedEntity { id: '0', tripUpdate: [TripUpdate] }, 
     FeedEntity { id: '1', tripUpdate: [TripUpdate] } ],
  header:
   FeedHeader {
     gtfsRealtimeVersion: '1.0',
     incrementality: 0,
     timestamp: Long { low: 1581568947, high: 0, unsigned: true } } }

Output of GRT feed in Nodejs

FeedMessage {
  entity:
   [ FeedEntity { id: '1985737', isDeleted: false, tripUpdate: [TripUpdate] },
     FeedEntity { id: '1985738', isDeleted: false, tripUpdate: [TripUpdate] },
     ... 31 more items ],
  header:
   FeedHeader {
     gtfsRealtimeVersion: '1.0',
     incrementality: 0,
     timestamp: Long { low: 1581568953, high: 0, unsigned: true } } }

Answer 1

I think you need to add the x-api-key to the URL connection header

to do that you need to open connection and adding it like that

url.openConnection().setRequestProperty("x-api-key", "YOUR API KEY")