1

I have been trying to configure offline unit tests for polymer web components that use the latest release of Firebase distributed database. Some of my tests are passing, but others—that look nigh identical to passing ones—are not running properly.

I have set up a project on github that demonstrates my configuration, and I'll provide some more commentary below.

Sample: https://github.com/doctor-g/wct-firebase-demo

In that project, there are two suites of tests that work fine. The simplest is offline-test, which doesn't use web components at all. It simply shows that it's possible to use the firebase database's offline mode to run some unit tests. The heart of this trick is the in the suiteSetup method shown below—a trick I picked up from nfarina's work on firebase-server.

suiteSetup(function() {
  app = firebase.initializeApp({
      apiKey: 'fake',
      authDomain: 'fake',
      databaseURL: 'https://fakeserver.firebaseio.com',
      storageBucket: 'fake'
  });
  db = app.database();

  db.goOffline();
});

All the tests in offline-test pass.

The next suite is wct-firebase-demo-app_test.html, which test the eponymous web component. This suite contains a series of unit tests that are set up like offline-test and that pass. Following the idea of dependency injection, the wct-firebase-demo-app component has a database attribute into which is passed the firebase database reference, and this is used to make all the firebase calls. Here's an example from the suite:

  test('offline set string from web component attribute', function(done) {
    element.database = db;
    element.database.ref('foo').set('bar');
    element.database.ref('foo').once('value', function(snapshot) {
      assert.equal(snapshot.val(), 'bar');
      done();
    });
  });

I have some very simple methods in the component as well, in my attempt to triangulate toward the broken pieces I'll talk about in a moment. Suffice it to say that this test passes:

test('offline push string from web component function', function(done) {
    element.database = db;
    let resultRef = element.pushIt('foo', 'bar');
    element.database.ref('foo').once('value', function(snapshot) {
      assert.equal(snapshot.val()[resultRef.key], 'bar');
      done();
    });
  });

and is backed by this implementation in wct-firebase-demo-app:

  pushIt: function(at, value) {
    return this.database.ref(at).push(value);
  },

Once again, these all pass. Now we get to the real quandary. There's a suite of tests for another element, x-element, which has a method pushData:

  pushData: function(at, data) {
    this.database.ref(at).push(data);
  }

The test for this method is the only test in its suite:

  test('pushData has an effect', function(done) {
    element.database = db;
    element.pushData('foo', 'xyz');
    db.ref('foo').once('value', function(snapshot) {
      expect(snapshot.val()).not.to.be.empty;
      done();
    });
  });

This test does not pass. While this test is running, the console comes up with an error message:

    Your API key is invalid, please check you have copied it correctly.

By setting some breakpoints and walking through the execution, it seems to me that this error comes up after the call to once but before the callback is triggered. Note, again, this doesn't happen with the same test structure described above that's in wct-firebase-demo-app.

That's where I'm stuck. Why do offline-test and wct-firebase-demo-app_test suites work fine, but I get this API key error in x-element_test? The only other clue I have is that if I copy in a valid API key into my initializeApp configuration, then I get a test timeout instead.

UPDATE:

Here is a (patched-together) image of my console log when running the tests.:

Console log showing the failing test

To illustrate the issue brought up by tony19 below, here's the console log with just pushData has an effect in x-element_test commented out:

Console log showing passing tests

Paul Gestwicki
  • 1,610
  • 1
  • 15
  • 18
  • Ah, very useful observation @tony19! I had been trusting the wct output without looking at the console logs. Perhaps it's more a house of cards than I had expected. – Paul Gestwicki May 29 '16 at 20:59

1 Answers1

0

The offline-test results are apparently false positives. If you check the Chrome console, offline-test actually throws the same error:

enter image description here

The error doesn't affect the test results most likely because the API key validation occurs asynchronously after the test has already completed. If you could somehow hook into that validation, you'd be able to to catch the error in your tests.

Commenting out all tests except for offline firebase is ok shows the error still occurring, which points to suiteSetup(). Narrowing the problem down further by commenting 2 of the 3 function calls in the setup, we'll see the error is caused by the call to firebase.initializeApp() (and not necessarily related to once() as you had suspected).

One workaround to consider is wrapping the Firebase library in a class/interface, and mocking that for unit tests.

tony19
  • 125,647
  • 18
  • 229
  • 307
  • That's not the same behavior I'm seeing here: if I comment out all the tests except `offline firebase is ok`, then everything works fine, without throwing any API error. In fact, if I only comment out `pushData has an effect` in `x-element_test`, then the whole suite runs without generating any warnings or errors in the console. – Paul Gestwicki May 30 '16 at 13:06
  • I'm curious how changes in x-element_test.html would have any effect on offline-test.html. The behavior I see is easily reproducible for me on OSX El Capitan, Chrome 51, at commit [`e5d1d55`](https://github.com/doctor-g/wct-firebase-demo/commit/e5d1d557eb25351d728835f3bca2a0dfbaddc315). – tony19 May 30 '16 at 16:51
  • Also to clarify, I'm running `offline-test` by opening `offline-test.html`. I'm assuming you're running it in a different way based on your screenshots. Do you see a difference when you open that file alone? – tony19 May 30 '16 at 17:23
  • I'm running the tests via `polymer test`, which is a wrapper around `wct`. – Paul Gestwicki May 31 '16 at 12:11
  • Running `offline-test.html` by itself in Chrome, I get a similar output to what you have posted here, although the specific errors and their placement are different. Probably most importantly, there's a 400 on the getProjectConfig call. It ends up with "Your API key is invalid." Chrome 51, Linux Mint 17. – Paul Gestwicki May 31 '16 at 12:42
  • Also, I wouldn't trust running any of the tests directly in the browser, based on what's explained at https://firebase.google.com/docs/web/setup#run_a_local_web_server_for_development: "Some parts of the Firebase SDK require that your website be served from a server rather than from a local filesystem." Of course, those same parts may be what are choking on my `x-element` tests. – Paul Gestwicki May 31 '16 at 12:50
  • A couple notes that may help: 1) The "Your API Key is invalid ..." error comes from Firebase Auth. When operating offline, auth doesn't really matter so that error may be benign. 2) When offline, firebase only raises events if it has *complete* data at the location. So if you write /foo/bar and listen on /foo, you won't get an event. But if you write /foo and then listen /foo, you will. So you *may* benefit from just doing .set(null) at the root ref after setting Firebase.goOffline() to make sure there's complete data everywhere and all of your listeners will fire. – Michael Lehenbauer Jun 15 '16 at 16:03