1

After integrating with Travis-CI I've noticed that false positives are possible when a spec file for some reason throws an error.

All the files linted on travis

Running "jshint:files" (jshint) task
Verifying property jshint.files exists in config...OK
Options: force=false, reporterOutput=null, jshintrc=".jshintrc", ignores=["bower_components/**"]
OK
>> 29 files lint free.

But when phantom starts up one of the specs fails and this only happens within travis. I an other coworkers can checkout a fresh repo and run the same grunt task without a hitch.

Testing jasmine specs via phantom
>> Error: xhrFailed at
>> http:/localhost:8000/bower_components/dojo/dojo.js:301 
>> http:/localhost:8000/bower_components/dojo/dojo.js:1509 
>> http:/localhost:8000/bower_components/dojo/dojo.js:93 
>> http:/localhost:8000/bower_components/dojo/dojo.js:728 
>> http:/localhost:8000/bower_components/dojo/dojo.js:1247 
>> http:/localhost:8000/bower_components/dojo/dojo.js:746 
>> http:/localhost:8000/bower_components/dojo/dojo.js:791 
>> http:/localhost:8000/bower_components/dojo/dojo.js:124 
>> http:/localhost:8000/widgets/tests/spec/SpecSearch.js:379 
Jasmine Runner Starting...

This error is swallowed, the other spec files are run, and if they all pass travis thinks all is well.

What ways do we have to combat this? Basically the entire spec files will not load or run yet we have a build:passing image. I want to be able to trust the CI without having to look at the output.

I can run 74 tests with grunt travis locally, but when run on travis it passes with 55.

Steve
  • 25,806
  • 2
  • 33
  • 43
  • As a workaround you can add jshint to your CI setup as well. – Lajos Veres May 13 '14 at 15:49
  • @LajosVeres linting is part of the CI setup. There are no lint errors. Like I said, I can run this locally without seeing this problem. It only happens when invoked on the travis server. – Steve May 13 '14 at 16:56
  • Maybe I misunderstood something. You wrote "spec file ...cannot be parsed". My idea is to detect this bad state with jshint. (it detects parse errors also.) Anyway this is only a workaround. If you can find a proper solution, you should choose it. – Lajos Veres May 13 '14 at 20:27
  • @LajosVeres I updated the wording and pasted the linting output as proof. – Steve May 13 '14 at 22:42
  • I've just found the problem in my test files: all suites passing silently when a problem occurs in the file itself. Even worse, I got false positives with linted file for some, not understood yet problem. Do you have any new info or clue on the topic after 2 years? – 108adams Jun 21 '16 at 14:20
  • Nope, sorry @108adams, no new news on my end. – Steve Jun 21 '16 at 15:32
  • @Steve I got green build with a spaghetti spam created by a crazy git merge :) The only solution I know as for now is to harden your tests with jshint/eslint pre-step in your gulp/grunt/whatever chain. Greetings :) – 108adams Jul 07 '16 at 08:10

0 Answers0