0

We have a number of WatiN tests running from CruiseControl, but we get intermittent failures. The most common failure point is when doing really simple things, such as clicking a link. Here are two example stack traces...

WatiN.Core.Exceptions.TimeoutException: Timeout while waiting for main document becoming available ---> System.IO.IOException: The operation is not allowed on non-connected sockets. at System.Net.Sockets.NetworkStream.InitNetworkStream(Socket socket, FileAccess Access) at WatiN.Core.Native.Mozilla.FireFoxClientPort.SendCommand(String data) at WatiN.Core.Native.Mozilla.FireFoxClientPort.SendAndRead(String data, Boolean resultExpected, Boolean checkForErrors, Object[] args) at WatiN.Core.Native.ClientPortBase.WriteAndReadAsBool(String data, Object[] args) at WatiN.Core.Native.JSBrowserBase.IsLoading() at WatiN.Core.Native.JSWaitForComplete.b__0() at WatiN.Core.UtilityClasses.TryFuncUntilTimeOut.Try[T](DoFunc1 func) --- End of inner exception stack trace --- at WatiN.Core.UtilityClasses.TryFuncUntilTimeOut.ThrowTimeOutException(Exception lastException, String message) at WatiN.Core.UtilityClasses.TryFuncUntilTimeOut.Try[T](DoFunc1 func) at WatiN.Core.WaitForCompleteBase.WaitUntil(DoFunc`1 waitWhile, BuildTimeOutExceptionMessage exceptionMessage) at WatiN.Core.Native.JSWaitForComplete.WaitWhileDocumentNotAvailable() at WatiN.Core.FireFox.WaitForComplete(Int32 waitForCompleteTimeOut) at WatiN.Core.Element.FireEvent(String eventName, Boolean waitForComplete, NameValueCollection eventProperties)
at WatiN.Core.Element.ClickImpl(Boolean waitforComplete) at ... [our code]

...and also...

WatiN.Core.Exceptions.TimeoutException: Timeout while waiting for main document becoming available at WatiN.Core.UtilityClasses.TryFuncUntilTimeOut.ThrowTimeOutException(Exception lastException, String message) at WatiN.Core.UtilityClasses.TryFuncUntilTimeOut.Try[T](DoFunc1 func) at WatiN.Core.WaitForCompleteBase.WaitUntil(DoFunc1 waitWhile, BuildTimeOutExceptionMessage exceptionMessage) at WatiN.Core.Native.JSWaitForComplete.WaitWhileDocumentNotAvailable() at WatiN.Core.FireFox.WaitForComplete(Int32 waitForCompleteTimeOut) at WatiN.Core.Element.FireEvent(String eventName, Boolean waitForComplete, NameValueCollection eventProperties)
at WatiN.Core.Element.ClickImpl(Boolean waitforComplete) at ... [our code]

The odd thing is that it mentions JSWaitForComplete, which seems odd as that exception is from a plain link that doesn't have any attached Javascript. In our test fixtures we make sure to close the browser, so there shouldn't be stale FF instances clogging things up. Its all a bit of a mystery.

Does anyone have any suggestions for things we can try to fix this issue?

This is running WatiN 2.0 Final and Firefox 3.6.13.

GoatInTheMachine
  • 3,583
  • 3
  • 25
  • 35
  • This guy had a similar problem, doesn't look like it was solved though: http://stackoverflow.com/questions/1145619/watin-tests-failing-with-timeoutexception-when-run-from-cruisecontrol – Grant Crofton Apr 07 '11 at 15:45
  • Yep I saw that, and had similar results searching the web for answers. Don't really want to have to re-write all of our tests using Selenium (which I wanted to do in the first place!) – GoatInTheMachine Apr 07 '11 at 15:47
  • Do you get the same issue with IE? Does a liberal spreading of WaitForComplete()s or Thread.sleep()s make any difference? (just stabbing in the dark, I've never used Watin with CC or FF). – Grant Crofton Apr 07 '11 at 15:56
  • @Grant, we get similar issues with IE, yes. I've also tried lots of sleeps between tests, before and after shutdowns and startups, everywhere...it certainly helps, but I'd rather a proper fix! – GoatInTheMachine Apr 07 '11 at 16:00

1 Answers1

1

One issue we had when running a series of functional tests in Firefox was that the test scripts were finishing one test, and then moving on to the next test before Firefox had finished shutting down.

When the next test tried to start running, the first thing it tried to do was fire up Firefox, but because Firefox was still busy shutting down the previous session, it would intermittently throw up an error box instead of starting the browser. This obviously then caused the test to fail.

The solutions we came up with were:

  1. add a delay to the end of each test so that the browser has time to shut down properly.

  2. Use a different browser, or a combination of browsers for the tests.

We haven't revisited this since FF4 was released, but I do note that FF4 has much quicker startup and shut-down times, so the issue may also be resolved by upgrading Firefox. This may or may not be an option, depending on whether you actually want to run the tests in FF3.6

Spudley
  • 166,037
  • 39
  • 233
  • 307
  • I think there are issues with WatiN and FF4 so we don't want to upgrade yet. Regards the tests, we're currently restarting the browser on a fixture basis rather than for each test, so I don't think its the startup that is the problem. Useful suggestions though! – GoatInTheMachine Apr 07 '11 at 15:35