6

Im having a very sneaky issue in Xcode and its "Log Navigator".

I've recreated the issue like so; I have a very simple OCUnit test:

-(void)testSimpleFailure{

    STFail(@"Please fail!");
}

I clean my build, I build my test scheme, and run my tests.

In the output console window I see all my test execute, including the one above. The console displays the test actually ran AND failed:

##teamcity[testFailed name='-|[SimpleTestClassTests.testSimpleFailure|]' message='Please fail!' details='']

HOWEVER, Xcode displays the "Tests Succeeded overlay" and the "Log Navigator" on the left says there are no issues and all is green.

Has anyone else had a similar issue? Does anyone know how to resolve this?

I would very much like to not get a false positive before I commit.

Edited to clarify my question.

Nick L
  • 145
  • 1
  • 8

6 Answers6

11

I'm running my application-tests and unit-tests on a device. I'm getting the same results as you - tests are failing but the Log Navigator says everything passes. Not only that, I can't set breakpoints in my code.

The solution for me was to "Edit Scheme...", chose the "Test" scheme, and change my Debugger from "LLDB" to "GDB". I'm using Xcode 4.3.2, with my compiler set for all targets to LLVM 3.1

After changing my debugger for the Test scheme to GDB, The Log Navigator reports the test failures, and I can set breakpoints and debug as well while the tests are running.

link: search for "Unit tests crash before executing any code" (LLDB bug) https://github.com/kstenerud/iOS-Universal-Framework

Shazron
  • 2,446
  • 1
  • 18
  • 30
7

Ok, after about 3 developers over 5 days looking at this problem, we have found a solution:

rm -rf Library/Application\ Support/iPhone\ Simulator

Or, alternatively, if blowing away an entire directory tree isn't your thing:

Go to your iOS Simulator application. Choose iOS Simulator from the top menu and choose "Reset Content and Settings..."

Apparently something gets corrupted between the simulator and Xcode, so Xcode cannot hook to the simulator to watch the tests. Xcode believes no tests ran, therefore the tests pass.

Exactly what causes this, we aren't sure, but after 5 grueling days and 1 solution later, we don't care because we're heading to the nearest bar.

Cheers.

Nick L
  • 145
  • 1
  • 8
  • 3
    Have you seen this behavior when unit testing on device? My "solution" was to set a break point at the top of the test I'd just written and step through. Did you file a radar? I'd like to dupe it. – edelaney05 Feb 08 '12 at 20:21
2

I just read this long post about the issue that also affects Mac apps:

http://masonmark.com/the-xcode-fairy/

The trick seems to be to quit Xcode and open the project again. The bug is intermittent and will sometimes be here, or never be here, and it stays in this state until the next restart of Xcode.

It's a good read too.

charles
  • 11,212
  • 3
  • 31
  • 46
0

What worked for me: Reset the Simulator like mentioned by Nick L in his answer PLUS closing XCode and restarting. Then every build with an STAssertTrue(false) or STFail will give me a negative result.

Community
  • 1
  • 1
EeKay
  • 6,494
  • 3
  • 24
  • 24
0

If you are using XCTest, make sure you are deploying to an iOS 7 simulator. If you deploy to iOS 6 it will say build succeeds, test fails but not give any logging information in the debugger.

Rudiger
  • 6,749
  • 13
  • 51
  • 102
0

Yes, I've seen this, intermittent and annoying.

Do make sure the top (most recent) log message is selected.

zaph
  • 111,848
  • 21
  • 189
  • 228
  • Luckily, each of the Log Navigator logs has a time stamp, so I'm sure I've selected the correct log. – Nick L Dec 09 '11 at 16:42
  • Yeah, I understand and I know that the error you mention is real, I get it too. It can be rather embarrassing (and to Apple in abstentia ) when I have that problem during demonstrating TDD. :-) – zaph Dec 09 '11 at 16:47
  • How do you get around this? This has been happening for the last day. – Nick L Dec 09 '11 at 17:52