1

My team has a battery of traditional, non-ui tests for a Xamarin application. We'd like to run these tests in the XTC but haven't found much documentation about running traditional tests in the XTC.

My best guess is to load our Xamarin Unit Test app into the cloud and then use the UI controls to run and check our unit tests in this app. Is there a more straight forward way that I'm missing?

And again, just to be clear, we don't want to UI test our app directly. We've architected from the ground up using IOC so that we get about 95% test coverage without full integration testing.

For example, here's one of our tests:

[Test]
public void ExecuteThreadSafeActionThrowsNoException()
{
    var test = new List<int> { 1, 2, 3 };

    Assert.DoesNotThrow(() =>
    {
        _concurrent.ExecuteThreadSafe(() =>
        {
            test.Add(4);

            foreach (var i in test)
            {
                Task.Delay(DelayTime).Wait();
            }

            test.Add(5);
        });
    });
}

Where _concurrent is an interface that has different implementations depending on Platform (iOS and Android) and OS version. All this tests is that our threading code works as expected on multiple devices.

Mark Rucker
  • 6,952
  • 4
  • 39
  • 65
  • So you want to run a `Unit Test Runner` via Xamarin Test Cloud? – Jon Douglas Jan 04 '17 at 20:39
  • I don't know. Do I? That's the only way I can think of running traditional unit tests in XTC. What we really want is a way to run our tests against many OS versions. XTC seemed the most logical way to do that. That's what we do locally to execute our Unit Tests against different OS versions. – Mark Rucker Jan 04 '17 at 20:40
  • Could you please specify exactly what you want to test and update your question accordingly? – Demitrian Jan 04 '17 at 20:41
  • 1
    I could see you using the `NUnit.Xamarin` test runners and uploading the binary to test on a plethora of devices: https://github.com/nunit/nunit.xamarin However I'm not so sure of your best course of action to determine if the `Test` should pass/fail given that this would be mixing `UI Testing` with `Unit Testing`. I would think that you could query a UI element to ensure a `Passed / Failed` state. – Jon Douglas Jan 04 '17 at 20:44
  • @JonDouglas Yeah, that's what I'm thinking. NUnit.Xamarin is what we already use locally to run our tests within our emulated devices. Still, it felt kind of wonky for XTC, so I thought I'd ask. – Mark Rucker Jan 04 '17 at 20:48
  • @JonDouglas Also, probably worth noting, I even tested out the Unit Test project type provided by Xamarin proper to see if it auto hooks up when loaded from an XTC project. As far as I can tell it doesn't. I still have to walk the app's UI in order to run and confirm test results. – Mark Rucker Jan 04 '17 at 20:53
  • I think it's a wonky situation as there is a clear distinction between `Unit` and `UI` testing. This is one of those edge cases where there isn't a straight-forward solution. – Jon Douglas Jan 04 '17 at 20:54
  • Which project are you referring to? Since XTC is a `UI Test` platform, you would have to adhere to the ideaologies of `UI Testing` to determine your `Unit Test` results given you run the `Unit Test Runner` in the binary you upload. – Jon Douglas Jan 04 '17 at 20:55
  • [this](https://developer.xamarin.com/guides/ios/deployment,_testing,_and_metrics/touch.unit/) project type – Mark Rucker Jan 04 '17 at 20:56
  • Yeah, iOS and Android default templates use `NUnitLite`, but I personally feel that `NUnit.Xamarin` is more feature rich and supported by the `NUnit` community. – Jon Douglas Jan 04 '17 at 21:00
  • @JonDouglas agreed :) That's why we use it. I also thought the UI was nicer looking. The cleaner UI was a selling point when first pitching unit tests. – Mark Rucker Jan 04 '17 at 21:02
  • 1
    @MarkRucker I've used XTC/MMC (and private test clouds) to run NUnit/xUnit for devices and have the *app* posting the results via an IP Writter back to our test collection server. This is a part of the build/release CI so the NUnit/xUnit tests (non-device) assemblies are built and tested locally, if pass, build test app for devices, post them to test cloud, collect results, if pass, build/post UI-based test apps, etc... – SushiHangover Jan 04 '17 at 23:25

0 Answers0