4

I'm testing my android application using calabash-android which provides it's own "test project", with a script that renames it's package to reflect the app under test, and then uses an InstrumentationTestRunner subclass:

adb shell am instrument -w -e class sh.calaba.instrumentationbackend.InstrumentationBackend #{ENV['TEST_PACKAGE_NAME']}/sh.calaba.instrumentationbackend.CalabashInstrumentationTestRunner

I'll accept any answer that allows me to generate a code coverage report similar to Emma or Cobertura for an Android app, with data collected while being tested in calabash-android.

In attempt to get Emma working I have...

  • Attempted to follow these instructions to get Maven to build my project (because it's been too long since I used ant). A coverage.em file is generated in target/emma
  • Modified the calabash-android script, adding "-e coverage true"
  • When I run calabash-android, I eventually see "Generated coverage data to /data/data/my.project/files/coverage.ec"
  • adb -e pull /data/data/my.project/files/coverage.ec coverage.ec

...so now I should be able to run:

  • java -cp $ANDROID_HOME/tools/lib/emma.jar emma report -r html -in target/emma/coverage.em,coverage.ec

but I get an error:

EMMA: processing input files...
java.io.UTFDataFormatException: malformed input around byte 107

...So I assume that there's something wrong with the android maven plugin, and I'm trying to figure out how to generate the coverage.em file. I've ran "android update project -p ." and "ant emma" and "ant emma debug", but I can't find coverage.em anywhere...

...The generated build.xml file seems to imply that the generation of the coverage.em file is only generated when you run "ant emma test", but I don't think that's going to work because the test app is controlled by calabash-android.

In attempt to get Cobertura working I have...

  • Googled various forms of "cobertura android", but it doesn't seem as if anyone has had any luck.
  • Attempted to configure a cobertura profile in my Maven pom file to instrument the classes, but (in Maven 3) I get
    • a whole heap of warnings about log4j and ant having "InnerClasses" attributes and that I should recompile them from source
    • an error that com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.3.0:dex "ANDROID-040-001: Could not execute: Command = /bin/sh -c -cd /path/to/myproject && java -jar $ANDOID_HOME/platform-tools/lib/dx.jar --dex ..."
EXCEPTION FROM SIMULATION:
local variable type mismatch: attempt to set or access a value of type 
java.lang.Class using a local variable of type java.lang.reflect.Type[].  
This is symptomatic of .class transformation tools that ignore local variable information.

... this is probably why nobody's been able to get cobertura working on Android?

Community
  • 1
  • 1
Nicholas Albion
  • 3,096
  • 7
  • 33
  • 56
  • I've only been able to get cobertura running for unit tests running in the JVM via Robotium. I'm pretty sure it messes with the byte code, which precludes it from being used on Dalvik. – Edward Dale Aug 24 '12 at 08:33

1 Answers1

5

The problem is that maven-android-plugin uses version 2.1.5320 of emma, whereas the Android tools use version 2.0.5312. As discussed here, those two versions are incompatible.

The fix is to only use a single version across the two tools. I've been able to get it to work by cloning the maven-android-plugin repo, setting the emma dependency version back to 2.0.5312, and installing it to my local repository. Make sure the emma dependency in your under-test project is also correct, and then you should be able to generate a coverage.

The alternative is to make sure all of the tools are using the newest version. I haven't tested it, but it might work if you do the report generation from maven is the version will be the same then. You could also download the newest version of emma and generate the report using the jar from that package.

Edward Dale
  • 29,597
  • 13
  • 90
  • 129
  • Ah, thanks! Maybe I can specify a specific Emma version in the plugin definition in my pom file. ...or use maven to generate the report instead of using the android tools version – Nicholas Albion Aug 24 '12 at 09:42
  • 1
    I've also added [an issue](https://github.com/calabash/calabash-android/issues/103) for calabash-android so you don't have to modify the the code. – Edward Dale Aug 24 '12 at 09:46
  • 1
    I can confirm that this works: java -cp ~/.m2/repository/emma/emma/2.1.5320/emma-2.1.5320.jar emma report -r html -in ../../my-project/target/emma/coverage.em,coverage.ec -sp ../../my-project/src – Nicholas Albion Aug 27 '12 at 00:33
  • Thanks for that command line! I was able to make my emma report generated with that one as well. – Richard Guion Apr 24 '13 at 21:23
  • Thank you very much!! I have spend almost one week tracing that problem. It felt pretty depressing to find out it was because of version number... – Karri Rasinmäki Feb 04 '14 at 19:10