MEASURING MANUAL/AUTOMATED FUNCTIONAL TESTS CODE COVERAGE

Question

We are trying to generate code coverage on performing manual tests for application hosted in Azure app services.

I have a trail account in portal.azure.com with 'user' privilege.

Tried the below options: A. Use VS tools in Azure console and local B. Use OpenCover in Azure console

Both the above mentioned options using VS tools and Opencover has been used in the localhost (IIS server) to successfully generate code coverage on runnign manual tests.

A. Steps for code coverage using VS tools in Azure: 1. cd D:\Program Files (x86)\Microsoft Visual Studio 15.0\Team Tools\Performance Tools 2. vsinstr /coverage D:\home\site\wwwroot\bin\abc.dll 3. vsperfcmd /start:coverage /output:D:\home\LogFiles\Testing123.coverage 4. vsperfcmd -shutdown

Issue1: The generated code coverage file has error: Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see https://go.microsoft.com/fwlink/?LinkID=253731

However I'm able to generate code coverage on the same application which has been hosted in my local IIS server. The same 4 steps mentioned above is working. The issue is that the instrumented dll doesn't log the transactions performed on the hosted application.

Reference: https://gurunadhduvvuru.wordpress.com/2014/08/01/generating-functional-testing-code-coverage/

PS: We have tried starting the coverage process before and after publish the application.   

B. Steps for code coverage using Open Cover in Azure 1. OpenCover.Console.exe -target:C:\Windows\System32\inetsrv\w3wp.exe -targetargs:"-debug -s 1" -targetdir:C:\inetpub\wwwroot\bin\ -filter:+[] -register:user -output:C:\Users\coverage2.xml

Issue 2: Access denied is displayed when the application is not running.
Issue 3: The generated coverage file is empty and doesn't has the same error mentioned in Issue 1, on executing the opencover command when the application is running.

Reference: https://automationrhapsody.com/code-coverage-manual-automated-tests-opencover-net-applications/

Any help on the reason for the generated coverage file to be empty ? or any reference to suffice my requirement would be much appreciated?

Answer 1

It appears that it is a limitation of Azure WebApp Sandbox, all Azure Web Apps run in a secure environment called a sandbox. Each app runs inside its own sandbox, isolating its execution from other instances on the same machine as well as providing an additional degree of security and privacy which would otherwise not be available.

You cannot install arbitrary software when running on Azure App Service. Generally, you cannot make any machine level changes (e.g. registry changes).

Reference: https://github.com/projectkudu/kudu/wiki/Azure-Web-App-sandbox. If your requirement fits, try this on an Azure VM running IIS, where you have greater control on managing the VM.